Medical image analysis method, electronic device and computer-readable storage medium

Abstract

The present application provides a medical image analysis method, an electronic device and a computer-readable storage medium. The method includes: obtaining a classification result of a medical image by a first model, and a detection result of the medical image by a second model, the classification result including a first disease grade of disease, and the detection result including lesion information, determining a second disease grade corresponding to the detection result according to a disease grading rule, determining analysis steps of the medical image according to a matching situation of the first disease grade and the second disease grade. By utilizing the present application, the accurate analysis of the medical image can be achieved.

Description

[Technical Field]

[0001] This application belongs to the biomedical field and relates to image processing technology, and in particular to a medical image analysis method, electronic device and computer-readable storage medium. [Previous Technology]

[0002] In the field of medical technology, medical imaging has become an important tool to assist doctors in diagnosing diseases. Taking fundus image analysis of diabetic retinopathy (DR) as an example, diabetic retinopathy is also known as diabetic retinopathy. Doctors can use fundus images to determine whether there are lesions such as microaneurysms and hemorrhages, thereby determining whether the patient has diabetic retinopathy and the severity of the disease.

[0003] In related technologies, models are often used to analyze medical images. However, this method is limited by the accuracy, generalization ability, and sufficiency of training data, and the analysis results of medical images are often inaccurate.

[0004] If medical images cannot be accurately analyzed, it will affect the subsequent diagnosis of the disease. [Summary of the Invention]

[0005] In view of the above, it is necessary to provide a medical image analysis method, electronic device and computer-readable storage medium that can solve the technical problem of difficulty in accurately analyzing medical images.

[0006] On one hand, this application provides a medical image analysis method, the method comprising: obtaining a classification result of a first model on a medical image and a detection result of a second model on the medical image, wherein the classification result includes a first disease level of the disease, the detection result includes lesion information, determining a second disease level corresponding to the detection result according to disease grading rules, and determining the analysis steps of the medical image based on the matching situation of the first disease level and the second disease level.

[0007] In some embodiments of this application, the classification result includes a first probability of the first disease level, the detection result also includes the confidence level of the lesion information, the lesion information includes lesion type and lesion location, and the method further includes: determining the key lesion type of the second disease level according to the key lesion rule, and determining the second probability corresponding to the detection result based on the confidence level of the lesion information corresponding to the key lesion type.

[0008] In some embodiments of this application, determining the second probability corresponding to the detection result based on the confidence level of the lesion information corresponding to the key lesion type includes: if the key lesion type is multiple, determining the highest confidence level from all confidence levels corresponding to the multiple key lesion types, and determining the second probability based on the highest confidence level; if the key lesion type is single, determining the confidence level corresponding to the key lesion type as the second probability.

[0009] In some embodiments of this application, when the first disease level and the second disease level do not match, the following analysis steps are performed: the first probability is compared with a first preset value, and when the first probability is less than the first preset value, the second probability is compared with a second preset value.

[0010] In some embodiments of this application, if the first probability is greater than or equal to the first preset value, or if the first probability is less than the first preset value and the second probability is less than the second preset value, the following analysis steps are performed: the detection result is adjusted according to the difference between the first disease level and the second disease level to obtain a corrected detection result, and the analysis result of the medical image is determined according to the first disease level and the corrected detection result.

[0011] In some embodiments of this application, adjusting the detection result based on the difference between the first disease level and the second disease level to obtain a corrected detection result includes: if the first disease level is higher than the second disease level, excluding the lesion type corresponding to the second disease level from the lesion types corresponding to the first disease level to obtain the remaining lesion types; adjusting the confidence level corresponding to each lesion type among the remaining lesion types until a confidence level that meets a preset condition is obtained; determining the corrected detection result based on the confidence level that meets the preset condition, the lesion type corresponding to the confidence level that meets the preset condition, and the lesion location; if the first disease level is lower than the second disease level, determining the corrected detection result based on the lesion type, lesion location, and confidence level corresponding to the first disease level.

[0012] In some embodiments of this application, if the first probability is less than the first preset value and the second probability is greater than or equal to the second preset value, the following analysis steps are performed: the analysis result of the medical image is determined based on the second disease level and the detection result.

[0013] In some embodiments of this application, determining the second disease level corresponding to the test result according to the disease grading rules includes: comparing the test result with lesion information corresponding to a plurality of preset disease levels to determine the second disease level, wherein the lesion information includes preset lesion types and lesion locations of preset lesion types.

[0014] In some embodiments of this application, if the second disease level is the lowest preset disease level among the plurality of preset disease levels, then the key lesion type of the second disease level is the preset lesion type corresponding to the second disease level; if the second disease level is not the lowest preset disease level, then the key lesion type of the second disease level includes the preset lesion type remaining after excluding the preset lesion type corresponding to the previous preset disease level from the preset lesion types corresponding to the second disease level.

[0015] In some embodiments of this application, when the first disease level matches the second disease level, the following analysis steps are performed: the analysis result of the medical image is determined based on the detection result, the first disease level, or the second disease level.

[0016] On the other hand, this application provides an electronic device, the electronic device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the electronic device implements the medical image analysis method.

[0017] On the other hand, this application provides a computer-readable storage medium storing a computer program that, when executed by a processor in an electronic device, implements the medical image analysis method.

[0018] In the medical image analysis scheme provided in this application embodiment, medical images are processed using a complex model to facilitate the verification of classification results, thereby reducing the risk of inaccurate analysis of medical images due to the limitations of a single model. Since the first disease level is obtained directly from classifying the medical image using the first model, while the second disease level is indirectly determined from the detection results of the medical image using the second model, the second disease level has higher uncertainty due to the uncertainty of the detection results. Therefore, the first disease level has higher reliability than the second disease level. By matching the first disease level with the second disease level, the accuracy of the classification results can be verified, thereby accurately determining the subsequent analysis steps for the medical image. Thus, by utilizing a multi-model verification mechanism, accurate analysis of medical images can be achieved.

Implementation Method

[0019] It should be noted that in this application, "at least one" refers to one or a plural, and "plural" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can mean: A exists alone, A and B exist simultaneously, or B exists alone, where A and B can be singular or plural. The terms "first," "second," "third," "fourth," etc. (if present) in the specification, scope of the invention application, and drawings of this application are used to distinguish similar objects, rather than to describe a specific order or sequence.

[0020] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design solutions. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a specific manner. Without conflict, the following embodiments and features described therein can be combined with each other.

[0021] In the field of medical technology, medical images have become an important tool to assist doctors in diagnosing diseases. Taking fundus image analysis of diabetic retinopathy (DR) as an example, doctors can determine the presence of lesions such as microaneurysms and hemorrhages based on fundus images, thereby determining whether the patient has DR and the severity of the disease. Related technologies often utilize models to analyze medical images. However, this method is limited by the accuracy, generalization ability, and sufficiency of training data of the model, often resulting in inaccurate analysis results for medical images.

[0022] If medical images cannot be accurately analyzed, it will affect the subsequent diagnosis of diseases. In order to solve the above-mentioned technical problems, this application provides a medical image analysis method that can achieve accurate analysis of medical images.

[0023] The medical image analysis method provided in this application embodiment can be applied to one or more electronic devices. The electronic devices can be computers, mobile phones, laptops, servers, etc. The server can be a cloud server or a server cluster. This application does not limit the type of electronic device.

[0024] As shown in Figure 1, this is a flowchart of a medical image analysis method provided in an embodiment of this application. Depending on different needs, the order of each step in the flowchart can be adjusted according to actual requirements, and some steps can be omitted. The medical image analysis method is applied to electronic devices, such as the electronic device 10 shown in Figure 4.

[0025] Step S11: Obtain the classification results of the first model on the medical image and the detection results of the second model on the medical image. The classification results include the first disease level and the first probability of the first disease level. The detection results include lesion information and its corresponding confidence level. The lesion information includes the lesion location and its corresponding lesion type.

[0026] In some embodiments of this application, the first model includes, but is not limited to: logistic regression network, support vector machine network, decision tree network, random forest network, extreme gradient boosting network, convolutional neural network, and semantic analysis network. The convolutional neural network can be RestNet, GoogleNet, VGG Net, etc. The semantic analysis network can be Vision Transformer and Swing Transformer, etc.

[0027] In some embodiments, the disease may be a retinal disease. The first model can be trained on one or more of the aforementioned support vector machines or similar networks using medical images and corresponding labels as training samples. The labels may include the disease level of the medical image, such as level 0, level 1, level 2, level 3, and level 4. The electronic device can use the first model to map medical images to a plurality of preset disease levels, obtain a first probability that the medical image corresponds to each preset disease level, and select the preset disease level with the highest first probability from the plurality of preset disease levels as the first disease level.

[0028] The medical image can be a fundus image or other types of medical images, such as pathological slide images, lung CT images, colonoscopy images, brain MRI images, corneal images and retinal images, etc. In order to clearly illustrate the medical image analysis method provided in the embodiments of this application, the fundus image will be used as an example for explanation below.

[0029] Medical images can be obtained by taking pictures of the inside and back of the eye using a fundus camera. Since the inside and back of the eye include structures such as the retina, macula, optic disc, and blood vessels, medical images obtained by taking pictures of the inside and back of the eye can determine whether there are lesions in the eye, and, if so, determine the corresponding lesion type, also known as lesion characteristics. Lesion types in medical images include, but are not limited to: microaneurysms, hard exudates, soft exudates, hemorrhages, venous beading, intraretinal microvascular abnormalities (IRMAs), neovascularization, vitreous hemorrhage, and premacular hemorrhage.

[0030] The preset disease level can be customized, and this application does not limit this. For example, the preset disease level may include level 0, level 1, level 2, level 3, and level 4, etc. For example, through the first model, the electronic device obtains a medical image with a first probability of 0.1 corresponding to disease level 0, a first probability of 0.2 corresponding to disease level 1, a first probability of 0.5 corresponding to disease level 2, a first probability of 0.1 corresponding to disease level 3, and a first probability of 0.1 corresponding to disease level 4. Since the first probability of 0.5 for disease level 2 is the largest, the electronic device can determine that disease level 2 is the first disease level.

[0031] In some embodiments of this application, the second model may be an object detection model, including but not limited to: YOLO, PP-YOLO, RetinaNet, RestNet+FPN and EfficientNet.

[0032] In some embodiments, the second model can be trained on the target detection model using medical images and corresponding detection results as training samples. The detection results in the training samples can be referred to the description of the detection results in step S11. The electronic device can input the medical image into the second model for detection to obtain the above-mentioned detection results. The lesion information in the detection results can be one or multiple, and the lesion location in each lesion information can be one or multiple; this application does not impose any limitations on this. The electronic device can use a preset graphic (e.g., a rectangle) to select the lesion location in the medical image.

[0033] Each lesion location may have a corresponding lesion type, and different characters may be used to represent the lesion type. The characters may be numbers, letters, or a combination of numbers and letters. This application does not restrict the type of characters. For example, the number 1 may be used to represent one lesion type, the number 2 may be used to represent another lesion type, the number 3 may be used to represent yet another lesion type, and so on.

[0034] Each lesion location and each lesion type can have a corresponding confidence level, and the confidence level corresponding to each lesion information can be determined by the confidence levels corresponding to the included lesion locations and lesion types. For example, if any lesion information includes a lesion location and its corresponding lesion type, then the confidence level corresponding to the any lesion information can be the average or weighted average of the confidence levels corresponding to the lesion locations and their corresponding lesion types included in the any lesion information.

[0035] The above examples of lesion information are merely illustrative and are not limited to practical applications. For example, in other embodiments, lesion information may also include information such as lesion size.

[0036] In this embodiment, by using a complex model to process medical images, the classification results can be verified, thereby reducing the risk of inaccurate analysis of medical images due to the limitations of a single model.

[0037] In other embodiments of this application, after obtaining the detection results, the electronic device may retain only the lesion information with a confidence level greater than a preset confidence threshold from the detection results, as the basis for determining the second disease level below. Each lesion information may have a corresponding confidence threshold. The confidence thresholds corresponding to the lesion information of different lesion types may be the same or different, and this application does not impose any restrictions on this. For example, the confidence threshold corresponding to the lesion information of the lesion type represented by the value 1 may be 0.2, and the confidence threshold corresponding to the lesion information of the lesion type represented by the value 8 may be 0.6.

[0038] In this embodiment, by retaining lesion information with a confidence level greater than a preset confidence level threshold in the detection results, the accuracy of the detection results can be improved.

[0039] Step S12: Determine the second disease level corresponding to the test result according to the disease grading rules, determine the key lesion type of the second disease level according to the key lesion rules, and determine the second probability corresponding to the test result according to the confidence level corresponding to the key lesion type.

[0040] In some embodiments of this application, the electronic device determines the second disease level corresponding to the test result according to the disease grading rules, including: comparing the test result with lesion information corresponding to a plurality of preset disease levels, and determining the preset disease level corresponding to the lesion information that matches the test result among the plurality of preset disease levels as the second disease level.

[0041] The lesion information in the detection results can be compared with the lesion information of the preset disease level. The disease grading rule can define multiple preset disease levels and lesion information corresponding to each preset disease level. The lesion information includes preset lesion type and lesion location of preset lesion type, etc. Figure 2 shows an example of the disease grading rule provided in an embodiment of this application.

[0042] In the disease grading rules shown in Figure 2, five preset disease levels are defined: 0, 1, 2, 3, and 4. Disease level 0 corresponds to a state without fundus lesions. Disease level 1 corresponds to a lesion type of microaneurysm. Disease level 2 corresponds to lesion types including microaneurysms and other lesions such as hemorrhage, hard exudates, and soft exudates. Disease level 3 corresponds to any one of the following three conditions: 1. Severe retinal hemorrhage (approximately 20 hemorrhage points) in all four quadrants; 2. Pearl vein changes in two or more quadrants; 3. Intraretinal microvascular abnormalities in one or more quadrants. The quadrants can be obtained by dividing the posterior pole core region of the eyeball, which can be a circular area centered on the macula with a preset diameter (e.g., 5mm~6mm). Disease level 4 corresponds to lesion types including any one of the following two: 1. Neovascularization; 2. Vitreous hemorrhage or preretinal hemorrhage.

[0043] In addition to the lesion type corresponding to each preset disease level, the disease grading rules can also define the retinopathy type corresponding to each preset disease level. For example, in Figure 2, disease level 0 corresponds to no diabetic retinopathy; disease level 1 corresponds to mild non-proliferative diabetic retinopathy (Mild NPDR); disease level 2 corresponds to moderate non-proliferative diabetic retinopathy (Moderate NPDR); disease level 3 corresponds to severe non-proliferative diabetic retinopathy (Severe NPDR); and disease level 4 corresponds to proliferative diabetic retinopathy (PDR).

[0044] The key lesion rule can define the key lesion type corresponding to each preset disease level. Figure 3 shows an example of the key lesion rule provided in an embodiment of this application. The key lesion type corresponding to each preset disease level in Figure 3 can refer to other lesion types besides those indicated in black font among all preset lesion types corresponding to each preset disease level. For example, there is no key lesion type for disease level 0; the key lesion type for disease level 1 is microaneurysm; the key lesion types for disease level 2 include: hemorrhage, hard exudate, and soft exudate; the key lesion types for disease level 3 include: pearly veins and internal microvascular abnormalities; the key lesion types for disease level 4 include: neovascularization, vitreous hemorrhage, and preretinal hemorrhage.

[0045] For example, the electronic device determines the key lesion type of the second disease level according to the key lesion rule, including: if the second disease level is the lowest preset disease level among the plurality of preset disease levels, then the key lesion type of the second disease level is the preset lesion type corresponding to the second disease level; if the second disease level is not the lowest preset disease level, then the key lesion type of the second disease level includes the preset lesion type remaining after excluding the preset lesion type corresponding to the previous preset disease level from the preset lesion types corresponding to the second disease level.

[0046] For example, if the second disease level is level 2, and the preset lesion types corresponding to level 2 include microaneurysms, hemorrhage, hard exudates, and soft exudates, and the preceding disease level is level 1, and the preset lesion types corresponding to level 1 include microaneurysms, then the key lesion types corresponding to the second disease level are lesions excluding microaneurysms, including hemorrhage, hard exudates, and soft exudates. By analogy, the key lesion types can be determined when the second disease level is any preset disease level.

[0047] For example, the electronic device determines the second probability corresponding to the detection result based on the confidence level corresponding to the key lesion type, including: if there are multiple key lesion types, determining the highest confidence level from all confidence levels corresponding to the multiple key lesion types, and determining the second probability based on the highest confidence level; if there is only one key lesion type, determining the confidence level corresponding to the key lesion type as the second probability. Wherein, the highest confidence level can be determined as the second probability.

[0048] For example, if the second disease grade is 2, and the confidence levels for the presence of three microaneurysms are 0.7, 0.5, and 0.9, respectively, the confidence levels for two hard exudates are 0.3 and 0.7, respectively, and the confidence levels for four soft exudates are 0.1, 0.8, 0.4, and 0.7, respectively. Since microaneurysms are not a key lesion type for grade 2, while hard and soft exudates are key lesion types for grade 2, and the highest confidence level among the confidence levels for hard and soft exudates is 0.8, the second probability can be 0.8.

[0049] In this embodiment, the second disease level corresponding to the detection result is determined according to the disease grading rules, so as to verify the reliability of the first disease level below. Since the key lesion type of the second disease level is any lesion type other than the lesion type of the preset disease level that precedes and is adjacent to the second disease level, the key lesion type of the second disease level can accurately reflect the characteristics of the second disease level. Therefore, the second probability is determined by the highest confidence level among the key lesion types corresponding to the second disease level, which can improve the accuracy of the second probability.

[0050] Step S13: Determine whether the first disease level and the second disease level match.

[0051] For example, if the first disease level and the second disease level are the same, it can be determined that the first disease level and the second disease level match; if the first disease level and the second disease level are not the same, it can be determined that the first disease level and the second disease level do not match.

[0052] In some embodiments of this application, the electronic device can determine the analysis steps for the medical image based on the matching situation of the first disease level and the second disease level. Specifically, the matching situation includes two situations: the first disease level and the second disease level match, and the first disease level and the second disease level do not match. If the first disease level and the second disease level match, step S14 is executed; if the first disease level and the second disease level do not match, step S15 is executed.

[0053] In this embodiment, by verifying whether the first disease level and the second disease level match, it is possible to preliminarily determine whether the first disease level is reliable.

[0054] Step S14: Determine the analysis result based on the test results, the first disease level, or the second disease level.

[0055] In this embodiment, when the first disease level matches the second disease level, the first disease level can be determined to be reliable. Since the first disease level and the second disease level are the same, the electronic device can determine the detection result, the first disease level, or the second disease level as the analysis result of the medical image.

[0056] In other embodiments, when the first disease level and the second disease level are the same, the electronic device may also determine the higher of the detection result, the first disease level or the second disease level, and the first probability and the second probability as the analysis result of the medical image.

[0057] Step S15: Determine whether the first probability is less than the first preset value.

[0058] In some embodiments of this application, the first preset value can be customized, and this application does not limit it. If the first probability is less than the first preset value, step S16 is executed; if the first probability is greater than or equal to the first preset value, step S17 is executed.

[0059] In this embodiment, the first disease level is obtained by directly classifying the medical image through the trained first model, while the second disease level is indirectly determined by the detection results of the medical image through the second model. Since the detection results have uncertainty, the second disease level has higher uncertainty. Therefore, compared to the second disease level, the first disease level has higher reliability. When the first disease level and the second disease level do not match, a first probability is preferentially compared with a first preset value to further determine whether the first disease level is reliable. Through double verification, errors can be reduced, so as to accurately determine the analysis results of the medical image in the following text.

[0060] Step S16: Determine whether the second probability is less than the second preset value.

[0061] In some embodiments of this application, the second preset value can be customized, and this application does not limit this. If the second probability is less than the second preset value, step S17 is executed; if the second probability is greater than or equal to the second preset value, step S18 is executed.

[0062] In this embodiment, when the first probability is less than the first preset value, it can be determined that the reliability of the first disease level is low. At this time, the second probability of the second disease level is compared with the second preset value, and it is possible to quickly determine whether the second disease level is reliable.

[0063] Step S17: Adjust the detection results according to the difference between the first disease level and the second disease level to obtain corrected detection results, and determine the analysis results of the medical image according to the first disease level and the corrected detection results.

[0064] In some embodiments of this application, the electronic device adjusts the detection result based on the difference between the first disease level and the second disease level to obtain a corrected detection result, including: if the first disease level is higher than the second disease level, excluding the lesion type corresponding to the second disease level from the lesion types corresponding to the first disease level, and obtaining the remaining lesion types; adjusting the confidence level corresponding to each lesion type among the remaining lesion types until a confidence level that meets a preset condition is obtained; determining the corrected detection result based on the confidence level that meets the preset condition, the lesion type corresponding to the confidence level that meets the preset condition, and the lesion location; if the first disease level is lower than the second disease level, determining the corrected detection result based on the lesion type, lesion location, and confidence level corresponding to the first disease level.

[0065] The adjustment of the confidence level can be achieved by an electronic device increasing the confidence level corresponding to each of the remaining lesion types according to a preset step size. The preset step size can be customized, and this application does not impose any restrictions on it. At least one confidence level must meet the preset condition, which can be greater than or equal to a preset threshold.

[0066] For example, following the above embodiment, if the first disease level is level 2 and the second disease level is level 1, since the first disease level is higher than the second disease level, the lesion type corresponding to the second disease level 1, namely microaneurysm, hemorrhage, hard exudate, and soft exudate, can be excluded from the lesion types corresponding to the first disease level 2: microaneurysm, hemorrhage, hard exudate, and soft exudate. Therefore, the remaining lesion types include hemorrhage, hard exudate, and soft exudate. The confidence level corresponding to the detection box for lesion types hemorrhage, hard exudate, and soft exudate can be increased according to a preset step size until a confidence level greater than a preset threshold is obtained. The lesion type and lesion location corresponding to the confidence level greater than the preset threshold are then determined as the corrected detection result.

[0067] For example, following the above embodiment, if the first disease level is level 1 and the second disease level is level 2, since the first disease level is lower than the second disease level, the lesion type (microaneurysm), detection frame and confidence level corresponding to the first disease level 1 can be determined as the corrected detection result.

[0068] In some embodiments, the electronic device may determine a first disease level and a correction detection result as the analysis result of the medical image. In other embodiments, the electronic device may determine a first disease level, a first probability, and a correction detection result as the analysis result of the medical image.

[0069] In this embodiment, when the first probability is greater than or equal to the first preset value, or when the first probability is less than the first preset value and the second probability is less than the second preset value, it can be determined that the first disease level and the first probability have higher reliability than the detection result and the second disease level and the second probability determined through the detection result. Therefore, by adjusting the detection result, the accuracy of the corrected detection result can be improved, thereby improving the accuracy of the analysis result.

[0070] In other embodiments, the confidence level can be adjusted multiple times. The electronic device can display the confidence level and the corresponding detection box and lesion type after each adjustment through the user interface for the user's reference.

[0071] Step S18: Determine the analysis results of the medical image based on the second disease level and the test results.

[0072] In some embodiments, the electronic device may determine the second disease level and the detection result as the analysis result of the medical image. In other embodiments, the electronic device may determine the second disease level, the second probability, and the detection result as the analysis result of the medical image.

[0073] In this embodiment, when the first disease level and the second disease level do not match, the first probability of the first disease level is less than the first preset value and the second probability of the second disease level is greater than or equal to the second preset value. It can be determined that the reliability of the first disease level is low and the reliability of the second disease level is high. Therefore, directly determining the second disease level and the detection result as the analysis result of the medical image can improve the accuracy of the analysis result, thereby achieving accurate analysis of the medical image.

[0074] In the medical image analysis scheme provided in this application embodiment, the first disease level is obtained by directly classifying the medical image through a trained first model, while the second disease level is indirectly determined by the detection results of the medical image through a second model. Since the detection results have uncertainty, the second disease level has higher uncertainty; therefore, the first disease level has higher reliability than the second disease level. In the case where the first disease level and the second disease level do not match, if the first probability is greater than or equal to a first preset value, or if the first probability is less than the first preset value and the second probability is less than the second preset value, the detection results are adjusted according to the difference between the first and second disease levels, which can improve the accuracy of the detection results. The analysis results of the medical image are determined based on the first disease level and the adjusted corrected detection results, which can improve the accuracy of the analysis results. In cases where the first disease level and the second disease level do not match, if the first probability is less than the first preset value and the second probability is greater than or equal to the second preset value, it means that the second disease level has higher reliability than the first disease level. Therefore, the analysis results of the medical image are determined based on the second disease level and the detection results, which can ensure the accuracy of the analysis results. In this way, through the multi-model validation mechanism, accurate analysis of medical images can be achieved.

[0075] As shown in Figure 4, this is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device 10 can be a computer, mobile phone, tablet computer, laptop computer, server, etc. The specific type of the electronic device 10 is not limited in this embodiment of the application.

[0076] In FIG4, the electronic device 10 may include a communication module 101, a memory 102, a processor 103, an input / output (I / O) interface 104, and a bus 105. The processor 103 is coupled to the communication module 101, the memory 102, and the input / output interface 104 through the bus 105.

[0077] Communication module 101 may include a wired communication module and / or a wireless communication module. The wired communication module may provide one or more wired communication solutions such as Universal Serial Bus (USB) and Controller Area Network (CAN). The wireless communication module may provide one or more wireless communication solutions such as Wireless Fidelity (Wi-Fi), Bluetooth (BT), mobile communication networks, frequency modulation (FM), near field communication (NFC), and infrared (IR).

[0078] Memory 102 may include one or more random access memory (RAM) and one or more non-volatile memory (NVM). The RAM can be directly read and written by the processor 103, and can be used to store executable programs (e.g., machine instructions) of other running programs, as well as user and application data. The RAM may include static random-access memory (SRAM), dynamic random-access memory (DRAM), synchronous dynamic random-access memory (SDRAM), double data rate synchronous dynamic random-access memory (DDR SDRAM), etc.

[0079] Non-volatile memory can also store executable programs and user and application data, etc., and can be pre-loaded into random access memory for direct reading and writing by the processor 103. Non-volatile memory may include disk memory and flash memory.

[0080] Memory 102 is used to store one or more computer programs. One or more computer programs are configured to be executed by processor 103. The one or more computer programs include a number of instructions, which, when executed by processor 103, can implement a medical image analysis method that can be executed on electronic device 10.

[0081] In other embodiments, the electronic device 10 shown in FIG4 also includes an external memory interface for connecting to an external memory to expand the storage capacity of the electronic device 10.

[0082] The processor 103 may include one or more processing units, such as an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video transcoder, a digital signal processor (DSP), and / or a neural network processing unit (NPU). The different processing units may be independent devices or integrated into one or more processors.

[0083] The processor 103 provides computing and control capabilities. For example, the processor 103 is used to execute computer programs stored in the memory 102 to implement the above-described medical image analysis method.

[0084] The input / output interface 104 is used to provide a channel for user input or output. For example, the input / output interface 104 can be used to connect various input / output devices, such as mice, keyboards, touch devices, and display screens, so that users can enter information or visualize information.

[0085] Bus 105 is used at least to provide a channel for communication between communication modules 101, memory 102, processor 103 and input / output interface 104 in electronic device 10.

[0086] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the electronic device 10. In other embodiments of this application, the electronic device 10 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0087] This application also provides a computer-readable storage medium storing a computer program, which includes program instructions. When the program instructions are executed, the method implemented can refer to the methods in the above embodiments of this application.

[0088] The computer-readable storage medium may be the internal memory of the electronic device described in the above embodiments, such as the hard drive or memory of the electronic device. The computer-readable storage medium may also be an external storage device of the electronic device, such as a plug-in hard drive, smart media card (SMC), secure digital (SD) card, flash memory card, etc. equipped on the electronic device.

[0089] In some embodiments, the computer-readable storage medium may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application program required for at least one function, etc.; and the data storage area may store data created based on the use of the electronic device, etc.

[0090] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for example, the division of modules is merely a logical functional division, and in actual implementation, there may be other division methods. The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0091] In addition, the functional modules in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware form or in the form of hardware plus software functional modules.

[0092] Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of this application is defined by the appended claims rather than the foregoing description. Thus, it is intended that all variations falling within the meaning and scope of equivalents fall within the claims of the invention be incorporated herein. No appended diagram markings in the claims of the invention should be construed as limiting the scope of the invention involved.

[0093] Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. A plural unit or device described in this application may also be implemented by a single unit or device through software or hardware. The terms "first," "second," etc., are used to indicate names and do not indicate any particular order.

[0094] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application and not to limit it. Although this application has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of this application without departing from the spirit and scope of the technical solutions of this application. 10: Electronic device 101: Communication module 102: Memory 103: Processor 104: Input / output interface 105: Bus S11~S18: Steps [Simplified Explanation of the Diagram]

[0095] Figure 1 is a flowchart of a medical image analysis method provided in an embodiment of this application.

[0096] Figure 2 is an example of a disease grading rule provided in an embodiment of this application.

[0097] Figure 3 is an example of a key lesion rule provided in an embodiment of this application.

[0098] Figure 4 is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

Claims

1. A medical image analysis method, applied to electronic devices, wherein, The method includes: acquiring classification results of a medical image by a first model and detection results of the medical image by a second model, wherein the classification results include a first disease level and a first probability of the first disease level, and the detection results include lesion information and the confidence level of the lesion information, wherein the lesion information includes lesion type and lesion location; determining a second disease level corresponding to the detection results according to disease grading rules; determining a key lesion type of the second disease level according to key lesion rules, and determining a second probability corresponding to the detection results based on the confidence level of the lesion information corresponding to the key lesion type; and determining the analysis steps of the medical image based on the matching situation between the first disease level and the second disease level, including: In the case where the first disease level and the second disease level do not match, the first probability is compared with a first preset value. When the first probability is less than the first preset value, the second probability is compared with a second preset value. If the first probability is greater than or equal to the first preset value, or if the first probability is less than the first preset value and the second probability is less than the second preset value, the detection result is adjusted according to the difference between the first disease level and the second disease level to obtain a corrected detection result. Based on the first disease level and the corrected detection result, the analysis result of the medical image is determined.

2. The medical image analysis method as described in claim 1, wherein, The step of determining the second probability corresponding to the detection result based on the confidence level of the lesion information corresponding to the key lesion type includes: if the key lesion type is multiple, determining the highest confidence level from all confidence levels corresponding to the multiple key lesion types, and determining the second probability based on the highest confidence level; if the key lesion type is single, determining the confidence level corresponding to the key lesion type as the second probability.

3. The medical image analysis method as described in claim 1, wherein, The step of adjusting the detection result based on the difference between the first disease level and the second disease level to obtain a corrected detection result includes: If the first disease level is higher than the second disease level, excluding the lesion type corresponding to the second disease level from the lesion types corresponding to the first disease level to obtain the remaining lesion types; adjusting the confidence level corresponding to each lesion type among the remaining lesion types until a confidence level that meets a preset condition is obtained; determining the corrected detection result based on the confidence level that meets the preset condition, the lesion type corresponding to the confidence level that meets the preset condition, and the lesion location; If the first disease level is lower than the second disease level, determining the corrected detection result based on the lesion type, lesion location, and confidence level corresponding to the first disease level.

4. The medical image analysis method as described in claim 1, wherein, If the first probability is less than the first preset value and the second probability is greater than or equal to the second preset value, the following analysis steps are performed: Based on the second disease level and the detection results, the analysis results of the medical image are determined.

5. The medical image analysis method as described in claim 1, wherein, The step of determining the second disease level corresponding to the test result according to the disease grading rules includes: comparing the test result with lesion information corresponding to a plurality of preset disease levels to determine the second disease level, wherein the lesion information includes preset lesion types and lesion locations of preset lesion types.

6. The medical image analysis method as described in claim 5, wherein, If the second disease level is the lowest preset disease level among the plurality of preset disease levels, then the key lesion type of the second disease level is the preset lesion type corresponding to the second disease level. If the second disease level is not the lowest preset disease level, then the key lesion type of the second disease level includes the preset lesion type remaining after excluding the preset lesion type corresponding to the previous preset disease level from the preset lesion types corresponding to the second disease level.

7. The medical image analysis method as described in claim 1, wherein, If the first disease level matches the second disease level, the following analysis steps are performed: Based on the detection results, the first disease level, or the second disease level, the analysis result of the medical image is determined.

8. An electronic device, wherein, The electronic device includes: a memory storing at least one instruction; and a processor executing the instructions stored in the memory to implement the medical image analysis method as described in any one of claims 1 to 7.

9. A computer-readable storage medium, wherein, The computer-readable storage medium stores a computer program that, when executed by a processor in an electronic device, implements the medical image analysis method as described in any one of claims 1 to 7.

Images