Medical image analysis method and apparatus

Abstract

The medical image analysis method includes: reading an original medical image; using complementary multiple artificial intelligence models to perform image classification and object detection on the original medical image to obtain a first classification result and multiple object detection results; using a feature integration and conversion module to integrate and quantitatively convert first and second detection results in the object detection results to obtain a quantitative result; and using a machine learning module to perform machine learning on the quantitative result and the first classification result to obtain and display an image interpretation result.

Description

Technical Field

[0001] This invention relates to a medical image analysis method and apparatus. Background Technology

[0002] To improve the efficiency of physicians' image interpretation and reduce human error, using artificial intelligence (AI) to assist physicians in interpreting images has become one of the key projects in the development of smart healthcare in the medical field.

[0003] AI-assisted image diagnosis primarily utilizes three technologies: object detection, segmentation, and classification. In terms of classification, it can distinguish between benign and malignant tumors, as well as classify relatively simple disease severity, enabling more efficient allocation of medical resources. Generally, classifiers such as Deep Neural Networks (DNN), Support Vector Machines (SVM, a type of machine learning), and Random Forests are used to classify the severity of lesions in medical images.

[0004] However, the classification, symptoms, and physiological structures of diseases are often interconnected, and the manner and degree of these connections are difficult to define clearly, even for experienced physicians, and are not easily determined through analytical methods. This type of problem is well-suited for application of AI (Artificial Intelligence) technology to learn from labeled data. However, past experience has shown that using only a single A1 classification model or an AI detection model for symptoms is insufficient to achieve optimal accuracy. If the shortcomings of the detection model can be compensated for by the classification model (and vice versa), that is, if the functions of these models complement each other, the drawbacks of using a single model can be mitigated, thereby improving the false positive rate.

[0005] Furthermore, if the amount of training data is small, or if the symptoms are very subtle in the early stages of a disease, using a classifier alone to classify medical images may fail to effectively identify these symptoms due to the extremely small range of variation between normal and abnormal images, leading to incorrect disease classification results. Summary of the Invention

[0006] According to one embodiment of this case, a medical image analysis method is proposed, comprising: reading original medical images; performing image classification and object detection on the original medical images using complementary multiple artificial intelligence models to obtain a first classification result and multiple object detection results; integrating and quantizing the object features of the first detection result and the second detection result in the object detection results by a feature integration and conversion module to obtain a quantization result; and performing machine learning on the quantization result and the first classification result by a machine learning module to obtain and display the image interpretation result.

[0007] According to another embodiment of this invention, a medical image analysis device is proposed, including a processor and a display unit coupled to the processor. The processor is configured to: read raw medical images; perform image classification and object detection on the raw medical images using complementary multiple artificial intelligence models to obtain a first classification result and multiple object detection results; integrate and quantize the object features of the first detection result and the second detection result in the object detection results by a feature integration and conversion module to obtain a quantization result; and perform machine learning on the quantization result and the first classification result by a machine learning module to obtain an image interpretation result, which is displayed on the display unit. Attached Figure Description

[0008] Figure 1 shows macular edema in diabetic patients;

[0009] Figure 2 shows a functional schematic diagram of a medical image analysis device according to an embodiment of the present invention;

[0010] Figure 3 shows a schematic diagram of medical image analysis according to an embodiment of this case;

[0011] Figure 4 shows a schematic diagram of medical image analysis according to another embodiment of this case;

[0012] Figure 5A shows a schematic diagram of dividing the fundus image into four quadrants with the optic disc as the center point in one embodiment of this case.

[0013] Figure 5B shows a view of a non-diabetic fundus image in another embodiment of this case;

[0014] Figure 6 shows a schematic diagram of the angular membership quadrant function according to an embodiment of this case;

[0015] Figure 7 shows the integrated symptom-physiological organization matrix according to one embodiment of this case;

[0016] Figure 8 shows a schematic diagram of merging the outputs of multiple AI models according to an embodiment of this case and inputting them into a machine learning module;

[0017] Figure 9 shows a schematic diagram of a machine learning algorithm according to an embodiment of this case;

[0018] Figure 10 shows an example of an image interpretation result displayed on a display unit according to an embodiment of this case;

[0019] Figure 11 shows a flowchart of a medical image analysis method according to an embodiment of this case.

[0020] [Explanation of Labels in the Attached Image]

[0021] 100: Original medical images

[0022] 110: Center of the macula

[0023] R: Sight plate diameter; HE: Hard leakage

[0024] 111: Video Disc

[0025] 200: Medical Image Analysis Device

[0026] 210: Processor 220: Database

[0027] 230: Display Unit

[0028] RMI: Original Medical Imaging

[0029] 310: Classification Model

[0030] 320: Symptom Detection Model

[0031] 330: Physiological Tissue Detection Model

[0032] 340: Disease Classification Results

[0033] 350: Symptom detection results

[0034] 360: Physiological Tissue Detection Results

[0035] 370: Feature Integration and Conversion Module

[0036] 380: Machine Learning Module

[0037] 390: Image Interpretation Results

[0038] 410: Classification Model

[0039] 420: Disease Classification Results

[0040] 810: Symptom-Physiological-Organic Relationship Matrix

[0041] 820: One-dimensional Pathological Physiological Organization Relationship Matrix

[0042] 910-940: Nodes

[0043] 1010: DR interpretation results (binary classification)

[0044] 1020: DR interpretation results (five-category classification)

[0045] 1030: DME interpretation results

[0046] 1105-1130: Steps Detailed Implementation

[0047] The technical terms used in this specification refer to those commonly used in the field. Where this specification provides explanations or definitions for certain terms, the interpretation of those terms shall be based on the explanations or definitions provided in this specification. Each embodiment of the present invention has one or more technical features. Where feasible, those skilled in the art may selectively implement some or all of the technical features in any embodiment, or selectively combine some or all of the technical features in these embodiments.

[0048] Currently, diabetic macular edema (DME) is defined as the presence of any hard exudates (HE) within one disc diameter from the center of the macula. Conversely, the absence of any hard exudates within one disc diameter from the center of the macula is considered non-referable diabetic macular edema.

[0049] In addition, preoperative computed tomography (CT) scans are used to assess the resectability of pancreatic cancer tumors, which are classified into five grades, from Grade 0 to Grade 4. The grade is related to the degree of tumor contact with blood vessels; the greater the degree of tumor contact with blood vessels, the higher the grade.

[0050] As shown in Figure 1, within the original medical image 100, a virtual circle is drawn with the center 110 of the macula as the center and the diameter R of the optic disc 111 as the radius. If hard leakage HE appears within this virtual circle, the condition is considered to be diabetic macular edema.

[0051] Figure 2 shows a functional schematic diagram of a medical image analysis device 200 according to an embodiment of this invention. The medical image analysis device 200 includes a processor 210, a database 220, and a display unit 230. The processor 210 is coupled to the database 220 and the display unit 230. The processor 210 reads raw medical images from the database 220, performs analysis and image interpretation, and then displays the completed image interpretation on the display unit 230. Processor 210 may be, for example, a central processing unit (CPU), or other programmable general-purpose or special-purpose microcontroller (MCU), microprocessor, digital signal processor (DSP), programmable controller, application-specific integrated circuit (ASIC), graphics processing unit (GPU), arithmetic logic unit (ALU), complex programmable logic device (CPLD), field programmable gate array (FPGA), or other similar elements or combinations thereof. Display unit 230 may be, for example, but not limited to, a liquid crystal display (LCD) or other device with display functionality.

[0052] Figure 3 shows a schematic diagram of medical image analysis according to an embodiment of this case. As shown in Figure 3, the raw medical image RMI is input to various artificial intelligence (AI) models for different tasks, such as, but not limited to, classification model 310, symptom detection model 320 and physiological tissue detection model 330.

[0053] Classification model 310 is a disease severity classification (class) model used to analyze the entire original medical image RMI to obtain disease classification results 340.

[0054] Symptom detection model 320, which is a disease-related symptom detection model, analyzes the original medical image RMI to obtain symptom detection results 350. Symptom detection results 350 include the location, area, confidence value, and number of each symptom type. Optionally, the area of ​​each symptom can be calculated using the vertical and horizontal lengths of the bounding box of the symptom selected by symptom detection model 320. Here, the symptom is, for example, but not limited to, hard leakage.

[0055] Physiological tissue detection model 330 is a disease-related physiological tissue (including organs and tissues) detection model that analyzes the original medical image RMI to obtain physiological tissue detection results 360. Physiological tissue detection results 360 include the location, vertical length (first length), horizontal length (second length), and confidence value of each physiological tissue. Optionally, the area of ​​each physiological tissue can be calculated using the vertical and horizontal lengths of the bounding box selected by the physiological tissue detection model 330. Here, physiological tissues include, but are not limited to, the optic disc, the center of the macula, etc.

[0056] Symptom detection results 350 and physiological tissue detection results 360 are input to the Feature Integration and Transformation module 370 for the integration and quantization transformation of object features (e.g., but not limited to symptoms and physiological tissues). Specifically, the Feature Integration and Transformation module 370 uses the symptom detection results and physiological tissue detection results to derive quanta or vectors through transformation. The quantization transformation results (quanta or vectors) from the Feature Integration and Transformation module 370, along with the disease classification results 340, are input to the machine learning module 380. The machine learning module 380 uses machine learning (ML) algorithms to find multiple judgment rules, obtaining image interpretation results 390. The image interpretation results 390 can be input to the display unit 230 for display. For example, the image interpretation results may indicate the probability of DME (Disease-Related Metastasis).

[0057] The object detection results (scalar or vector) and disease classification results 340 can be used as multiple training data (each training data includes the object detection results (scalar or vector) and disease classification results 340) to train the machine learning module 380 to obtain image interpretation results.

[0058] In one embodiment of this invention, the feature integration and conversion module 370 and the machine learning module 380 can be implemented, for example, by using a chip, a circuit block within the chip, a firmware circuit, a circuit board containing several electronic components and wires, or a storage medium storing multiple sets of program code. They can also be implemented by executing corresponding software or programs through electronic devices such as computer systems or servers. The feature integration and conversion module 370 and the machine learning module 380 are executed by the processor 210 in Figure 2.

[0059] Figure 4 shows a schematic diagram of medical image analysis according to another embodiment of the present invention. As shown in Figure 4, the raw medical image RMI is input to various artificial intelligence (AI) models for different tasks, such as, but not limited to, classification model 310, symptom detection model 320, physiological tissue detection model 330 and classification model 410.

[0060] As shown in Figure 4, the physiological tissue detection result 360 (i.e., the target region) generated by the physiological tissue detection model 330 is input to the classification model 410. For example, the classification model 410 uses the central macula region of the original medical image RMI as input. Optionally, the central macula region can be selected by the bounding box previously defined by the physiological tissue detection result 360, or by the macula center and optic disc diameter, which are calculated using the physiological tissue detection result 360 and analyzed by the classification model 410 to generate the disease classification result 420. One of the key points of this embodiment is that the central macula region is defined by the physiological tissue detection model 330 and used as input to the classification model 410. In particular, during the training and / or prediction process of the classification model 410, the defined central macula region is used as the input image, rather than the entire original medical image RMI. Compared to using the entire raw medical image RMI as input to classification model 410, even after image reduction operations (often used to save computational time in deep learning and / or prediction), defining the central macular region in this way makes it more likely to be detected by classification model 410. This effect is particularly useful when lesions or anatomic landmarks are very small relative to the entire raw medical image RMI. This ensures that classification model 410 is capable of learning to identify small objects within the central macular region.

[0061] As shown in Figure 4, the disease classification result 420 of classification model 410 is also input into machine learning module 380.

[0062] The operating principles of the feature integration and conversion module 370 will now be explained.

[0063] Figure 5A shows a schematic diagram of dividing the fundus image into four quadrants with the optic disc center as the center point, in one embodiment of this invention. In Figure 5A, S1-S4 represent the first to fourth quadrants, respectively. Furthermore, Sn represents the nth physiological tissue, such as the center of the macula, a quadrant, etc. (n is a positive integer). In one embodiment of this invention, n = 1 to 6, but the range of n can be adjusted according to actual application. That is, in this embodiment, a quadrant also represents a physiological tissue. Represents the i-th symptom in the m-th type of disease, such as Represents the first hard leak. This represents the third hard leak. Representative symptoms The relative distance from the physiological tissue Sn, For example, Representative symptoms The relative distance from the physiological tissue Sn.

[0064] In one embodiment of this case, taking the relationship between quadrants (n=1~4) and symptoms as an example, the quantification method of the feature integration and conversion module 370 considers any combination of the correlation between the symptom and this quadrant, the area of ​​the symptom, and the confidence value. In one embodiment of this case, the degree of quantification is positively correlated with the degree of membership in the quadrant, the area, the confidence value, or any combination thereof. In one embodiment of this case, for example but not limited to, the first positive relationship between multiple quadrants and symptoms can be expressed by equation (I):

[0065] In equation (i), the parameters p, q, and r can be obtained through data learning. The degree of membership of an angle to a quadrant can be represented using a fuzzy function (for example, but not limited to, as shown in Figure 6), the angle Symptoms The angle (in radians) with the horizontal axis. The degree to which a representative angle belongs to a quadrant. Representative symptoms The relative area, This represents confidence level. The angle represents the quadrant function. Among them, In one embodiment of this case, the planar coordinates used are with the center of the optic disc as the origin, the horizontal line as the horizontal axis, and the vertical line as the vertical axis. That is, the first positive relationship between these quadrants and the symptom is a first calculation result relating the degree of quadrant membership of the angle to a first parameter (parameter p). The area and a second operation result with a second parameter (parameter q) And, the result of a third operation on this confidence value and a third parameter (parameter r).

[0066] In one embodiment of this case, taking the relationship between general physiological tissue and symptoms as an example, the quantification method of the feature integration and conversion module 370 considers any combination of the correlation between the symptom and the physiological tissue (e.g., the reciprocal of the distance between the symptom and the physiological tissue), the symptom area, and the confidence value. In one embodiment of this case, the quantification degree value is positively correlated with (1) the reciprocal of the distance between the symptom and the physiological tissue, (2) the area, (3) the confidence value, or any combination thereof. In one embodiment of this case, for example but not limited to, the second positive relationship between physiological tissue and symptoms can be expressed by equation (ii):

[0067]

[0068] In equation (ii), the parameters p', q', and r' can be obtained through data learning. That is, the second positive relationship between physiological tissue and symptoms is the result of a fourth operation on the reciprocal of the distance between the symptom and the physiological tissue and the fourth parameter (parameter p'). The area and the result of a fifth operation with the fifth parameter (parameter q') And, the result of a sixth operation between the confidence value and the sixth parameter (parameter r').

[0069] Figure 5B shows a portion of a non-diabetic fundus image in another embodiment of this case. In Figure 5B, S1 and S2 represent two physiological tissues, such as blood vessels and the pancreas, respectively. This represents the i-th symptom in the m-th type of disease (e.g., pancreatic tumor). Representative of illness The relative distance from the physiological tissue Sn, For example, Representative symptoms The relative distance from physiological tissue S1.

[0070] Similarly, taking Figure 5B as an example, in one embodiment of this case, taking the relationship between general physiological tissues (such as blood vessels or the pancreas) and symptoms (such as pancreatic tumors) as an example, the quantification method of the feature integration and conversion module 370 takes into account any combination of the correlation between the symptom and this physiological tissue (for example, the reciprocal of the distance between the symptom and the physiological tissue), the area of ​​the symptom, confidence value, etc. In one embodiment of this case, the quantification degree value is positively correlated with (1) the reciprocal of the distance between the symptom and the physiological tissue, (2) the area, (3) the confidence value, or any combination thereof. The second positive relationship between physiological tissues and symptoms can be expressed by Equation (II), the details of which will not be repeated here.

[0071] The aforementioned angular membership quadrant function can be implemented using fuzzy theory. It should also be noted that, as shown in Figure 6, four fuzzy sets can be defined, where the function output lies between 0 and 1. Furthermore, the shape of this angular membership quadrant function can be, but is not limited to, a trapezoid, a triangle, or any combination thereof, and this shape can be trained (the shape is trainable).

[0072] In this embodiment, when n = 1 to 4 (quadrants), the parameters in equation (I) This represents the confidence value output by the symptom detection model 320 for that symptom. That is, when n = 1–4 (quadrants), equation (I) uses the angular membership quadrant function, area parameter, and confidence parameter to calculate the quantified value. Confidence parameter The higher the value, the greater the degree of symptom presence.

[0073] In this embodiment, when n = 5 to 6 (physiological tissue), Equation (II) uses a distance parameter (especially the reciprocal of the distance parameter), an area parameter, and a confidence parameter to calculate a quantitative value.

[0074] In this embodiment, the parameters p, q, r, p', q', and r' are real numbers and can be updated during model training, for example, but not limited to using backpropagation to find the optimal solution for parameters p, q, r, p', q', and r'. Alternatively, Bayesian optimization can be used to find the optimal combination of parameters p, q, r, p', q', and r'. When using Bayesian optimization, the conditions for p, q, r, p', q', and r' can be, but are not limited to: 0 ≤ p, q, r, p', q', r'. The parameters p, q, r, p', q', and r' can be updated using either backpropagation or Bayesian optimization.

[0075] In one embodiment of this case, the feature integration and conversion module 370 converts the symptom detection result 350 and the physiological tissue detection result 360 into feature vectors or scalars, that is, the function f(S n L m ).

[0076] In detail, the symptom detection result 350 obtained by the symptom detection model 320 can be a symptom result matrix, as shown in Figure 7. In this symptom result matrix, each column element includes: [Lesion, X, Y, W, H, C], where Lesion, X, Y, W, H, and C represent: the symptom type (Lesion) predicted by the symptom detection model, the X coordinate (X) of the symptom location, the Y coordinate (Y) of the symptom location, the horizontal length (W) of the symptom, the vertical length (H) of the symptom, and the confidence value (C) of the symptom.

[0077] The physiological tissue detection result 360 obtained by the physiological tissue detection model 330 is a physiological tissue result matrix, as shown in Figure 7. In this physiological tissue result matrix, each column element includes: [Structure, X, Y, W, H, C], where Structure, X, Y, W, H, and C represent: the physiological tissue type (Structure) predicted by the physiological tissue detection model, the X coordinate (X) of the physiological tissue location, the Y coordinate (Y) of the physiological tissue location, the horizontal length (W) of the physiological tissue, the vertical length (H) of the physiological tissue, and the confidence value (C) of the physiological tissue.

[0078] The feature integration and transformation module 370 integrates the symptom result matrix and the physiological tissue result matrix into a symptom-physiological tissue relationship matrix (as shown in Figure 7). The correspondence and severity of various types of physiological tissues and various types of symptoms in the symptom-physiological tissue relationship matrix can be generated using quantitative methods. In Figure 7, it is assumed that there are 5 types of symptoms: L1 represents hard leakage (HE), L2 represents hemorrhage (H), L3 represents soft leakage (SE), L4 represents neovascularization (NE), and L5 represents microaneurysm (MA).

[0079] Figure 8 shows a schematic diagram of merging the outputs of multiple AI models according to an embodiment of this case and inputting them into the machine learning module 380. As shown in Figure 8, the disease classification result 340 of the classification model 310 can be represented as a one-dimensional confidence value matrix. As previously mentioned, the feature integration and transformation module 370 integrates the symptom detection result 350 (which can be represented as a symptom result matrix) and the physiological tissue detection result 360 (which can be represented as a physiological tissue result matrix) into a symptom-physiological tissue relationship matrix 810. Then, the feature integration and transformation module 370 flattens the symptom-physiological tissue relationship matrix 810 into a one-dimensional symptom-physiological tissue relationship matrix 820. The disease classification result 340 (one-dimensional confidence value matrix) and the one-dimensional symptom-physiological tissue relationship matrix 820 are input into the machine learning module 380 to perform machine learning algorithms to obtain the image interpretation result 390.

[0080] In one embodiment of this invention, the machine learning algorithm is, for example, but not limited to, a decision tree. Figure 9 shows a schematic diagram of a machine learning algorithm according to an embodiment of this invention. As shown in Figure 9, a confidence value F is determined at node 910. cls Is it less than 0.158? If node 910 is false, jump to node 915 to determine if DME exists. If node 910 is true, jump to node 920 to determine the confidence value F. cls Is it less than 0.016? If node 920 is true, jump to node 930 to determine if there is no DME. If node 920 is false, jump to node 925 to determine the confidence value F. SL Is it less than 0.221? If node 925 is false, jump to node 935 to determine if a DME exists. If node 925 is true, jump to node 940 to determine if a DME does not exist.

[0081] Figure 10 shows an example of an image interpretation result 390 displayed on the display unit 230 according to an embodiment of this invention. As shown in Figure 10, the image interpretation result 390 includes: original medical image RMI (e.g., but not limited to, an image of one eye, in which symptoms and physiological tissues are also displayed), DR interpretation result (binary classification) 1010, DR interpretation result (five-class classification) 1020, and DME interpretation result 1030. Of course, Figure 10 is an exemplary example of displaying the image interpretation result 390, and it should be understood that this invention is not limited thereto.

[0082] Figure 11 shows a flowchart of a medical image analysis method according to an embodiment of this case. As shown in Figure 11, in step 1105, the original medical image is read. In step 1110, it is determined whether the size of the original medical image is less than a predetermined size threshold. If step 1110 is negative (i.e., not less than), then in step 1115, the size of the original medical image is adjusted to be less than the predetermined size threshold. If step 1110 is true, then in step 1120, multiple complementary AI models (such as classification models, symptom detection models, physiological tissue detection models, etc.) are used to perform image classification and object detection on the medical image to obtain a first classification result and multiple object detection results. In step 1125, two of the detection results are integrated and quantized using object features (e.g., but not limited to, symptoms, physiological tissues, etc.) to obtain a quantized result. In step 1130, machine learning is performed on the quantized result and the first classification result to obtain an image interpretation result, and the image interpretation result is displayed.

[0083] The original medical images may be fundus images (as shown in Figure 5A) or non-fundus images (as shown in Figure 5B), both of which are within the scope of this case.

[0084] The details of steps 1120, 1125 and 1130 are as described above and will not be repeated here.

[0085] The performance differences between the prior art and the embodiments of this case are compared based on the severity of DME:

[0086]

[0087] As can be seen from the table above, the embodiments in this case have improved in terms of sensitivity, specificity and accuracy.

[0088] This case discloses a method and system for assisting medical image analysis based on symptom detection and physiological tissue detection, and in particular, a method and system for medical image analysis combining a multi-task AI model.

[0089] In this embodiment, by combining AI models with different tasks and interrelationships (such as classification models, symptom detection models, and physiological tissue detection models) in a complementary manner, the overall system's performance in classifying the severity of lesions is effectively improved, thereby enhancing the accuracy of medical image interpretation. In this embodiment, the classification model compensates for the shortcomings of the detection model (and vice versa), meaning that the functions of these models complement each other, which can mitigate the drawbacks of using a single model and thus reduce the false positive rate.

[0090] In this embodiment, by combining the extensive interpretation results of the classification model with the results based on pathological analysis, the correlation information between symptoms and physiological tissues is obtained. Then, the best decision is found through machine learning. Therefore, it can overcome common problems in the present (such as the fact that the simple CNN classification model has shift invariance and cannot accurately determine the relative positional relationship between symptoms and physiological tissues, and is prone to misjudgment when the symptoms are small, or, in certain situations (such as when the symptoms completely cover the physiological tissues, the physiological tissue model cannot identify them), it is easy to cause misjudgment).

[0091] This invention uses machine learning to find the optimal output prediction rules, which can also be applied to applications beyond medical image recognition.

[0092] The embodiments of this case can learn the data-driven structure of the data, and can learn high-dimensional and highly abstract data distribution structures; it does not require subjective human definition of rules, and is relatively objective; through expert annotation, it can directly learn the experience and knowledge of experts, avoiding situations where experts cannot describe their knowledge and experience in words; and it can independently adapt to new data.

[0093] In summary, the embodiments of this case can improve the efficiency of the lesion screening system; have high interpretability; and these rules can be extended to fuzzy systems in the future and used as fuzzy rules.

[0094] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A medical image analysis method, comprising: Read raw medical images; The original medical image was classified and object detected using the first classification model, the symptom detection model, and the physiological tissue detection model, respectively, to obtain the disease classification results, symptom detection results, and physiological tissue detection results. The feature integration and conversion module integrates and quantifies the object features of the symptom detection results and the physiological tissue detection results to obtain the quantitative results. as well as The machine learning module performs machine learning on the quantitative result and the disease classification result to obtain and display the image interpretation result; Specifically, the feature integration and conversion module converts the symptom detection results and the physiological tissue detection results into pure quantities or vectors through quantization conversion; when the feature integration and conversion module performs quantization, for the relationship between multiple quadrants and the symptom, the degree of quantization is positively correlated with the degree of angular membership in the quadrant, the area, the confidence value, or any combination thereof; and when the feature integration and conversion module performs quantization, for the relationship between the physiological tissue and the symptom, the degree of quantization is positively correlated with the reciprocal of the distance between the symptom and the physiological tissue, the area, the confidence value, or any combination thereof.

2. The medical image analysis method according to claim 1 further includes: When it is determined that the size of the original medical image is not less than a predetermined size threshold, the size of the original medical image is adjusted to be less than the predetermined size threshold.

3. The medical image analysis method according to claim 1, wherein, The first classification model is used to analyze the original medical image to obtain the disease classification result; The symptom detection model analyzes the original medical image to obtain the symptom detection results, which include the location, area, confidence value, and total number of symptoms of each symptom type. as well as The physiological tissue detection model analyzes the original medical image to obtain the physiological tissue detection results, which include the location, first length, second length, and confidence value of each physiological tissue.

4. The medical image analysis method according to claim 3, wherein: The area of ​​each symptom is calculated using the vertical and horizontal lengths of the bounding box detected by the symptom detection model.

5. The medical image analysis method according to claim 3, wherein: The physiological tissue detection results are input into the second classification model. The second classification model selects the target region from the original medical image based on the physiological tissue detection results, analyzes the selected target region, and obtains the second classification result. The second classification result is then input into the machine learning module.

6. The medical image analysis method according to claim 1, wherein: The first positive relationship between these quadrants and the symptom is related to the first calculation result of the degree of the angle's membership in the quadrant and the first parameter, the second calculation result of the area and the second parameter, and the third calculation result of the confidence value and the third parameter; as well as The second positive relationship between the physiological tissue and the symptom is related to the fourth operation result of the reciprocal of the distance between the symptom and the physiological tissue and the fourth parameter, the fifth operation result of the area and the fifth parameter, and the sixth operation result of the confidence value and the sixth parameter.

7. The medical image analysis method according to claim 6, wherein: the first parameter, the second parameter, the third parameter, the fourth parameter, the fifth parameter, and the sixth parameter are obtained through data learning.

8. The medical image analysis method according to claim 7, wherein: the first parameter, the second parameter, the third parameter, the fourth parameter, the fifth parameter, and the sixth parameter are updated by backpropagation or Bayesian optimization.

9. The medical image analysis method according to claim 1, wherein: When the degree of membership of an angle to a quadrant is realized using fuzzy theory, multiple fuzzy sets are defined. When the degree to which this angle belongs to a quadrant is represented by a function, the output of that function is between 0 and 1; and The shape of the quadrant function to which this angle belongs is a trapezoid, a triangle, or any combination thereof, and this shape is trainable.

10. The medical image analysis method according to claim 1, wherein: The symptom detection results are presented as a symptom result matrix. Each column of this matrix contains the following elements: The type of the symptom, its location, its horizontal length, its vertical length, and its confidence score, and The physiological tissue detection results are presented as a physiological tissue result matrix. Each column of the physiological tissue result matrix includes: the type of physiological tissue, the location of the physiological tissue, the horizontal length of the physiological tissue, the vertical length of the physiological tissue, and the confidence value of the physiological tissue.

11. The medical image analysis method according to claim 10, wherein: The feature integration and transformation module integrates the symptom result matrix and the physiological tissue result matrix into a symptom-physiological-tissue relationship matrix; and The disease classification result is a one-dimensional confidence value matrix. The feature integration and transformation module flattens the symptom-physiological-organization relationship matrix into a one-dimensional symptom-physiological-organization relationship matrix.

12. The medical image analysis method according to claim 11, wherein: the machine learning module performs machine learning on the one-dimensional confidence value matrix and the one-dimensional symptom-physiological-tissue relationship matrix to obtain the image interpretation result.

13. The medical image analysis method according to claim 12, wherein: the image interpretation result includes a medical image having the symptom and the physiological tissue and at least one interpretation result; and the original medical image includes fundus image or non-fundus image.

14. A medical image analysis device, comprising: processor; as well as The display unit is coupled to the processor. in, The processor is configured as follows: Read raw medical images; The original medical image was classified and object detected using the first classification model, the symptom detection model, and the physiological tissue detection model, respectively, to obtain the disease classification results, symptom detection results, and physiological tissue detection results. The feature integration and conversion module integrates and quantifies the object features of the symptom detection results and the physiological tissue detection results to obtain the quantitative results. as well as The machine learning module performs machine learning on the quantification result and the disease classification result to obtain the image interpretation result, which is displayed on the display unit. Specifically, the feature integration and conversion module converts the symptom detection results and the physiological tissue detection results into pure quantities or vectors through quantization conversion; when the feature integration and conversion module performs quantization, for the relationship between multiple quadrants and the symptom, the degree of quantization is positively correlated with the degree of angular membership in the quadrant, the area, the confidence value, or any combination thereof; and when the feature integration and conversion module performs quantization, for the relationship between the physiological tissue and the symptom, the degree of quantization is positively correlated with the reciprocal of the distance between the symptom and the physiological tissue, the area, the confidence value, or any combination thereof.

15. The medical image analysis device according to claim 14, wherein, The processor is configured to adjust the size of the original medical image to be smaller than the predetermined size threshold when it is determined that the size of the original medical image is not smaller than the predetermined size threshold.

16. The medical image analysis device according to claim 14, wherein, The first classification model is used to analyze the original medical image to obtain the disease classification result; The symptom detection model analyzes the original medical image to obtain the symptom detection results, which include the location, area, confidence value, and total number of symptoms of each symptom type. as well as The physiological tissue detection model analyzes the original medical image to obtain the physiological tissue detection results, which include the location, first length, second length, and confidence value of each physiological tissue.

17. The medical image analysis device according to claim 16, wherein, The area of ​​each symptom is calculated using the vertical and horizontal lengths of the bounding box detected by the symptom detection model.

18. The medical image analysis device according to claim 17, wherein, The physiological tissue detection results are input into the second classification model. The second classification model selects the target region from the original medical image based on the physiological tissue detection results, analyzes the selected target region, and obtains the second classification result. The second classification result is then input into the machine learning module.

19. The medical image analysis device according to claim 14, wherein, The first positive relationship between these quadrants and the symptom is related to the first calculation result of the degree of the angle's membership in the quadrant and the first parameter, the second calculation result of the area and the second parameter, and the third calculation result of the confidence value and the third parameter; as well as The second positive relationship between the physiological tissue and the symptom is related to the fourth operation result of the reciprocal of the distance between the symptom and the physiological tissue and the fourth parameter, the fifth operation result of the area and the fifth parameter, and the sixth operation result of the confidence value and the sixth parameter.

20. The medical image analysis device according to claim 19, wherein, The first parameter, the second parameter, the third parameter, the fourth parameter, the fifth parameter, and the sixth parameter are obtained through learning from materials.

21. The medical image analysis device according to claim 20, wherein, The first parameter, the second parameter, the third parameter, the fourth parameter, the fifth parameter, and the sixth parameter are updated using backpropagation or Bayesian optimization.

22. The medical image analysis device according to claim 14, wherein, When the degree of membership of an angle to a quadrant is realized using fuzzy theory, multiple fuzzy sets are defined. When the degree to which this angle belongs to a quadrant is represented by a function, the output of that function is between 0 and 1; and The shape of the quadrant function to which this angle belongs is a trapezoid, a triangle, or any combination thereof, and this shape is trainable.

23. The medical image analysis device according to claim 14, wherein, The symptom detection results are presented as a symptom result matrix. Each column of this matrix includes: the type of the symptom, the location of the symptom, the horizontal length of the symptom, the vertical length of the symptom, and the confidence score of the symptom. The physiological tissue detection results are presented as a physiological tissue result matrix. Each column of the physiological tissue result matrix includes: the type of physiological tissue, the location of the physiological tissue, the horizontal length of the physiological tissue, the vertical length of the physiological tissue, and the confidence value of the physiological tissue.

24. The medical image analysis device according to claim 23, wherein, The feature integration and conversion module integrates the symptom result matrix and the physiological tissue result matrix into a symptom-physiological-tissue relationship matrix. as well as The disease classification result is a one-dimensional confidence value matrix. The feature integration and transformation module flattens the symptom-physiological-organization relationship matrix into a one-dimensional symptom-physiological-organization relationship matrix.

25. The medical image analysis device according to claim 24, wherein, The machine learning module performs machine learning on the one-dimensional confidence value matrix and the one-dimensional symptom physiological tissue relationship matrix to obtain the image interpretation result.

26. The medical image analysis device according to claim 25, wherein, The image interpretation results include medical images with the symptoms and physiological tissues and at least one interpretation result; and the original medical images include fundus images or non-fundus images.

Images