Weakly supervised learning diabetic retinopathy grading and lesion identification method and system

Abstract

The present invention relates to a weakly supervised learning diabetic retinopathy grading and lesion identification method and system, the method comprising: S1: inputting a diabetic retinopathy fundus color image into a CNN network to obtain a feature map F; inputting F into a lesion point capture module based on weakly supervised object positioning, selecting to enter a branch module, calculating a score map, multiplying the score map with the feature map F to obtain a new feature map F'; S2: using an optimal network structure search module based on reinforcement learning to calculate the insertion position, selection probability, feature discarding threshold and retention threshold of the weakly supervised lesion point capture module in the CNN network, and finally outputting the feature map F. NA S3: Lesion attribute prediction and disease classification are obtained through attribute mining and lesion identification modules. S4: Based on the results of S3, the final lesion identification result is obtained through multiple iterative erasure calculations. The method provided by the present invention can improve the effectiveness of lesion capture and use lesion mining methods to determine lesion type, providing a basis for disease classification.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to a method and system for weakly supervised learning of diabetic retinopathy grading and lesion recognition. Background Art

[0002] In recent years, the incidence of diabetes has been increasing annually. Diabetic retinopathy (DR), a major complication of diabetes, has become the leading cause of blindness among working-age people worldwide. According to statistics, approximately 500 million people in my country are in prediabetes, 110 million have diabetes, and approximately 37 million have diabetic retinopathy. Diagnosis of diabetic retinopathy requires a professional ophthalmologist to observe lesions on fundus color images and determine the patient's condition and grade based on a diabetic retinopathy grading standard. This process is time-consuming and labor-intensive. Furthermore, in remote or underdeveloped areas, the lack of specialized ophthalmologists prevents patients from receiving timely diagnosis, delaying treatment. According to relevant statistics, 87% of diabetic patients currently seek medical treatment at county-level or lower-level medical institutions, yet basic diagnostic and treatment measures and appropriate technologies for diabetic retinopathy are implemented in tertiary medical institutions. Therefore, intelligent and automated diabetic retinopathy diagnosis technology can effectively alleviate physician workload and improve diagnostic efficiency.

[0003] Currently, leveraging deep learning technology to assist with diabetic retinopathy (DR) diagnosis is a hot topic. Computer vision techniques within deep learning can extract features from color images of the retinal fundus, abstractly understand lesion characteristics, and leverage the powerful summarization and induction capabilities of neural networks to complete DR diagnosis. Deep learning-based disease grading and lesion identification have achieved significant success in addressing clinical challenges. Among numerous technologies, disease grading and lesion detection are two fundamental tasks that provide significant support for clinical diagnosis. Disease grading predicts the disease category, assisting doctors in early diagnosis, while lesion detection directly locates lesions, providing interpretable diagnostic evidence. Extensive research has examined disease grading and lesion localization, often treating the two tasks separately. However, disease grading and lesion detection complement each other. Lesion detection comprehensively locates potential lesions, providing a reliable basis for grading, while classification models used for disease grading can learn differential information across categories, which helps improve detection performance. Furthermore, accurately labeled data is a prerequisite for applying deep learning algorithms in the medical image field. Disease grading tasks require image-level labeled data, but lesion detection requires fine-grained annotation with labeled boxes, which is a time-consuming and laborious task for professional doctors. Therefore, how to complete disease grading and lesion detection while using limited annotations is a challenge in medical image processing. Summary of the Invention

[0004] In order to solve the above technical problems, the present invention provides a weakly supervised learning method and system for grading diabetic retinopathy and identifying lesions.

[0005] The technical solution of the present invention is: a weakly supervised learning method for grading diabetic retinopathy and identifying lesions, comprising:

[0006] Step S1: Input the diabetic retinopathy fundus color image into the convolutional neural network to obtain a feature map F; input F into the lesion point capture module based on weakly supervised object positioning to reduce its dimension into a one-dimensional feature map. According to the selection probability, one of the three branch modules is selected: the attention branch module, the salient feature erasure module, and the salient feature retention module. The score map is calculated and multiplied by the feature map F to obtain a new feature map F';

[0007] Step S2: Use the optimal network structure search module based on reinforcement learning to calculate the insertion position of the weakly supervised lesion point capture module in the convolutional neural network model, the selection probability of the three branch modules, the feature discarding threshold and the retention threshold in the significant feature erasing module and the significant feature retention module, and finally output the feature map F NA ;

[0008] Step S3: Based on the diabetic retinopathy grading standard, the lesion categories used to distinguish the diabetic retinopathy grades are obtained, which are used as prior knowledge for annotation in weakly supervised multi-label classification. The lesion attribute prediction results and disease grading results are obtained through attribute mining and lesion recognition modules;

[0009] Step S4: Based on the lesion attribute prediction results and the disease grading results, a class activation map method is used to obtain a visualization map of the lesion location, and the final lesion recognition result is obtained through multiple iterative erasing calculations.

[0010] Compared with the prior art, the present invention has the following advantages:

[0011] 1. The method disclosed in the present invention uses a lesion point capture module for weakly supervised object positioning, which can capture more potential lesion information during the disease grading process, that is, using category-specific information to obtain the location of potential lesions, and using lesion mining methods to determine the lesion type. The visualization results can provide an explanatory diagnostic basis for disease grading.

[0012] 2. The method disclosed in the present invention optimizes the configuration of the lesion point capture module of weakly supervised object positioning through the optimal network structure search module based on reinforcement learning, thereby improving the effect of lesion capture.

[0013] 3. The method disclosed in the present invention takes the basis for lesion classification as a priori knowledge attribute label, proposes an attribute mining method to allow the model to fit the label, further improves the disease classification performance, and uses the attribute category obtained in the attribute mining process as the lesion detection result. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 This is a flow chart of a weakly supervised learning method for grading diabetic retinopathy and identifying lesions in an embodiment of the present invention;

[0015] Figure 2 Schematic diagram of the structure of a lesion point capture module based on weakly supervised object positioning in an embodiment of the present invention;

[0016] Figure 3 Schematic diagram of the calculation process of the optimal network structure search module based on reinforcement learning in an embodiment of the present invention;

[0017] Figure 4 Schematic diagram of the process of lesion attribute mining based on prior knowledge and iterative attribute mining in an embodiment of the present invention;

[0018] Figure 5 This is a diagram showing the effect of lesion identification in an embodiment of the present invention;

[0019] Figure 6 This is a module block diagram of a weakly supervised learning diabetic retinopathy grading and lesion identification system in an embodiment of the present invention. DETAILED DESCRIPTION

[0020] The present invention provides a weakly supervised learning method for diabetic retinopathy grading and lesion identification, which can capture more potential lesion information during the disease grading process, improve the effect of lesion capture, and use lesion mining methods to determine the lesion type. The visualization results can provide an explanatory diagnostic basis for disease grading.

[0021] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention is further described in detail below through specific implementation and in conjunction with the accompanying drawings.

[0022] Example 1

[0023] like Figure 1 As shown, an embodiment of the present invention provides a weakly supervised learning method for grading diabetic retinopathy and identifying lesions, comprising the following steps:

[0024] Step S1: Input the diabetic retinopathy fundus color image into the convolutional neural network to obtain a feature map F; input F into the lesion point capture module based on weakly supervised object positioning to reduce its dimension into a one-dimensional feature map. Based on the selection probability, one of the three branch modules is selected: the attention branch module, the salient feature erasure module, and the salient feature retention module. The score map is calculated and multiplied by the feature map F to obtain a new feature map F';

[0025] Step S2: Use the optimal network structure search module based on reinforcement learning to calculate the insertion position of the weakly supervised lesion capture module in the convolutional neural network model, the selection probability of the three branch modules, the feature discarding threshold and the retention threshold in the significant feature erasing module and the significant feature retention module, and finally output the feature map F NA ;

[0026] Step S3: Based on the diabetic retinopathy grading standard, the lesion categories used to distinguish the diabetic retinopathy grades are obtained, which are used as prior knowledge for annotation in weakly supervised multi-label classification. The lesion attribute prediction results and disease grading results are obtained through attribute mining and lesion recognition modules;

[0027] Step S4: Based on the lesion attribute prediction results and disease grading results, the class activation map method is used to obtain a visualization of the lesion location, and the final lesion recognition result is obtained through multiple iterative erasing calculations.

[0028] In one embodiment, the above step S1: inputs the diabetic retinopathy fundus color image into the convolutional neural network to obtain a feature map F; inputs F into the lesion point capture module based on weakly supervised object positioning, reduces its dimension into a one-dimensional feature map, and selects one of the three branch modules according to the selection probability: the attention branch module, the significant feature erasure module, and the significant feature retention module to calculate a score map, and multiplies the score map with the feature map F to obtain a new feature map F', which specifically includes:

[0029] Step S11: Input the color image of the retinal fundus into the convolutional neural network CNN to obtain the feature map F, F∈R H×W×D , where H, W and D are the height, width and depth of feature map F respectively;

[0030] During the computational process, convolutional neural networks tend to use the most discriminative features to complete image classification, which results in the neglect of other potentially useful information. However, in medical images, disease classification often requires a comprehensive evaluation of multiple lesions, and the diagnosis of diabetic retinopathy requires the integration of multiple lesion attributes. First, weakly supervised lesion localization is performed on the feature map F extracted by the convolutional neural network. This can more comprehensively extract subtle lesions in medical images, improve the accuracy of disease classification, and serve as the basis for lesion capture and visualization.

[0031] Step S12: After F is average pooled in a depth dimension, the attention feature map Att is obtained, Att∈R H×W , Att is mapped to [0,1] after Sigmoid operation to obtain a one-dimensional feature map;

[0032] The value of Att after being mapped to [0, 1] reflects the discriminativeness of the feature. The salient feature erasure structure suppresses the most discriminative features. Portions of Att with values ​​greater than the parameter dropout threshold are set to 0, indicating that the most discriminative features in this spatial dimension are discarded. The salient feature preservation structure prevents the network from over-focusing on potentially invalid features, which can lead to reduced classification performance. Portions of Att with values ​​less than the parameter dropout threshold are set to 0, indicating that features in this spatial dimension that are potentially irrelevant to the classification result are discarded.

[0033] Step S13: According to the selection probability p, the one-dimensional feature map is input into one of the attention branch module, the salient feature erasure module and the salient feature retention module to obtain a score map; wherein the selection probabilities of the attention branch module, the salient feature erasure module and the salient feature retention module are P and P, respectively. A , P D and P H , and P A +P D +P H =1;

[0034] Step S14: Multiply the score map by the feature map F to obtain a new feature map F'. Formula (1) is as follows;

[0035]

[0036] Among them, p is the selection probability, × is the point-by-point multiplication, A is the output of the attention branch module, D is the output of the salient feature erasing module, and H is the output of the salient feature retention module.

[0037] like Figure 2 As shown in the figure, a structural diagram of the lesion point capture module based on weakly supervised object positioning is shown. The lesion point capture module based on weakly supervised object positioning is only used in the network training process and is not used in testing and actual applications.

[0038] In one embodiment, the above step S2: uses the optimal network structure search module based on reinforcement learning to calculate the insertion position of the weakly supervised lesion point capture module in the convolutional neural network model, the selection probability of the three branch modules, the feature discarding threshold and the retention threshold in the significant feature erasing module and the significant feature retention module, and finally outputs the feature map F NA , specifically including:

[0039] LSTM is used as the controller to generate the search parameter values, and ResNet50 is used as the backbone network for feature extraction. The various parameters of the lesion point capture module based on weakly supervised object positioning are obtained by selecting probability p. The gradient of p is scaled by the performance of the validation set on the lesion point capture module based on weakly supervised object positioning to update the controller; thereby determining the insertion position of the weakly supervised lesion point capture module in the convolutional neural network model, the selection probability of the three branch modules, and the feature discarding threshold and retention threshold in the salient feature erasure module and the salient feature retention module.

[0040] In the lesion capture module based on weakly supervised object localization, many parameters and settings are directly related to disease classification and lesion capture performance, including the insertion position of the lesion capture module in the CNN network, the selection probability of the three branches, and the feature discarding and retention thresholds. When the lesion capture module based on weakly supervised object localization is inserted in a shallow layer of the CNN network, the CNN extracts generalized features, and feature discarding will cause these features to be lost, affecting the feature abstraction performance of subsequent layers. When the lesion capture module based on weakly supervised object localization is inserted in a deeper layer of the CNN network, the features are highly correlated with the class, and inappropriate feature discarding can lead to the loss of important information or the extraction of erroneous information. Similarly, the selection probability of the three branches and the feature discarding and retention thresholds are also important. The probability of selecting the salient feature erasing structure and the feature discarding threshold affect the network's ability to capture non-discriminative lesions, while the probability of selecting the salient feature preserving structure is related to the feature retention threshold and the network's generalization performance. Therefore, the neural network structure search method can maximize the role of the weakly supervised lesion capture structure.

[0041] Therefore, the embodiment of the present invention constructs an optimal network structure search module based on reinforcement learning, uses a recurrent neural network LSTM controller to sample the microstructure of the network in a given search set, splices them together to build an end-to-end network architecture, converges it through training, and uses reinforcement learning methods to update the controller based on the performance of the model on the validation set so that the controller can generate a better network structure. LSTM is used as the controller to generate search parameter values, and ResNet50, which is widely used in other diabetic retinopathy classification studies, is used as the backbone network for feature extraction to calculate the insertion position of the weakly supervised lesion capture module in the CNN network, the selection probability of the three branch modules, and the feature discarding threshold and retention threshold in the significant feature erasure module and the significant feature retention module.

[0042] like Figure 3 As shown, a schematic diagram of the calculation process of the optimal network structure search module based on reinforcement learning is shown.

[0043] After completing steps S1 and S2 above, the CNN network trained using the lesion point capture module based on weakly supervised object positioning and the optimal network structure search module based on reinforcement learning already has a relatively comprehensive and accurate lesion capture capability. On this basis, the following lesion attribute mining and lesion recognition modules can further improve the accuracy of disease grading and complete lesion recognition.

[0044] In one embodiment, the above S3: based on the diabetic retinopathy grading standard, obtains the lesion category used to distinguish the diabetic retinopathy grade, uses it as prior knowledge for annotation in weakly supervised multi-label classification, and obtains the lesion attribute prediction result and disease grading result through the attribute mining and lesion recognition module, specifically including:

[0045] Step S31: taking the lesion type of each disease level in diabetic retinopathy as prior knowledge, six prior knowledge attributes are obtained: microaneurysm, hemorrhage, venous beading, intraretinal microvascular abnormality, neovascularization, and preretinal hemorrhage, which are used as prior knowledge for annotation in weakly supervised multi-label classification;

[0046] Although the above attributes can be used to obtain multi-classification labels, the characteristics of lesions represented by the same fundus image are not exactly the same, so the prior knowledge attribute labels may not be completely correct. Therefore, different lesion attributes have different effects on the disease classification of the current image. This paper uses attribute mining methods to solve this problem and further improve the disease classification performance.

[0047] Step S32: Based on the input feature map F NA , the attribute mining and lesion identification module uses cross entropy loss to classify diseases, and then uses formula (2) to calculate the attribute prediction q:

[0048] q=f(F NA ) (2)

[0049] Where f is a function containing a fully connected layer and a sigmoid layer, and q is a vector of dimension 6, representing the six predicted lesion attributes;

[0050] Step S33: Use the feature class activation map method to obtain a heat map to obtain lesion information; use a threshold t to obtain K activation areas on the heat map, thereby forming K connected components, each connected component as a lesion area; perform K iterations on the heat map, each time at F NA Cover the affected area with F NA Multiply and share the weight of f to get the new attribute estimate q k ,q k ∈R 6The softmax layer can be used to obtain the damage attribute weight of this iteration, and the average value is taken as the attribute weight of the lesion. The calculation formula (3) is as follows:

[0051]

[0052] Where softmax(·) is the softmax layer, k = 0 is calculated on the original feature map F, and weigh is the attribute weight of the lesion, which is used to solve the problem of incomplete attribute labels;

[0053] Step S34: Multiply the attribute weight weigh and the attribute prediction q to obtain the weighted attribute prediction. The calculation formula (4) is as follows:

[0054] q weight =q×weight (4)

[0055] q weight Finally, after passing through the fully connected layer and the sigmoid layer, the binary cross entropy loss is calculated with the attribute labels, thereby obtaining the lesion attribute prediction results and disease grading results.

[0056] Figure 4 A schematic diagram of the lesion attribute mining process based on prior knowledge and iterative attribute mining is presented;

[0057] In one embodiment, the above step S4: based on the lesion attribute prediction results and the disease grading results, a class activation map method is used to obtain a visualization map of the lesion location, and a final lesion recognition result is obtained through multiple iterative erasing calculations, specifically including:

[0058] Based on the K-times iterative calculation process in step S33, the lesion attributes of the current occluded area can be calculated by formula (5), and the K results are combined to obtain the lesion recognition result of the entire image;

[0059]

[0060] The final lesion recognition effect is shown in the figure Figure 5 As shown in the figure, the input image is a color fundus image, the lesion recognition is the lesion recognition effect, and the potential lesion is a visualization heat map of the potential lesion, where the highlighted part indicates the presence of potential lesions.

[0061] The method disclosed in the present invention uses a lesion point capture module for weakly supervised object positioning, which can capture more potential lesion information during the disease grading process, that is, it uses category-specific information to obtain the potential lesion location, and uses a lesion mining method to determine the lesion type, and can provide an explanatory diagnostic basis for disease grading through visualization results. The method disclosed in the present invention optimizes the configuration of the lesion point capture module for weakly supervised object positioning through an optimal network structure search module based on reinforcement learning, thereby improving the effect of lesion capture. The method disclosed in the present invention uses the basis for lesion grading as a priori knowledge attribute label, proposes an attribute mining method to allow the model to fit the label, further improves the disease grading performance, and uses the attribute category obtained in the attribute mining process as the lesion detection result.

[0062] Example 2

[0063] like Figure 6 As shown, the embodiment of the present invention provides a weakly supervised learning diabetic retinopathy grading and lesion identification system, including the following modules:

[0064] A feature map acquisition module 51 is used to input the diabetic retinopathy fundus color image into a convolutional neural network to obtain a feature map F; input F into a lesion point capture module based on weakly supervised object positioning to reduce its dimension into a one-dimensional feature map. Based on the selection probability, one of the three branch modules is selected: an attention branch module, a salient feature erasure module, and a salient feature retention module. A score map is calculated and the score map is multiplied by the feature map F to obtain a new feature map F';

[0065] The optimal network structure search module 52 is used to calculate the insertion position of the weakly supervised lesion point capture module in the convolutional neural network model, the selection probability of the three branch modules, the feature discarding threshold and the retention threshold in the significant feature erasing module and the significant feature retaining module using the optimal network structure search module based on reinforcement learning, and finally output the feature map F NA ;

[0066] Lesion attribute mining module 53 is used to obtain lesion categories for distinguishing diabetic retinopathy grades based on the diabetic retinopathy grading standard, use this as prior knowledge for annotation in weakly supervised multi-label classification, and obtain lesion attribute prediction results and disease grading results through attribute mining and lesion recognition modules;

[0067] The lesion identification module 54 is used to obtain a visualization of the lesion location using a class activation map method based on the lesion attribute prediction result and the disease grading result, and obtain the final lesion identification result through multiple iterative erasing calculations.

[0068] The above embodiments are provided for the purpose of describing the present invention only and are not intended to limit the scope of the present invention. The scope of the present invention is defined by the appended claims. Various equivalent substitutions and modifications made without departing from the spirit and principles of the present invention are intended to be within the scope of the present invention.

Claims

1. A weakly supervised learning method for classifying diabetic retinopathy image features and identifying lesion image features, characterized by: include: Step S1: Input the color image of the diabetic retinopathy into the convolutional neural network to obtain the feature map ;Will Input the lesion point capture module based on weakly supervised object positioning, reduce its dimension to a one-dimensional feature map, and select one of the three branch modules according to the selection probability: attention branch module, salient feature erasure module and salient feature retention module, calculate the score map, and compare the score map with the feature map. Multiply to get a new feature map ; Step S2: Use the optimal network structure search module based on reinforcement learning to calculate the insertion position of the weakly supervised lesion point capture module in the convolutional neural network model, the selection probability of the three branch modules, the feature discarding threshold and the retention threshold in the significant feature erasing module and the significant feature retention module, and finally output the feature map , specifically including: Use LSTM as the controller to generate the search parameter value, use ResNet50 as the backbone network for feature extraction, and select the probability Obtain various parameters of the lesion point capture module based on weakly supervised object positioning, and scale them by the performance of the validation set on the lesion point capture module based on weakly supervised object positioning The gradient of is used to update the controller; thereby determining the insertion position of the weakly supervised lesion point capture module in the convolutional neural network model, the selection probability of the three branch modules, the feature discarding threshold and the retention threshold in the significant feature erasing module and the significant feature retaining module; Step S3: Based on the diabetic retinopathy grading standard, the lesion categories used to distinguish the diabetic retinopathy grades are obtained, which are used as prior knowledge for annotation in weakly supervised multi-label classification. The lesion attribute prediction results and disease grading results are obtained through attribute mining and lesion recognition modules; Step S4: Based on the lesion attribute prediction results and the disease grading results, a class activation map method is used to obtain a visualization map of the lesion location, and the final lesion recognition result is obtained through multiple iterative erasing calculations.

2. The weakly supervised learning diabetic retinopathy image feature classification and lesion image feature recognition method according to claim 1 is characterized in that: Step S1: Input the diabetic retinopathy color image into the convolutional neural network to obtain the feature map ;Will Input the lesion point capture module based on weakly supervised object positioning, reduce its dimension to a one-dimensional feature map, and select one of the three branch modules according to the selection probability: attention branch module, salient feature erasure module and salient feature retention module, calculate the score map, and compare the score map with the feature map. Multiply to get a new feature map , specifically including: Step S11: Input the diabetic retinopathy color image into the convolutional neural network CNN to obtain the feature map , , where H, W and D are the feature maps height, width and depth; Step S12: After average pooling in a depth dimension, the attention feature map is obtained , , After the Sigmoid operation, the value is mapped to [0, 1] to obtain the one-dimensional feature map; Step S13: According to the selection probability , the one-dimensional feature map is input into one of the attention branch module, the salient feature erasure module and the salient feature retention module to obtain a score map; wherein the selection probabilities of the attention branch module, the salient feature erasure module and the salient feature retention module are respectively , and ,and + + =1; Step S14: Compare the score map with the feature map Multiply to get a new feature map , formula (1) is as follows; (1) in, is the selection probability, × is the point-by-point multiplication, is the output of the attention branch module, is the output of the salient feature erasure module, is the output of the salient feature retention module.

3. The weakly supervised learning diabetic retinopathy image feature classification and lesion image feature recognition method according to claim 1 is characterized in that: Step S3: obtaining lesion categories for distinguishing diabetic retinopathy grades based on the diabetic retinopathy grading standard, using them as prior knowledge for annotation in weakly supervised multi-label classification, and obtaining lesion attribute prediction results and disease grading results through attribute mining and lesion recognition modules, specifically including: Step S31: taking the lesion type of each disease level in diabetic retinopathy as prior knowledge, six prior knowledge attributes are obtained: microaneurysm, hemorrhage, venous beading, intraretinal microvascular abnormality, neovascularization, and preretinal hemorrhage, which are used as prior knowledge for annotation in weakly supervised multi-label classification; Step S32: Based on the input feature map The attribute mining and lesion identification module uses cross entropy loss to classify diseases, and then uses formula (2) to calculate the attribute prediction : (2) in, It is a function that includes a fully connected layer and a sigmoid layer. is a vector of 6 dimensions, representing the six predicted lesion attributes; Step S33: Use the feature class activation map method to obtain a heat map to obtain lesion information; use a threshold t to obtain K activation areas on the heat map, thereby forming K connected components, each of which is a lesion area; perform K iterative calculations on the heat map, each time at Cover the affected area with Multiply and share The weight of , The softmax layer can be used to obtain the damage attribute weight of this iteration, and the average value is taken as the attribute weight of the lesion. The calculation formula (3) is as follows: (3) Among them, softmax(·) is the softmax layer, and k=0 is in the original feature map. Calculate on, is the attribute weight of the lesion, which is used to solve the problem of incompletely correct attribute labels; Step S34: weight the attribute and the attribute prediction Multiplying them together yields the weighted attribute prediction, and the calculation formula (4) is as follows: (4)。 4. The method for weakly supervised learning of diabetic retinopathy image feature classification and lesion image feature recognition according to claim 3, characterized in that: The step S4: based on the lesion attribute prediction result and the disease grading result, a class activation map method is used to obtain a visualization map of the lesion location, and a final lesion recognition result is obtained through multiple iterative erasing calculations, specifically including: Based on the K-times iterative calculation process in step S33, the lesion attributes of the current occluded area can be calculated by formula (5), and the K results are combined to obtain the lesion recognition result of the entire image; (5)。 5. A weakly supervised learning diabetic retinopathy image feature classification and lesion image feature recognition system, characterized by: Includes the following modules: The feature map acquisition module is used to input the diabetic retinopathy color image into the convolutional neural network to obtain the feature map ;Will Input the lesion point capture module based on weakly supervised object positioning, reduce its dimension to a one-dimensional feature map, and select one of the three branch modules according to the selection probability: attention branch module, salient feature erasure module and salient feature retention module, calculate the score map, and compare the score map with the feature map. Multiply to get a new feature map ; The optimal network structure search module is used to calculate the insertion position of the weakly supervised lesion point capture module in the convolutional neural network model, the selection probability of the three branch modules, the feature discarding threshold and the retention threshold in the significant feature erasing module and the significant feature retention module using the optimal network structure search module based on reinforcement learning, and finally output the feature map , specifically including: Use LSTM as the controller to generate the search parameter value, use ResNet50 as the backbone network for feature extraction, and select the probability Obtain various parameters of the lesion point capture module based on weakly supervised object positioning, and scale them by the performance of the validation set on the lesion point capture module based on weakly supervised object positioning The gradient of is used to update the controller; thereby determining the insertion position of the weakly supervised lesion point capture module in the convolutional neural network model, the selection probability of the three branch modules, the feature discarding threshold and the retention threshold in the significant feature erasing module and the significant feature retaining module; The lesion attribute mining module is used to obtain lesion categories for differentiating diabetic retinopathy grades based on the diabetic retinopathy grading standards. This lesion category is used as prior knowledge for annotation in weakly supervised multi-label classification. Lesion attribute prediction results and disease grading results are obtained through attribute mining and lesion recognition modules. The lesion recognition module is used to obtain a visualization of the lesion location using a class activation map method based on the lesion attribute prediction results and the disease grading results, and obtain the final lesion recognition result through multiple iterative erasing calculations.

Images