Disease image recognition method based on lesion localization and feature iterative refinement technology

Abstract

The present invention relates to a disease image recognition method based on lesion localization and feature iterative refinement technology. Compared with the prior art, it solves the defect that the small lesion difference features between similar diseases cannot be effectively extracted under complex backgrounds, resulting in low disease recognition accuracy. The present invention includes the following steps: establishment of a disease data set; construction of a disease recognition network; training of a disease recognition network; acquisition of disease images to be identified; and acquisition of disease image recognition results. The present invention effectively removes the influence of background features on recognition results through unsupervised lesion localization technology, and feature iterative refinement technology amplifies the differences between similar diseases, which is beneficial for the model to learn the difference features between different diseases, thereby greatly improving the recognition accuracy of similar diseases in complex field environments and realizing crop disease recognition in field environments.

Description

Technical Field

[0001] The present invention relates to the technical field of disease image recognition, and in particular to a disease image recognition method based on lesion positioning and feature iterative refinement technology. Background Art

[0002] Traditional methods of identifying crop diseases usually rely on growers to observe the color, texture, shape and location of the disease. This manual method is time-consuming and labor-intensive, and requires growers to have professional knowledge of disease identification, and cannot provide timely and effective disease identification results.

[0003] In recent years, with the popularity of smartphones and the great success of machine learning (especially deep learning) in the field of image recognition, researchers have begun to explore the application of deep learning algorithms in the field of crop disease identification, and have successfully developed a crop disease identification system that can be deployed on embedded devices such as smartphones to guide growers in daily crop disease management.

[0004] In the application of these systems, it was found that the recognition accuracy of diseases with large visual differences in simple backgrounds is high, exceeding 95%. However, for similar diseases with small visual differences in complex backgrounds (such as early symptoms of similar crop diseases), the recognition accuracy of the model will drop sharply, and it is difficult to meet the needs of daily disease management of a certain type of crop. This is because the model does not effectively learn the characteristics of small lesions between similar diseases, and the complex background features will interfere with the model decision, especially when the background contains features similar to the lesions.

[0005] How to effectively remove the influence of background on recognition results and enhance the subtle difference features of similar diseases to improve the recognition accuracy of the model for similar diseases has become a technical problem that needs to be solved urgently. Summary of the invention

[0006] The purpose of the present invention is to solve the defect in the prior art that the tiny lesion difference features between similar diseases cannot be effectively extracted under complex backgrounds, resulting in low disease recognition accuracy, and to provide a disease image recognition method based on lesion positioning and feature iterative refinement technology to solve the above problem.

[0007] In order to achieve the above object, the technical solution of the present invention is as follows:

[0008] A disease image recognition method based on lesion localization and feature iterative refinement technology comprises the following steps:

[0009] 11) Establishment of disease dataset: Obtain crop disease images with artificial labels, select crop images of the same type to construct disease dataset, and perform preprocessing;

[0010] 12) Construction of disease recognition network: The disease recognition network is set to include a backbone network, a category response map generation module, a main lesion positioning and recognition module, and a lesion detail suggestion module. The input of the disease recognition network is the original image in the preprocessed disease data set, and the output is the category score corresponding to the category; the backbone network is used to extract the features of the original image, the main lesion area image and the lesion detail image, the category response map generation module is used to generate a category response map, the main lesion positioning and recognition module uses the category response map of the original image to realize the positioning of the main lesion area in the original image, and the lesion detail suggestion module uses the category response map of the main lesion area and the area suggestion method to locate the lesion details in the main lesion area image;

[0011] 13) Training of disease recognition network: input the preprocessed disease data set into the disease recognition network to train the disease recognition network;

[0012] 14) Acquisition of disease images to be identified: Acquisition of disease images to be identified and preprocessing;

[0013] 15) Obtaining the disease image recognition result: Input the preprocessed disease image to be identified into the trained disease recognition network to obtain the disease image recognition result.

[0014] The construction of the disease identification network includes the following steps:

[0015] 21) Constructing the backbone network: The backbone network is set to be composed of a number of convolutional neural network layers, pooling layers and activation function layers;

[0016] 22) Constructing a category response map generation module: setting the category response map to be calculated by the output feature map of the last convolutional layer of the backbone network and the weight of the fully connected layer;

[0017] 23) Construct the main lesion location and identification module;

[0018] 24) Construct a lesion detail suggestion module.

[0019] The construction of the category response graph generation module comprises the following steps:

[0020] 31) Set the feature map of the last layer of the backbone network output to be defined as S∈R C×H×W After the average pooling layer and the fully connected layer, the feature map obtains a one-dimensional vector corresponding to the score of each category Where N c is the number of categories, and the weight of the fully connected layer is recorded as

[0021] 32) Set the weight matrix W fc Perform matrix multiplication with the feature map S to obtain the category response map corresponding to all categories, that is, Take the graph M corresponding to the true category label c in M c =M[c], which is the category response diagram.

[0022] The construction of the main lesion location identification module comprises the following steps:

[0023] 41) Set the input of the main lesion location recognition module to the original image I raw Category response plot for

[0024] 42) Calculate the mean of the category response map using the following formula:

[0025]

[0026] Among them, H and W are the height and width of the feature map, and (i, j) is a pixel in the map;

[0027] 43) The category response plot All values ​​on are compared with the mean as follows, and the expression is as follows:

[0028]

[0029] Get a mask matrix, define the connected area with the largest true value range as the main lesion area, and the image in this area contains the most relevant information to category c;

[0030] 44) Mapping the position of the largest connected area in the mask matrix back to the coordinate position in the original image, that is, obtaining the main lesion position parameter, thereby realizing the positioning of the main lesion area;

[0031] 45) The area corresponding to the main lesion position parameter is changed from the original image I raw Crop it out and enlarge it to the original image size to obtain the image of the main lesion area.

[0032] The construction of the lesion detail suggestion module comprises the following steps:

[0033] 51) Set the input of the lesion detail suggestion module to be the category response map of the main lesion area image

[0034] 52) At each position, use three scales (112, 56, 28) and three aspect ratios (1:2, 1:1, 2:1), generate 9 anchor boxes, and calculate the category score in each anchor box;

[0035] 53) Using the non-maximum suppression algorithm, the five anchor box regions with the highest category scores are selected as the final lesion detail regions;

[0036] 54) These regional position parameters are mapped back to the main lesion region image, and the lesion images within the parameters are cropped and enlarged to obtain the corresponding lesion detail images.

[0037] The training of the disease recognition network includes the following steps:

[0038] 61) Input the original images in the preprocessed disease dataset into the backbone network, and input the feature map output by the last convolutional layer of the backbone network into its average pooling layer, fully connected layer and softmax layer to calculate the probability y of each category raw ; At the same time, the feature map is input into the category response map generation module to calculate the original image category response map

[0039] 62) Convert the original image category response map Input the main lesion location identification module to calculate the main lesion location parameters;

[0040] 63) Crop the largest connected area on the original image and enlarge it to the input image size to obtain the main lesion area image;

[0041] 64) inputting the main lesion area image into the backbone network to obtain a feature map of the main lesion area image, and inputting the feature map of the main lesion area image into the category response map generation module to obtain a category response map of the main lesion area image,

[0042] The feature map of the main lesion area image is input into the corresponding average pooling layer and fully connected layer to obtain the score vector z of each category main ; Then pass it into the softmax layer to calculate the probability y of each category main The feature map of the main lesion area image is input into the category response map generation module, and the category response map of the main lesion area image is output.

[0043] 65) The category response map of the main lesion area image Input the lesion detail suggestion module and calculate the top 5 lesion detail position parameters as the lesion detail information;

[0044] 66) cutting out the lesion detail information from the main lesion area image and enlarging it to obtain a lesion detail image;

[0045] 67) Input the lesion detail map into the backbone network to extract the lesion detail features and generate a lesion detail feature map. The lesion detail feature map is input into the corresponding average pooling layer and the fully connected layer to obtain the score vector z for each category. details , and then pass it into the softmax layer to calculate the probability y of each category details ;

[0046] 68) The disease detail information learned by the lesion detail suggestion module is propagated to the main lesion location and recognition module through knowledge distillation;

[0047] For a given category c calculate:

[0048]

[0049] Where T is a temperature parameter, is the score of the main lesion area image corresponding to category c, is the soft probability of the main lesion area image corresponding to category c, N c is the number of categories;

[0050] The disease details information is propagated to the main lesion location and recognition module through the following loss function;

[0051]

[0052] Among them, L soft (y soft ,y details ) is defined as the soft cross entropy loss function, is the soft probability of the main lesion area image corresponding to category k, is the probability that the lesion detail image corresponds to category k;

[0053] 69) For a given image I, the cross entropy loss function L cls and L in 64) soft Combination L total Optimizing the model:

[0054]

[0055] Among them, λ is a hyperparameter used to balance the weights of the two losses;

[0056] 610) Repeat the above process until L total No longer changes, the model converges, and the training ends.

[0057] Beneficial Effects

[0058] Compared with the prior art, the disease image recognition method based on lesion localization and feature iterative refinement technology of the present invention effectively removes the influence of background features on the recognition result through unsupervised lesion localization technology, and the feature iterative refinement technology amplifies the differences between similar diseases, which is conducive to the model learning the difference characteristics between different diseases, thereby greatly improving the recognition accuracy of similar diseases in complex field environments and realizing crop disease recognition in field environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 is a method sequence diagram of the present invention;

[0060] Figure 2a , Figure 2b , Figure 2c The main lesion area image generated by the main lesion positioning and identification module of the present invention;

[0061] Figure 3a , Figure 3b , Figure 3c This is a lesion detail positioning (disease) result map generated using the method described in the present invention. DETAILED DESCRIPTION

[0062] In order to have a further understanding and recognition of the structural features and the effects achieved by the present invention, a preferred embodiment and accompanying drawings are used for detailed description as follows:

[0063] like Figure 1 As shown, the disease image recognition method based on lesion localization and feature iterative refinement technology described in the present invention includes the following steps:

[0064] The first step is to establish a disease dataset: obtain crop disease images with artificial labels, select the same type of crop images to build a disease dataset, and perform preprocessing. Define diseases with visually similar color, texture, shape, disease location, and context information as similar diseases. Based on various public crop disease datasets, obtain crop disease images with artificial labels on various agricultural consulting websites, and select a type of crop (crop diseases of the same type usually have similar context information) to build a similar disease dataset.

[0065] The second step is to construct a disease recognition network: the disease recognition network is set to include a backbone network, a category response map generation module, a main lesion positioning and recognition module, and a lesion detail suggestion module. The input of the disease recognition network is the original image in the preprocessed disease data set, and the output is the category score corresponding to the category; the backbone network is used to extract the features of the original image, the main lesion area image and the lesion detail image, the category response map generation module is used to generate a category response map, the main lesion positioning and recognition module uses the category response map of the original image to realize the positioning of the main lesion area in the original image, and the lesion detail suggestion module uses the category response map of the main lesion area and the area suggestion method to locate the lesion details in the main lesion area image.

[0066] In terms of lesion localization, the present invention relates to two types of lesion localization, namely, main lesion area and lesion details, both of which utilize category response maps. The theoretical basis for achieving localization is that the value in the category response map characterizes the degree of its correlation with the category, and the larger the value, the greater the correlation. The areas in the category response map of the original disease image that are larger than the mean value are all target areas, and the largest one of these target areas is defined as the main lesion area; multiple area suggestion boxes are generated on the category response map corresponding to the main lesion area, and the category scores and sums of all the points therein are calculated. In order to eliminate the redundancy of detail features, the non-maximum suppression algorithm is used to take the positions of the five largest suggestion boxes as the lesion detail positions.

[0067] In the extraction of lesion features, the present invention adopts a coarse to fine iterative refinement strategy and uses the backbone network to sequentially extract the original disease image features, the main lesion area features, and the lesion detail features.

[0068] In terms of feature refinement, after obtaining the main lesion area and the lesion detail position, in order to better extract the detail features, these areas are cropped and enlarged to further extract and refine the lesion detail features.

[0069] The construction of the disease recognition network includes the following steps:

[0070] (1) Constructing a backbone network: The backbone network can be constructed in a traditional way, that is, the backbone network is set to be composed of a plurality of convolutional neural network layers, pooling layers and activation function layers.

[0071] (2) Constructing a category response map generation module: The category response map is calculated by the output feature map of the last convolutional layer of the backbone network and the weight of the fully connected layer. The category response map generation module is used to locate the main lesion area and lesion details. Compared with the saliency map method, the category response map can pay more attention to the area related to the category and achieve more accurate positioning of the main lesion area and lesion details.

[0072] The specific steps for building a category response graph generation module are as follows:

[0073] A1) Set the feature map output by the last layer of the backbone network to be defined as S∈R C×H×W After the average pooling layer and the fully connected layer, the feature map obtains a one-dimensional vector corresponding to the score of each category Where N c is the number of categories, and the weight of the fully connected layer is recorded as

[0074] A2) Set the weight matrix W fc Perform matrix multiplication with the feature map S to obtain the category response map corresponding to all categories, that is, Take the graph M corresponding to the true category label c in M c =M[c], which is the category response diagram.

[0075] (3) Construct the main lesion location and recognition module. The main lesion location and recognition module locates the main lesion according to the category response map of the original image, and crops and enlarges it from the original image to obtain the main lesion area image; this module can effectively eliminate the influence of complex background features on disease recognition, making the main lesion features more obvious, while retaining the contextual information of the lesion area in the original image to the greatest extent.

[0076] The specific steps are as follows:

[0077] B1) Set the input of the main lesion location recognition module to the original image I raw Category response plot for

[0078] B2) Calculate the mean of the category response map using the following formula:

[0079]

[0080] Among them, H and W are the height and width of the feature map, and (i, j) is a pixel in the map;

[0081] B3) The categorical response plot All values ​​on are compared with the mean as follows, and the expression is as follows:

[0082]

[0083] Get a mask matrix, define the connected area with the largest true value range as the main lesion area, and the image in this area contains the most relevant information to category c;

[0084] B4) mapping the position of the largest connected area in the mask matrix back to the coordinate position in the original image, that is, obtaining the main lesion position parameter, thereby realizing the positioning of the main lesion area;

[0085] B5) The area corresponding to the main lesion position parameters is changed from the original image I raw Crop it out and enlarge it to the original size to get the image of the main lesion area. Figure 2a , Figure 2b , Figure 2c As shown in the figure, it can be seen that the main lesion positioning and identification module effectively realizes the positioning of the main lesion area.

[0086] (4) Constructing the lesion detail suggestion module. Although the main lesion location and recognition module effectively eliminates the influence of complex background on disease recognition and amplifies the lesion features, this feature is not enough to distinguish similar diseases. It is still necessary to further amplify and learn the lesion detail features. The lesion detail suggestion module is mainly designed to achieve this function.

[0087] C1) Set the input of the lesion detail proposal module to be the category response map of the main lesion area image

[0088] C2) At each position, use three scales (112, 56, 28) and three aspect ratios (1:2, 1:1, 2:1), generate 9 anchor boxes, and calculate the category score in each anchor box;

[0089] C3) Using the non-maximum suppression algorithm, the five anchor box regions with the highest category scores are selected as the final lesion detail regions;

[0090] C4) These regional position parameters are mapped back to the main lesion area image, and the lesion images within the parameters are cropped and enlarged to obtain the corresponding lesion detail images. Figure 3a , Figure 3b , Figure 3c As shown in the figure, it can be seen that the lesion detail proposal module can effectively locate the detail lesion area that determines the disease category.

[0091] The third step is to train the disease recognition network: input the preprocessed disease data set into the disease recognition network to train the disease recognition network. The network adopts the training strategy of iterative refinement of lesion features, combines the category response map generation module, the main lesion location recognition module and the lesion detail suggestion module to achieve "coarse to fine" lesion features. At the same time, the influence of lesion detail features and lesion upper and lower features on disease recognition is taken into account. The lesion detail features learned by the lesion detail suggestion module are propagated to the main lesion location recognition module by using knowledge distillation, which effectively improves the overall recognition accuracy of the network.

[0092] The training of the disease recognition network includes the following steps:

[0093] (1) The original images in the preprocessed disease dataset are input into the backbone network. The feature map output by the last convolutional layer of the backbone network is input into its average pooling layer, fully connected layer and softmax layer to calculate the probability y of each category. raw ; At the same time, the feature map is input into the category response map generation module to calculate the original image category response map

[0094] (2) The original image category response map Input the main lesion positioning and identification module to calculate the main lesion position parameters.

[0095] (3) The largest connected area is cropped on the original image and enlarged to the input image size to obtain the main lesion area image. The cropping operation can effectively remove complex background information, and the enlargement operation is conducive to the backbone network to better extract lesion features. In the present invention, after completing the positioning operation, the main lesion positioning and identification module and the lesion detail suggestion module crop the corresponding main lesion area and lesion details from the original image and the main lesion area through the cropping operation, and then enlarge the main lesion area and lesion details to the original image size through the enlargement operation and input them into the backbone network to extract features.

[0096] (4) Inputting the main lesion area image into the backbone network to obtain a feature map of the main lesion area image, and inputting the feature map of the main lesion area image into the category response map generation module to obtain a category response map of the main lesion area image.

[0097] The feature map of the main lesion area image is input into the corresponding average pooling layer and fully connected layer to obtain the score vector z of each category main ; Then pass it into the softmax layer to calculate the probability y of each category main The feature map of the main lesion area image is input into the category response map generation module, and the category response map of the main lesion area image is output.

[0098] (5) The category response map of the main lesion area image Input the lesion detail suggestion module and calculate the top 5 lesion detail position parameters as the lesion detail information.

[0099] (6) The lesion detail information is cropped out from the main lesion area image and amplified to obtain a lesion detail map.

[0100] (7) The lesion detail map is input into the backbone network to extract the lesion detail features and generate a lesion detail feature map. The lesion detail feature map is input into the corresponding average pooling layer and fully connected layer to obtain the score vector z for each category. details , and then pass it into the softmax layer to calculate the probability y of each category details .

[0101] (8) Propagate the disease detail information learned by the lesion detail suggestion module to the main lesion location and recognition module through knowledge distillation;

[0102] For a given category c calculate:

[0103]

[0104] Where T is a temperature parameter, is the score of the main lesion area image corresponding to category c, is the soft probability of the main lesion area image corresponding to category c, N c is the number of categories;

[0105] The disease details information is propagated to the main lesion location and recognition module through the following loss function;

[0106]

[0107] Among them, L soft (y soft ,y details ) is defined as the soft cross entropy loss function, is the soft probability of the main lesion area image corresponding to category k, is the probability that the lesion detail image corresponds to category k.

[0108] (9) For a given image I, the cross entropy loss function L cls and L in (4) soft Combination L total Optimizing the model:

[0109]

[0110] Among them, λ is a hyperparameter used to balance the weights of the two losses;

[0111] (10) Repeat the above process until L total No longer changes, the model converges, and the training ends.

[0112] The fourth step is to obtain the disease image to be identified: obtain the disease image to be identified and perform preprocessing.

[0113] Step 5: Obtaining the disease image recognition results: Input the pre-processed disease image to be identified into the trained disease recognition network to obtain the disease image recognition results.

[0114] Here, the crop disease image to be identified is input into the trained disease recognition network, and the main lesion features are extracted by the main lesion positioning and recognition module. The final disease recognition basis of the crop with this feature is obtained through average pooling and a fully connected layer, and the category corresponding to the maximum score is selected as the final classification result. The original disease image features contain too many background features that are not conducive to classification, and the overly detailed lesion detail features lose contextual information. Both features are not conducive to disease recognition. Therefore, the main lesion area features that have removed the background information and learned the lesion detail features are used as the final classification basis. The main lesion classification features are averaged pooled, fully connected layers, and softmax to obtain the score vector of each category; the category corresponding to the item with the highest score is the final recognition result.

[0115] Table 1 Comparison of recognition accuracy between the method of the present invention and the prior art method

[0116] method mean Precision mean Recall mean F1-score Resnet50 83.14 81.67 81.67 RA-CNN 83.37 82.42 82.63 MA-CNN 83.71 83.26 82.75 MMAL-Net 85.73 83.82 84.38 SSN 83.73 82.07 82.07 TASN 85.14 83.91 84.20 S3N 85.86 83.09 83.79 The method of the present invention 88.49 87.22 88.54

[0117] As can be seen from Table 1, the detection results of the method of the present invention and other prior art methods on crop similar disease data sets use the detection accuracy evaluation methods well-known in the industry, such as mean Precision, mean Recall, and mean F1-score. From Table 1, it can be seen that the recognition accuracy of the method described in this patent is significantly ahead of the prior art methods.

[0118] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions only describe the principles of the present invention. The present invention may be subject to various changes and improvements without departing from the spirit and scope of the present invention. These changes and improvements fall within the scope of the present invention. The scope of protection claimed by the present invention is defined by the attached claims and their equivalents.

Claims

1. A disease image recognition method based on lesion localization and feature iterative refinement technology, characterized in that: The following steps are involved: 11) Establishment of disease dataset: Obtain crop disease images with artificial labels, select crop images of the same type to construct disease dataset, and perform preprocessing; 12) Construction of disease recognition network: The disease recognition network is set to include a backbone network, a category response map generation module, a main lesion positioning and recognition module, and a lesion detail suggestion module. The input of the disease recognition network is the original image in the preprocessed disease data set, and the output is the category score corresponding to the category; the backbone network is used to extract the features of the original image, the main lesion area image and the lesion detail image, the category response map generation module is used to generate a category response map, the main lesion positioning and recognition module uses the category response map of the original image to realize the positioning of the main lesion area in the original image, and the lesion detail suggestion module uses the category response map of the main lesion area and the area suggestion method to locate the lesion details in the main lesion area image; 13) Training of disease recognition network: input the preprocessed disease data set into the disease recognition network to train the disease recognition network; The training of the disease recognition network includes the following steps: 131) The original images in the preprocessed disease dataset are input into the backbone network, and the feature map output by the last convolutional layer of the backbone network is input into its average pooling layer, fully connected layer and softmax layer to calculate the probability y of each category raw ; At the same time, the feature map is input into the category response map generation module to calculate the original image category response map 132) The original image category response map Input the main lesion location identification module to calculate the main lesion location parameters; 133) cropping the largest connected area on the original image and enlarging it to the input image size to obtain an image of the main lesion area; 134) inputting the main lesion area image into the backbone network to obtain a feature map of the main lesion area image, and inputting the feature map of the main lesion area image into the category response map generation module to obtain a category response map of the main lesion area image, The feature map of the main lesion area image is input into the corresponding average pooling layer and fully connected layer to obtain the score vector z of each category main ; Then pass it into the softmax layer to calculate the probability y of each category main The feature map of the main lesion area image is input into the category response map generation module, and the category response map of the main lesion area image is output. 135) The category response map of the main lesion area image Input the lesion detail suggestion module and calculate the top 5 lesion detail position parameters as the lesion detail information; 136) cutting out the lesion detail information from the main lesion area image and performing a magnification process to obtain a lesion detail map; 137) Input the lesion detail map into the backbone network to extract the lesion detail features and generate a lesion detail feature map. The lesion detail feature map is input into the corresponding average pooling layer and the fully connected layer to obtain the score vector z of each category. details , and then pass it into the softmax layer to calculate the probability y of each category details ; 138) The disease detail information learned by the lesion detail suggestion module is propagated to the main lesion location and recognition module through knowledge distillation; For a given category c calculate: Where T is a temperature parameter, is the score of the main lesion area image corresponding to category c, is the soft probability of the main lesion area image corresponding to category c, N c is the number of categories; The disease details information is propagated to the main lesion location and recognition module through the following loss function; Among them, L soft (y soft ,y details ) is defined as the soft cross entropy loss function, is the soft probability of the main lesion area image corresponding to category k, is the probability that the lesion detail image corresponds to category k; 139) For a given image I, the cross entropy loss function L cls and L in 64) soft Combination L total Optimizing the model: Among them, λ is a hyperparameter used to balance the weights of the two losses; 1310) Repeat the above process until L total No longer changes, the model converges, and the training ends; 14) Acquisition of disease images to be identified: Acquisition of disease images to be identified and preprocessing; 15) Obtaining the disease image recognition result: Input the preprocessed disease image to be identified into the trained disease recognition network to obtain the disease image recognition result.

2. The disease image recognition method based on lesion localization and feature iterative refinement technology according to claim 1 is characterized in that: The construction of the disease identification network includes the following steps: 21) Constructing the backbone network: The backbone network is set to be composed of several convolutional neural network layers, pooling layers and activation function layers; 22) Constructing a category response map generation module: setting the category response map to be calculated by the output feature map of the last convolutional layer of the backbone network and the weight of the fully connected layer; 23) Construct the main lesion location and identification module; 24) Construct a lesion detail suggestion module.

3. The disease image recognition method based on lesion localization and feature iterative refinement technology according to claim 2 is characterized in that: The construction of the category response graph generation module comprises the following steps: 31) Set the feature map of the last layer of the backbone network output to be defined as S∈R C×H×W After the average pooling layer and the fully connected layer, the feature map obtains a one-dimensional vector corresponding to the score of each category Where N c is the number of categories, and the weight of the fully connected layer is recorded as 32) Set the weight matrix W fc Perform matrix multiplication with the feature map S to obtain the category response map corresponding to all categories, that is, Take the graph M corresponding to the true category label c in M c =M[c], which is the category response diagram.

4. The disease image recognition method based on lesion localization and feature iterative refinement technology according to claim 2 is characterized in that: The construction of the main lesion location identification module comprises the following steps: 41) Set the input of the main lesion location recognition module to the original image I raw Category response plot for 42) Calculate the mean of the category response map using the following formula: Among them, H and W are the height and width of the feature map, and (i, j) is a pixel in the map; 43) The category response plot All values ​​on are compared with the mean as follows, and the expression is as follows: Get a mask matrix, define the connected area with the largest true value range as the main lesion area, and the image in this area contains the most relevant information to category c; 44) Mapping the position of the largest connected area in the mask matrix back to the coordinate position in the original image, that is, obtaining the main lesion position parameter, thereby realizing the positioning of the main lesion area; 45) The area corresponding to the main lesion position parameter is changed from the original image I raw Crop it out and enlarge it to the original image size to obtain the image of the main lesion area.

5. The disease image recognition method based on lesion localization and feature iterative refinement technology according to claim 2 is characterized in that: The construction of the lesion detail suggestion module comprises the following steps: 51) Set the input of the lesion detail suggestion module to be the category response map of the main lesion area image 52) At each position, use three scales (112, 56, 28) and three aspect ratios (1:2, 1:1, 2:1), generate 9 anchor boxes, and calculate the category score in each anchor box; 53) Using the non-maximum suppression algorithm, the five anchor box regions with the highest category scores are selected as the final lesion detail regions; 54) These regional position parameters are mapped back to the main lesion region image, and the lesion images within the parameters are cropped and enlarged to obtain the corresponding lesion detail images.

Images