Strawberry Disease Image Recognition Method Based on Self-Attention Mechanism

Abstract

The present invention relates to a strawberry disease image recognition method based on the self-attention mechanism, which solves the defect that it is difficult to recognize strawberry disease images compared with the prior art. The present invention includes the following steps: obtaining a strawberry disease image data set and performing preprocessing; constructing a strawberry disease classification and recognition model; constructing a strawberry disease self-attention mechanism module; training the strawberry disease classification and recognition model; obtaining the strawberry disease image to be recognized; and obtaining the result of the strawberry disease image to be recognized. The present invention solves the problem of the recognition accuracy of strawberry diseases at the present stage, uses data augmentation to process strawberry disease images, and proposes a self-attention mechanism module, and combines the use of a strawberry disease classification and recognition model as a backbone network to accelerate the recognition speed and accuracy of strawberry diseases.

Description

Technical Field

[0001] The present invention relates to the technical field of disease identification, and specifically to a strawberry disease image recognition method based on the self-attention mechanism. Background Art

[0002] Diseases of crops have a huge impact on agricultural yields. If the types of agricultural diseases cannot be identified in a timely manner, agricultural yields will be greatly affected. Especially for fruits like strawberries that have high environmental requirements, if the types of strawberry diseases are not discovered in a timely manner, the strawberry yield reduction can reach more than 50%. Therefore, the timely identification of strawberry diseases is the basis for the prevention and control of strawberry diseases.

[0003] In traditional agricultural work, agricultural workers are often at a loss when faced with strawberry crop diseases. On the one hand, it is due to the lack of knowledge about professional strawberry diseases, and on the other hand, it is because the identification environment of strawberry diseases is complex. All these lead to difficulties in identifying strawberry diseases. Correctly identifying the type of strawberry disease and dealing with the corresponding category has become the primary problem to be solved currently. The traditional identification environment of strawberry diseases is complex, resulting in difficulties in identifying strawberry diseases. Therefore, how to design an image recognition method for strawberry diseases has become an urgent technical problem to be solved. Summary of the Invention

[0004] The purpose of the present invention is to solve the defect that it is difficult to identify strawberry disease images in the prior art, and to provide a strawberry disease image recognition method based on the self-attention mechanism to solve the above problems.

[0005] In order to achieve the above purpose, the technical solution of the present invention is as follows:

[0006] A strawberry disease image recognition method based on the self-attention mechanism includes the following steps:

[0007] Obtain a strawberry disease image data set and perform preprocessing: Obtain the original strawberry disease images, perform data augmentation on them to obtain enhanced strawberry disease images, and obtain a strawberry disease image data set;

[0008] Construct a strawberry disease classification and recognition model: Construct a strawberry disease classification and recognition model, which includes a dimensionality reduction module, a sampling module, and a classification module;

[0009] Construct a strawberry disease self-attention mechanism module: The strawberry disease self-attention mechanism module is composed of a window module and a sliding window module connected in series;

[0010] Train the strawberry disease classification and recognition model: Input the strawberry disease image data set into the strawberry disease classification and recognition model for training;

[0011] Obtaining the strawberry disease image to be recognized: Obtain the strawberry disease image to be recognized and perform preprocessing;

[0012] Obtaining the result of the strawberry disease image to be recognized: Input the preprocessed strawberry disease image to be recognized into the trained strawberry disease classification and recognition model to obtain the recognition result of the strawberry disease image.

[0013] The obtaining of the strawberry disease image dataset and performing preprocessing includes the following steps:

[0014] Enhance the contrast and intensity of the obtained original strawberry disease image, and use the flipping method to enhance the number of the dataset;

[0015] Adopt the cutout data augmentation method, that is, fill a square area in the strawberry disease image with 0 pixel values to achieve random occlusion; then perform central normalization operation on the randomly occluded strawberry disease image to eliminate the influence of 0 value filling on training;

[0016] Fuse and enhance the strawberry disease image, and its expression is as follows:

[0017] λ = Beta(α,β)

[0018] mix_batch x = λbatch x1 +(1 - λ)batch x2 ,

[0019] where Beta represents the beta distribution, α and β are two calculation parameters, λ is the mixing coefficient calculated by the beta distribution of parameters α and β, and mix_batch x represents the mixed strawberry disease samples, batch x1 is a batch of strawberry disease samples, batch x2 is another batch of strawberry disease samples;

[0020] After the input strawberry disease image is preprocessed, the strawberry disease enhanced image is output.

[0021] The construction of the strawberry disease classification and recognition model includes the following steps:

[0022] Set that the strawberry disease classification and recognition model includes a dimensionality reduction module, a sampling module and a classification module,

[0023] Input the strawberry disease enhanced image, segment it through blocks of size 4×4, and output the strawberry segmentation feature map; first output the dimensionality-reduced strawberry disease information through the dimensionality reduction module for the strawberry segmentation feature map, then output the hierarchical strawberry disease feature map through three sampling modules, and finally classify and recognize the hierarchical strawberry disease feature map through the classification module to output the strawberry disease classification and recognition information;

[0024] Set a dimensionality reduction module, which includes a convolutional layer and a self-attention mechanism module;

[0025] Set that the convolutional layer consists of a 1×1 convolution, which is used for dimensionality reduction of the strawberry disease feature map; the self-attention mechanism module is composed of a window module and a sliding window module connected in series;

[0026] After the strawberry segmentation feature map enters the dimensionality reduction module, it first passes through a 1×1 convolutional layer to obtain a strawberry dimensionality reduction feature map, and then inputs the strawberry dimensionality reduction feature map into the self-attention mechanism module to output a layer of strawberry disease information;

[0027] Set a sampling module, and set that there are three sampling modules, and the three sampling modules are all the same;

[0028] The sampling module includes a block splicing and a self-attention mechanism module. The block splicing consists of the splicing of strawberry disease information and a 1×1 convolution to achieve downsampling of the strawberry disease feature map. The splicing of strawberry disease feature information divides each 2×2 adjacent pixel into a block, and then stitches together the pixels at the same position in each block to obtain four strawberry splicing feature maps. These four strawberry splicing feature maps are stitched together in the depth direction and then output a strawberry block splicing feature map through a 1×1 convolution;

[0029] Input the strawberry block splicing feature map into the self-attention mechanism module. After the strawberry block splicing feature map passes through three sampling modules, a hierarchical strawberry disease feature map is output;

[0030] Set a classification module,

[0031] The classification module consists of an LN normalization and a global average pooling. The LN normalization is used to perform overall normalization of the feature data by translation and scaling on the extracted hierarchical strawberry disease feature map; the global average pooling averages the two-dimensional images of each channel in the hierarchical strawberry disease feature map and outputs a C×1 feature matrix. The feature matrix is multiplied by a weight matrix G of size t×C to obtain the probability of each class of strawberry disease, where the weight matrix G is obtained through training, t is the number of strawberry disease categories, and finally the strawberry disease classification information is output through the classification module.

[0032] The construction of the self-attention mechanism module includes the following steps:

[0033] Set that the self-attention mechanism module is composed of a window module and a sliding window module connected in series;

[0034] Set the window module: the input is the initial feature map, and the initial feature map is the strawberry dimensionality reduction feature map or the strawberry block splicing feature map,

[0035] First, perform LN normalization, then output the window self-attention feature map through the window self-attention mechanism. Fuse the initial feature map with the obtained window self-attention feature map to get the fused feature map. Perform LN normalization on the fused feature map again and input it into the multi-layer perceptron module to output the multi-layer perceptron feature map. Then fuse the fused feature map with the obtained multi-layer perceptron feature map to form the final output window feature map;

[0036] Set the window self-attention mechanism;

[0037] Set the sliding window module:

[0038] Replace the window-based self-attention mechanism in the window module with the shifted window self-attention mechanism to obtain the sliding window module. Use the final output window feature map of the window module as the input of the sliding window module, and output either a layer of strawberry disease information or a layer of hierarchical strawberry disease feature maps;

[0039] Set that the shifted window self-attention mechanism consists of window shifting and setting a mask,

[0040] First, input the final output window feature map of the window module. The window shifting moves the window in the feature map downward by half the window size, i.e., the window is offset to the right and downward from the upper left corner by to obtain the shifted window, where M is the window size;

[0041] For the obtained shifted window, use the self-attention calculation method in the window self-attention mechanism to calculate the information of each sliding window;

[0042] Use the method of shift configuration to isolate the information of different regions by setting a mask.

[0043] The training of the strawberry disease classification and recognition model includes the following steps:

[0044] Train the strawberry disease classification and recognition model:

[0045] Input the enhanced strawberry disease images into the strawberry disease classification and recognition model. First, divide the enhanced strawberry disease images into blocks through block segmentation. For the obtained segmented strawberry feature maps, first train them through the dimensionality reduction module to obtain strawberry disease information;

[0046] For the strawberry disease dimensionality reduction information, perform training and sampling through three sampling modules to obtain hierarchical strawberry disease feature maps, and input the hierarchical strawberry disease feature maps into the classification module for classification;

[0047] The hierarchical strawberry disease feature map is separately trained in the global average pooling of the classification module, and the trained weight matrix G is used to calculate the probability of each class of strawberry disease information. The predicted strawberry disease classification information is finally output through the classification training of the classification module;

[0048] The self-attention mechanism module is separately trained. Its input is the strawberry dimensionality-reduced feature map in the dimensionality reduction module or the strawberry patch stitching feature map in the sampling module. The trained coefficient matrix S is used for the calculation of self-attention;

[0049] The predicted strawberry disease classification information is compared with the correct strawberry disease classification information, and then the gradient backpropagation algorithm is used to adjust the weights of the model;

[0050] Training loss function;

[0051] Define the loss function L of the strawberry disease classification recognition model as:

[0052]

[0053] where y is the true strawberry disease classification information, is the predicted strawberry disease classification information.

[0054] The setting of the window self-attention mechanism includes the following steps:

[0055] The setting based on the window self-attention mechanism consists of window segmentation and self-attention calculation. First, input the strawberry dimensionality-reduced feature map or the strawberry patch stitching feature map. Window segmentation divides it into H×W blocks, and every M×M blocks form a window, obtaining a total of windows, and then calculate the self-attention value of each window through self-attention;

[0056] The self-attention calculation method is set as follows:

[0057] First, input the image to be calculated with size . Among them, C is the image dimension. The input image to be calculated passes through the coefficient matrix S with size C×C to obtain three vectors Q, K, and V. Among them, the coefficient matrix S is obtained through training. Q is the query vector, K is the key vector, and V is the value vector. Then calculate the self-attention matrix A as follows:

[0058]

[0059] where, is the standard deviation of the matrix elements in QK T in,

[0060] Then the computational complexity Ω based on the window self-attention mechanism is:

[0061] Ω = 4HW + 2M2 HWC

[0062] Among them, H represents the height of the feature map, W represents the width of the feature map, C represents the depth of the feature map, and M represents the size of each window.

[0063] Beneficial effects

[0064] The strawberry disease image recognition method based on the self-attention mechanism of the present invention solves the problem of the recognition accuracy of strawberry diseases at the present stage compared with the prior art. It uses data augmentation to process strawberry disease images, and proposes a self-attention mechanism module, and combines the use of a strawberry disease classification recognition model as the backbone network to accelerate the recognition speed and accuracy of strawberry diseases.

[0065] The method of the present invention aims at the situation where various types of strawberry data are unevenly distributed. It fills the number of the dataset through methods such as inversion to solve the overfitting problem that may occur during the training process due to the large difference in the number of strawberry disease types. And it improves the generalization ability of the data and increases the amount of image features through methods such as data shearing and data fusion to improve the recognition accuracy of strawberry diseases. Then, it mainly uses a special self-attention mechanism, which effectively reduces the amount of calculation and quickly learns rich strawberry disease feature information by constructing a hierarchical feature network, aiming to improve the learning ability of strawberry micro-lesion symptoms. Finally, the global average pooling method is used for classification recognition to solve the input-output limitations, increase the robustness of the module, and lightweight the model to accelerate the recognition speed.

[0066] The present invention constructs a strawberry disease self-attention mechanism module to extract strawberry disease classification feature information, and then constructs a dimensionality reduction module and a sampling module based on the strawberry disease self-attention mechanism module. The dimensionality reduction module adjusts the dimension of the strawberry disease feature map, and the sampling module outputs a hierarchical strawberry disease feature map to obtain a strawberry disease feature map with rich information. Finally, the final strawberry disease classification recognition information is obtained through the classification module.

[0067] Through experimental verification, the method proposed by the present invention has a significant improvement in the recognition accuracy compared with the current strawberry disease recognition method. This shows that the method of the present invention can be applied to the agricultural strawberry disease recognition work under multi-category and complex backgrounds. Brief description of the drawings

[0068] Figure 1 It is the sequence diagram of the method of the present invention;

[0069] Figure 2 It is the step diagram of the mobile window self-attention mechanism involved in the present invention;

[0070] Figure 3 It is the framework diagram of the strawberry disease self-attention mechanism module involved in the present invention;

[0071] Figure 4 This is the comparison chart of the accuracy curves between the present invention and the prior art;

[0072] Figure 5 This is the prediction result chart of the method of the present invention. Detailed implementation manners

[0073] To have a further understanding and recognition of the structural features and achieved effects of the present invention, the following is a detailed description with preferred embodiments and accompanying drawings:

[0074] As Figure 1 shown, the strawberry disease image recognition method based on the self-attention mechanism of the present invention includes the following steps:

[0075] The first step is to obtain a strawberry disease image dataset and perform preprocessing: Obtain the original strawberry disease images, perform data augmentation on them to obtain the enhanced strawberry disease images, and obtain the strawberry disease image dataset.

[0076] The specific steps are as follows:

[0077] (1) Strengthen the contrast and intensity of the strawberry disease images in the dataset. For the problem of uneven data volume of each type of strawberry disease in the original strawberry disease dataset, methods such as flipping are used to enhance the dataset in terms of quantity to solve the overfitting problem that may occur due to the large difference in the number of strawberry disease types during the training process.

[0078] (2) To improve the generalization ability of each type of data in the strawberry disease dataset, the cutout data augmentation method is used, that is, a square area in the strawberry disease image is filled with 0 pixel values to achieve random occlusion; then the randomly occluded strawberry disease image is subjected to central normalization operation to eliminate the influence of 0 value filling on training.

[0079] (3) Perform fusion enhancement on the strawberry disease images, and its expression is as follows:

[0080] λ = Beta(α,β)

[0081] mix_batch x = λbatch x1 +(1 - λ)batch x2 ,

[0082] where Beta represents the beta distribution, α and β are two calculation parameters, λ is the mixing coefficient calculated from the beta distribution of parameters α and β, mix_batch x represents the strawberry disease samples after mixing, batch x1 is a batch of strawberry disease samples, batchx2 These are another batch of strawberry disease samples.

[0083] (4) After the input strawberry disease image is preprocessed, an enhanced strawberry disease image is output.

[0084] Second step, construct a strawberry disease classification and recognition model: Construct a strawberry disease classification and recognition model, which includes a dimensionality reduction module, a sampling module, and a classification module.

[0085] In the classification of strawberry diseases, there are dimensionality limitations for the input, requiring a lower input dimension. Therefore, a dimensionality reduction module is introduced to reduce the dimension of the enhanced strawberry disease image. To obtain a large amount of rich strawberry disease feature information for classification, a sampling module is introduced. After passing through 3 sampling modules, the strawberry disease information will be downsampled by 4 times, 8 times, and 16 times respectively. The hierarchical strawberry disease feature map obtained in this way has rich strawberry disease feature information. Finally, a classification module is introduced for strawberry disease classification and recognition. In the classification module, global average pooling is selected for strawberry disease classification and recognition because global average pooling is more suitable for the recognition of various types of strawberry diseases. Compared with the limitations of the input of the fully connected layer, global average pooling greatly reduces the number of parameters, reduces overfitting, makes the model more robust, and makes strawberry disease recognition more stable and rapid.

[0086] The specific steps are as follows:

[0087] (1) Set that the strawberry disease classification and recognition model includes a dimensionality reduction module, a sampling module, and a classification module.

[0088] Input the enhanced strawberry disease image, output a strawberry segmentation feature map through segmentation by blocks of size 4×4; for the strawberry segmentation feature map, first output the dimensionality-reduced strawberry disease information through the dimensionality reduction module, then output a hierarchical strawberry disease feature map through three sampling modules, and finally classify and recognize the hierarchical strawberry disease feature map through the classification module to output strawberry disease classification and recognition information.

[0089] (2) Set the dimensionality reduction module, and the dimensionality reduction module includes a convolutional layer and a self-attention mechanism module;

[0090] Set that the convolutional layer consists of a 1×1 convolution, which is used for the dimensionality reduction of the strawberry disease feature map; the self-attention mechanism module is composed of a window module and a sliding window module connected in series;

[0091] After the strawberry segmentation feature map enters the dimensionality reduction module, first obtain a strawberry dimensionality-reduced feature map through a 1×1 convolutional layer, and then input the strawberry dimensionality-reduced feature map into the self-attention mechanism module to output a layer of strawberry disease information.

[0092] (3) Set the sampling module. Set that there are three sampling modules, and the three sampling modules are all the same;

[0093] The sampling module includes a block splicing and a self-attention mechanism module. The block splicing consists of the splicing of strawberry disease information and a 1×1 convolution to achieve downsampling of the strawberry disease feature map. The splicing of strawberry disease feature information divides each 2×2 adjacent pixel into a block, and then stitches together the pixels at the same position in each block to obtain four strawberry splicing feature maps. These four strawberry splicing feature maps are stitched together in the depth direction and then output the strawberry block splicing feature map through a 1×1 convolution;

[0094] The strawberry block splicing feature map is input into the self-attention mechanism module. After passing through three sampling modules, the hierarchical strawberry disease feature map is output.

[0095] (4) Set the classification module,

[0096] The classification module consists of an LN normalization and a global average pooling. The LN normalization is used to perform overall normalization of the feature data by translation and scaling on the extracted hierarchical strawberry disease feature map. Among them, the LN normalization is used to perform overall normalization of the feature data of the extracted hierarchical strawberry disease feature map by translation and scaling to remove the problems of missing sample data and uneven channel distribution; the global average pooling averages the two-dimensional images of each channel in the hierarchical strawberry disease feature map and outputs a C×1 feature matrix. The feature matrix is multiplied by a weight matrix G of size t×C to obtain the probability of each category of strawberry disease. Among them, the weight matrix G is obtained through training, and t is the number of strawberry disease categories. Finally, the strawberry disease classification information is output through the classification module.

[0097] In the third step, construct the strawberry disease self-attention mechanism module. The strawberry disease self-attention mechanism module is composed of a window module and a sliding window module connected in series.

[0098] In the self-attention mechanism module, the computational complexity is too high when calculating self-attention originally. In order to reduce the computational complexity, the window module is introduced. However, while solving the computational complexity problem, information blockage occurs between each window. Therefore, in order to enable information interaction between windows, solve the model limitation and expand the receptive field, the sliding window module is introduced.

[0099] The construction of the self-attention mechanism module includes the following steps:

[0100] (1) Set that the self-attention mechanism module is composed of a window module and a sliding window module connected in series.

[0101] (2) Set the window module: The input is the initial feature map, and the initial feature map is the strawberry dimensionality-reduced feature map or the strawberry block splicing feature map,

[0102] First, perform layer normalization (LN), then output the window self-attention feature map through the window self-attention mechanism. Fuse the initial feature map with the obtained window self-attention feature map to get the fused feature map. Then, perform layer normalization on the fused feature map again and input it into the multi-layer perceptron module to output the multi-layer perceptron feature map. Finally, fuse the fused feature map with the obtained multi-layer perceptron feature map to form the final output window feature map.

[0103] (3) Set the window self-attention mechanism. The setting of the window self-attention mechanism includes the following steps:

[0104] A1) Set that the window self-attention mechanism consists of window splitting and self-attention calculation. First, input the strawberry dimensionality-reduced feature map or the strawberry patch stitching feature map. Window splitting divides it into H×W blocks, and every M×M blocks form a window, obtaining a total of windows, and then calculate the self-attention value of each window through self-attention calculation;

[0105] A2) Set the self-attention calculation method as follows:

[0106] First, input the image to be calculated with a size of . Among them, C is the image dimension. The input image to be calculated passes through the coefficient matrix S with a size of C×C to obtain three vectors Q, K, and V. Among them, the coefficient matrix S is obtained through training. Q is the query vector, K is the key vector, and V is the value vector. Then, calculate the self-attention matrix A as follows:

[0107]

[0108] Among them, is the standard deviation of the matrix elements in QK T .

[0109] Then, the computational complexity Ω of the window self-attention mechanism is:

[0110] Ω = 4HW + 2M 2 HWC,

[0111] where H represents the height of the feature map, W represents the width of the feature map, C represents the depth of the feature map, and M represents the size of each window.

[0112] (4) Set the sliding window module:

[0113] Replace the window-based self-attention mechanism in the window module with the moving window self-attention mechanism to obtain the sliding window module, and use the final output window feature map of the window module as the input of the sliding window module, and output a layer of strawberry disease information or a layer of hierarchical strawberry disease feature maps.

[0114] (5) The set moving window self-attention mechanism consists of window movement and mask setting.

[0115] First, input the final output window feature map of the window module. The window movement moves the windows in it downward by half the window distance, that is, the windows are offset to the right and downward from the upper left corner by to obtain the moving windows, where M is the window size.

[0116] For the obtained moving windows, use the self-attention calculation method in the window self-attention mechanism to calculate the information of each sliding window, where the calculation method of the self-attention of the sliding window module is the same as that of the window module.

[0117] Use the method of shift configuration to isolate the information of different regions by setting masks.

[0118] Here, the self-attention calculated for each moving window includes the information of several previous windows, thus solving the problem of the model's limitation in communicating with each window and expanding the receptive field. To reduce the computational complexity of window self-attention, an efficient batch calculation is adopted for the method of shift configuration, so that the information of different regions can be isolated by setting masks.

[0119] Step 4, training of the strawberry disease classification and recognition model: Input the strawberry disease image dataset into the strawberry disease classification and recognition model for training.

[0120] The training of the strawberry disease classification and recognition model includes the following steps:

[0121] (1) Training the strawberry disease classification and recognition model:

[0122] B1) Input the enhanced strawberry disease images into the strawberry disease classification and recognition model. First, divide the enhanced strawberry disease images into blocks through block segmentation. For the obtained segmented strawberry segmentation feature maps, first train the strawberry disease information through the dimensionality reduction module.

[0123] B2) For the strawberry disease dimensionality reduction information, perform training sampling through three sampling modules to obtain a hierarchical strawberry disease feature map, and input the hierarchical strawberry disease feature map into the classification module for classification.

[0124] B3) Train separately in the global average pooling of the classification module to obtain the weight matrix G for calculating the probability of each class of strawberry disease information, and finally output the predicted strawberry disease classification information through the classification training of the classification module.

[0125] B4) Train the self-attention mechanism module separately. Its input is the strawberry dimensionality reduction feature map in the dimensionality reduction module or the strawberry block splicing feature map in the sampling module, and train to obtain the coefficient matrix S for self-attention calculation.

[0126] (2) Compare the obtained predicted strawberry disease classification information with the correct strawberry disease classification information, and then use the gradient backpropagation algorithm to adjust the weights of the model.

[0127] (3) Train the loss function; to make the model tend to make the predicted output closer to the true sample label y, define the loss function L of the strawberry disease classification recognition model as:

[0128]

[0129] where y is the true strawberry disease classification information, is the predicted strawberry disease classification information.

[0130] Compare the obtained predicted strawberry disease classification information with the correct strawberry disease classification information, and then use backpropagation to adjust the weights of the model to make the output probability distribution closer to the correct output; train the model with a large number of strawberry disease images. After sufficient training, the strawberry disease classification recognition model will continuously correct the distance from the correct strawberry disease classification information and finally reach a classification recognition accuracy of up to 97.2%.

[0131] Fifth step, obtaining the strawberry disease image to be recognized: Obtain the strawberry disease image to be recognized and perform preprocessing.

[0132] Sixth step, obtaining the result of the strawberry disease image to be recognized: Input the preprocessed strawberry disease image to be recognized into the trained strawberry disease classification recognition model to obtain the recognition result of the strawberry disease image.

[0133] As Figure 2 shown, it is the step diagram of the moving window self-attention mechanism using the method of the present invention. When using the window-based self-attention mechanism module, self-attention calculation is only performed within each window, so information cannot be transmitted between windows. To solve this problem, a moving window self-attention mechanism module is introduced, that is, an offset window-based self-attention mechanism.

[0134] As Figure 2 shown, the window-based self-attention mechanism is used on the left (assuming it is the L-th layer). Then, according to the previously introduced window-based self-attention mechanism and the moving window self-attention mechanism being used in pairs, the moving window self-attention mechanism is used in the (L + 1)-th layer. It can be found from the comparison of the left and right figures that the window (Windows) has shifted, that is, the window has shifted Pixels. The window after the downward offset, for example, for the 2x4 window in the second column of the first row, can enable the exchange of information between two windows in the first row of the L-th layer. For the 4x4 window in the second column of the second row, it can enable the exchange of information between four windows in the L-th layer, and the same applies to others. In this way, the problem that information cannot be exchanged between different windows is solved.

[0135] As Figure 3 shown, it is the strawberry disease self-attention mechanism module of the present invention. The strawberry disease self-attention mechanism module is composed of a window module (window module) and a sliding window module (sliding window module) connected in series. The window module first inputs the feature map, first passes through LN normalization, then performs self-attention mechanism based on the window to output the window self-attention feature map, fuses the initial feature map with the obtained window self-attention feature map to obtain the fused feature map, passes the fused feature map through LN normalization first, and then inputs it into the multi-layer perceptron module to output the multi-layer perceptron feature map, and then fuses the fused feature map with the obtained multi-layer perceptron feature map to finally obtain the window feature map; the basic structure of the sliding window module is the same. Replace the self-attention mechanism based on the window in the window module with the moving window self-attention mechanism to obtain the sliding window module, and use the output of the window module as the input of the sliding window module, and finally output the strawberry disease self-attention feature map.

[0136] Table 1 is the experimental comparison result table of this method and several commonly used classification methods

[0137]

[0138] As shown in Table 1, it shows the experimental comparison results of this method and several commonly used classification methods. It can be seen that the data augmentation method adopted by this method has a significant effect. The recognition accuracies of several commonly used classification methods have been improved by at least about 1%. The method of the present invention has also been improved by 1.13%. At the same time, the recognition accuracy of the method of the present invention is the highest, up to 97.25%. At the same time, the recognition speed is also the fastest among the same kind. This shows that the method of the present invention has a good effect in multi-category and high-complexity strawberry monitoring and can well solve the problems related to strawberry disease recognition.

[0139] By Figure 4It can be seen that compared with the original network model of the Swin Transformer, the method of the present invention climbs faster in terms of accuracy, and the finally recognized accuracy is relatively higher than that of the original network model. The accuracy of the method of the present invention in recognizing strawberry disease data is as high as 97.25%. There is a relatively obvious improvement compared with the accuracy of other classification network models, and the recognition accuracy is also improved by 1.1% compared with the Swin Transformer. The accuracy of the test set of the method of the present invention is not less than 97% in multiple experiments, which fully shows that the method of the present invention has a relatively obvious advantage in recognizing images of strawberries with diseases of small scale and high complexity.

[0140] It can be seen from Figure 5 that the method of the present invention is used to recognize strawberry diseases for 4 randomly selected strawberry disease images, and it can be seen that the result predictions are all successful and the prediction accuracy is as high as 100%, indicating that the method of the present invention has good recognition accuracy for strawberry diseases in complex environments.

[0141] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited by the above embodiments, and the above embodiments and the principles described in the specification are only the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention will have various changes and improvements, and these changes and improvements all fall within the scope of the present invention claimed. The scope of protection claimed by the present invention is defined by the appended claims and their equivalents.

Claims

1. A strawberry disease image recognition method based on the self-attention mechanism, characterized in that, It includes the following steps: 11) Obtain the strawberry disease image dataset and preprocess it: Obtain the original strawberry disease images, perform data augmentation on them to obtain the enhanced strawberry disease images, and obtain the strawberry disease image dataset; 12) Construct a strawberry disease classification and recognition model: Construct a strawberry disease classification and recognition model, which includes a dimensionality reduction module, a sampling module, and a classification module; 13) Construct a strawberry disease self-attention mechanism module: The strawberry disease self-attention mechanism module is composed of a window module and a sliding window module connected in series; Constructing the strawberry disease self-attention mechanism module includes the following steps: 131) Set that the strawberry disease self-attention mechanism module is composed of a window module and a sliding window module connected in series; 132) Set the window module: The input is the initial feature map, and the initial feature map is the strawberry dimensionality reduction feature map or the strawberry block stitching feature map. First, perform LN normalization, then output the window self-attention feature map through the window self-attention mechanism, fuse the initial feature map with the obtained window self-attention feature map to obtain the fused feature map, perform LN normalization on the fused feature map again, and then input it into the multi-layer perceptron module to output the multi-layer perceptron feature map. Fuse the fused feature map with the obtained multi-layer perceptron feature map again to form the final output window feature map; 133) Set the window self-attention mechanism; 134) Set the sliding window module: Replace the window-based self-attention mechanism in the window module with the moving window self-attention mechanism to obtain the sliding window module, and use the final output window feature map of the window module as the input of the sliding window module to output one layer of strawberry disease information or one layer of hierarchical strawberry disease feature maps; 135) Set that the moving window self-attention mechanism consists of window movement and setting a mask. First, input the final output window feature map of the window module. Move the window in it downward by half the window distance, that is, the window is offset to the right and downward from the upper left corner by to obtain a moving window, where M is the window size; For the obtained moving window, calculate the information of each sliding window using the self-attention calculation method in the window self-attention mechanism; Use the method of shift configuration to isolate the information in different regions by setting a mask; 14) Training of the strawberry disease classification and recognition model: Input the strawberry disease image dataset into the strawberry disease classification and recognition model for training; 15) Obtaining the strawberry disease image to be recognized: Obtain the strawberry disease image to be recognized and preprocess it; 16) Obtaining the result of the strawberry disease image to be recognized: Input the preprocessed strawberry disease image to be recognized into the trained strawberry disease classification and recognition model to obtain the recognition result of the strawberry disease image.

2. The strawberry disease image recognition method based on the self-attention mechanism according to claim 1, wherein, The obtaining of the strawberry disease image dataset and preprocessing includes the following steps: 21) Strengthen the contrast and intensity of the obtained original strawberry disease images, and use the flipping method to enhance the number of datasets; 22) Use the cutout data augmentation method, that is, fill a square area in the strawberry disease image with 0 pixel values to achieve random occlusion; then perform central normalization on the randomly occluded strawberry disease image to eliminate the influence of 0 value filling on training; 23) Perform fusion enhancement on the strawberry disease images, and its expression is as follows: λ = Beta(α,β) mix_batch x = λbatch x1 + (1 - λ)batch x2 , Among them, Beta represents the beta distribution, α and β are two calculation parameters, λ is the mixing coefficient calculated from the beta distribution of parameters α and β, and mix_batch x represents the strawberry disease samples after mixing, and batch x1 is a batch of strawberry disease samples, and batch x2 is another batch of strawberry disease samples; 24) After the input strawberry disease images are preprocessed, output the enhanced strawberry disease images.

3. The strawberry disease image recognition method based on the self-attention mechanism according to claim 1, characterized in that, The construction of the strawberry disease classification and recognition model includes the following steps: 31) It is assumed that the strawberry disease classification and recognition model includes a dimensionality reduction module, a sampling module, and a classification module. Input the enhanced strawberry disease image, and through block segmentation with a size of 4×4, output the strawberry segmentation feature map. First, pass the strawberry segmentation feature map through the dimensionality reduction module to output the dimensionality-reduced strawberry disease information, then pass it through three sampling modules to output the hierarchical strawberry disease feature map, and finally, pass the hierarchical strawberry disease feature map through the classification module for classification and recognition to output the strawberry disease classification and recognition information. 32) Set the dimensionality reduction module, which includes a convolutional layer and a self-attention mechanism module. It is assumed that the convolutional layer consists of a 1×1 convolution for dimensionality reduction of the strawberry disease feature map. The self-attention mechanism module is composed of a window module and a sliding window module in series. After the strawberry segmentation feature map enters the dimensionality reduction module, first obtain the strawberry dimensionality-reduced feature map through a 1×1 convolutional layer, and then input the strawberry dimensionality-reduced feature map into the self-attention mechanism module to output the strawberry disease information at one level. 33) Set the sampling module, and assume there are three sampling modules, and all three sampling modules are the same. The sampling module includes a block concatenation and a self-attention mechanism module. The block concatenation consists of the concatenation of strawberry disease information and a 1×1 convolution to achieve downsampling of the strawberry disease feature map. The concatenation of strawberry disease feature information divides each 2×2 adjacent pixel into a block, then combines the pixels at the same position in each block to obtain four strawberry concatenated feature maps, concatenates these four strawberry concatenated feature maps in the depth direction, and then outputs the strawberry block concatenated feature map through a 1×1 convolution. Input the strawberry block concatenated feature map into the self-attention mechanism module. After passing through the three sampling modules, the hierarchical strawberry disease feature map is output. 34) Set the classification module. The classification module consists of an LN normalization and a global average pooling. The LN normalization is used to globally normalize the feature data by translation and scaling for the extracted hierarchical strawberry disease feature map. The global average pooling averages the two-dimensional images of each channel in the hierarchical strawberry disease feature map and outputs a C×1 feature matrix. The feature matrix is multiplied by a weight matrix G of size t×C to obtain the probability of each class of strawberry disease, where the weight matrix G is obtained through training, and t is the number of strawberry disease categories. Finally, the strawberry disease classification information is output through the classification module.

4. The strawberry disease image recognition method based on the self-attention mechanism according to claim 1, wherein The training of the strawberry disease classification and recognition model includes the following steps: 41) Train the strawberry disease classification and recognition model: 411) Input the enhanced strawberry disease image into the strawberry disease classification and recognition model. First, divide the enhanced strawberry disease image into blocks through block segmentation. For the obtained strawberry segmentation feature map in blocks, first train through the dimensionality reduction module to obtain the strawberry disease information. 412) For the strawberry disease dimensionality-reduced information, train and sample through the three sampling modules to obtain the hierarchical strawberry disease feature map, and input the hierarchical strawberry disease feature map into the classification module for classification. 413) The hierarchical strawberry disease feature map is trained separately in the global average pooling of the classification module, and the weight matrix G is obtained through training for calculating the probability of each type of strawberry disease information. The predicted strawberry disease classification information is finally output through the classification training of the classification module; 414) The self-attention mechanism module is trained separately. Its input is the strawberry dimensionality-reduced feature map in the dimensionality reduction module or the strawberry patch concatenated feature map in the sampling module, and the coefficient matrix S is obtained through training for self-attention calculation; 42) Compare the predicted strawberry disease classification information with the correct strawberry disease classification information, and then use the gradient backpropagation algorithm to adjust the weights of the model; 43) Train the loss function; Define the loss function L of the strawberry disease classification recognition model as: Among them, y is the true strawberry disease classification information, is the predicted strawberry disease classification information.

5. The strawberry disease image recognition method based on the self-attention mechanism according to claim 1, characterized in that The setting of the window self-attention mechanism includes the following steps: 51) The setting is based on the window self-attention mechanism, which consists of window partitioning and self-attention calculation. First, the input is the strawberry dimensionality-reduced feature map or the strawberry patch stitching feature map. Window partitioning divides it into H×W patches, and every M×M patches form a window, resulting in a total of windows. Then, the self-attention value of each window is calculated through self-attention. 52) Set the self-attention calculation method as follows: First, input an image to be calculated with a size of where C is the image dimension. The input image to be calculated passes through a coefficient matrix S of size C×C to obtain three vectors Q, K, and V. Among them, the coefficient matrix S is obtained through training. Q is the query vector, K is the key vector, and V is the value vector. Then, calculate the self-attention matrix A as follows: Among them, is the standard deviation of T the matrix elements in QK Then the computational complexity Ω based on the window self-attention mechanism is: Ω = 4HW + 2M 2 HWC, Where, H represents the height of the feature map, W represents the width of the feature map, C represents the depth of the feature map, and M represents the size of each window.

Images