A crop leaf disease detection method based on a transformer
By employing a Transformer-structured encoder and decoder in crop leaf disease detection, the problems of slow detection speed and large parameter quantity in existing technologies are solved, achieving faster and more accurate disease detection, which is suitable for edge devices.
Patent Information
- Application Number
- CN202211385848.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-11-07
- Publication Date
- 2025-12-19
- Estimated Expiration
- 2042-11-07
AI Technical Summary
Existing methods for detecting crop leaf diseases rely on high-performance convolutional neural networks, which result in slow detection speeds, difficulty in deployment on edge devices, and limited coverage and inaccuracy of manual identification methods.
A Transformer-based method for detecting crop leaf diseases is adopted, which uses an encoder for feature extraction and a decoder for feature fusion. By constructing a leaf disease detection model with encoder and decoder structures, the number of parameters is reduced and the detection speed is improved.
It achieves faster disease detection speed with fewer parameters, improves detection accuracy and coverage, and is suitable for edge device deployment.
Smart Images
Figure CN115620146B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of crop disease detection, and more particularly to a crop leaf disease detection method based on a Transformer. BACKGROUND
[0002] During the growth of crops, various diseases are often infected due to weather, microorganisms, and growth environment factors. These diseases often occur in the stems and leaves of plants, and the disease characteristics of the leaf part are the most obvious and serious. Because the disease spot affects the photosynthesis of the crop leaf, it leads to slow growth or even leaf fall, thereby affecting the yield of the crop.
[0003] The most common crop disease detection method at present is that farmers judge whether the crop has a disease according to their years of planting experience, distinguish the disease type and the degree of the disease, and take corresponding prevention and control measures according to their judgment. In the case of a wide area of crop planting, manual methods are difficult to monitor large-scale crop diseases, and it is unrealistic to identify diseases by artificial means, and such identification method has a small coverage, and its accuracy cannot be guaranteed. Common crop detection methods based on deep learning usually rely on convolutional neural networks, which have high requirements for the computing performance of hardware and cannot well meet the deployment needs of edge devices.
[0004] The prior art discloses an apple tree trunk disease identification and classification method based on a convolutional neural network. The method uses a VGG model of deep learning to detect apple tree trunk diseases and saves manpower, but the model parameter quantity and the calculation amount are high, and therefore the detection speed is slow.
[0005] Therefore, in combination with the above needs and the technical defects of the prior art that the computing performance of hardware is required to be high and the detection speed is slow, the present application provides a crop leaf disease detection method based on a Transformer. SUMMARY
[0006] The present application provides a crop leaf disease detection method based on a Transformer, which can achieve faster disease detection speed with less parameter quantity.
[0007] The primary object of the present application is to solve the above technical problems, and the technical solution of the present application is as follows:
[0008] The present application provides a crop leaf disease detection method based on a Transformer, which can achieve faster disease detection speed with less parameter quantity.
[0009] S1, acquire n crop leaf images for model training, mark the disease area in the crop leaf image, and obtain a sample set of crop leaf images with completed marking.
[0010] S2, construct an initial leaf disease detection model, the initial leaf disease detection model comprising an encoder and a decoder, the encoder adopting a Transformer structure for high-level feature extraction, and the decoder adopting a channel attention mechanism to fuse the encoder Transformer output feature map.
[0011] S3, training the initial leaf disease detection model using the sample set of step S1, thereby obtaining a trained leaf disease detection model.
[0012] S4, using the trained leaf disease detection model to detect the crop leaf image to be detected, and obtaining the crop leaf disease detection result in the crop leaf image.
[0013] Further, the method for marking the crop leaf image in step S1 is specifically:
[0014] S11, acquire n crop leaf images for model training, wherein the crop leaf image has RGB three channels; the crop leaf image includes healthy leaf crop leaf image and disease leaf crop leaf image.
[0015] S12, mark the disease area and disease type of the disease area in each disease leaf crop leaf image, mark the healthy leaf in each healthy leaf crop leaf image, derive a mask image according to the marking data, each crop leaf image corresponds to a mask image; wherein the mask image and the crop leaf image have the same length and width, the mask image is a single-channel gray image, and each pixel point in the mask image corresponds to a label of disease type or healthy leaf.
[0016] Further, the encoder of the leaf disease detection model in step S2 comprises: an input layer, a convolution layer and a Transformer layer, specifically:
[0017] The input layer inputs the crop leaf image and transmits the crop leaf image to the first convolution layer; the size of the crop leaf image is HxWx3, wherein H and W are the length and width of the crop leaf image respectively, and 3 represents that the crop leaf image has RGB three channels.
[0018] The crop leaf image passes through the first convolution layer to obtain a first encoding feature map, and the size of the first encoding feature map is HxWx24.
[0019] The first encoding feature map is first input into a pooling layer for pooling processing, and then a second encoding feature map is obtained after being processed by a second convolutional layer, the size of the second encoding feature map being
[0020] The crop image is subjected to four times down-sampling processing by a pooling layer, and then a third encoding feature map is obtained by inputting the crop image into a first Transformer layer, the size of the third encoding feature map being
[0021] The third encoding feature map is input into a second Transformer layer to obtain a fourth encoding feature map, the size of the fourth encoding feature map being
[0022] The fourth encoding feature map is input into a third Transformer layer to obtain a fifth encoding feature map, the size of the fifth encoding feature map being
[0023] The fifth encoding feature map is input into a fourth Transformer layer to obtain a sixth encoding feature map, the size of the sixth encoding feature map being
[0024] Further, the specific structure of the first Transformer layer, the second Transformer layer, the third Transformer layer and the fourth Transformer layer is as follows:
[0025] The first Transformer layer comprises a pooling layer, a block division operation module, a linear encoding operation module and a Transformer Block; and the crop image is sequentially subjected to the pooling layer, the block division operation module, the linear encoding operation module and the Transformer Block to obtain the third encoding feature map.
[0026] The block division is to divide the picture into several parts.
[0027] The second Transformer layer, the third Transformer layer and the fourth Transformer layer are sequentially connected by a down-sampling operation module and a Transformer Block.
[0028] The down-sampling operation module in the Transformer layer is to divide the encoding feature map into four regions, then splice them in the depth direction to become [0, 1, 2, 3], and then linearly transform them to become [4, 5].
[0029] The down-sampling process can reduce the parameter quantity of the model.
[0030] Further, the Transformer Block specifically comprises: an LN layer normalization module, a window multi-head attention module, an MLP multi-layer perceptron, a sliding window multi-head attention module, and a skip connection module.
[0031] The LN layer normalization module, the window multi-head attention module, and the skip connection module are sequentially connected; next, the LN layer normalization module, the MLP multi-layer perceptron, and the skip connection module are sequentially connected; next, the LN layer normalization module, the sliding window multi-head attention module, and the skip connection module are sequentially connected; and finally, the LN layer normalization module, the MLP multi-layer perceptron, and the skip connection module are sequentially connected.
[0032] Further, the convolution layer comprises two convolution kernels, wherein the convolution kernel size is 3x3, the convolution kernel step is 1, the convolution kernel padding is 1, and the convolution kernel Mish activation function is Mish=x*tanh(ln(1+e x ))).
[0033] Further, the decoder of the leaf disease detection model in step S2 comprises a convolution kernel, a channel attention module, and a spatial attention module, and specifically comprises:
[0034] The first layer of the sixth encoding feature map input decoder is spliced and fused with the fifth encoding feature map after 1x1 convolution kernel, and then passes through a 3x3 convolution kernel and a channel attention module to output a first decoding feature map with a size of .
[0035] The second layer of the first decoding feature map input decoder is spliced and fused with the fourth encoding feature map after 1x1 convolution kernel, and then passes through a 3x3 convolution kernel and a channel attention module to output a second decoding feature map with a size of .
[0036] The third layer of the second decoding feature map input decoder is spliced and fused with the third encoding feature map after 1x1 convolution kernel, and then passes through a 3x3 convolution kernel and a channel attention module to output a third decoding feature map with a size of .
[0037] The fourth layer of the third decoding feature map input decoder is spliced and fused with the second encoding feature map after 1x1 convolution kernel, and then passes through a 3x3 convolution kernel and a spatial attention module to output a fourth decoding feature map with a size of .
[0038] The fourth decoding feature map is input into the fifth layer of the decoder, is spliced and fused with the first encoding feature map after 1*1 convolution kernel, and then is output as a fifth decoding feature map with a size of H*W*24 after 3*3 convolution kernel and a spatial attention module.
[0039] The fifth decoding feature map is input into the sixth layer of the decoder, and is output as a sixth decoding feature map with a size of H*W*N after 1*1 convolution kernel, where N is the total number of pest species and backgrounds.
[0040] Further, the process of training the initial disease detection model by using the sample set in step S3 is specifically: the sample set includes a training set and a test set, the training set is input into the convolutional neural network to obtain a sixth decoding feature map, the difference between the sixth decoding feature map and each sub-pixel point in the corresponding labeled mask image is calculated by a Dice loss function, and the calculated difference is updated to the convolution kernel in the leaf disease detection model through back propagation; the training accuracy is calculated by using the test set after each training, and the training is completed when the training accuracy reaches the training target, and the weight parameter is saved to obtain the trained leaf disease detection model.
[0041] Further, the calculation method of the training accuracy is: inputting the pictures in the test set into the leaf disease detection model to obtain the sixth decoding feature map corresponding to each picture in the test set The intersection over union IoU between the sixth decoding feature map and the labeled mask image V k k and the loss value The average intersection over union MIoU and the average loss value of all pictures in the test set are calculated, and the training target is reached when the MIoU is greater than 0.9 or the change amplitude of the average loss value is less than 3%, and the training is ended and the model weight parameter Weight is saved.
[0042] The calculation formula of the average intersection over union is: The calculation formula of the average loss value is: i wherein, Loss i represents the loss value of the ith test set picture.
[0043] Further, the process of detecting the crop leaf image to be detected by using the trained disease detection model in step S4 is specifically.
[0044] S41, converting the crop leaf image im to be detected into a tensor between 0 and 1 to obtain an input matrix P.
[0045] S42, input the input matrix P into the leaf disease detection model, and obtain an output matrix Q through the leaf disease detection model; the output matrix Q is calculated in the following manner wherein is a convolution symbol.
[0046] S43, convert the matrix Q into a picture through the PLT library, and the obtained picture is a predicted disease area and disease type distribution map.
[0047] Compared with the prior art, the beneficial effects of the technical scheme of the present application are:
[0048] The present application provides a crop leaf disease detection method based on a Transformer, which detects crop leaf disease conditions by using a leaf disease detection model. The model adopts an encoder and a decoder structure. The encoder of the model mainly uses a Transformer to complete feature extraction. Compared with a traditional convolutional neural network, the Transformer is faster and has fewer parameters. In the decoder part, the channel attention mechanism is used for feature fusion in the Transformer output feature map, which can well reduce the model parameter quantity without causing performance degradation. BRIEF DESCRIPTION OF DRAWINGS
[0049] Figure 1 is a step flowchart of the crop leaf disease detection method based on a Transformer of the present application.
[0050] Figure 2 is a leaf disease detection model structure diagram of the present application.
[0051] Figure 3 is a Transformer Block network structure diagram.
[0052] Figure 4 is a down-sampling flowchart in the present application. DETAILED DESCRIPTION
[0053] In order to more clearly understand the above-mentioned purposes, features and advantages of the present application, the present application will be further described in detail below in combination with the drawings and specific embodiments. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.
[0054] In the following description, many specific details are set forth in order to provide a thorough understanding of the present application, however, the present application can also be implemented in other ways different from those described herein, therefore, the protection scope of the present application is not limited by the specific embodiments disclosed below.
[0055] Embodiment 1
[0056] AsFigure 1 As shown, the present application provides a crop leaf disease detection method based on Transformer, which comprises the following steps:
[0057] S1, obtain n crop leaf images for model training, mark the disease area in the crop leaf image, and obtain a sample set of the labeled crop leaf image.
[0058] S2, construct an initial leaf disease detection model, the initial leaf disease detection model comprises an encoder and a decoder, the encoder adopts a Transformer structure for high-level feature extraction, and the decoder adopts a channel attention mechanism to fuse the feature map output by the encoder Transformer.
[0059] S3, train the initial leaf disease detection model using the sample set of step S1, thereby obtaining a trained leaf disease detection model.
[0060] S4, using the trained leaf disease detection model to detect the crop leaf image to be detected, and obtaining the crop leaf disease detection result in the crop leaf image.
[0061] The embodiment proposes a crop leaf disease detection method based on Transformer, which uses a leaf disease detection model to detect crop leaf disease conditions. The model adopts an encoder and a decoder structure, and the encoder of the model mainly uses Transformer to complete feature extraction. Compared with traditional convolutional neural networks, it is faster and has fewer parameters. In the decoder part, the channel attention mechanism is used for feature fusion of the Transformer output feature map, which can well reduce the model parameter quantity without performance degradation.
[0062] Embodiment 2
[0063] Based on the above embodiment 1, combined with Figures 2-3 The embodiment details the specific content of the crop leaf disease detection method based on Transformer.
[0064] Further, the method for obtaining the labeled crop leaf image in step S1 is specifically:
[0065] S11, obtain n crop leaf images for model training, wherein the crop leaf image has three channels of RGB; the crop leaf image includes healthy leaf crop leaf image and diseased leaf crop leaf image.
[0066] S12, marking the disease area and the disease type of the disease area in each disease leaf crop leaf image, marking the healthy leaf in each healthy leaf crop leaf image, deriving a mask image according to the marking data, each crop leaf image corresponding to a mask image; wherein the mask image and the crop leaf image have the same length and width, the mask image is a single-channel gray image, and each pixel point in the mask image corresponds to a label of a disease type or a healthy leaf.
[0067] Further, the encoder of the leaf disease detection model in step S2 is as shown in the figure, and specifically includes an input layer, a convolutional layer and a Transformer layer, and specifically comprises: Figure 2
[0068] The input layer inputs the crop leaf image and transmits the crop leaf image to the first convolutional layer; the size of the crop leaf image is HxWx3, wherein H and W are the length and width of the crop leaf image respectively, and 3 represents that the crop leaf image has RGB three channels.
[0069] The crop leaf image is processed by the first convolutional layer to obtain a first encoding feature map, and the size of the first encoding feature map is HxWx24.
[0070] The first encoding feature map is first input into the pooling layer for pooling processing, and then is processed by the second convolutional layer to obtain a second encoding feature map, and the size of the second encoding feature map is
[0071] The crop leaf image is processed by the pooling layer for four times of downsampling, and then is input into the first Transformer layer to obtain a third encoding feature map, and the size of the third encoding feature map is
[0072] The third encoding feature map is input into the second Transformer layer to obtain a fourth encoding feature map, and the size of the fourth encoding feature map is
[0073] The fourth encoding feature map is input into the third Transformer layer to obtain a fifth encoding feature map, and the size of the fifth encoding feature map is
[0074] The fifth encoding feature map is input into the fourth Transformer layer to obtain a sixth encoding feature map, and the size of the sixth encoding feature map is
[0075] Further, the specific structure of the first Transformer layer, the second Transformer layer, the third Transformer layer and the fourth Transformer layer is as follows:
[0076] The first Transformer layer comprises a pooling layer, a block division operation module, a linear coding operation module and a Transformer Block; the crop leaf image sequentially passes through the pooling layer, the block division operation module, the linear coding operation module and the Transformer Block to obtain a third coding feature map.
[0077] The block division is to divide the picture into several parts.
[0078] The second Transformer layer, the third Transformer layer and the fourth Transformer layer are sequentially connected with a down-sampling operation module and a Transformer Block.
[0079] The down-sampling operation module in the Transformer layer is to divide the coding feature map into four regions, then splice in the depth direction to become [0, 1, 2, 3], and then linearly transform to become [4, 5].
[0080] The down-sampling process can reduce the parameter quantity of the model.
[0081] Further, the structure of the Transformer Block is as shown in Figure 3 , and specifically comprises an LN layer normalization module, a window multi-head attention module, an MLP multi-layer perceptron, a sliding window multi-head attention module and a skip connection module.
[0082] The LN layer normalization module, the window multi-head attention module and the skip connection module are sequentially connected; then the LN layer normalization module, the MLP multi-layer perceptron and the skip connection module are sequentially connected; then the LN layer normalization module, the sliding window multi-head attention module and the skip connection module are sequentially connected; finally, the LN layer normalization module, the MLP multi-layer perceptron and the skip connection module are sequentially connected.
[0083] Further, the convolution layer comprises two convolution kernels, wherein the convolution kernel size is 3x3, the convolution kernel step is 1, the convolution kernel padding is 1, and the convolution kernel Mish activation function is Mish=x*tanh(ln(1+e x )).
[0084] Further, the decoder of the leaf disease detection model in step S2 comprises a convolution kernel, a channel attention module and a spatial attention module, and specifically comprises:
[0085] The sixth encoded feature map is input into the first layer of the decoder, passed through a 1×1 convolutional kernel, and then concatenated with the fifth encoded feature map. After passing through a 3×3 convolutional kernel and a channel attention module, the output size is [size missing]. The first decoded feature map.
[0086] The first decoded feature map is input into the second layer of the decoder, passed through a 1×1 convolutional kernel, and then concatenated with the fourth encoded feature map. After passing through a 3×3 convolutional kernel and a channel attention module, the output size is... The second decoding feature map.
[0087] The second decoded feature map is input into the third layer of the decoder, passed through a 1×1 convolutional kernel, and then concatenated and fused with the third encoded feature map. After passing through a 3×3 convolutional kernel and a channel attention module, the output size is... The third decoding feature map.
[0088] The third decoded feature map is input into the fourth layer of the decoder, passed through a 1×1 convolutional kernel, and then concatenated with the second encoded feature map. After passing through a 3×3 convolutional kernel and a spatial attention module, the output size is... The fourth decoding feature map.
[0089] The fourth decoded feature map is input into the fifth layer of the decoder. After passing through a 1×1 convolutional kernel, it is concatenated and fused with the first encoded feature map. Then, it passes through a 3×3 convolutional kernel and a spatial attention module to output a fifth decoded feature map of size H×W×24.
[0090] The fifth decoded feature map is input into the sixth layer of the decoder. After passing through a 1×1 convolution kernel, it outputs a sixth decoded feature map of size H×W×N, where N is the sum of the types of pests and diseases and the background.
[0091] Furthermore, the process of training the initial disease detection model using the sample set in step S3 is as follows: the sample set includes a training set and a test set. The training set is input into the convolutional neural network to obtain the sixth decoded feature map. The difference between the sixth decoded feature map and each sub-pixel in the corresponding labeled mask image is calculated using the Dice loss function. The calculated difference is then used to update the convolutional kernel in the leaf disease detection model through backpropagation. After each training iteration, the training accuracy is calculated using the test set. When the training accuracy reaches the training target, the training is complete. The weight parameters are saved to obtain the trained leaf disease detection model.
[0092] Furthermore, the method for calculating the training accuracy is as follows: input the images in the test set into the leaf disease detection model to obtain the sixth decoded feature map corresponding to each image in the test set. Calculate the sixth decoding feature map With the mask image V of the markerk Intersection over Union (IoU) k and loss value The average intersection-over-union (MIoU) and average loss value of all images in the test set are calculated. The training objective is achieved when MIoU > 0.9 or the average loss value changes by less than 3%. The training ends and the model weight parameter is saved.
[0093] The formula for calculating the average intersection-union ratio is: Where Vi represents the intersection-union ratio of the i-th test set image; the formula for calculating the average loss value is: Among them, Loss i Let represent the loss value of the i-th test set image.
[0094] Furthermore, the process of using the trained disease detection model to detect the crop leaf images in step S4 is as follows:
[0095] S41. Convert the crop leaf image im to be detected into a tensor between 0 and 1 to obtain the input matrix P.
[0096] S42. Input the input matrix P into the leaf disease detection model, and obtain the output matrix Q through the leaf disease detection model; the output matrix Q is calculated as follows: in This is the convolution symbol.
[0097] S43. Convert matrix Q into an image using the PLT library. The resulting image is the predicted distribution map of disease areas and disease types.
[0098] Example 3
[0099] Based on the above embodiments 1 and 2, combined with Figure 4 The table and other materials used in this embodiment, with grape diseases as actual samples, further illustrate the technical solution of the present invention.
[0100] like Figure 4 As shown, the downsampling process used in this invention is as follows:
[0101] In a specific embodiment, the process of using grape diseases as actual samples for disease detection is as follows:
[0102] S1. Collect sample images, identify learning samples, and classify and save them.
[0103] To obtain the model, a large number of pest and disease samples need to be input into the network to train the neural network. The actual grape leaf disease samples used for training include: ring spot disease, black rot disease, brown spot disease, and healthy leaves as negative samples.
[0104] First, the disease pictures are polygonally labeled using Labelme, and the required masks for training are exported and converted to grayscale images ranging from 0 to 4. Second, the total sample is R i and the corresponding mask picture M i , i is the sample number. Where R i is the picture size HxWx3, H and W are the length and width of the picture respectively, and 3 is the three channels of the RGB picture. M i is the picture size HxWx1. R i and M i Each channel value is between 0 and 255, where M i Each pixel value in M i represents a leaf spot, black rot, brown spot, and healthy leaf, respectively, using 1-4, and 0 represents the background.
[0105] S2, divide the data set.
[0106] After classification and labeling, the total number of grape disease and pest sample pictures is 3143, which are divided into four categories. First, the samples in R i are randomly divided into training set T k and test set V k according to a certain proportion, and the corresponding mask pictures are mT j and mV k , where j=0.9x i; k=i-j; k, i∈Z.
[0107] S3, train the model.
[0108] Use the Transformer network to learn the sample picture autonomously, and get the best effect through continuous learning. The pictures in T j are input into the network one by one. The convolutional neural network structure contains two parts, encoder and decoder.
[0109] The encoder part uses the Transformer structure for high-level feature extraction, and the convolutional neural network for low-level feature extraction.
[0110] The Transformer Block structure is shown in Figure 1 , which includes window multi-head attention (W-MSA), layer normalization (LN), perception (MLP), and sliding window multi-head attention (SW-MSA). The steps can be divided into four steps:
[0111] 1. The feature map passes through the LN layer, then through W-MSA, and then performs a jump connection.
[0112] 2. The connected feature map passes through the LN layer again, then through the fully connected layer MLP, and then performs a jump connection.
[0113] 3. The feature map goes through LN layer, SW-MSA, and then skip connection.
[0114] 4. The connected feature map goes through LN layer again, fully connected layer MLP, and then skip connection.
[0115] The convolution layer is composed of two convolution kernels with a size of 3x3, a stride of 1, and padding of 1, and a Mish activation function, whose function is Mish = x*tanh(ln(1+e x )). The down-sampling uses a max-pooling layer with a size of 2x2 and a stride of 2.
[0116] Through the stacking of convolution layers and Transformer structures, an encoder is obtained, whose structure is shown in the left part of Figure 2
[0117] The first layer is the input layer, with a size of HxWx3.
[0118] The second layer is the convolution layer, and the output size is HxWx24 after two convolution operations. Its output is denoted as X1.
[0119] The third layer is the convolution layer, and the output size is after the feature map after the pooling layer is operated by two convolution operations. Its output is denoted as X2.
[0120] The fourth layer is the Transformer layer, which includes block division operation, linear encoding operation, and TransformerBlock. After the input layer data is down-sampled by 4, it is sequentially operated by block division, linear encoding, and Transformer Block, and the output size is Its output is denoted as X3.
[0121] The fifth layer is the Transformer layer, which includes down-sampling operation and Transformer Block. The input feature map is input to the Transformer Block after down-sampling. The output size is Its output is denoted as X4.
[0122] The sixth layer is the Transformer layer, which includes down-sampling operation and Transformer Block. The input feature map is input to the Transformer Block after down-sampling. The output size is Its output is denoted as X5.
[0123] The seventh fifth layer is a Transformer layer, comprising a downsampling operation and a Transformer Block. The input feature map is input to the Transformer Block after the downsampling operation. The output size is The output thereof is denoted as X6.
[0124] The decoder is composed of a 3x3 convolution and attention. The upsampling operation uses a linear interpolation method. The specific structure thereof is as follows from bottom to top: Figure 2 as shown in the right half
[0125] The first layer input is X6, which is spliced with X5 after 1x1 convolution. The fused vector is subjected to 3x3 convolution and channel attention module, and the output size is denoted as U1.
[0126] The second layer input is U1, which is spliced with X4 after 1x1 convolution. The fused vector is subjected to 3x3 convolution and channel attention module, and the output size is denoted as U2.
[0127] The third layer input is U2, which is spliced with X3 after 1x1 convolution. The fused vector is subjected to 3x3 convolution and channel attention module, and the output size is denoted as U3.
[0128] The fourth layer input is U3, which is spliced with X2 after 1x1 convolution. The fused vector is subjected to 3x3 convolution and spatial attention module, and the output size is denoted as U4.
[0129] The fifth layer input is U4, which is spliced with X1 after 1x1 convolution. The fused vector is subjected to 3x3 convolution and spatial attention module, and the output size is HxWx24, denoted as U5.
[0130] The sixth layer is an output layer, which mainly functions to output the pixel position of each category. U5 is output as M after 1x1 convolution, with a size of HxWxN, wherein N is the total number of disease and pest categories and background, which is 5 in this paper.
[0131] S4, calculate the loss function.
[0132] M i is compressed to 0-1 by normalization compression, and the resulting matrix is A. The average value of each channel number of the output layer M is compressed to the size of HxWx1, and the output matrix is B. The Dice loss function can be expressed as After that, the network parameters are updated through back propagation, and step 3 is continued until the picture in T j is input into the network.
[0133] S5, calculate accuracy.
[0134] Input the picture in V k into the network to obtain the output corresponding to each picture Calculate the intersection over union IoU k between V and k and the loss value Calculate the average intersection over union k of all verification sets V and the average loss value When MIoU> 0.9 or the change amplitude of the loss value is less than 3%, end the training and save the model weight parameter W. Otherwise, return to step 3.
[0135] S6, predict unknown diseases
[0136] Read the prediction picture im and convert it into a tensor between 0 and 1 to obtain the input matrix P. The output matrix Q is calculated as Where is the convolution symbol. Finally, convert the matrix Q into a picture through the PLT library to obtain the predicted disease area and disease type distribution map.
[0137] Table 1 shows the comparison of the identification performance of different models for leaf disease. It can be seen that the SW-Unet model using the method has smaller parameter quantity, and the inference speed and identification effect are better than those of other models.
[0138] Table 1 Comparison of identification performance of different methods for grape leaf disease
[0139]
[0140] The icons in the drawings that describe the positional relationship of the structure are only used for illustrative description, and cannot be understood as a limitation on the patent.
[0141] Obviously, the above embodiments of the present application are only examples for clearly illustrating the present application, and are not a limitation on the embodiments of the present application. For those skilled in the art, other different forms of changes or variations can be made on the basis of the above description. Here, all the embodiments are not required to be exhausted. Any modification, equivalent replacement and improvement made within the spirit and principles of the present application should be included in the protection scope of the claims of the present application.
Claims
1. A method for detecting crop leaf disease based on Transformer, characterized in that, The method comprises the following steps: S1, obtaining n crop leaf images for model training, marking the disease area in the crop leaf image, and obtaining a sample set of the marked crop leaf image; S2, constructing an initial leaf disease detection model, the initial leaf disease detection model comprising an encoder and a decoder, the encoder adopting a Transformer structure for high-level feature extraction, and the decoder adopting a channel attention mechanism to fuse the encoder Transformer output feature map; the encoder of the leaf disease detection model comprises an input layer, a convolution layer and a Transformer layer, specifically: The input layer inputs the crop leaf image and transmits the crop leaf image to the first convolution layer; the size of the crop leaf image is HxWx3, wherein H and W are the length and width of the crop leaf image, respectively, and 3 represents that the crop leaf image has RGB three channels; The crop leaf image passes through the first convolution layer to obtain a first encoding feature map, and the size of the first encoding feature map is HxWx24; The first encoded feature map is first input into a pooling layer for pooling processing, and then is processed by a second convolutional layer to obtain a second encoded feature map, the size of the second encoded feature map being The crop leaf image is subjected to four times down-sampling processing through a pooling layer, and then input into a first Transformer layer to obtain a third encoding feature map, and the third encoding feature map has a size of The third encoded feature map is input into a second Transformer layer to obtain a fourth encoded feature map, and the size of the fourth encoded feature map is The fourth encoded feature map is input into a third Transformer layer to obtain a fifth encoded feature map, and the size of the fifth encoded feature map is The fifth encoded feature map is input into a fourth Transformer layer to obtain a sixth encoded feature map, and the size of the sixth encoded feature map is The specific structure of the first Transformer layer, the second Transformer layer, the third Transformer layer and the fourth Transformer layer is as follows: The first Transformer layer comprises a pooling layer, a block division operation module, a linear coding operation module and a Transformer Block; the crop leaf image sequentially passes through the pooling layer, the block division operation module, the linear coding operation module and the Transformer Block to obtain a third encoding feature map; The second Transformer layer, the third Transformer layer and the fourth Transformer layer are sequentially connected by a down-sampling operation module and a Transformer Block; The down-sampling operation modules in the Transformer layers are all to divide the encoding feature map into Four regions, then spliced in the depth direction to become [0, 1, 2, 3], and then linearly transformed to become [4, 5]; the Transformer Block specifically comprises an LN layer normalization module, a window multi-head attention module, an MLP multi-layer perceptron, a sliding window multi-head attention module and a skip connection module. Wherein, the LN layer normalization module, the window multi-head attention module and the skip connection module are sequentially connected; next, the LN layer normalization module, the MLP multi-layer perceptron and the skip connection module are sequentially connected; next, the LN layer normalization module, the sliding window multi-head attention module and the skip connection module are sequentially connected; finally, the LN layer normalization module, the MLP multi-layer perceptron and the skip connection module are sequentially connected; The decoder of the leaf disease detection model comprises a convolution kernel, a channel attention module and a spatial attention module, specifically comprising: The sixth encoding feature map is input into the first layer of the decoder, and after 1*1 convolution kernel, it is spliced and fused with the fifth encoding feature map, and then passes through 3*3 convolution kernel and channel attention module, and outputs the first decoding feature map with a size of The first decoding feature map is input into a second layer of the decoder, is spliced and fused with the fourth encoding feature map after a 1*1 convolution kernel, and then is output as a second decoding feature map with a size of The second decoding feature map is input into a third layer of the decoder, is spliced and fused with the third encoding feature map after 1*1 convolution kernel, and then is output as a third decoding feature map with a size of The third decoding feature map is input into a fourth layer of the decoder, is spliced and fused with the second encoding feature map after 1*1 convolution kernel, and then is output with a size of the fourth decoding feature map after 3*3 convolution kernel and a spatial attention module. The fourth decoding feature map is input into the fifth layer of the decoder, and after passing through the 1x1 convolution kernel, it is spliced and fused with the first encoding feature map, and then passes through the 3x3 convolution kernel and the spatial attention module to output the fifth decoding feature map with a size of HxWx24; The fifth decoding feature map is input into the sixth layer of the decoder, and after passing through the 1x1 convolution kernel, the sixth decoding feature map with a size of HxWxN is output, wherein N is the total number of disease and pest types and background; S3, training the initial leaf disease detection model using the sample set of step S1 to obtain a trained leaf disease detection model; S4, using the trained leaf disease detection model to detect the crop leaf image to be detected, and obtaining a crop leaf disease detection result in the crop leaf image.
2. The method of claim 1, wherein the method is based on a Transformer. The method for obtaining the marked crop leaf image in step S1 is specifically: S11, obtaining n crop leaf images for model training, wherein the crop leaf images have three RGB channels; the crop leaf images include healthy leaf crop leaf images and diseased leaf crop leaf images; S12, marking a disease area and a disease type of the disease area in each diseased leaf crop leaf image, marking a healthy leaf in each healthy leaf crop leaf image, deriving a mask image according to the marking data, and each crop leaf image corresponds to a mask image; The mask image and the crop leaf image have the same length and width, the mask image is a single-channel grayscale image, and each pixel point in the mask image corresponds to a label of a disease type or a healthy leaf.
3. The method of claim 1, wherein the method is based on a Transformer. The convolution layer includes two convolution kernels, wherein the convolution kernel size is 3x3, the convolution kernel step is 1, the convolution kernel padding is 1, the convolution kernel Mish activation function is Mish=x*tanh(ln(1+e x )).
4. The method of claim 1, wherein the method is based on a Transformer. The process of training the initial disease detection model using the sample set in step S3 is specifically: the sample set includes a training set and a test set, the training set is input into a convolutional neural network to obtain a sixth decoding feature map, the difference between the sixth decoding feature map and each sub-pixel point in the corresponding labeled mask image is calculated through a Dice loss function, and the calculated difference is updated through back propagation to update the convolution kernel in the leaf disease detection model; the training accuracy is calculated using the test set after each training, and the training is completed when the training accuracy reaches the training target, and the weight parameters are saved to obtain the trained leaf disease detection model.
5. The method of claim 4, wherein the method is based on a Transformer. The method for calculating the training accuracy is as follows: input the images in the test set into the leaf disease detection model to obtain the sixth decoded feature map corresponding to each image in the test set. Calculate the sixth decoding feature map With the mask image V of the marker k Intersection over Union (IoU) k and loss value The average intersection-union ratio (MIoU) and average loss value of all images in the test set are calculated. The training objective is achieved when MIoU > 0.9 or the average loss value changes by less than 3%. The training ends and the model weight parameter is saved. The calculation formula of the average intersection over union is: Wherein, V i represents the intersection over union of the ith test set picture; the calculation formula of the average loss value is: Wherein, Loss i represents the loss value of the ith test set picture.
6. The method of claim 5, wherein the method is a method of detecting crop leaf disease based on a Transformer. The process of detecting the crop leaf image to be detected using the trained disease detection model in step S4 is specifically: S41, converting the crop leaf image to be detected into a tensor between 0 and 1 to obtain an input matrix P; S42, inputting the input matrix P into the leaf disease detection model to obtain an output matrix Q through the leaf disease detection model; The output matrix Q is calculated as where is the convolution symbol; S43, converting the matrix Q into a picture through the PLT library to obtain a predicted disease area and disease type distribution map.