A method for constructing a lightweight attention mechanism network for crop disease identification

Abstract

The application relates to a method for constructing a lightweight attention mechanism network for crop disease identification, and relates to the field of deep learning. The method is based on a MobileViT model, channel attention mechanisms are added to part of MobileViT blocks of the MobileViT model, and channel attention mechanisms and spatial attention mechanisms are added to the last MobileViT block. A lightweight Transformer model, namely a MobileViT network model, is built. The model can effectively learn local representation and global representation, and can better capture crop disease information. Improved attention mechanisms are added to the model, all data of a PlantVillage public dataset are used for training and testing, a total of 38 categories are contained, and 99.60% of an identification accuracy is obtained through evaluation and verification on the PlantVillage public dataset, so that crop diseases can be effectively identified.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of deep learning, in particular to a method for constructing a lightweight attention mechanism network for crop disease identification, and belongs to the application of deep learning model in the field of crop disease identification. BACKGROUND

[0002] Agriculture is an important industry, and the growth and yield of crops have a significant impact on people's living standards. Ensuring the growth and yield of crops is of great significance. Crop diseases affect the growth of crops, reduce crop yields, and affect quality. Because agriculture has an important position, crop diseases have a serious harm to crops, so it is particularly important to quickly and accurately identify crop diseases.

[0003] To solve the problem of crop diseases, a key aspect is to quickly and accurately identify crop diseases, and then take appropriate measures. Accurate and efficient judgment of crop diseases is a major challenge. In recent years, deep learning technology has developed rapidly and is widely used in various fields. In the field of image recognition, convolutional neural networks have achieved very good results and can effectively extract image features for classification. Researchers have proposed various convolutional neural networks, such as VGGNet, GoogLeNet, and ResNet. VGGNet is a relatively deep model with a relatively simple network structure and has achieved good results. GoogLeNet uses Inception modules to build, which use a multi-branch structure to extract different information and improve the network's representation ability. In ResNet, the residual connection technology is proposed, which can construct a relatively deep convolutional neural network and achieve good results. In the field of image recognition, convolutional neural networks have dominated in the past, and in recent years, Transformer models have also been applied to computer vision. Transformer models have achieved outstanding results in natural language processing (NLP), and the main core of Transformer models is self-attention mechanism, which is different from convolutional neural networks and recurrent neural networks. Vision Transformer (ViT) models have been applied to computer vision, and Vision Transformer (ViT) models use Transformer structure to process visual tasks and achieve good results. ViT model divides the image into non-overlapping patches, then linearly maps these patches, and then inputs them into the Transformer model for calculation. The patch of the image is similar to the token in the NLP task.

[0004] With the rapid development of deep learning, the application field of deep learning technology is also more and more extensive, and the deep learning technology is gradually used in crop disease identification. Many deep learning models have outstanding effects and can accurately identify crop diseases, but they often have large parameter quantities and high requirements for computing and storage resources, so it is difficult to deploy and use on some mobile terminals and embedded devices. In addition, the Transformer model also has good effect in the visual field, which can learn global representation, but its parameter quantity is also large, and it is also difficult to use on mobile terminals and embedded devices. Therefore, it is very meaningful to design a lightweight model that can effectively identify crop diseases. SUMMARY

[0005] The purpose of the present application is to better identify crop diseases, while considering the parameter quantity and complexity of the model, and to be able to be deployed and used on some mobile terminals and embedded devices, a method for constructing a lightweight attention mechanism network for crop disease identification is provided, which is based on a lightweight network model and adds attention mechanism.

[0006] The method for constructing a lightweight attention mechanism network for crop disease identification is based on a MobileViT model, and a channel attention mechanism is added in part of the MobileViT block of the MobileViT model, and a channel attention mechanism and a spatial attention mechanism are added at the end of the MobileViT model.

[0007] The channel attention mechanism is based on the CBAM attention mechanism and further includes a one-dimensional convolution; the channel attention mechanism is used to analyze the relationship between image channels, gives each channel a weight, and obtains key information to improve the performance of the network;

[0008] The spatial attention mechanism is based on the CBAM attention mechanism and further includes a multi-branch network structure and a dilated convolution; the multi-branch network structure is constructed by different sizes of convolution kernels, and the dilated convolution layer is used to increase the receptive field; the spatial attention mechanism is used to analyze the relationship between image spaces, and gives each pixel point a weight, so as to obtain important information in the spatial dimension.

[0009] The MobileViT model can effectively learn local representation and global representation, and the improved attention mechanism added in the model can better capture crop disease information. The all data of the PlantVillage public data set is used for training and testing, and the training method is as follows:

[0010] The model is trained using the PlantVillage public dataset, which is a public crop disease dataset containing 38 categories; the PlantVillage dataset is randomly divided into a training set, a validation set, and a test set according to a ratio of 6:2:2; the training set is used for model training, the validation set is used to check the model state during training, and the test set is used for final testing of the model effect;

[0011] During model training, each sample in the training set is composed of an input image and the real class label corresponding to the image; the sample data in the training set is input into the model to obtain the prediction output of the model, and the output of the model is a vector; assuming there are C categories, a vector with a total of C elements will be output, and each position represents the probability of this category; the real label of the input image is also a vector containing C elements, and only one element of the vector is 1 and the others are 0, and the position of 1 represents the real class label of the image; the output result of the model (i.e. the predicted label) is compared with the real label of the input image, and the loss is calculated by the cross-entropy loss function, and the calculation formula of the cross-entropy loss function is where C is the number of categories, p c is a variable with a value of 0 or 1 (if c is the real category, then p c = 1, otherwise p c = 0), and q c is the probability of the model predicting the category c; after calculating the loss, the gradient of the model parameters is calculated by back propagation, and the network model parameters are updated using the AdamW optimizer; in the actual training process, the mini-batch method is used, that is, a batch of sample data is input into the model each time, and the specific training process is as follows:

[0012] (1) input a batch of image sample data into the model.

[0013] (2) calculate the predicted category of the batch of data by the model.

[0014] (3) compare the predicted category obtained by the model output with the real category, and calculate the loss by the cross-entropy loss function.

[0015] (4) perform back propagation operation to calculate the gradient of the model parameters, and update the network model parameters using the AdamW optimizer.

[0016] (5) repeat the above steps until the training is completed.

[0017] The details and key points of the network model of the present application are described below. The core part of the network model in the present application includes an attention mechanism and a MobileViT block, wherein the attention mechanism comprises a channel attention mechanism and a spatial attention mechanism.

[0018] 1) constructing a channel attention mechanism for analyzing the relationship between image channels, giving each channel a weight to obtain key information and thus improving the performance of the network;

[0019] 2) constructing a spatial attention mechanism, giving each pixel a weight, compressing the channel information through global average pooling and maximum pooling for the input image, using convolution kernels of different sizes to construct a multi-branch network to better fuse information and improve the representation ability of the network to capture crop disease information, using a dilated convolution to increase the receptive field, processing the compressed image through multiple convolution kernels, adding and fusing the results, and then calculating the attention score through a sigmoid function;

[0020] 3) constructing a lightweight attention mechanism network for crop disease recognition, using a MobileViT model as the basic model, adding a channel attention mechanism in some MobileViT blocks of the MobileViT model, and adding channel attention and spatial attention mechanisms at the end of the MobileViT model to better capture channel information and spatial information.

[0021] In step 1), the channel attention mechanism is constructed based on the CBAM attention mechanism. The channel attention mechanism uses a multi-layer perceptron (MLP) containing two fully connected layers, and a one-dimensional convolution is introduced to alleviate the problem of too many parameters:

[0022] Suppose the input image is where H, W, and C are the height, width, and number of channels of the image, respectively;

[0023] The spatial information of the image is compressed through global average pooling and maximum pooling operations to obtain where and represent the results obtained by using global average pooling and maximum pooling on the image in the channel attention mechanism, respectively. In the channel attention, the calculation process of the global average pooling is for each channel of the image. Each channel has HxW values, and the sum of all elements in each channel is taken and averaged. The calculation process of the maximum pooling operation is similar, which is to find the maximum value of all elements in each channel. Concatenate and fuse through a one-dimensional convolution, and calculate the attention score through a sigmoid function. The specific calculation process is as follows:

[0024]

[0025] wherein, is a one-dimensional convolution with kernel size k, and σ is a sigmoid function, The attention score is multiplied by the input image to obtain the output

[0026] In step 2), the spatial attention mechanism is constructed, which is different from the channel attention mechanism. The spatial attention focuses on the relationship between spaces, giving each pixel a weight, thereby obtaining important information in the spatial dimension. In the spatial attention mechanism, for the input image, the channel information is compressed by global average pooling and maximum pooling, respectively, to obtain wherein and respectively represent the results obtained by global average pooling and maximum pooling in the spatial dimension of the image. In the spatial attention mechanism, the calculation process of global average pooling is to calculate the position of each pixel point in the image. There are a total of C channels at each pixel point position, i.e. a total of C values, and then the average value of the C values is obtained to obtain the output of the average pooling. The calculation process of maximum pooling is similar, which is to change the average value process to maximum value. Then, the obtained is calculated by a convolution layer; in order to better fuse information, improve the representation ability of the network, and capture crop disease information, a multi-branch structure is adopted. Some classic models, such as GoogLeNet and ResNet, use a multi-branch structure to improve the performance of the model. Therefore, in the spatial attention mechanism, in order to better extract information, a multi-branch network structure with different sizes of convolution kernels is considered, i.e. the input image is input into the multiple branches for calculation, and then the output results of the multiple branches are added, and a dilated convolution is used to increase the receptive field; after the pooling operation, the are spliced, then processed by multiple convolution kernels, and the results are added and fused, and then the attention score is calculated by sigmoid; the specific calculation process of the spatial attention score is as follows:

[0027]

[0028] wherein, s sp is the attention score, represents a two-dimensional convolution with kernel size k i × k i , the dilated value is d i , σ is a sigmoid function, Then the attention score is multiplied by the input to obtain the output

[0029] In step 3), the MobileViT model comprises an MV2 block and a MobileViT block; MV2 is an inverted residual structure in MobileNetv2; the MobileViT block is a core module of the MobileViT model, used to learn local representations and global representations; and the construction of the crop disease identification model comprises local modeling and global modeling:

[0030] (1) Local modeling: for an input image where H, W, and C are height, width, and channel number, respectively; first, a local modeling operation F local is performed, which comprises an n x n convolution module f n×n to learn local representations, and then a 1 x 1 convolution module f 1×1 to increase the dimension, to obtain the output X local of local modeling, and the specific calculation process is as follows:

[0031]

[0032] where d is the dimension of X local .

[0033] (2) Global modeling: comprising Unfold, Transformer module calculation, and Fold operation; the output X local of local modeling is subjected to Unfold operation to change the image into sequence data that can be processed by the Transformer, and self-attention operation is performed; X local is divided into non-overlapping patches, and after the Unfold operation, X is obtained, where P = p h x p w , p h , and p w are the height and width of each patch, respectively, P is the group number, and N is the number of patches in each group; the data X p in each group is input into the Transformer module for calculation to learn global representations; and Fold operation is performed to restore X . The dimension is reduced by a 1 x 1 convolution operation, spliced with the original input X, and fused by an n x n convolution operation, and the specific operation is as follows:

[0034] X o = f n×n ([X, f 1×1 (X fold )])

[0035] where f n×nFor n*n convolution operation, f 1×1 For 1*1 convolution operation.

[0036] Compared with the prior art, the present application has the following outstanding technical effects and advantages:

[0037] The present application is based on a lightweight Transformer model, namely a MobileViT network model, which can effectively learn local and global representations, and can better capture crop disease information by adding an improved attention mechanism in the model. The model is trained and tested using all the data in the PlantVillage public dataset, which contains a total of 38 categories. Through evaluation and verification on the PlantVillage public dataset, the model in the present application achieves an identification accuracy of 99.60%, indicating the effectiveness of the network model in the present application.

[0038] The present application is compared with some existing works, such as in the literature "Using Deep Learning for Image-Based Plant Disease Detection", the authors use GoogLeNet model to achieve 99.35% accuracy on PlantVillage public dataset. In the literature "Tomato crop disease classification using pre-trained deep learning algorithm", the authors Rangarajan et al. select tomato images in PlantVillage dataset and use VGG16 model to achieve 96.19% accuracy. In the literature "Grape disease image classification based on lightweight convolution neural networks and channelwise attention", the channel attention mechanism is added to ShuffleNet, and grape images in PlantVillage dataset are selected for recognition, achieving an accuracy of 99.14%. Compared with the prior art, the present application has the advantages that on the one hand, the MobileViT model is used, which can effectively learn local and global representations. On the other hand, considering that some crop diseases are small and difficult to identify, an improved attention mechanism is added, including channel attention mechanism and spatial attention mechanism, considering the channel dimension and spatial dimension, so that the model can focus on the disease area in the crop disease picture, thereby effectively identifying crop diseases.

[0039] The application studies the convolutional neural network and the Transformer model, and the convolutional neural network and the Transformer model have good effects in image recognition and can be used for crop disease identification. Meanwhile, considering that some deep learning models have a large parameter quantity and are difficult to use on some embedded and mobile devices, a lightweight model is selected for crop disease identification. In addition, the attention mechanism is studied, the attention mechanism is added to the model for learning important information, and the representation ability of the model is improved, so that the crop diseases can be effectively identified. BRIEF DESCRIPTION OF DRAWINGS

[0040] Figure 1 The input crop disease picture.

[0041] Figure 2 The structure diagram of the MobileViT block with channel attention mechanism.

[0042] Figure 3 The division mode of the Transformer module calculation in the (improved) MobileViT block is shown, wherein the same color is a group.

[0043] Figure 4 The attention mechanism structure diagram includes channel attention and spatial attention.

[0044] Figure 5 The result diagram of the network model prediction output. DETAILED DESCRIPTION

[0045] In order to more specifically explain the application, the following will be described in detail in combination with the drawings and examples.

[0046] Firstly, the training mode of the network model in the application is explained.

[0047] In the application, the model is trained by using the public dataset PlantVillage, and the PlantVillage is a public crop disease dataset containing 38 categories. The PlantVillage dataset is randomly divided into a training set, a validation set and a test set according to a ratio of 6:2:2. The training set is used for model training, the validation set is used for checking the model state during the training process, and the test set is used for finally testing the effect of the model.

[0048] During the model training process, each sample in the training set is composed of an input image and the real class label corresponding to the image. The sample data in the training set is input into the model to obtain the prediction output of the model. The output of the model is a vector. Assuming that there are C categories, a vector with a total of C elements will be output, and each position represents the probability of this category. The real label of the input image is also a vector containing C elements. The element of this vector is only 1, and the others are 0. The position of 1 represents the real class label of the image. The output result (i.e. the predicted label) of the model is compared with the real label of the input image, and the loss is calculated by the cross-entropy loss function. The calculation formula of the cross-entropy loss function is where C is the number of categories, p c is a variable with a value of 0 or 1 (if c is the real category, then p c = 1, otherwise p c = 0), and q c is the probability of the model predicting the category c. After calculating the loss, the gradient of the model parameters is calculated by back propagation, and the network model parameters are updated using the AdamW optimizer. In the actual training process, the mini-batch method is used, that is, a batch of sample data is input into the model each time. The specific training process is as follows.

[0049] (1) Input a batch of image sample data into the model.

[0050] (2) Calculate the predicted category of the batch of data by the model.

[0051] (3) Compare the predicted category obtained by the model output with the real category, and calculate the loss by the cross-entropy loss function.

[0052] (4) Perform back propagation operation to calculate the gradient of the model parameters, and update the network model parameters using the AdamW optimizer.

[0053] (5) Repeat the above steps until the training is completed.

[0054] Next, the details and key points of the network model of the application will be described in detail.

[0055] The application constructs an effective attention mechanism, which is added to the network model to improve the performance of the network and effectively capture crop disease information. Attention mechanism is widely used in natural language processing, computer vision and other fields. Attention mechanism can indicate which information is important and ignore unimportant information, so that the model can make more accurate judgments. In the crop disease recognition model, the attention mechanism is introduced to enable the model to effectively capture disease information and improve the recognition accuracy.

[0056] The application mainly improves the CBAM attention mechanism, which will not be too complex while ensuring performance. CBAM includes channel attention and spatial attention mechanisms. The channel attention mechanism mainly analyzes the relationship between image channels, gives each channel a weight, and obtains key information to improve the performance of the network. The attention mechanism constructed by the application is shown in Figure 4 As shown in Figure 4 The upper part is the calculation process of the channel attention mechanism, and the lower part is the calculation process of the spatial attention mechanism. The output obtained after the input image is calculated by the channel attention and spatial attention mechanisms is added to the input image to obtain the final output. The specific calculation processes of the channel attention mechanism and the spatial attention mechanism are described as follows.

[0057] The channel attention constructed by the application mainly improves the CBAM attention mechanism. In CBAM, the channel attention mechanism uses a multi-layer perceptron (MLP) containing two fully connected layers, resulting in a large number of parameters. To solve this problem, the application introduces one-dimensional convolution to alleviate the problem of too many parameters according to the idea in ECA-Net. Assuming that the input image is where H, W, and C are the height, width, and channel number of the image, respectively. First, the spatial information of the image is compressed by global average pooling and maximum pooling operations. The average pooling operation can represent some information of the whole, and the maximum pooling operation can represent significant information. Combining the two can better improve the performance of the model. After global average pooling and maximum pooling calculation, respectively Then and are spliced, and then fused by one-dimensional convolution. Finally, the attention score is calculated by the sigmoid function. The specific calculation process is as follows:

[0058]

[0059] where is a one-dimensional convolution with a kernel size of k, and σ is a sigmoid function, Then the attention score is multiplied by the input image to obtain the output

[0060] After considering the attention of the channel dimension, the spatial attention mechanism is constructed. Unlike the channel attention mechanism, the spatial attention focuses on the relationship between spaces, giving each pixel a weight to obtain important information in the spatial dimension. In the spatial attention mechanism, for the input image, first, the channel information is compressed by global average pooling and maximum pooling to obtain Then the obtained The calculation is performed. In order to better fuse information, improve the representation ability of the network, and capture crop disease information, a multi-branch structure is adopted. Some classic models, such as GoogLeNet and ResNet, adopt a multi-branch structure to improve the performance of the model. Therefore, in the spatial attention mechanism, in order to better extract information, a multi-branch network structure is considered to be constructed using different sizes of convolution kernels, and a dilated convolution is selected to increase the receptive field. After the pooling operation, the output is obtained The two are spliced, then processed by multiple convolution kernels, and the results are added and fused, and then the attention score is calculated by sigmoid. The specific calculation process is as follows:

[0061]

[0062] wherein, represents a two-dimensional convolution with a convolution kernel size of k i x k i , a dilated value of d i , and a sigmoid function σ, Then the attention score is multiplied by the input to obtain the output

[0063] (2) The MobileViT model is used as the basis of the model. For the Transformer model, it can learn global representation based on self-attention mechanism, which is its advantage. For the convolutional neural network, it has spatial inductive bias and can learn local representation with fewer parameters. Considering the advantages of convolutional neural networks and Transformer models, the MobileViT model is selected to construct the crop disease recognition model in the present application, which combines the advantages of convolutional neural networks and Transformers. The main modules included in MobileViT are MV2 blocks and MobileViT blocks. The MV2 block is the MobileNetv2 block, which is the inverted residual structure in MobileNetv2. In MobileViT, the core module is the MobileViT block, which combines the advantages of CNN and Transformer, and is used to learn local representation and global representation. For the input image wherein H, W and C are height, width and channel number respectively. First, the local modeling operation F local is performed, which first learns local representation through an n x n convolution module f n×n , and then performs dimensionality increasing through a 1 x 1 convolution module f 1×1 to obtain the output X local after local modeling, and the specific calculation process is as follows:

[0064]

[0065] where d is X local dimension. Then, global modeling is performed by the Transformer model. The main steps include Unfold, Transformer module calculation and Fold operation. For the output X local , first, the Unfold operation is performed to change the image into sequence data that can be processed by the Transformer, and self-attention operation is performed. X local is divided into non-overlapping patches, and after the Unfold operation, X where P = p h p w , p h p w are the height and width of each patch, respectively, P is the number of groups, and N is the number of patches in each group. Then, the data X p in each group is input into the Transformer module for calculation to learn global representation. Then, the Fold operation is performed for restoration to obtain X Finally, a 1x1 convolution operation is performed for dimension reduction, and then the original input X is spliced through a splicing operation, and an n x n convolution operation is performed for fusion. The specific operation is as follows:

[0066] X o = f n×n ([X,f 1×1 (X fold )])

[0067] where f n×n is an n x n convolution operation, and f 1×1 is a 1 x 1 convolution operation.

[0068] On the basis of MobileViT, in order to better capture information, the channel attention mechanism is added in part of the MobileViT blocks of MobileViT.

[0069] The following is the process of inputting an image and predicting the recognition output by the network model.

[0070] (1) An image is input, as shown in Figure 1 , the size is 224x224x3, which represents that the height and width of the image are 224, there are 3 channels, and the category is "Apple___Apple_scab". First, a 3x3 convolution layer is used for calculation, and then a BatchNorm layer and a SiLU activation function are used.

[0071] (2) In the first stage, the calculation is performed through a MobileNetv2 module. In the module, first, the calculation is performed through a 1x1 convolution module, and the dimension is increased through the 1x1 convolution layer. Then, the calculation is performed through a 3x3 deep convolution, the input channel number and the output channel number of the deep convolution are equal, and only one convolution kernel is used for each channel to reduce the parameter quantity. Finally, the dimension is reduced through the 1x1 convolution.

[0072] (3) In the second stage, the calculation is performed through three MobileNetv2 modules, and the calculation process of each MobileNetv2 module is similar to step (2). The difference lies in that the channel number of the output image in the current stage is 48, and the size is 56x56; in step (2), the channel number of the output image is 32, and the size is 112x112.

[0073] (4) In the third stage, first, the calculation is performed through a MobileNetv2 module, and the specific process is similar to step (2). The difference lies in that the channel number of the output image in the current stage is 64, and the size is 28x28. The calculation is performed through the improved MobileViT block, and the structure is as shown in Figure 2 In the improved MobileViT block, first, the calculation is performed through the channel attention mechanism, assuming that the input is The global average pooling and the maximum pooling are used to calculate the input X, respectively, and the spatial information is compressed into the channel to obtain Then, the and are spliced, fused through one-dimensional convolution, and the attention score is calculated through the sigmoid function Finally, the attention score and the input are multiplied to obtain the output After the channel attention calculation, the local representation calculation is performed. First, the calculation is performed through the 3x3 convolution module, and then the calculation is performed through the 1x1 convolution module. Then, the global representation calculation is performed. First, the Unfold operation is performed to divide the previous output into a group of non-overlapping patches. After the Unfold operation, the output is obtained After that, the data X p in each group is input into the Transformer module for calculation to learn the global representation. The grouping division method is as shown in Figure 3 The same color is a group. Then, the output is restored through the Fold operation, and then the calculation is performed through the 1x1 convolution. The result after the calculation is spliced with the input, and then a 3x3 convolution is used to fuse the information to obtain the output.

[0074] (5) In the fourth stage, first calculate through a MobileNetv2 module, and then calculate through an improved MobileViT block. The specific process is similar to step (4), and the difference is that the output image channel number in the current stage is 80, and the size is 14x14; the output image channel number in step (4) is 64, and the size is 28x28.

[0075] (6) In the fifth stage, first calculate through a MobileNetv2 module, and then calculate through a MobileViT block. The specific process is similar to step (4), and the difference is that no channel attention mechanism is added, and the output image channel number in the current stage is 96, and the size is 7x7.

[0076] (7) The output obtained in the previous step is first calculated using a 1x1 convolution, and then input into the attention mechanism. Assuming that the previous output is where H, W, and C are height, width, and channel number, respectively. First, calculate the channel attention. After calculating through global average pooling and maximum pooling, respectively, obtain After that, and are spliced, and then fused through one-dimensional convolution. Finally, the attention score is calculated through the sigmoid function. The specific calculation process is as follows:

[0077]

[0078] Then multiply the attention score and the input image to obtain the output After the channel attention is completed, the spatial attention is calculated. The output of the channel attention is first processed through the pooling operation to obtain After that, the two are spliced, then processed through multiple convolution kernels, and the results are added and fused, and then the attention score is calculated through the sigmoid function. The specific calculation process is as follows:

[0079]

[0080] Then multiply the attention score and the channel attention output to obtain the output Finally, add and the input X l to obtain the output of the attention mechanism.

[0081] (8) the output obtained in the previous step is input into the final classifier, in which the output obtained previously is first pooled in the channel dimension through global average pooling, then calculated through a fully connected layer, and finally the number of elements output is the number of categories, each element is the probability that the input image belongs to the category, the maximum value in these elements is the category predicted by the network model, and finally the category is output, as shown in Figure 5 .

[0082] The application combines the MobileViT model and the attention mechanism to construct a lightweight attention mechanism network for crop disease identification. For the basic MobileViT model, the application adds a channel attention mechanism in part of the MobileViT block of the MobileViT, and further adds a channel attention and spatial attention mechanism at the end of the MobileViT model, so as to better capture channel information and spatial information.

Claims

1. A method for constructing a lightweight attention mechanism network for crop disease recognition, characterized in that Based on the MobileViT model, an improved attention mechanism is added, which specifically includes the following steps: 1) Construct a channel attention mechanism based on the CBAM attention mechanism, which also includes one-dimensional convolution for analyzing the relationship between image channels, giving each channel a weight to obtain key information and improve the performance of the network; 2) Construct a spatial attention mechanism based on the CBAM attention mechanism, which gives each pixel a weight. For the input image, the channel information is compressed by global average pooling and maximum pooling, a multi-branch network is constructed using different size convolution kernels to better fuse information and improve the representation ability of the network to capture crop disease information. Use the dilated convolution to increase the receptive field; the compressed image is processed through multiple convolution kernels, the results are added and fused, and the attention score is calculated through sigmoid; 3) Construct a lightweight attention mechanism network for crop disease recognition, use the MobileViT model as the base model, add channel attention mechanism in part of the MobileViT block of the MobileViT model, and add channel attention and spatial attention mechanism at the end of the MobileViT model to better capture channel information and spatial information; The MobileViT model includes MV2 blocks and MobileViT blocks; MV2 is the inverted residual structure in MobileNetv2; MobileViT block is the core module of MobileViT model, which is used to learn local representation and global representation; the MobileViT block of the crop disease recognition model includes local modeling and global modeling: (1) Local modeling: for the input image where H, W, C are height, width and channel number respectively; first, a local modeling operation F local is performed, which first passes through an n x n convolution module f n×n to learn local representation, n = 3, and then passes through a 1 x 1 convolution module f 1×1 to increase dimension, to obtain the output X local after local modeling, the specific calculation process is as follows: where d is X local dimensionality, i.e., the number of channels of the output after the local modeling computation; (2) Global modeling: including Unfold, Transformer module calculation and Fold operation; the output X obtained after local modeling local Unfold operation is performed to change the image into sequence data that can be processed by the Transformer, and self-attention operation is performed; X local is divided into non-overlapping patches, and after the Unfold operation P = p h × p w , p h , p w are the height and width of each patch respectively, P is the number of groups, and N is the number of patches in each group; the data X p in each group is input into the Transformer module for calculation to learn global representation; and Fold operation is performed to obtain The dimension is reduced through a 1×1 convolution operation, spliced with the original input X through a splicing operation, and fused through an n×n convolution operation, and the specific operation is as follows: where f n×n is an n x n convolution operation, f 1×1 is a 1 x 1 convolution operation.

2. The method of claim 1, wherein the method of constructing a lightweight attention mechanism network for crop disease identification is characterized by The improved attention mechanism based on the MobileViT model is trained and tested using all the data in the PlantVillage public dataset, and the training method is as follows: Use the public dataset PlantVillage to train the model, PlantVillage is a public crop disease dataset containing 38 categories; randomly divide the PlantVillage dataset into training set, validation set and test set according to the ratio of 6:2:2; The training set is used for model training, the validation set is used to check the model state during training, and the test set is used to test the effect of the model finally; In the model training process, each sample in the training set is composed of an input image and a real class label corresponding to the image; the sample data in the training set is input into the model to obtain the prediction output of the model, the output of the model is a vector, assuming that there are C categories, a vector with a total of C elements will be output, each position represents the probability of this category; the real label of the input image is also a vector containing C elements, only one element of the vector is 1 and the others are 0, the position of 1 represents the real class label of the image; the predicted label output by the model is compared with the real label of the input image, the loss is calculated by the cross-entropy loss function, the calculation formula of the cross-entropy loss function is Where C is the number of categories, p c is a variable with a value of 0 or 1, if c is the real category, then p c = 1, otherwise p c = 0, q c is the probability of the model predicting the category c. After calculating the loss, the gradient of the model parameters is calculated by back propagation, and the AdamW optimizer is used to update the network model parameters; in the actual training process, the mini-batch method is used, that is, a batch of sample data is input into the model each time, and the specific training process is as follows: (1) input a batch of image sample data into the model; (2) calculate the predicted class of the batch of data through the model; (3) compare the predicted class obtained by the model output with the real class, and calculate the loss through the cross-entropy loss function; (4) perform back propagation operation to calculate the gradient of the model parameters, and use the AdamW optimizer to update the network model parameters; (5) Repeat the above steps, and end when the training set number is reached.

3. The method of claim 1, wherein the method of constructing a lightweight attention mechanism network for crop disease identification is characterized by In step 1), the specific steps of constructing the channel attention mechanism are as follows: Assume the input image is where H, W, C are the height, width and channel number of the image, respectively; The spatial information of the image is compressed by global average pooling and max pooling operations, respectively, to obtain wherein and respectively represent the results obtained by global average pooling and max pooling of the image in the channel in the channel attention mechanism; in the channel attention, the calculation process of the global average pooling is for each channel of the image, each channel has HxW values, the sum of all elements in each channel is summed and averaged; the calculation process of the max pooling operation is similar, that is, the maximum value of all elements in each channel is obtained; the calculated and are spliced, fused by one-dimensional convolution, and the attention score is calculated by a sigmoid function, and the specific calculation process is as follows: wherein, is a one-dimensional convolution with kernel size k, and σ is a sigmoid function, The attention score is multiplied with the input image to get the output 4. The method of claim 1, wherein the method of constructing a lightweight attention mechanism network for crop disease identification is characterized by In step 2), the construction space attention mechanism gives each pixel point a weight, thereby obtaining important information in the spatial dimension; in the spatial attention mechanism, for the input image, the channel information is compressed by global average pooling and maximum pooling to obtain wherein and respectively represent the results obtained by global average pooling and maximum pooling in the spatial dimension of the image; in the spatial attention mechanism, the calculation process of global average pooling is to calculate the position of each pixel point in the image, and there are C channels at each pixel point position, that is, there are C values in total, and then the average value of the C values is obtained to obtain the output of the average pooling; the calculation process of maximum pooling is similar, that is, the process of averaging is changed to finding the maximum value; then the obtained is calculated through a convolution layer; in order to better fuse information, improve the representation ability of the network, and capture crop disease information, a multi-branch structure is adopted; some classic models, such as GoogLeNet and ResNet, adopt a multi-branch structure to improve the performance of the model; therefore, in the spatial attention mechanism, in order to better extract information, a multi-branch structure network is considered to be constructed using different sizes of convolution kernels, that is, the input image is input into the multiple branches to calculate, and then the output results of the multiple branches are added, and an empty convolution is used to increase the receptive field; after the pooling operation, the result is obtained; the two are spliced, then processed by multiple convolution kernels, and the results are added and fused, and then the attention score is calculated by sigmoid; the calculation process of the spatial attention score s sp is as follows: where s sp is the attention score, represents a two-dimensional convolution with a kernel size of k i x k i and a dilation value of d i , and σ is a sigmoid function, and the output is obtained by multiplying the attention score and the input

Images