A mango leaf spot disease recognition method based on a MSA-DViTNet lightweight model

Abstract

The application discloses a mango leaf spot disease recognition method based on an MSA-DViTNet lightweight model, and comprises the following steps: step 1, classifying and making a data set according to disease categories for the obtained mango leaf spot disease data pictures; step 2, using data enhancement to expand the obtained data set, and obtaining a training set and a verification set after preprocessing; step 3, constructing a mango leaf spot disease recognition network based on the MSA-DViTNet lightweight model; step 4, inputting the training set obtained in step 2 into the mango leaf spot disease recognition network based on the MSA-DViTNet lightweight model established in step 3 to perform model training, and obtaining optimal model weights; and step 5, using the optimal model weights obtained in step 4 as a pre-training model to perform image classification verification on the verification set, so as to evaluate the performance of the model. The application can improve the recognition accuracy of the mango leaf spot disease and realize lightweight deployment of the mango leaf spot disease recognition.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of leaf disease identification, and in particular to a mango leaf spot disease identification method based on a MSA-DViTNet lightweight model. BACKGROUND

[0002] Mango is an important economic crop, but leaf disease is one of the main factors affecting the yield and quality of mangoes. Leaf diseases include diseases caused by fungal, bacterial and viral pathogens, such as powdery mildew, anthracnose, etc. These diseases can cause mango leaves to wilt, turn yellow, have spots and rot, etc., which seriously affects the growth and development of mangoes.

[0003] In the prior art, the diagnosis and monitoring of mango leaf diseases mainly rely on manual observation and experience-based judgment, which has the problems of subjectivity and inaccuracy. Therefore, it is necessary to classify the images of mango leaf diseases to improve the efficiency of diagnosis and monitoring of mango leaf diseases and provide technical support for the sustainable development of mango cultivation. SUMMARY

[0004] The technical problem to be solved by the present application is to provide a mango leaf spot disease identification method based on a MSA-DViTNet lightweight model, which can improve the recognition accuracy of mango leaf spot disease and realize lightweight deployment of mango leaf spot disease identification.

[0005] To solve the above technical problems, the present application provides a mango leaf spot disease identification method based on a MSA-DViTNet lightweight model, comprising the following steps:

[0006] Step 1: classify the obtained mango leaf spot disease data pictures according to the disease categories and make data sets;

[0007] Step 2: use data augmentation to expand the sample of the obtained data set, and obtain a training set and a validation set after preprocessing;

[0008] Step 3: build a mango leaf spot disease identification network based on a MSA-DViTNet lightweight model, including a preliminary feature extraction module, a local feature extraction module, a global feature extraction module and a fusion module; wherein the preliminary feature extraction module is used for shallow feature extraction of the input image, the local feature extraction module extracts local features by sliding in different positions of the image and captures local details in the image, the global feature extraction module analyzes the entire image using the context information between different regions in the image, and the fusion module is used for integrating all the extracted features to output the final mango leaf spot disease identification classification result;

[0009] Step 4, input the training set obtained in step 2 into the mango leaf spot disease recognition network based on the MSA-DViTNet lightweight model established in step 3 for model training to obtain the optimal model weight;

[0010] Step 5, using the optimal model weight obtained in step 4 as a pre-training model, image classification verification is performed on the verification set to evaluate the performance of the model.

[0011] Preferably, in step 1, the mango leaf spot disease is divided into eight categories, including anthracnose, bacterial wilt, cutting weevil disease, dead leaf, choline midge disease, healthy leaf, powdery mildew, and smoke haze disease.

[0012] Preferably, in step 2, the sample expansion of the mango leaf spot disease dataset is based on an image operation data enhancement method, and a random probability and appropriate combination strategy is used to perform data enhancement on the dataset.

[0013] Preferably, in step 3, the preliminary feature extraction module is used to extract the input feature information and reduce the size of the feature mapping. The convolution and maximum pooling operation are fused together through two times of Concat connection and one time of Add connection. The shallow layer feature extraction is realized by using dilated convolution combined with multi-scale extraction. In the first stage of the input classification picture, a two-layer 3x3 convolution is used instead of a 5x5 convolution to obtain a larger receptive field. Then, a 3x3 convolution is used in parallel with a max pooling layer for downsampling operation, and Concat connection is used to increase the number of channels. In the second stage, two branches are used. The left branch is connected with a 1x1 convolution and a 3x3 convolution in sequence, and the right branch is connected with a 1x1 convolution and three 3x3 dilated convolutions in sequence. Concat connection is used to increase the number of channels. In the third stage, a 3x3 convolution is used in parallel with a max pooling layer for downsampling operation, and Add connection is used to output the feature map.

[0014] Preferably, in step 3, the local feature extraction module includes three sequentially connected feature extraction modules: MSA_I module, MSA_C module, and MSA_I module. The MSA_I module enables the model to focus on important features in both channel and spatial dimensions and suppress unnecessary features. The MSA_C module has different receptive fields and is used for feature extraction of different scales for the input mango leaf spot disease picture.

[0015] Preferably, the MSA_I module first passes through a 1x1 convolution layer to obtain a mango feature map, then performs feature extraction through a 3x3 deep separable convolution and an improved attention mechanism, and finally passes through a 1x1 convolution layer to reduce the dimension to obtain the initial mango feature map.

[0016] Preferably, the MSA_C module adopts three branches to construct a multi-scale feature extraction module. The first branch adopts a 1x1 convolutional layer for downsampling operation without changing the channel dimension. The second branch adopts a dimension increasing and then dimension decreasing operation, and a 3x3 deep separable convolution is used in the middle to extract features, forming an inverted bottle neck structure. The third branch also adopts a dimension increasing and then dimension decreasing operation, and two 3x3 deep separable convolutions are used in the middle to obtain a larger receptive field. The feature maps obtained by the three branches are connected by Add to realize feature fusion to obtain the final feature map.

[0017] Preferably, in step 3, the global feature extraction module includes four sequentially connected feature extraction modules: MSA_C1 module, DViT module, MSA_C2 module and DViT module. The structures of the MSA_C1 module, MSA_C2 module and MSA_C module are consistent. The pure Transformer network model is prone to lose the inductive bias information of the image, and needs to consume more parameters for learning. The global feature extraction module has both convolution properties and combines the establishment of global dependence on input feature map information in ViT, and uses the context information between different regions in the image to analyze the entire image.

[0018] Preferably, the DViT module first extracts local representation using a 3x3 convolution, adjusts the channel using a 1x1 convolution, then divides the feature map into tokens and expands them into a sequence through Unflod, inputs them into several parallel Transformer modules for global feature extraction, folds the feature map into the original feature map form through Flod, then adjusts the channel to the original size using a 1x1 convolution, then concatenates the original input feature map through a shortcut branch, and finally performs feature fusion through a 3x3 convolution to obtain the final output.

[0019] Preferably, in step 4, the network model training needs to be iteratively trained multiple times until the loss function of the network model converges, that is, the loss function decreases to a certain extent and tends to be stable, and a network model with high disease recognition rate is obtained.

[0020] The beneficial effects of the present application are: (1) the present application adopts a depth separable convolution to build a feature extraction module, which reduces the parameter quantity and calculation quantity of the model, reduces the complexity of the model, and further reduces the risk of overfitting; (2) the present application adopts multi-scale feature fusion to extract image features, which increases the depth and width of the network while improving the receptive field, thereby improving the expression ability of the model; (3) the present application is based on the improved CBAM attention mechanism, which changes from the original serial structure to the parallel addition mode, and the two modules can independently learn from the original input feature mapping, decoupling the dependency between them, improving the feature representation ability of the model; (4) the present application adopts a parallel Transformer module, which models the global dependency of all elements in the input sequence through a multi-head attention mechanism, can better handle long-distance dependencies, and solves some limitations and problems of traditional sequence models. BRIEF DESCRIPTION OF DRAWINGS

[0021] Figure 1 The present application is a method flowchart.

[0022] Figure 2 The present application is a mango leaf spot disease recognition network model structure diagram.

[0023] Figure 3 The present application is a preliminary feature extraction module structure diagram.

[0024] Figure 4 The present application is a MSA_I module structure diagram.

[0025] Figure 5 The present application is a MSA_C module structure diagram.

[0026] Figure 6 The present application is a structure diagram of the improved CBAM (BX_CBAM) attention mechanism module.

[0027] Figure 7 The present application is a DViT module structure diagram. DETAILED DESCRIPTION

[0028] As shown in Figure 1 A mango leaf spot disease recognition method based on a MSA-DViTNet lightweight model, comprising the following steps:

[0029] Step 1: Classify the acquired mango leaf spot disease data pictures and artificially mark them to make a data set.

[0030] The mango leaf spot disease image data is divided into eight categories according to the leaf spot disease categories, including mango anthracnose, mango cutting weevil disease, mango dead leaf, mango gall midge disease, mango powdery mildew, mango soot smog disease and mango healthy leaf.

[0031] Step 2: Use the image operation-based method to perform sample expansion on the obtained data set, and divide it into a training set and a validation set according to an 8:2 ratio. The processed data set contains a total of 20,000 images, of which 16,000 are in the training set and 4,000 are in the validation set.

[0032] (1) The specific process of data enhancement based on image operation is as follows:

[0033] With the expansion of the training set, the performance of the network model will be further improved. The present application adopts the strategy of random probability and combination transformation for data enhancement, that is, 12 image enhancement methods are combined into an enhancement sequence, and 3 to 5 methods are randomly selected in each enhancement to be applied to the image. By adjusting the proportion and order of each method, the effect and diversity of enhancement are controlled. Table 1 is a detailed description of the image data enhancement method.

[0034] Table 1 Detailed description of image data enhancement method

[0035]

[0036] (2) Image preprocessing is performed on the data set after sample expansion:

[0037] The data preprocessing operation method for the training set is as follows: randomly crop the picture to 224*224, randomly flip it in the horizontal direction, convert it to tensor data type, and normalize it. The data preprocessing operation method for the validation set is as follows: scale the picture to 1.143 times of 224, crop the picture center to 224*224, convert it to tensor data type, and normalize it.

[0038] Step 3: Mango leaf spot recognition method based on MSA-DViTNet lightweight model, such as Figure 2As shown, it includes a preliminary feature extraction module, a local feature extraction module, a global feature extraction module and a fusion module. First, the input feature vector (3, 224, 224) is subjected to the preliminary feature extraction module to obtain the output feature vector (48, 56, 56); second, the input feature vector is subjected to the local feature extraction module, and MSA_I, MSA_C and MSA_I are sequentially connected, using deep classifiable convolution and multi-scale feature fusion construction module, increasing the receptive field while significantly reducing the parameters and computational complexity, improving the generalization ability of the model, and obtaining output feature vectors (64, 56, 56), (64, 28, 28) and (64, 28, 28) respectively; third, the input feature vector is subjected to the global feature extraction module, and MSA_C1, DViT, MSA_C2 and DViT are sequentially connected, and the introduction of the parallel Transformer module enables the input image features to be globally correlated, and the self-attention mechanism enables the model to have better translation invariance, obtaining output vectors (80, 14, 14), (80, 14, 14), (80, 7, 7) and (80, 7, 7) respectively; finally, the input feature vector is subjected to 1x1 convolution for feature fusion and changing the number of channels, obtaining the output feature vector (320, 7, 7), using adaptive pooling layer to dynamically adjust the size of the pooling operation, and using the full connection layer to integrate the previous information to output the final classification result.

[0039] (1) The preliminary feature extraction module is as shown in Figure 3 As shown, the main branch is designed with a two-layer 3x3 small convolution instead of a 5x5 large convolution, reducing the parameter amount while increasing the receptive field; then a max pooling layer is designed in parallel with a 3x3 convolution for downsampling, avoiding information loss, realizing nonlinear transformation of a large receptive field, and performing Concat connection; then two parallel branches, the first branch is composed of a 1x1 convolution connected in series with a 3x3 convolution, and the second branch is composed of a 1x1 convolution connected in series with three 3x3 dilated convolutions with expansion factors of 1, 2 and 5 respectively, and through Concat connection, the model can capture feature information at different levels and different scales; finally, a max pooling layer is used in parallel with a 3x3 convolution for downsampling and Add connection, better capturing details and context information in the image.

[0040] (2) The MSA_I module is as shown in Figure 4 As shown, it is composed of deep separable convolution and BX_CBAM, first using 1x1 PW convolution to increase dimension, connecting in series with 3x3 DW convolution and BX_CBAM attention mechanism, and finally using 1x1 PW convolution to reduce dimension, reducing the computational complexity and improving the expression ability of the features.

[0041] (3) The improved CBAM attention mechanism (BX_CBAM) is as shown in Figure 5As shown, since CBAM is a serial structure, the spatial attention and the channel attention are affected by the content processed by the previous module regardless of the order. Therefore, the original serial is changed to parallel addition, so that both attention mechanisms can learn what they need from the original input feature map without affecting each other.

[0042] The calculation formula is:

[0043]

[0044]

[0045] F3 = F1 + F2

[0046]

[0047]

[0048] In the formula, F is the input feature map, is the dot product, M c (F) is the channel attention operation, F1 is the channel attention output feature map, M s (F) is the spatial attention operation, F2 is the spatial attention output feature map, F3 is the final output feature map, AvgPool is the global average pooling, MaxPool is the global maximum pooling, MLP is the shared network of the convolutional neural network, the maximum pooling and the average pooling share a neural network to extract features, σ is the ReLU activation, f 7*7 is a 7x7 convolution.

[0049] (4) The MSA_C module is as shown in Figure 6 , which adopts a deep separable convolution and multi-scale feature fusion to construct a feature extraction module, which consists of three branches. The first branch is a 1x1 convolution for down-sampling. The second branch is connected in turn with a 1x1 PW convolution, a 3x3 DW convolution and a 1x1 PW convolution. The third branch is connected in turn with a 1x1 PW convolution, two 3x3 DW convolutions and a 1x1 PW convolution. Multiple convolution kernels of different sizes are used to extract features from the feature map, which increases the adaptability of the network to scales and widens the network.

[0050] (5) The DViT module is as shown in Figure 7As shown in the DViT, first, the feature map is passed through a convolution layer with a convolution kernel size of 3*3, and then through a convolution layer with a convolution kernel size of 1*1 for channel adjustment, then sequentially through the Unfold, parallel Transformer, Fold structure for global feature modeling, the number of parallel Transformer modules is consistent, and when the DViT module is repeated twice, the number of parallel Transformer modules is 2, 3, respectively, and then through a convolution layer with a convolution kernel size of 1*1 to adjust the channel to the original size, then through the shortcut shortcut branch and the original input feature map are concatenated by channel, and finally through a convolution layer with a convolution kernel size of 3*3 for feature fusion to obtain the final output;

[0051] The Unfold structure is to change the input [B, H, W, C] to [B, P, N, d], that is, to divide the input feature map into a patch; wherein B is batch_size, H is the length of the input feature map, W is the width of the input feature map, C is the number of channels, note that P is w*h (the width and height of each patch), N is the number of patches, d>C, d is the number of blocks obtained by sliding and cropping the length and width of the input feature map;

[0052] The Fold structure is to change [B, P, N, d] to [B, H, W, d];

[0053] The Transformer structure is as shown in the Transformer of Figure 7 As shown in the Transformer, it is used for modeling and extracting the features of the image block, and captures the global information in the image through the multi-head self-attention mechanism and the feedforward neural network, first, the normalization layer, the multi-head self-attention mechanism, and the dropout layer are sequentially connected, then the output feature map is obtained by Add splicing through the shortcut shortcut branch and the original input feature map, then the output feature map is input to the normalization layer, the multi-layer perceptron, and the dropout layer which are sequentially connected, and then the output feature map is Add spliced through the shortcut shortcut branch;

[0054] The multi-head self-attention mechanism is sequentially connected to a fully connected layer, a discard layer, another fully connected layer, and a Softmax activation function. The self-attention process is as follows: Each input vector is linearly projected into three different new vectors (query Q, key K, and value V), the score is calculated, then divided by a scaling factor (the square root of the key vector dimension), then standardized by the Softmax function, then multiplied by each V, and finally the weight vector is accumulated to obtain the self-attention output of the input vector. Multi-head attention splits the three vectors Q, K, and V multiple times, and each set of vectors is mapped to a different subspace. Then, self-attention calculation is performed in parallel, and finally the results of the separate calculations are combined to obtain the output of the entire multi-head attention layer.

[0055] The calculation process is as follows:

[0056]

[0057] MultiHead(Q,K,V)=Concat(head1,...,head h W O

[0058] where head h =Attention(QW i O ,KW i K VW i V )

[0059] in, d k =d v =d modle / h

[0060] The multilayer sensing machine, such as Figure 7 As shown in the MLP Block diagram, the layers connected in sequence are a normalization layer, a fully connected layer, a ReLU6 activation function, a discard layer, two fully connected layers, and another discard layer. These layers are used to perform non-linear transformations and extract features from each image patch. By stacking multiple fully connected layers, the generalization ability of the model is increased.

[0061] (6) Repeatedly stack the above modules to achieve image feature fusion and obtain the final mango leaf spot disease identification network model. Table 2 is the overall structural framework of the model.

[0062] Table 2 Overall structural framework of the model

[0063]

[0064]

[0065] Step 4: Train the built mango leaf spot disease recognition network based on MSA-DViTNet lightweight model. First, input the processed training set into the built model for feature extraction. Through the iterative forward propagation and back propagation process, combine the cosine annealing adjustment learning rate, momentum gradient descent and weight decay to optimize the network parameters. Finally, use adaptive pooling layer and full connection layer to classify the trained results.

[0066] (1) The cross-entropy loss function is selected as the target loss function, which makes the model prediction results continuously approach the true label. The formula of cross-entropy loss function is:

[0067]

[0068] In the formula: y i represents the true probability value of the i-th dimension of the picture category, represents the probability value of the i-th dimension of the picture category calculated by the neural network;

[0069] (2) SGD is used as the optimizer to minimize the above cross-entropy loss function to gradually optimize the network parameters and make the loss function converge more stably. The update formula of the optimizer is:

[0070]

[0071] In the formula: w t is the model parameter at time step t, is the gradient of the loss function L with respect to the model parameter w on (x i , y i ), η is the learning rate, which controls the update step size;

[0072] (3) The cosine annealing method is used to dynamically adjust the learning rate. Specifically, it gradually reduces the initial learning rate to a minimum value, and then gradually increases it to form a cosine function curve. The update formula is:

[0073]

[0074] In the formula: η t is the current learning rate, η min is the minimum value of the learning rate, η max is the maximum value of the learning rate, T cur is the current epoch, T max is the maximum epoch, and π is the circular constant;

[0075] Step 5: The mango leaf spot disease recognition network based on the MSA-DViTNet lightweight model is trained for multiple iterations, and by adjusting the training parameters such as batch size, iteration number, learning rate, etc., until the optimal model weight is found; the obtained optimal model weight is used as a pre-training model to perform image classification verification on the validation set to evaluate the performance of the model.

[0076] The performance evaluation indexes of the present application adopt accuracy (Accuracy), precision (Precision), recall (Recall), and comprehensive evaluation index (F1-Measure), and the formulas of the evaluation indexes are as follows:

[0077]

[0078]

[0079]

[0080]

[0081] Among them, TP represents the number of positive class predictions; TN represents the number of negative class predictions; FP represents the number of negative class predictions; and FN represents the number of positive class predictions.

[0082] Table 3 compares the experimental results of the present application with the performance of the traditional model

[0083]

[0084] As can be seen from Table 3, for the recognition of mango leaf diseases with similar morphologies between categories, the comprehensive performance of the MSA-DViTNet model proposed by the present application is optimal, with the highest accuracy of 99.6%, and the model size is only 6.19 MB, and the precision, recall and F1 score are also the highest. Compared with some existing models, the accuracy of the present application is higher, the robustness is stronger, and the recognition rate is higher.

Claims

1. A method for identifying mango leaf spot disease based on a lightweight MSA-DViTNet model, characterized in that, Includes the following steps: Step 1: Classify the acquired mango leaf spot disease data images according to disease type and create a dataset; Step 2: Use data augmentation on the dataset obtained in Step 1 to expand the samples in the dataset. After preprocessing, the training set and validation set are obtained. Step 3: Construct a mango leaf spot disease identification network based on the MSA-DViTNet lightweight model, including a preliminary feature extraction module, a local feature extraction module, a global feature extraction module, and a fusion module. The preliminary feature extraction module performs shallow feature extraction on the input image. The local feature extraction module captures local details by sliding and extracting local features at different positions in the image. The global feature extraction module analyzes the entire image using contextual information between different regions. The fusion module integrates all extracted features and outputs the final mango leaf spot disease identification and classification result. The preliminary feature extraction module extracts input feature information, reduces the size of the feature mapping, and fuses convolution and max pooling operations through two Concat connections and one Add connection. It uses dilated convolution combined with multi-scale extraction to achieve shallow feature extraction. In the first stage, the preliminary feature extraction module uses a two-layer 3×3 convolution instead of a 5×5 convolution to obtain a larger receptive field. Then, it uses a 3×3 convolution parallel max pooling layer for downsampling and performs Concat connections to increase the number of channels. The second stage uses two branches. The left branch connects a 1×1 convolution and a 3×3 convolution in sequence, and the right branch connects a 1×1 convolution and three 3×3 dilated convolutions in sequence. Concat connections are used to increase the number of channels. The third stage uses 3×3 convolutions and parallel max pooling layers to perform downsampling operations and add connections to output feature maps. The local feature extraction module comprises three sequentially connected feature extraction modules: MSA_I, MSA_C, and MSA_I. The MSA_I module suppresses unnecessary features, while the MSA_C module extracts features at different scales from the input mango leaf spot image. The MSA_I module first uses a 1×1 convolutional layer to obtain an upscaled mango feature map, then uses a 3×3 depthwise separable convolution and an improved CBAM attention mechanism for feature extraction, and finally uses a 1×1 convolutional layer for dimensionality reduction to obtain the initial mango feature map. The MSA_C module uses three branches to construct a multi-scale feature extraction module. The first branch uses a 1×1 convolutional layer for downsampling without changing the channel count. The second branch uses an upscale-then-downscale operation, with a 3×3 depthwise separable convolution for feature extraction, forming an inverted bottleneck structure. The third branch also uses an upscale-then-downscale operation, with two 3×3 depthwise separable convolutions to obtain a larger receptive field. The feature maps obtained from the three branches are connected using an Add connection to achieve feature fusion and obtain the final feature map. The global feature extraction module consists of four sequentially connected feature extraction modules: MSA_C1, DViT, MSA_C2, and DViT. The MSA_C1, MSA_C2, and MSA_C modules have the same structure. The DViT module first uses 3×3 convolution to extract local representations, then uses 1×1 convolution to adjust the channels. Next, it uses Unflod to segment the feature map into tokens and unfold them into sequences, which are then input into several parallel Transformer modules for global feature extraction. After FLOD, the feature map is folded back to its original form. Then, 1×1 convolution is used to adjust the channels to their original size. Next, it is concatenated with the original input feature map by channel through a shortcut branch. Finally, a 3×3 convolution is used to fuse the features to obtain the final output. Step 4: Use the training set obtained in Step 2 to input into the mango leaf spot disease identification network based on the MSA-DViTNet lightweight model established in Step 3 for model training to obtain the optimal model weights. Step 5: Use the optimal model weights obtained in Step 4 as the pre-trained model, and perform image classification validation on the validation set to evaluate the model's performance.

2. The mango leaf spot disease identification method based on the MSA-DViTNet lightweight model as described in claim 1, characterized in that, In step 1, mango leaf spot diseases are classified into eight categories, including anthracnose, bacterial canker, cutting weevil disease, dead leaves, midge, healthy leaves, powdery mildew, and soot fog disease.

3. The mango leaf spot disease identification method based on the MSA-DViTNet lightweight model as described in claim 1, characterized in that, In step 2, the mango leaf spot disease dataset is augmented using image manipulation-based data augmentation methods, employing random probability and appropriate combinations of strategies to augment the dataset.

4. The mango leaf spot disease identification method based on the MSA-DViTNet lightweight model as described in claim 1, characterized in that, In step 4, the network model training needs to be iterated multiple times until the loss function of the network model converges, that is, the loss function tends to stabilize after it is reduced to a certain extent, and a network model with a high disease recognition rate is obtained.

Images