A citrus Huanglong disease recognition method based on CNN and ViT

Abstract

The present invention relates to a citrus Huanglong disease recognition method based on CNN and ViT, comprising: obtaining and preprocessing citrus Huanglong disease image samples; forming a data set, dividing the data set into a training set, a validation set and a test set; constructing a citrus Huanglong disease image recognition model; preprocessing the training set, inputting the preprocessed training set into the citrus Huanglong disease recognition model for training; obtaining and preprocessing citrus Huanglong disease images to be detected, and inputting the trained citrus Huanglong disease recognition model to obtain citrus Huanglong disease recognition results. The present invention realizes accurate recognition of citrus Huanglong disease images by connecting a CNN branch model to a ViT branch model; using three cascaded residual modules, the number of parameters is greatly reduced compared to traditional ResNet, and combined with a lightweight attention module, the number of parameters is greatly reduced while the model accuracy is improved, the background interference is effectively reduced, and the recognition performance of pests and diseases is improved.

Description

Technical Field

[0001] The invention relates to the technical field of agricultural pest image processing, in particular to a citrus Huanglong disease recognition method based on CNN and ViT. Background Art

[0002] Crop diseases and insect pests are one of the major agricultural disasters in my country. They are diverse, have a wide impact, and often break out into disasters, causing huge losses to my country's national economy, especially agricultural production. Different diseases may present different symptoms and characteristics at different stages of development and under different environmental conditions, making accurate identification very difficult.

[0003] The data on crop pests and diseases in farmland is huge and growing, involving a large amount of images and related information. The need to efficiently process and manage these large-scale data has become a challenge, including data collection, storage, transmission and analysis. However, to date, most pest and disease identification relies on experienced experts, which is not only a disaster for inexperienced growers, but also time-consuming and laborious. Therefore, it is urgent to find a method that can quickly and accurately identify citrus Huanglongbing in a natural environment.

[0004] In recent years, the citrus Huanglongbing recognition method based on deep learning and convolutional neural network has replaced manual recognition and traditional machine learning methods. Many researchers have conducted corresponding research on citrus Huanglongbing and achieved good results. However, some studies have shown that the data-driven deep learning model will have a decline in accuracy when the training and test data are different images taken under different conditions, especially in natural scenes rather than laboratory scenes. Summary of the invention

[0005] In order to solve the problem of low accuracy of traditional citrus Huanglong disease detection methods, the purpose of the present invention is to provide a citrus Huanglong disease recognition method based on CNN and ViT, which can achieve lightweight, reduce background interference, improve model generalization ability, accurately identify pests and diseases, and can comprehensively utilize local features and global context information to improve the accuracy of crop pest and disease identification.

[0006] To achieve the above object, the present invention adopts the following technical solution: a citrus Huanglong disease identification method based on CNN and ViT, the method comprising the following steps in order:

[0007] (1) Acquisition and preprocessing of citrus Huanglong disease image samples: Collect pictures of citrus Huanglong disease in orchards under real background as citrus Huanglong disease image samples, and preprocess the citrus Huanglong disease image samples;

[0008] (2) The preprocessed citrus Huanglong disease image samples are used to form a dataset, and the dataset is divided into a training set, a validation set, and a test set;

[0009] (3) constructing a citrus Huanglong disease image recognition model, wherein the citrus Huanglong disease image recognition model is composed of a CNN branch model and a ViT branch model;

[0010] (4) Preprocessing the training set, inputting the preprocessed training set into the citrus Huanglong disease recognition model for training, and obtaining a trained citrus Huanglong disease recognition model;

[0011] (5) Obtain the citrus Huanglong disease image to be identified and preprocess it, input the preprocessed citrus Huanglong disease image to be identified into the trained citrus Huanglong disease recognition model to obtain the citrus Huanglong disease recognition result.

[0012] In step (1), step (4) and step (5), the preprocessing refers to deleting unclear images, performing random rotation, random cropping and image enhancement, and the random cropping refers to cropping with a size of 224 224 in size.

[0013] In step (2), the dataset is divided into training set, validation set, and test set in a ratio of 8:1:1.

[0014] In step (3), the CNN branch model includes a seven-layer structure, namely:

[0015] The first layer is a 7 7 convolutions, the input image size of the first layer is 224 224 3. The output feature map size of the first layer is 112 112 64;

[0016] The second layer is a maximum pooling layer, and the input feature map size of the second layer is 112 112 64, the output feature map size of the second layer is 56 56 64;

[0017] The third layer is the first cascaded residual module, which includes two branches, one of which consists of 1 1 convolution, 3 3 convolutions, 1 1 convolution is composed of a series of branches, which is connected in parallel with a residual edge branch. The two branch feature maps are added. The input feature map size of the third layer is 56 56 64, the output feature map size of the third layer is 56 56 256;

[0018] The fourth layer is a second cascaded residual module, and the second cascaded residual module includes two branches, one of which consists of 1 1 convolution, 3 3 convolutions, 1 1 convolution is composed of a series of branches with 1 1 The residual edge branches of the convolutional layer are connected in parallel, and the two branch feature maps are added. The input feature map size of the fourth layer is 56 56 256, the output feature map size of the fourth layer is 56 56 256;

[0019] The structure of the fifth layer is the same as that of the fourth layer, and the input feature map size of the fifth layer is 56 56 256, the output feature map size of the fifth layer is 14 14 1024;

[0020] The sixth layer is a 1 1 convolution, the input feature map size of the sixth layer is 14 1024, the output feature map size of the sixth layer is 14 14 320;

[0021] The seventh layer is an adaptive average pooling layer;

[0022] In step (3), the ViT branch model includes seven layers, namely:

[0023] The first layer consists of a convolution kernel of 3 for dimensionality reduction. 3 convolution and a MV2 module for feature extraction are connected in series;

[0024] The second layer consists of a MV2 module with a step size of 1 and a MV2 module with a step size of 2 connected in series;

[0025] The third layer consists of a MV2 module with a stride of 2 and a lightweight attention module in series;

[0026] The structures of the fourth and fifth floors are the same as the third floor;

[0027] The sixth layer is a convolution kernel of 1 Convolution of 1;

[0028] The seventh layer is an adaptive average pooling layer.

[0029] The step (4) specifically includes the following steps:

[0030] (4a) Perform forward propagation: input the preprocessed training set and perform forward propagation through the citrus Huanglong disease image recognition model;

[0031] (4b) Calculate the loss according to the loss function:

[0032]

[0033] in, The probability distribution vector representing the true label elements, represents the predicted probability distribution vector of the citrus Huanglong disease image recognition model. elements, N represents the number of elements; λ is the regularization parameter, which is used to control the strength of regularization; represents the weight parameter;

[0034] (4c) Based on the loss results, back propagate and update the parameters;

[0035] (4d) Repeat steps (4a) to (4c) until the loss converges and the weight with the best prediction effect is obtained.

[0036] The MV2 module includes a downward projection convolution with a convolution kernel size of 1×1, a depth convolution with a convolution kernel size of 3×3, and an upward projection convolution with a convolution kernel size of 1×1. The input dimension is reduced by the downward projection convolution, features are extracted by the depth convolution, and the dimension is restored by the upward projection convolution. A residual edge branch is introduced before the downward projection convolution and after the upward projection convolution.

[0037] The lightweight attention module includes two 3×3 convolutions, two 1×1 convolutions, a Transformer module and a splicing module. First, the feature map is locally modeled through a 3×3 convolution, and then the number of channels is adjusted through a 1×1 convolution. Then, the global feature modeling is performed through the Transformer module, and then the number of channels is adjusted back to the original size through a convolution with a convolution kernel size of 1×1. Then, it is spliced ​​with the original input feature map through the residual edge branch, and finally, a 3×3 convolution is performed to perform feature fusion to obtain the output.

[0038] It can be seen from the above technical scheme that the beneficial effects of the present invention are: first, the present invention realizes accurate recognition of citrus Huanglong image diseases by connecting the CNN branch model to the ViT branch model; second, the present invention uses three cascaded residual modules, which greatly reduces the number of parameters compared to the traditional ResNet, and combines with the lightweight attention module to greatly reduce the number of parameters while improving the model accuracy, effectively reducing background interference, and improving the recognition performance of pests and diseases; third, the lightweight attention module used in the present invention, the series-connected Transformer modules in its structure can enhance the modeling ability of the model for global contextual information, and use the CNN branch model to extract local features, thereby realizing comprehensive modeling of global and local information of citrus Huanglong disease; fourth, the present invention proposes a new loss function that integrates the cross entropy loss function and the regularization term, which can reduce the loss function more quickly and improve the model accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 is a flow chart of the method of the present invention;

[0040] Figure 2 , 3 , 4, and 5 are schematic diagrams of the structures of the MV2 module, the lightweight attention module, the first cascaded residual module, and the second cascaded residual module, respectively;

[0041] Figure 6 This is a schematic diagram of Tokens calculation;

[0042] Figure 7 It is a structural schematic diagram of the citrus Huanglong disease image recognition model in the present invention. DETAILED DESCRIPTION

[0043] like Figure 1 As shown, a citrus Huanglong disease recognition method based on CNN and ViT includes the following steps in sequence:

[0044] (1) Acquisition and preprocessing of citrus Huanglong disease image samples: Collect pictures of citrus Huanglong disease in orchards under real background as citrus Huanglong disease image samples, and preprocess the citrus Huanglong disease image samples;

[0045] (2) The preprocessed citrus Huanglong disease image samples are used to form a dataset, and the dataset is divided into a training set, a validation set, and a test set;

[0046] (3) Construct an image recognition model for citrus Huanglong disease, such as Figure 7 As shown, the citrus Huanglong disease image recognition model is composed of a CNN branch model and a ViT branch model;

[0047] (4) Preprocessing the training set, inputting the preprocessed training set into the citrus Huanglong disease recognition model for training, and obtaining a trained citrus Huanglong disease recognition model;

[0048] (5) Obtain the citrus Huanglong disease image to be identified and preprocess it, input the preprocessed citrus Huanglong disease image to be identified into the trained citrus Huanglong disease recognition model to obtain the citrus Huanglong disease recognition result. The citrus Huanglong disease image to be identified is an image of a citrus leaf infected with Huanglong disease in its natural state.

[0049] In step (1), step (4) and step (5), the preprocessing refers to deleting unclear images, performing random rotation, random cropping and image enhancement, and the random cropping refers to cropping with a size of 224 The size of the dataset is 224. The sufficient amount of data after expansion is conducive to model training and can improve the accuracy, precision, Recall, and F1 score of the model output results to a certain extent.

[0050] In step (2), the dataset is divided into training set, validation set, and test set in a ratio of 8:1:1.

[0051] In step (3), the CNN branch model includes a seven-layer structure, namely:

[0052] The first layer is a 7 7 convolutions, the input image size of the first layer is 224 224 3. The output feature map size of the first layer is 112 112 64, the first layer is used for dimensionality reduction and feature extraction;

[0053] The second layer is a maximum pooling layer, and the input feature map size of the second layer is 112 112 64, the output feature map size of the second layer is 56 56 64; The second layer is used for dimensionality reduction and feature extraction;

[0054] like Figure 4 As shown, the third layer is the first cascaded residual module, and the first cascaded residual module includes two branches, one of which consists of 1 1 convolution, 3 3 convolutions, 1 1 convolution is composed of a series of branches, which is connected in parallel with a residual edge branch. The two branch feature maps are added. The input feature map size of the third layer is 56 56 64, the output feature map size of the third layer is 56 56 256; previous 1 1 convolution to ensure the consistency of the subsequent branch fusion form, 3 3 convolutions are used to extract features, and the last 1 1. Convolution is used to increase the dimension, and the residual edge branch is used to reduce the gradient disappearance, making the extracted feature map information more comprehensive;

[0055] like Figure 5 As shown, the fourth layer is the second cascaded residual module, and the second cascaded residual module includes two branches, one of which consists of 1 1 convolution, 3 3 convolutions, 1 1 convolution is composed of a series of branches with 1 1 The residual edge branches of the convolutional layer are connected in parallel, and the two branch feature maps are added. The input feature map size of the fourth layer is 56 56 256, the output feature map size of the fourth layer is 56 56 256; previous 1 1 convolution is used to reduce the dimension of feature maps, 3 3 convolutions are used to extract features, and the last 1 1 convolution is used to increase the dimension, and the residual edge is to reduce the gradient disappearance, so that the extracted feature map information is more comprehensive. The residual edge carries 1 1. The convolutional layer is used to keep the sizes of the two branches consistent so as to perform the addition operation;

[0056] The structure of the fifth layer is the same as that of the fourth layer, and the input feature map size of the fifth layer is 56 56 256, the output feature map size of the fifth layer is 14 14 1024;

[0057] The sixth layer is a 1 1 convolution, the input feature map size of the sixth layer is 14 1024, the output feature map size of the sixth layer is 14 14 320; The sixth layer is used to keep the number of output channels consistent with the ViT branch model to facilitate subsequent addition operations;

[0058] The seventh layer is an adaptive average pooling layer; it uses the best operation among the maximum pooling and the average pooling, and changes the feature map size to 1. 1 can be input to the fully connected layer;

[0059] Assuming W is the size of the input image, F is the size of the convolution kernel, P is the padding size, S is the step size, and N is the output size after the convolution layer, the calculation formula is as follows:

[0060] +1

[0061] This is a general formula that applies to all convolution and pooling layers. In the formula, W refers to the width of the input image of this layer. Since all images are square, the width and height are consistent; F is the convolution kernel of all convolution modules, and P is the padding of the input of this layer. S is the step size of the convolution kernel sliding, and N is the size of the image output after the convolution kernel.

[0062] Three cascaded residual modules are used in the CNN branch model, with a total parameter amount of 1.3M. The traditional ResNet uses 19 ResBlocks, with a total parameter amount of 20.4M. Compared with the traditional ResNet, the CNN branch model of the present invention is very lightweight and through operations such as dimensionality increase, dimensionality reduction, and the use of residual edges, the extracted feature map information is more complete and the gradient vanishing problem is alleviated; the output result of the present invention is better than the classic ResNet network.

[0063] In step (3), the ViT branch model includes seven layers, namely:

[0064] The first layer consists of a convolution kernel of 3 for dimensionality reduction. 3 convolution and a MV2 module for feature extraction are connected in series;

[0065] The second layer consists of a MV2 module with a step size of 1 and a MV2 module with a step size of 2 connected in series; the second layer is used to extract features;

[0066] The third layer consists of a MV2 module with a stride of 2 and a lightweight attention module in series; the MV2 module is used to extract features, and the lightweight attention module aims to model local and global information in the input tensor with fewer parameters;

[0067] The structures of the fourth and fifth floors are the same as the third floor;

[0068] The sixth layer is a convolution kernel of 1 The convolution of 1 keeps the number of output channels consistent with the CNN branch model to facilitate subsequent addition operations;

[0069] The seventh layer is an adaptive average pooling layer, which uses the best operation of maximum pooling and draw pooling, and changes the feature map size to 1 1 can be input to the fully connected layer;

[0070] The output feature maps of the CNN branch model and the ViT branch model are added, and then the added result is input into the fully connected layer, and then the recognition result is output through the softmax layer.

[0071] The step (4) specifically includes the following steps:

[0072] (4a) Perform forward propagation: input the preprocessed training set and perform forward propagation through the citrus Huanglong disease image recognition model;

[0073] (4b) Calculate the loss according to the loss function:

[0074]

[0075] in, The probability distribution vector representing the true label elements, represents the predicted probability distribution vector of the citrus Huanglong disease image recognition model. elements, N represents the number of elements; λ is the regularization parameter, which is used to control the strength of regularization; represents the weight parameter;

[0076] (4c) Based on the loss results, back propagate and update the parameters;

[0077] (4d) Repeat steps (4a) to (4c) until the loss converges and the weight with the best prediction effect is obtained.

[0078] A regularization term is added to the traditional cross entropy loss function. The main reason is that Transformer is more difficult to train than CNN. Adding the regularization term can make Transformer converge faster, thereby reducing the difficulty of training and improving the performance of the model. This is an operation that most CNN+Transformer models do not have.

[0079] like Figure 2 As shown in the figure, the MV2 module includes a down-projection convolution with a convolution kernel size of 1×1, a depth convolution with a convolution kernel size of 3×3, and an up-projection convolution with a convolution kernel size of 1×1. The input dimension is reduced by the down-projection convolution, and features are extracted by the depth convolution. The dimension is restored by the up-projection convolution, and a residual edge branch is introduced before the down-projection convolution and after the up-projection convolution. The MV2 module is influenced by the lightweight attention module and uses an inverted residual structure, which can effectively make the high-dimensional information lose less information after passing through the ReLU activation function.

[0080] like Figure 3As shown in the figure, the lightweight attention module includes two 3×3 convolutions, two 1×1 convolutions, a Transformer module and a splicing module. First, the feature map is locally modeled through a 3×3 convolution, and then the number of channels is adjusted through a 1×1 convolution. Then, the global feature modeling is performed through the Transformer module, and then the number of channels is adjusted back to the original size through a convolution with a convolution kernel size of 1×1. Then, it is spliced ​​with the original input feature map through the residual edge branch, and finally, a 3×3 convolution is performed to perform feature fusion to obtain the output.

[0081] In the Transformer module, the feature map is first divided into 2×2 patches, such as Figure 5 As shown in the figure, each Patch is composed of 4 different pixels (color blocks). When performing multi-head attention mechanism calculations, each Token (each small pixel or color block) only performs attention calculations with the Token of the same color as itself, thereby achieving the purpose of reducing the amount of calculations.

[0082] The traditional ViT branch has 86M parameters, while the lightweight attention module used in this application has only 1.9M parameters. In the lightweight attention module of the ViT branch model, the original multi-head attention mechanism calculates each Token and needs to perform Attention with all Tokens. Assuming that the height, width and number of channels of the feature map are H, W, and C respectively, the calculation cost is recorded as Cost=O(WHC). If each Token only performs Attention with the Token of the same color as itself, assuming that the Patch size is 2×2, then the calculation cost is Cost=O(WHC / 4), that is, the theoretical calculation cost is only 1 / 4 of the original one. Therefore, the ViT branch is very lightweight and uses fewer parameters to model local and global information in the input tensor, making the model extract richer features. The Tokens calculation diagram is shown below. Figure 6 shown.

[0083] The effect of the present invention can be illustrated by the following simulation experiment:

[0084] 1. Experimental conditions

[0085] This study mainly uses the citrus Huanglongbing disease image dataset published by Chi Meixiang et al. This dataset contains 10,154 citrus Huanglongbing field symptom recognition pictures, with a total of 5 categories and 14 subcategories. Among them, 7,516 pictures were collected in the field using mobile phones or cameras, 149 pictures were searched through the Internet, and 2,638 pictures were obtained through data enhancement, including test set data. At the same time, training and testing were also carried out on the Plantvillage public dataset with 38 categories of common plant diseases and insect pests, which contains 29,833 pictures.

[0086] For the Huanglongbing dataset, we extracted images and converted them from labeled detection task data to data for image classification tasks. We randomly divided the training set and validation set into a ratio of 8:2, and used built-in images that were different from the training and validation sets as the test set. For the Plantvillage dataset, we randomly divided it into a training set, a validation set, and a test set in a ratio of 8:1:1.

[0087] The optimization method is set to AdamW optimization method, the initial learning rate is set to 0.002, and considering that the Transformer model usually requires more iterations to converge than CNN, a total of 30 training cycles are selected for the Transformer related model. All experiments follow the method of training from scratch without using transfer learning.

[0088] 2. Experimental results

[0089] As shown in Table 1 below, compared with the current mainstream models, the model proposed in the present invention has higher accuracy than other advanced recognition models while maintaining a lower parameter amount.

[0090] Table 1 Comparison of the accuracy results of the present invention and other detection models (unit: %)

[0091] Model Parameter quantity (M) Accuracy(%) ResNet34 21.8M 97.30 AlexNet 60.9M 94.80 Vgg16 138.3M 96.00 GoogLeNet 500M 97.10 MobileNetv3 5.4M 94.00 The present invention 3.2M 97.60

[0092] In summary, the present invention realizes accurate recognition of citrus Huanglong image diseases by connecting the CNN branch model to the ViT branch model; the present invention uses three cascaded residual modules, which greatly reduces the number of parameters compared to the traditional ResNet, and combines with the lightweight attention module to greatly reduce the number of parameters while improving the model accuracy, effectively reducing background interference, and improving the recognition performance of pests and diseases; the lightweight attention module used in the present invention has a structure in which the series-connected Transformer modules can enhance the model's modeling ability for global contextual information, and uses the CNN branch model to extract local features, thereby realizing comprehensive modeling of global and local information of citrus Huanglong disease; the present invention proposes a new loss function that integrates the cross entropy loss function and the regularization term, which can reduce the loss function more quickly and improve the model accuracy.

[0093] The above description introduces the basic principles and specific implementations of the present invention. It should be pointed out that the implementations of the present invention are not limited to the above implementation examples. Any modifications, equivalent substitutions, etc. made without departing from the scope of the principles of the present invention shall be included in the protection scope of the present invention. The protection scope required by the present invention is defined by the attached claims and their equivalents.

Claims

1. A citrus Huanglong disease recognition method based on CNN and ViT, characterized in that: The method comprises the following steps in order: (1) Acquisition and preprocessing of citrus Huanglong disease image samples: Collect pictures of citrus Huanglong disease in orchards under real background as citrus Huanglong disease image samples, and preprocess the citrus Huanglong disease image samples; (2) The preprocessed citrus Huanglong disease image samples are used to form a data set, and the data set is divided into a training set, a validation set, and a test set; (3) constructing a citrus Huanglong disease image recognition model, wherein the citrus Huanglong disease image recognition model is composed of a CNN branch model and a ViT branch model; (4) preprocessing the training set, inputting the preprocessed training set into the citrus Huanglong disease recognition model for training, and obtaining a trained citrus Huanglong disease recognition model; (5) obtaining an image of citrus Huanglong disease to be identified and preprocessing it, inputting the preprocessed image of citrus Huanglong disease to be identified into a trained citrus Huanglong disease recognition model to obtain a citrus Huanglong disease recognition result; In step (1), step (4) and step (5), the preprocessing refers to deleting unclear images, performing random rotation, random cropping and image enhancement, and the random cropping refers to cropping the image to a size of 224×224; In step (2), the dataset is divided into training set, validation set and test set in a ratio of 8:1:1; In step (3), the CNN branch model includes a seven-layer structure, namely: The first layer is a 7×7 convolution. The input image size of the first layer is 224×224×3, and the output feature map size of the first layer is 112×112×64. The second layer is a maximum pooling layer. The input feature map size of the second layer is 112×112×64, and the output feature map size of the second layer is 56×56×64. The third layer is the first cascaded residual module, which includes two branches, one of which is composed of 1×1 convolution, 3×3 convolution, and 1×1 convolution connected in series in sequence, and the branch is connected in parallel with a residual edge branch. The two branch feature maps are added, the input feature map size of the third layer is 56×56×64, and the output feature map size of the third layer is 56×56×256; The fourth layer is a second cascaded residual module, and the second cascaded residual module includes two branches, one of which is composed of a 1×1 convolution, a 3×3 convolution, and a 1×1 convolution in series, and the branch is connected in parallel with a residual edge branch with a 1×1 convolution layer. The two branch feature maps are added, the input feature map size of the fourth layer is 56×56×256, and the output feature map size of the fourth layer is 56×56×256; The structure of the fifth layer is the same as that of the fourth layer. The input feature map size of the fifth layer is 56×56×256, and the output feature map size of the fifth layer is 14×14×1024. The sixth layer is a 1×1 convolution, the input feature map size of the sixth layer is 14×14×1024, and the output feature map size of the sixth layer is 14×14×320; The seventh layer is an adaptive average pooling layer; In step (3), the ViT branch model includes seven layers, namely: The first layer consists of a 3×3 convolution for dimensionality reduction and a MV2 module for feature extraction in series; The second layer consists of a MV2 module with a step size of 1 and a MV2 module with a step size of 2 connected in series; The third layer consists of a MV2 module with a stride of 2 and a lightweight attention module in series; The structures of the fourth and fifth floors are the same as the third floor; The sixth layer is a convolution with a convolution kernel of 1×1; The seventh layer is an adaptive average pooling layer; The step (4) specifically comprises the following steps: (4a) Perform forward propagation: input the preprocessed training set and perform forward propagation through the citrus Huanglong disease image recognition model; (4b) Calculate the loss according to the loss function: Among them, y i The i-th element in the probability distribution vector representing the true label, p i represents the i-th element in the predicted probability distribution vector of the citrus Huanglong disease image recognition model, N represents the number of elements; λ is the regularization parameter used to control the strength of regularization; w represents the weight parameter; (4c) Based on the loss results, back propagate and update the parameters; (4d) Repeat steps (4a) to (4c) until the loss converges and the weight with the best prediction effect is obtained; The MV2 module includes a down-projection convolution with a convolution kernel size of 1×1, a depth convolution with a convolution kernel size of 3×3, and an up-projection convolution with a convolution kernel size of 1×1. The input dimension is reduced by the down-projection convolution, features are extracted by the depth convolution, and the dimension is restored by the up-projection convolution. A residual edge branch is introduced before the down-projection convolution and after the up-projection convolution. The lightweight attention module includes two 3×3 convolutions, two 1×1 convolutions, a Transformer module and a splicing module. First, the feature map is locally modeled through a 3×3 convolution, and then the number of channels is adjusted through a 1×1 convolution. Then, the global feature modeling is performed through the Transformer module, and then the number of channels is adjusted back to the original size through a convolution with a convolution kernel size of 1×1. Then, it is spliced ​​with the original input feature map through the residual edge branch, and finally, a 3×3 convolution is performed to perform feature fusion to obtain the output.

Images