Tomato disease detection method and system based on hybrid self-attention mechanism

Abstract

This invention discloses a method and system for detecting tomato diseases based on a hybrid self-attention mechanism. The method includes inputting an image of a tomato leaf disease to be identified into a PACA-ConvNeXt model. The PACA-ConvNeXt model includes a ConvNeXt convolutional neural network. PACA attention mechanism modules are inserted into each level of the ConNeXt Block module of the ConvNeXt convolutional neural network. The PACA attention mechanism module includes pixel attention blocks and coordinate attention blocks. For the input feature map, each level of the ConNeXt Block module calculates the pixel correlation and coordinate information correlation of the feature map based on the PACA attention mechanism module to obtain a hybrid attention feature map containing pixel correlation and coordinate information correlation. Based on the hybrid attention feature map output by the last level of the ConNeXt Block module, the type of tomato disease is predicted.

Description

Technical Field

[0001] This invention relates to the field of machine learning technology, specifically to a method and system for detecting tomato diseases based on a hybrid self-attention mechanism. Background Technology

[0002] Currently, the main method for identifying tomato diseases in my country is through manual testing. However, in the early stages of plant diseases, lesions mostly appear on leaves and have similar characteristics. Due to the similarity of lesion characteristics, lack of personal experience, and the influence of the surrounding environment, growers are prone to misjudging the type of disease when using manual testing. This can lead to problems such as farmers using pesticides improperly or missing the best time to apply them. Therefore, it is very important to be able to quickly and accurately identify the types of tomato diseases in the early stages of disease.

[0003] With the rise of deep learning technology, many excellent convolutional neural network (CNN) models have emerged, such as LeNet, AlexNet, VGGNet, and GoogleNet. These classic CNN models allow for direct feature extraction, and computer vision has been widely applied in agriculture. However, these models have certain limitations, such as requiring high-precision image acquisition equipment, uniform lighting, and uncomplicated image backgrounds, making them difficult to extend to real-world production. Furthermore, tomato diseases are characterized by complex backgrounds, rich textures, and small differences in features between different diseases. Combined with the limitations of traditional classic CNN models in feature extraction, they cannot fully extract the characteristic information of tomato diseases during detection.

[0004] In related technologies, patent application CN113963279A proposes a system for identifying rice diseases using a lightweight attention network. This system uses a lightweight network as the backbone and incorporates an attention mechanism to learn the importance of input feature channels and spatial points, thereby identifying rice diseases in rice images. This scheme utilizes channel attention, focusing on information interaction across channels. By weighting features along the channel dimension, it retains important feature information.

[0005] While channel attention mechanisms excel at uncovering correlations between different feature channels and effectively enhancing the expressive power of deep learning models, they primarily focus on global information, potentially neglecting local image structure and spatial relationships. This is particularly crucial in tasks involving spatial information, such as crop disease identification. For instance, the morphology of certain diseases (e.g., the shape, size, and arrangement of spots) may contain important spatial relationship information. While attention mechanisms mentioned in relevant literature consider spatial and channel information, they may not emphasize pixel-level importance or adequately consider coordinate context. This can lead to poor model performance when identifying diseases with obvious local features or global distribution characteristics. For example, if a disease is only present in a small area of ​​a leaf, such local details are extremely important. Without pixel-level attention mechanisms, the model may miss this crucial information. Similarly, if the disease distribution on a leaf follows a specific pattern, then this global context becomes critical. Summary of the Invention

[0006] This invention aims to solve the problem of low accuracy in identifying tomato leaf diseases in complex environments.

[0007] The present invention solves the above-mentioned technical problems through the following technical means:

[0008] This invention proposes a method for detecting tomato diseases based on a hybrid self-attention mechanism, the method comprising:

[0009] The tomato leaf disease image to be identified is input into the trained PACA ConvNeXt model, wherein the PACAConvNeXt model includes a ConvNeXt convolutional neural network, and PACA attention mechanism modules are inserted into each level of the ConNeXtBlock module of the ConvNeXt convolutional neural network. The PACA attention mechanism module includes pixel attention blocks and coordinate attention blocks.

[0010] For each ConNeXt Block module, the pixel correlation and coordinate information correlation of the input feature map are calculated based on the PACA attention mechanism module to obtain a hybrid attention feature map containing pixel correlation and coordinate information correlation.

[0011] Based on the hybrid attention feature map output by the last-level ConNeXt Block module, the types of tomato diseases are predicted.

[0012] Furthermore, the ConvNeXt convolutional neural network includes an image processing layer, a first-level ConNeXt Block module, a second-level ConNeXt Block module, a third-level ConNeXt Block module, a fourth-level ConNeXt Block module, and an output prediction layer connected in sequence.

[0013] The second-level ConNeXt Block module, the third-level ConNeXt Block module, and the fourth-level ConNeXt Block module are all connected to the downsampling layer.

[0014] Furthermore, the image processing layer includes a convolutional layer Conv2d and a normalization layer Layer Norm_1. The input of the convolutional layer Conv2d is the tomato leaf disease image, and its output is connected to the input of the normalization layer Layer Norm_1. The output of the normalization layer Layer Norm_1 is connected to the first-level ConNeXt Block module.

[0015] Furthermore, the output prediction layer includes a global average pooling layer, a normalization layer LayerNorm_2, and a fully connected layer connected in sequence. The input of the global average pooling layer is connected to the four-level ConNeXt Block module, and the output of the fully connected layer is the classification result of tomato diseases.

[0016] Furthermore, each ConNeXt Block module includes a depthwise separable convolutional layer, a normalization layer Batch Norm_1, a convolutional layer Conv2d_1, a first activation function, a convolutional layer Conv2d_2, a DropPath layer, and a PACA attention mechanism module connected in sequence. The output of the PACA attention mechanism module is connected to the feature map input to the ConNeXt Block module via a concatenation operation.

[0017] Furthermore, the pixel attention block includes a second activation function and two convolutional kernels for performing linear transformations. The outputs of the two convolutional kernels are multiplied and concatenated before being passed through the second activation function to obtain the pixel attention feature weights.

[0018] The coordinate attention block includes global average pooling kernels for encoding features in the horizontal and vertical directions, respectively. A pair of orientation-aware feature maps output from the two global average pooling kernels are concatenated and then output to the convolutional layer Conv2d_3. The output of Conv2d_3 is sequentially passed through a normalization layer Batch Norm_2, a non-linear activation function, and a split operation to obtain two independent feature maps. These two independent feature maps are then passed through a convolutional layer Conv2d_4 and a third activation function to convert their dimensions to match the features input to the current ConNeXt Block module. Figure 1 The two independent feature maps after transformation are multiplied and then connected to obtain the coordinate attention feature weights.

[0019] Furthermore, for the input feature map, each ConNeXt Block module calculates the pixel correlation and coordinate information correlation of the feature map based on the PACA attention mechanism module, resulting in a feature map containing pixel correlation and coordinate information correlation, including:

[0020] The input feature map is processed using pixel attention blocks to obtain pixel correlations, expressed by the formula:

[0021]

[0022] In the formula, p ij Indicates pixel F i and F j Correlation between them; z ij =q(F i ) T k(F j ), q(F)=W q F,k(F)=W k F, W q and W k The weight matrix, This represents the number of channels after reducing the original number of channels C; F represents the input feature map; exp() represents the exponential function used to represent the power of e; A is the total number of pixels.

[0023] The input feature map is processed using coordinate attention blocks to obtain the attention feature weights for the c-th channel, expressed by the formula:

[0024]

[0025] In the formula, The value of F is obtained by processing the input feature map F through coordinate attention in the c-th channel; c (i,j) represents the value of the c-th channel of the input feature map; Let i be the channel attention weight in the i-th row of the c-th channel; Let be the channel attention weight in the j-th column of the c-th channel; i is the row index of the pixel in the c-th channel of the feature map F; j is the column index of the pixel in the c-th channel of the feature map F.

[0026] Based on the pixel correlation and the attention feature weights of each channel, the hybrid attention weight matrix is ​​calculated, as expressed by the formula:

[0027]

[0028] In the formula, B = (p ij ) (W×H)×(W×H) This represents the pixel attention feature matrix, where W is the channel width and H is the channel height. This represents the coordinate attention feature matrix, which is converted to C×N dimensions before matrix multiplication, where N = H×W, and C is the number of channels in the input feature map. This is a mixed attention weight matrix;

[0029] The hybrid attention weight matrix is ​​multiplied by the feature map input from the current level ConNeXt Block module to obtain the hybrid attention feature map, expressed as follows:

[0030]

[0031] In the formula, X is the feature map input to the current level ConNeXt Block module; β is a hyperparameter with a value of 0.5; and ο is the hybrid attention feature.

[0032] Furthermore, the formula for the hybrid attention feature map output by each ConNeXt Block module is as follows:

[0033]

[0034] In the formula, X is the feature map input to the current level ConNeXt Block module; C dw It is a depth-separable convolutional layer; For the normalization layer Batch Norm_1; For the attention mechanism module PACA; For DropPath layer, C1 represents the activation function; C2 is the convolutional layer Conv2d_1; C2 is the convolutional layer Conv2d_2.

[0035] Furthermore, before inputting the tomato leaf disease image to be identified into the trained PACA ConvNeXt model, the method further includes:

[0036] Obtain the tomato leaf disease dataset;

[0037] The PACA ConvNeXt model was trained using the tomato leaf disease dataset, and the loss between the true values ​​and the predicted values ​​output by the model was calculated. The loss calculation formula is expressed as follows:

[0038]

[0039] In the formula, M is the number of samples in each category; y ic p is a symbolic function; ic Let N be the probability that observed sample i belongs to class c; N is the total number of classes; and L is the loss.

[0040] When the loss between the true value and the predicted value meets expectations, the configuration parameters corresponding to the PACA ConvNeXt model are taken as the optimal configuration parameters to obtain the trained PACA ConvNeXt model.

[0041] Furthermore, this invention also proposes a tomato disease detection system based on a hybrid self-attention mechanism, the system comprising:

[0042] The input module is used to input the tomato leaf disease image to be identified into the trained PACA ConvNeXt model, wherein the PACA ConvNeXt model includes a ConvNeXt convolutional neural network, and PACA attention mechanism modules are inserted into each level of the ConNeXt Block module of the ConvNeXt convolutional neural network. The PACA attention mechanism module includes pixel attention blocks and coordinate attention blocks.

[0043] The image processing module is used to calculate the pixel correlation and coordinate information correlation of the input feature map based on the PACA attention mechanism module in each ConNeXt Block module, so as to obtain a hybrid attention feature map containing pixel correlation and coordinate information correlation.

[0044] The prediction module is used to predict the types of tomato diseases based on the hybrid attention feature map output by the last-level ConNeXt Block module.

[0045] The advantages of this invention are:

[0046] (1) The PACA ConvNeXt model used in this invention is based on the ConNeXt model. The self-built PACA attention mechanism module is inserted into each level of the ConNeXt Block module in the ConNeXt model. The PACA attention mechanism module includes pixel attention block and coordinate attention block. By parallel computing of pixel and coordinate attention in the input feature mapping, a hybrid attention mechanism is constructed, so that the features of the output feature map are a combination of pixel and coordinate information, which greatly enhances the global information and the attention to feature position information, and effectively improves the accuracy of the model in identifying diseases of crops with complex backgrounds, such as tomatoes. Compared with other identification networks, the network model proposed in this invention can significantly improve the identification accuracy, so that growers can achieve good prevention and control effects by targeting different situations.

[0047] (2) The PACA attention mechanism module first performs a linear transformation on the input feature map using two feature spaces q and v, and then performs convolution operations using 1×1 convolution kernels respectively, thereby reducing the number of channels and reducing the parameters and computational load.

[0048] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0049] Figure 1 This is a flowchart of a tomato disease detection method based on a hybrid self-attention mechanism proposed in an embodiment of the present invention;

[0050] Figure 2 This is a schematic diagram of the forward propagation of the PACA ConvNeXt network model in an embodiment of the present invention;

[0051] Figure 3 This is a network structure diagram of each ConNeXt Block module in this embodiment of the invention;

[0052] Figure 4 This is a network structure diagram of the PACA attention mechanism module in an embodiment of the present invention;

[0053] Figure 5 This is a flowchart of the PACA ConvNeXt model training process in an embodiment of the present invention;

[0054] Figure 6 This is a structural diagram of a tomato disease detection system based on a hybrid self-attention mechanism proposed in an embodiment of the present invention. Detailed Implementation

[0055] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below in conjunction with the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0056] like Figure 1 As shown, the first embodiment of the present invention proposes a method for detecting tomato diseases based on a hybrid self-attention mechanism, the method comprising the following steps:

[0057] S10. Input the tomato leaf disease image to be identified into the trained PACA ConvNeXt model, wherein the PACA ConvNeXt model includes a ConvNeXt convolutional neural network, and PACA attention mechanism modules are inserted into each level of the ConNeXt Block module of the ConvNeXt convolutional neural network. The PACA attention mechanism module includes pixel attention blocks and coordinate attention blocks.

[0058] S20. For the input feature map, each ConNeXt Block module calculates the pixel correlation and coordinate information correlation of the feature map based on the PACA attention mechanism module to obtain a hybrid attention feature map containing pixel correlation and coordinate information correlation.

[0059] S30. Based on the hybrid attention feature map output by the last-level ConNeXt Block module, predict the types of tomato diseases.

[0060] In this embodiment, for the input tomato leaf disease image, in each ConNeXt Block module of the PACA ConvNeXt model, the input feature map is computed in parallel with pixel and coordinate attention to construct a hybrid attention mechanism. This makes the output feature map a combination of pixel and coordinate information, which can simultaneously capture the spatial structure of the image and the correlation between channels. Specifically, the pixel attention mechanism focuses on mining local details in the image, such as the size, color, and shape of the disease; the coordinate attention mechanism focuses on capturing the global contextual information of the image, such as the distribution of the disease on the entire leaf, which helps the model understand the global characteristics of the disease. The PACA attention mechanism emphasizes the importance of pixel-level and coordinate contextual information, greatly enhancing the attention to global information and feature location information, and effectively improving the accuracy of the model in identifying diseases in crops with complex backgrounds, such as tomatoes.

[0061] It should be noted that the ConvNeXt convolutional neural network improves the efficiency and accuracy of feature extraction by using grouped convolution and cross-channel pooling, which can effectively capture the features of tomato diseases. This embodiment is based on the ConvNeXt convolutional neural network, and PACA attention mechanism modules are inserted into its ConNeXt Block modules at each level. This allows the new network model to better distinguish different disease types and improve accuracy in its response and extraction of different disease features.

[0062] It should be understood that those skilled in the art can also select other convolutional neural networks as a basis, add a self-designed PACA attention mechanism module, and construct a network model for tomato disease detection based on the actual situation.

[0063] In one embodiment, such as Figure 2 As shown, the ConvNeXt convolutional neural network includes an image processing layer, a first-level ConNeXt Block module, a second-level ConNeXt Block module, a third-level ConNeXt Block module, a fourth-level ConNeXt Block module, and an output prediction layer connected in sequence.

[0064] The second-level ConNeXt Block module, the third-level ConNeXt Block module, and the fourth-level ConNeXt Block module are all connected to the downsampling layer.

[0065] Furthermore, the image processing layer includes a convolutional layer Conv2d and a normalization layer Layer Norm_1. The input of the convolutional layer Conv2d is the tomato leaf disease image, and its output is connected to the input of the normalization layer Layer Norm_1. The output of the normalization layer Layer Norm_1 is connected to the first-level ConNeXt Block module.

[0066] Furthermore, the output prediction layer includes a global average pooling layer (Global AvgPooling), a normalization layer (Layer Norm_2), and a fully connected layer (FC) connected in sequence. The input of the global average pooling layer (Global Avg Pooling) is connected to the four-level ConNeXt Block module, and the output of the fully connected layer (FC) is the classification result of tomato diseases.

[0067] The specific forward propagation process of the PACA ConvNeXt network model constructed in this embodiment is as follows: Figure 2As shown, the overall network model structure includes one convolutional module, four ConvNeXt Attention Block convolutional modules, three downsampling modules, one global average pooling layer, two LN layers, and one fully connected layer. The construction of the ConvNeXt Attention Block convolutional module is as follows: Figure 3 As shown, the PACA attention mechanism module in the ConvNeXt Attention Block convolutional module is as follows: Figure 4 As shown.

[0068] In one embodiment, such as Figure 3 As shown, each level of the ConNeXt Block module includes a depthwise separable convolutional layer Depth Wise Conv2d, a normalization layer Batch Norm_1, a convolutional layer Conv2d_1, a first activation function, a convolutional layer Conv2d_2, a DropPath layer, and a PACA attention mechanism module connected in sequence. The output of the PACA attention mechanism module is connected to the feature map input to the ConNeXt Block module through a concatenation operation.

[0069] Specifically, the workflow of each ConNeXt Block module is as follows: Input feature map X → Extract features using a depthwise separable convolutional layer (Depth Wise Conv2d) with a kernel size of 7×7 → Pass through a normalization layer (Batch Norm_1) → Pass through a convolutional layer (Conv2d_1) with a kernel size of 1×1 → Pass through the first activation function (using the GELU activation function) → Increase the number of channels using a convolutional layer (Conv2d_2) with a kernel size of 1×1 → Obtain the feature map → Extract feature weights using the hybrid attention mechanism module (PACA) → Multiply the feature map with the hybrid attention feature weights → Obtain the hybrid attention feature map → Perform regularization through a DropPath layer → Add the obtained features to the original feature F → Obtain the hybrid attention feature map.

[0070] Specifically, the formula for the hybrid attention feature map output by each ConNeXt Block module is as follows:

[0071]

[0072] In the formula, X is the feature map input to the current level ConNeXt Block module; C dw It is a depth-separable convolutional layer; For the normalization layer Batch Norm_1; For the attention mechanism module PACA; For DropPath layer, C1 represents the activation function; C2 is the convolutional layer Conv2d_1; C2 is the convolutional layer Conv2d_2.

[0073] In one embodiment, such as Figure 4 As shown, the pixel attention block includes a second activation function and two convolutional kernels for linear transformation. The outputs of the two convolutional kernels are multiplied and concatenated, and then passed through the second activation function to obtain the pixel attention feature weights.

[0074] The input feature map F is first linearly transformed using two feature spaces q and v, and then convolved using 1×1 kernels. By reducing the number of channels, the parameters and computational cost are reduced, resulting in two sets of weight matrices. The two sets of weight matrices are then multiplied, and finally the normalized result is calculated using the second activation function (using the Softmax activation function) to obtain the pixel attention feature weights.

[0075] It's important to clarify that pixel attention focuses on information at the individual pixel level, emphasizing the importance of each pixel within the image. Spatial attention, on the other hand, refers to the spatial relationships between pixels, considering not only the information of individual pixels but also their interactions with surrounding pixels. Pixel attention is highly effective for processing images rich in detail, capturing subtle but impactful changes. Spatial attention, however, can capture a wide range of contextual information, making it invaluable for understanding the overall structure of an image.

[0076] In one embodiment, the coordinate attention block includes a 1×1 convolutional kernel, global average pooling kernels for encoding horizontal and vertical features respectively, and a pair of directionally known feature maps output by the two global average pooling kernels are concatenated and output to the convolutional layer Conv2d_3. The output of the convolutional layer Conv2d_3 is sequentially passed through a normalization layer BatchNorm_2, a non-linear activation function, and a split operation to obtain two independent feature maps. These two independent feature maps are then passed through a convolutional layer Conv2d_4 and a third activation function to convert their dimensions to match the features input to the current level ConNeXt Block module. Figure 1 The two independent feature maps after transformation are multiplied and then connected to obtain the coordinate attention feature weights.

[0077] Specifically, the input feature map F is first encoded using 1D global flat pooling kernels (H,1) and (1,W) to encode the horizontal and vertical features respectively (corresponding to...). Figure 4The X Avg Pooling and Y Avg Pooling processes are used to concatenate the resulting pair of directional feature maps. Then, a 1×1 convolutional layer Conv2d_3, a normalization layer Batch Norm_2, and a non-linear activation function Non Linear are used for feature transformation. Subsequently, a Split operation is performed to divide the feature maps into two independent feature maps. The feature maps are then transformed using two more 1×1 convolutional layers Conv2d_4 and a third activation function (using the Sigmoid function) to make their dimensions consistent with the input. Finally, they are merged into coordinate attention feature weights.

[0078] Coordinate attention mechanisms offer a better understanding of positional relationships and relative positions of objects in images, capturing global contextual information. They also provide richer information, including spatial contextual information, global contextual information, and fine-grained feature weights.

[0079] In one embodiment, step S20: For the input feature map, each ConNeXt Block module calculates the pixel correlation and coordinate information correlation of the feature map based on the PACA attention mechanism module to obtain a feature map containing pixel correlation and coordinate information correlation, including the following steps:

[0080] The input feature map is processed using pixel attention blocks to obtain pixel attention feature weights, expressed by the following formula:

[0081]

[0082] In the formula, p ij Indicates pixel F i and F j Correlation between them; z ij =q(F i ) T k(F j ), q(F)=W q F,k(F)=W k F, W q and W k The weight matrix, This represents the number of channels after reducing the original number of channels C; F represents the input feature map; exp() represents the exponential function used to represent the power of e; A is the total number of pixels.

[0083] The input feature map is processed using coordinate attention blocks to obtain the attention feature weights for the c-th channel, expressed by the formula:

[0084]

[0085] In the formula, The value of F is obtained by processing the input feature map F through coordinate attention in the c-th channel; c (i,j) represents the value of the c-th channel of the input feature map; Let i be the channel attention weight in the i-th row of the c-th channel; Let be the channel attention weight in the j-th column of the c-th channel; i is the row index of the pixel in the c-th channel of the feature map F; j is the column index of the pixel in the c-th channel of the feature map F.

[0086] Based on the pixel correlation and the attention feature weights of each channel, the hybrid attention weight matrix is ​​calculated, as expressed by the formula:

[0087]

[0088] In the formula, B = (p ij ) (W×H)×(W×H) This represents the pixel attention feature matrix, where W is the channel width and H is the channel height. This represents the coordinate attention feature matrix, which is converted to C×N dimensions before matrix multiplication, where N = H×W, and C is the number of channels in the input feature map. This is a mixed attention weight matrix;

[0089] The hybrid attention weight matrix is ​​multiplied by the feature map input from the current level ConNeXt Block module to obtain the hybrid attention feature map, expressed as follows:

[0090]

[0091] In the formula, X is the feature map input to the current level ConNeXt Block module; β is a hyperparameter with a value of 0.5; and ο is the hybrid attention feature map.

[0092] In one embodiment, such as Figure 5 As shown, in step S10: before inputting the tomato leaf disease image to be identified into the trained PACA ConvNeXt model, the method further includes the following steps:

[0093] Obtain the tomato leaf disease dataset;

[0094] The PACA ConvNeXt model was trained using the tomato leaf disease dataset, and the loss between the true values ​​and the predicted values ​​output by the model was calculated. The loss calculation formula is expressed as follows:

[0095]

[0096] In the formula, M is the number of samples in each category; y ic p is a symbolic function;ic Let N be the probability that observed sample i belongs to class c; N is the total number of classes; and L is the loss.

[0097] When the loss between the true value and the predicted value meets expectations, the configuration parameters corresponding to the PACA ConvNeXt model are taken as the optimal configuration parameters to obtain the trained PACA ConvNeXt model.

[0098] In this embodiment, tomato disease images are acquired as a dataset. The dataset is then divided into training, validation, and test sets and preprocessed. Finally, a PACA-ConNext model is constructed, and the extracted features are input into a fully connected layer to obtain prediction results. The model is then updated using the cross-entropy loss function to obtain the optimal configuration parameters. The trained model is then used to identify the tomato diseases and pests to be identified, and the identification results are obtained.

[0099] Specifically, the process of training the PACA ConvNeXt model and performing image detection is as follows:

[0100] (1) Preprocessing of tomato leaf disease image data:

[0101] Step 1.1, Obtain the tomato leaf disease dataset:

[0102] Tomato disease data was collected using Python web scraping, public datasets, and high-definition camera images. The collected tomato image data was manually labeled according to the types of tomato diseases. The dataset contains 10 types of diseases. The dataset was divided into training, validation, and test sets.

[0103] Step 1.2: In order to expand the dataset, the training set data is preprocessed, including random color jitter, random addition of Gaussian noise, random flipping, image resizing and normalization. The dataset is divided into training set, validation set and test set in a ratio of 8:1:1 to train the PACA-ConvNeXt model.

[0104] (2) Constructing the PACA-ConvNeXt model:

[0105] Step 2.1: The tomato leaf disease image obtained in Step 1.2 is used as the network input. The size of a single image is 224×224×3. A convolutional layer with a kernel size of 4×4 and a kernel stride of 4 is used, and then a normalized LN layer is passed to obtain the output feature map F2.1 with a dimension of 56×56×96.

[0106] Step 2.2: The feature map F2.1 is input into the first-level ConvNeXt Attention Block module (i.e., the ConNeXt Block module with the PACA attention mechanism inserted) for feature extraction. First, a depthwise separable convolution with a kernel size of 3×3, a stride of 1, and padding of 3 is used. The rest are convolution kernels with a size of 1×1 and a stride of 1. The output feature map F2.2 with dimensions of 56×56×96 is obtained by using a BN layer, a GELU activation function, a PACA attention module, and a DropPath layer.

[0107] Step 2.3 involves performing a 2×2 pooling kernel downsampling operation on the feature map F2.2. The sampled feature map is then input into the second-level ConvNeXt Attention Block module for feature extraction. First, a depthwise separable convolution with a kernel size of 3×3, a stride of 1, and padding of 3 is used. The remaining convolutions use 1×1 kernels with a stride of 1. A BN layer, a GELU activation function, a PACA attention module, and a DropPath layer are used to obtain the output feature map F2.3 with dimensions of 28×28×192.

[0108] Step 2.4: Perform a downsampling operation on the feature map F2.3 with a pooling kernel size of 2×2. Then, input the sampled feature map into the third-level ConvNeXt Attention Block module for feature extraction. First, use a depthwise separable convolution with a kernel size of 3×3, a stride of 1, and padding of 3. Then, use a 1×1 convolution kernel with a stride of 1. Use BN layer, GELU activation function, PACA attention module, and DropPath layer to obtain the output feature map F2.4 with a dimension of 14×14×384.

[0109] Step 2.5: Perform a downsampling operation on the feature map F2.4 with a pooling kernel size of 2×2. Then, input the sampled feature map into the fourth-level ConvNeXt Attention Block module for feature extraction. First, use a depthwise separable convolution with a kernel size of 3×3, a stride of 1, and padding of 3. Then, use a 1×1 convolution kernel with a stride of 1. Use BN layer, GELU activation function, PACA attention module, and DropPath layer to obtain the output feature map F2.5 with a dimension of 7×7×768.

[0110] Step 2.6: Perform global average pooling and LN operations on feature map F2.5 to obtain output feature map 2.6 with dimensions of 1×1×37632.

[0111] Step 2.7: The feature map F2.6 is passed through a fully connected layer and a Softmax activation function to obtain the output classification result Out, as shown in the following formula:

[0112]

[0113] Where: is a fully connected operation with an output dimension of 10, and Softmax is the activation function.

[0114] (3) Use the cross-entropy loss function to calculate the loss between the predicted value and the true value output in step 2.7. Use the constructed loss function to train the PACA ConvNeXt model, obtain the model parameters that make the test set the most accurate, and save them.

[0115] (4) Input the tomato leaf disease image to be identified into the trained PACA ConvNeXt model to realize the identification of tomato leaf disease image category.

[0116] Furthermore, the specific workflow of the PACA attention mechanism module is as follows:

[0117] (1) Constructing pixel attention: using two feature spaces q and k to pair the input image A linear transformation is performed as shown in the formula below to reduce the computational complexity of the model.

[0118] q(x)=W q x,k(x)=W k x

[0119] in, The weight matrix is ​​learned through 1×1 convolution operation. This represents the number of channels after reducing the original number of channels C. Represents real numbers. This reduces the number of parameters and computational cost without affecting the model's ability to extract features;

[0120] Two pixels x i and x j The correlation between them can be calculated by multiplying them using the following formula:

[0121] r ij =q(x i ) T k(x j )

[0122] Finally, the normalization result can be calculated using the Softmax function.

[0123]

[0124] Where, p ij Indicates pixel F iand F j The correlation between them.

[0125] Pixel attention B by p ij Arranged according to their corresponding positions:

[0126] B = (p ij ) (W×H)×(W×H)

[0127] (2) Constructing coordinate attention: Given a feature x, each channel is encoded along the horizontal and vertical coordinates using two spatially defined pooling kernels (H,1) and (1,W), respectively. Therefore, the output of the c-th channel at height H can be formulated as follows: The output of the c-th channel with width W can be written as:

[0128]

[0129]

[0130] In the formula, This represents the output of the c-th channel in a specific height direction; This represents the output of the c-th channel in the specific width direction; x c represents the input of channel c; h represents the position index in the height direction, with values ​​from 1 to H; w represents the position index in the width direction, with values ​​from 1 to W; i represents the position index in the width direction, with values ​​from 1 to W; j represents the position index in the height direction, with values ​​from 1 to H; W represents the width of the pooling core; and H represents the height of the pooling core.

[0131] The generated aggregated feature maps are then concatenated and passed to a shared 1×1 convolutional transform function F1 to obtain...

[0132] f=δ(F1([z h ,z w ]))

[0133] Where f = [] represents the concat operation of spatial dimensions, and δ is a non-linear activation function. It refers to the intermediate feature mapping in the horizontal and vertical directions.

[0134] Then, f is split along the spatial dimension into two independent tensors. The other two 1×1 convolutions are used to transform the input into tensors with the same number of channels, resulting in:

[0135] g h =σ(F h (f h ))

[0136] g w =σ(Fw (f w ))

[0137] Where σ is the Sigmoid function.

[0138] Finally, the output of the coordinate attention block can be written as:

[0139]

[0140] After obtaining the coordinate information correlation and pixel correlation respectively, the two types of attention information are fused into the hybrid module by multiplying the correlation matrix of the pixel attention module with the correlation matrix of the channel attention module.

[0141] First, The dimension is transformed Where N = H × W. The purpose of this transformation is to obtain a new feature map through matrix multiplication with the pixel correlation matrix a:

[0142]

[0143] Then, The calculated values ​​are then transformed back to the original dimensions. This yields a result containing the correlation between pixel and coordinate information. Finally, the concept of a residual network is introduced to multiply the newly obtained Feature Map by β. Then, the original input Feature Map is added as the final output, as shown below:

[0144]

[0145] This involves implementing attention operations on the input feature map of the PACA attention mechanism module in terms of coordinates and spatial pixels.

[0146] In addition, such as Figure 6 As shown, the second embodiment of the present invention proposes a tomato disease detection system based on a hybrid self-attention mechanism, the system comprising:

[0147] Input module 10 is used to input the tomato leaf disease image to be identified into the trained PACA ConvNeXt model, wherein the PACA ConvNeXt model includes a ConvNeXt convolutional neural network, and PACA attention mechanism modules are inserted into each level of the ConNeXt Block module of the ConvNeXt convolutional neural network. The PACA attention mechanism module includes pixel attention blocks and coordinate attention blocks.

[0148] Image processing module 20 is used to calculate the pixel correlation and coordinate information correlation of the input feature map based on the PACA attention mechanism module in each ConNeXt Block module, so as to obtain a hybrid attention feature map containing pixel correlation and coordinate information correlation.

[0149] Prediction module 30 is used to predict the types of tomato diseases based on the hybrid attention feature map output by the last-level ConNeXt Block module.

[0150] In one embodiment, the system further includes a pre-training module, specifically used for:

[0151] Obtain the tomato leaf disease dataset;

[0152] The PACA ConvNeXt model was trained using the tomato leaf disease dataset, and the loss between the true values ​​and the predicted values ​​output by the model was calculated. The loss calculation formula is expressed as follows:

[0153]

[0154] In the formula, M is the number of samples in each category; y ic p is a symbolic function; ic Let N be the probability that observed sample i belongs to class c; N is the total number of classes; and L is the loss.

[0155] When the loss between the true value and the predicted value meets expectations, the configuration parameters corresponding to the PACA ConvNeXt model are taken as the optimal configuration parameters to obtain the trained PACA ConvNeXt model.

[0156] It should be noted that other embodiments or implementation methods of the tomato disease detection system based on a hybrid self-attention mechanism described in this invention can refer to the above-described method embodiments and... Figure 2 , Figure 3 and Figure 4 The network structure shown is not repeated here.

[0157] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0158] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0159] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A method for detecting tomato diseases based on a hybrid self-attention mechanism, characterized in that, The method includes: The tomato leaf disease images to be identified are input into the trained PACA. The ConvNeXt model, wherein the PACA The ConvNeXt model includes a ConvNeXt convolutional neural network, in which a PACA attention mechanism module is inserted into each ConNeXtBlock module of the ConvNeXt convolutional neural network. The PACA attention mechanism module includes pixel attention blocks and coordinate attention blocks. For each ConNeXt Block module, the pixel correlation and coordinate information correlation of the input feature map are calculated based on the PACA attention mechanism module to obtain a hybrid attention feature map containing pixel correlation and coordinate information correlation. Based on the hybrid attention feature map output by the last-level ConNeXt Block module, the types of tomato diseases are predicted; For each ConNeXt Block module, the input feature map is processed by the PACA attention mechanism module to calculate the pixel correlation and coordinate information correlation of the feature map, resulting in a hybrid attention feature map containing both pixel correlation and coordinate information correlation, including: The input feature map is processed using pixel attention blocks to obtain pixel attention feature weights, expressed by the following formula: In the formula, Represents pixels and The correlation between them; , , and The weight matrix, , , This indicates a reduction in the original number of channels. The number of channels after that, F The input feature map is represented by exp(); exp() represents the exponential function, used to represent the power of e. A Total number of pixels; The input feature map is processed using coordinate attention blocks to obtain the attention feature weights for the c-th channel, expressed by the formula: In the formula, Input feature map F The value obtained by processing the c-th channel through coordinate attention; The value of the c-th channel of the input feature map; For the c-th channel i Channel attention weights for rows; For the c-th channel j Channel attention weights for columns; i For feature map F The row index of the pixel in the c-th channel; j For feature map F The column index of the pixel in the c-th channel; Based on the pixel correlation and the attention feature weights of each channel, the hybrid attention weight matrix is ​​calculated, as expressed by the formula: In the formula, Represents the pixel attention feature matrix. W For channel width, H For the channel height, Represents the coordinate attention feature matrix. This is a hybrid attention weight matrix; The hybrid attention weight matrix is ​​multiplied by the feature map input from the current level ConNeXt Block module to obtain the hybrid attention feature map, expressed as follows: In the formula, X The feature map input to the current level ConNeXt Block module; For hyperparameters; This is a hybrid attention feature.

2. The tomato disease detection method based on a hybrid self-attention mechanism as described in claim 1, characterized in that, The ConvNeXt convolutional neural network includes an image processing layer, a first-level ConNeXt Block module, a second-level ConNeXt Block module, a third-level ConNeXt Block module, a fourth-level ConNeXt Block module, and an output prediction layer connected in sequence. The second-level ConNeXt Block module, the third-level ConNeXt Block module, and the fourth-level ConNeXt Block module are all connected to the downsampling layer.

3. The tomato disease detection method based on a hybrid self-attention mechanism as described in claim 2, characterized in that, The image processing layer includes a convolutional layer Conv2d and a normalization layer Layer Norm_1. The input of the convolutional layer Conv2d is the tomato leaf disease image, and the output is connected to the input of the normalization layer Layer Norm_1. The output of the normalization layer Layer Norm_1 is connected to the first-level ConNeXt Block module.

4. The tomato disease detection method based on a hybrid self-attention mechanism as described in claim 2, characterized in that, The output prediction layer includes a global average pooling layer, a normalization layer (Layer Norm_2), and a fully connected layer connected in sequence. The input of the global average pooling layer is connected to the four-level ConNeXt Block module, and the output of the fully connected layer is the classification result of tomato diseases.

5. The tomato disease detection method based on a hybrid self-attention mechanism as described in claim 1, characterized in that, Each ConNeXt Block module includes a depthwise separable convolutional layer, a normalization layer Batch Norm_1, a convolutional layer Conv2d_1, a first activation function, a convolutional layer Conv2d_2, a DropPath layer, and a PACA attention mechanism module connected in sequence. The output of the PACA attention mechanism module is concatenated with the feature map input to the ConNeXt Block module.

6. The tomato disease detection method based on a hybrid self-attention mechanism as described in claim 1, characterized in that, The pixel attention block includes a second activation function and two convolutional kernels for linear transformation. The outputs of the two convolutional kernels are multiplied and concatenated, and then passed through the second activation function to obtain the pixel attention feature weights. The coordinate attention block includes global average pooling kernels for encoding horizontal and vertical features, respectively. The pair of directionally known feature maps output by the two global average pooling kernels are concatenated and then output to the convolutional layer Conv2d_3. The output of the convolutional layer Conv2d_3 is sequentially passed through the normalization layer Batch Norm_2, a non-linear activation function, and a split operation to obtain two independent feature maps. These two independent feature maps are then passed through the convolutional layer Conv2d_4 and a third activation function to convert their dimensions to be consistent with the feature map input to the current level ConNeXt Block module. The two converted independent feature maps are then concatenated through a multiplication operation to obtain the coordinate attention feature weights.

7. The tomato disease detection method based on a hybrid self-attention mechanism as described in claim 5, characterized in that, The formula for the hybrid attention feature map output by each ConNeXt Block module is as follows: In the formula, X is the feature map input to the ConNeXt Block module; It is a depth-separable convolutional layer; For the normalization layer Batch Norm_1; For the attention module PACA; For DropPath layer, Indicates the activation function; It is a convolutional layer Conv2d_1; It is a convolutional layer Conv2d_2.

8. The tomato disease detection method based on a hybrid self-attention mechanism as described in claim 1, characterized in that, The tomato leaf disease images to be identified are input into the trained PACA. Prior to the ConvNeXt model, the method also includes: Obtain the tomato leaf disease dataset; Using the tomato leaf disease dataset to study the PACA The ConvNeXt model is trained, and the loss between the true value and the model's predicted output is calculated. The loss calculation formula is expressed as follows: In the formula, M Number of samples in each category; It is a symbolic function; For observation samples belong The probability of a class; N Total number of categories; L For loss; When the loss between the actual and predicted values ​​matches expectations, PACA will be applied. The configuration parameters corresponding to the ConvNeXt model are used as the optimal configuration parameters to obtain the trained PACA. ConvNeXt model.

9. A tomato disease detection system based on a hybrid self-attention mechanism, characterized in that, The system includes: The input module is used to input the tomato leaf disease images to be identified into the trained PACA. The ConvNeXt model, wherein the PACA The ConvNeXt model includes a ConvNeXt convolutional neural network, and PACA attention mechanism modules are inserted into each ConNeXt Block module of the ConvNeXt convolutional neural network. The PACA attention mechanism modules include pixel attention blocks and coordinate attention blocks. The image processing module is used to calculate the pixel correlation and coordinate information correlation of the input feature map based on the PACA attention mechanism module in each ConNeXt Block module, so as to obtain a hybrid attention feature map containing pixel correlation and coordinate information correlation. The prediction module is used to predict the types of tomato diseases based on the hybrid attention feature map output by the last-level ConNeXt Block module. For each ConNeXt Block module, the input feature map is processed by the PACA attention mechanism module to calculate the pixel correlation and coordinate information correlation of the feature map, resulting in a hybrid attention feature map containing both pixel correlation and coordinate information correlation, including: The input feature map is processed using pixel attention blocks to obtain pixel attention feature weights, expressed by the following formula: In the formula, Represents pixels and The correlation between them; , , and The weight matrix, , , This indicates a reduction in the original number of channels. The number of channels after that, F The input feature map is represented by exp(); exp() represents the exponential function, used to represent the power of e. A Total number of pixels; The input feature map is processed using coordinate attention blocks to obtain the attention feature weights for the c-th channel, expressed by the formula: In the formula, Input feature map F The value obtained by processing the c-th channel through coordinate attention; The value of the c-th channel of the input feature map; For the c-th channel i Channel attention weights for rows; For the c-th channel j Channel attention weights for columns; i For feature map F The row index of the pixel in the c-th channel; j For feature map F The column index of the pixel in the c-th channel; Based on the pixel correlation and the attention feature weights of each channel, the hybrid attention weight matrix is ​​calculated, as expressed by the formula: In the formula, Represents the pixel attention feature matrix. W For channel width, H For the channel height, Represents the coordinate attention feature matrix. This is a hybrid attention weight matrix; The hybrid attention weight matrix is ​​multiplied by the feature map input from the current level ConNeXt Block module to obtain the hybrid attention feature map, expressed as follows: In the formula, X The feature map input to the current level ConNeXt Block module; For hyperparameters; This is a hybrid attention feature.

Images