Apple leaf disease classification method based on regional perception and lightweight hybrid Transform network

Abstract

The invention discloses an apple leaf disease classification method based on a regional perception and lightweight hybrid Transform network. The method comprises the following steps: firstly, inhibiting interference of a low-frequency background and dynamic illumination by using a region extraction and fine-grained convolutional coding module, and adaptively sensing a leaf region and a disease region; secondly, a lightweight Transform block of a parallel cascade structure is designed to extract multi-scale texture and edge features, and adaptive selection of an optimal feature scale, mining of a long-distance dependency relationship and cross-scale information interaction and fusion are realized in a jump connection mode; then, extracting rich texture detail features of the apple leaf region by adopting feature weighting based on adaptive learning, an inverse residual mechanism and the attention degree of a channel attention enhancement network on region feature information; and finally, the generalization ability of the network is improved by jointly adopting a weighted cross entropy loss function and a focus loss function. Experimental results verify the effectiveness of the method.

Description

Technical Field

[0001] The present invention relates to the technical field of agriculture and computer vision analysis, and in particular to an apple leaf disease classification method based on region perception and a lightweight hybrid Transformer network. Background Art

[0002] my country is the world's largest apple producer and consumer, with the planting area and output accounting for more than 40% of the world's total output. Apple yield and quality have an important impact on fruit farmers' income and fruit exports. However, the growth environment of apple trees is complex and changeable, which makes the prevention and control of leaf diseases a problem that troubles apple growers. At the same time, traditional manual identification methods are not only time-consuming and labor-intensive, but also obviously inaccurate, which can easily lead to the large-scale spread of apple leaf diseases, inevitably causing economic losses to fruit farmers. In this case, accurate identification and classification of apple leaf diseases is of great significance to ensuring the quality and safety of agricultural products and meeting people's growing demand for green agriculture and food safety.

[0003] At present, my country's agricultural modernization is facing an urgent need for digital transformation and upgrading. As an important part of smart agriculture, intelligent disease prevention and control can effectively improve the efficiency and accuracy of disease diagnosis and help fruit farmers take effective prevention and control measures in a timely manner. Therefore, the automatic classification of apple leaf diseases aims to quickly and accurately locate and classify various diseases on apple leaves (such as black spot disease, spot disease, etc.) through digital image processing, computer vision and artificial intelligence technology, so as to achieve real-time monitoring and accurate forecasting of large-scale apple orchards.

[0004] According to the different overall technical frameworks, the existing automatic classification methods for apple leaf diseases mainly include three categories, namely, the apple leaf disease classification method based on traditional image processing and machine learning, the apple leaf disease classification method based on deep learning, and the apple leaf disease classification method based on transfer learning.

[0005] First, the classification method of apple leaf diseases based on traditional image processing and machine learning mainly relies on classic digital image processing technology to extract features such as color, texture and shape of apple leaves, and then uses machine learning algorithms such as support vector machines and decision trees for classification. However, this type of method is highly dependent on feature extraction and performs poorly in complex backgrounds.

[0006] Secondly, the apple leaf disease classification method based on deep learning aims to use deep convolutional neural networks (such as ResNet, VGG, etc.), generative adversarial networks and other models to identify various apple leaf diseases, and has achieved performance significantly better than traditional classification methods. Liu et al. proposed an apple leaf disease recognition method based on AlexNet neural network, which can identify four common apple leaf diseases. Yan et al. used global average pooling layer instead of fully connected layer to reduce parameters, and added batch normalization layer to improve convergence speed, thus proposing an apple leaf disease classification method (AICNN) based on improved VGG16 model, which can handle three common apple leaf diseases: scab, downy mildew and fir rust. Ding et al. used the dual attention mechanism to suppress the interference of complex background on the recognition process, and combined the class balancing technology with the focal loss function to reduce the adverse effects of imbalanced data sets on classification accuracy, and then proposed an apple leaf disease classification method based on RFCA ResNet. Li et al. designed a vegetation index selection method using a saliency attention mechanism, and used the multi-label ReliefF feature selection algorithm to realize the automatic selection of vegetation index, and then constructed an apple disease area multi-label classification model AMMFNet, which can combine the channel attention mechanism to explore the complementarity between RGB and multispectral image data. Li Ling et al. proposed a classification method for apple leaf diseases based on MobileNet by modifying the size of the fully connected layer of the output layer of the MobileNet network model. Shen Hengyu used Keras to build a MobileNetV3 neural network, and then adjusted the network structure to make it suitable for the classification and recognition task of apple leaf diseases. On this basis, a hybrid attention mechanism and a channel attention mechanism were introduced, and then a classification method for apple leaf diseases based on the improved MobileNetV3 model was proposed. Liang Qianqian et al. used Fourier transform to design a learnable filter layer to replace the multi-head attention module of the original MobileViT model, and then used deep convolutional layers and residual structures to enhance the model's feature learning ability for disease images, thus proposing an apple leaf disease classification and recognition method based on the improved lightweight network MobileViT. Although the above method has achieved a high classification accuracy on specific data sets, in practical applications, the image data sets obtained from orchards usually have the characteristics of complex background, unbalanced distribution of disease categories, dynamic scene changes (such as leaf occlusion, fruit occlusion, diverse lighting and weather conditions, etc.), etc., resulting in the insufficient generalization ability of the above method and it is still difficult to meet the needs of practical applications.

[0007] Finally, since the training of deep learning models requires a large amount of data, transfer learning becomes an effective means. Therefore, the apple leaf disease classification method based on transfer learning aims to detect and classify apple leaf diseases using pre-trained models (such as VGG16, ResNet50, Inception V3, etc.) and transfer learning technology. Chen Hao et al. proposed an apple leaf disease image classification method based on transfer learning and VGG19 network through parameter transfer and model fine-tuning, and achieved good accuracy. Zhang et al. introduced the coordinate attention mechanism and weight adaptive multi-scale feature fusion module in the ResNet-50 model to enhance the feature extraction ability of the original ResNet-50 model, and used transfer learning and online data enhancement to improve the generalization ability of the model, thus proposing an apple leaf disease classification method based on the improved ResNet-50 neural network. Zhou Hongwei et al. transferred the model weights trained on ImageNet to a new model with a redesigned output layer, and then used apple leaf disease sample data to train the new model, and disclosed an apple leaf disease classification and recognition method based on transfer learning. However, the data on apple leaf diseases is often limited and lacks diversity, and the lack of sufficient labeled data is likely to cause such methods to suffer from overfitting or poor generalization. At the same time, changes in crop growth conditions (such as light, humidity, etc.) can make certain features unstable in different environments, thus affecting the robustness and adaptability of the migration model.

[0008] In summary, it is still a very challenging task to achieve accurate automatic classification of apple leaf diseases. At present, there is no automatic classification method for apple leaf diseases that can effectively resist complex background and dynamic lighting interference, with high accuracy, good robustness, strong generalization ability, and low computational complexity. Summary of the invention

[0009] According to the technical problems mentioned in the above background technology, a regional perception and lightweight hybrid Transformer network apple leaf disease classification method is provided. The present invention can effectively resist complex background and dynamic lighting interference, and is a regional perception and lightweight hybrid Transformer network apple leaf disease classification method with high accuracy, good robustness, strong generalization ability, low computational complexity.

[0010] The technical means adopted by the present invention are as follows:

[0011] The apple leaf disease classification method based on region perception and lightweight hybrid Transformer network is characterized by comprising the following steps:

[0012] Step 1: Build and initialize a deep convolutional neural network N for apple leaf disease classification apple; The deep convolutional neural network N apple Includes: target area perception subnetwork N obj , image fine-grained feature extraction sub-network N stem , feature fusion sub-network N fusion , 4 image local feature extraction sub-networks, denoted as N local1 、N local2 、N local3 、N local4 , 3 image global feature extraction sub-networks, denoted as N global1 、N global2 、N global3 , 3 local features and global features fusion sub-networks, respectively denoted as N Merge1 、N Merge2 With N Merge3 , channel attention module M ECA And the classification head module M classifier ;

[0013] Step 2: Input the training set of apple leaf disease images and the manually annotated label set C label , for the deep convolutional neural network N apple Perform training to obtain the trained deep convolutional neural network N apple ; The training set has a total of N testImg images, and let the number of iterations iter←1, the counter ind←1;

[0014] Step 3: Input the apple leaf disease image I to be processed, wherein the color space of the apple leaf disease image I to be processed is RGB; according to the trained deep convolutional neural network N obtained in step 2 apple Predict the disease category of the apple leaf disease image I to be processed.

[0015] Furthermore, the step 1 comprises the following steps:

[0016] Step 11: Establish the target area perception subnetwork N obj And initialize it; the target area perception subnetwork N obj Contains: 1 target area extraction module M extraction and a target region encoding module M stride1 ; The M extraction Contains: 1 layer of adaptive contrast enhancement module M conenc , 1-layer image segmentation module M based on maximum inter-class variance seg , 1 layer of morphological closing operation module M based on an elliptical structure element of size (10,10) morph ;

[0017] The Mstride1 Contains 3 groups of convolution operations, namely in, Contains 1 set of 18 convolution kernels of size 1×1, 1 BatchNorm2d data normalization layer, and 1 ReLU6 activation function layer; Contains 1 group of 18 convolution kernels of size 3×3 and group number 18, 1 BatchNorm2d data normalization layer, and 1 ReLU6 activation function layer; Contains 1 set of 32 convolution kernels of size 1×1 and 1 BatchNorm2d layer;

[0018] Step 12: Establish image fine-grained feature extraction subnetwork N stem And initialize it; the image fine-grained feature extraction subnetwork N stem Includes: Convolution layer Conv stem , BatchNorm2d data normalization layer and ReLU6 activation function layer; the convolution layer Conv stem Includes: 32 convolution kernels of size 3×3, each convolution kernel performs convolution operation with a step size of 1 pixel;

[0019] Step 13: Establish feature fusion sub-network N fusion And initialize it; the feature fusion sub-network N fusion Includes: 2 learnable weight coefficients w1 and w2;

[0020] Step 14: Establish image local feature extraction subnetwork N local1 And initialize it; the image local feature extraction subnetwork N local1 It includes: 3 groups of convolution operations; the convolution operations are respectively Said Contains 1 set of 192 convolution kernels of size 1×1; Contains 1 group of 192 convolution kernels of size 3×3 and group number 192, and each convolution kernel performs convolution operation with a step size of 2 pixels, 1 BatchNorm2d data normalization layer, and 1 ReLU6 activation function layer; Contains 1 group of 64 convolution kernels of size 1×1 and 1 BatchNorm2d layer;

[0021] Step 15: Establish image global feature extraction subnetwork N global1 And initialize it; the image global feature extraction subnetwork N global1 Contains 1 set of pooling operations And 1 self-attention module The pooling operation Contains 1 layer of average pooling with a size of 2×2 and a stride of 2, and 1 layer of maximum pooling with a size of 2×2 and a stride of 2;

[0022] Step 16: Establish image local feature extraction subnetwork N local2 And initialize it; the image local feature extraction subnetwork N local2 Contains 3 groups of convolution operations, namely Said Contains 1 set of 384 convolution kernels of size 1×1; Contains 1 group of 384 convolution kernels of size 3×3 and group number 384, and each convolution kernel performs convolution operation with a step size of 2 pixels, 1 BatchNorm2d data normalization layer, and 1 ReLU6 activation function layer; Contains 1 set of 128 convolution kernels of size 1×1 and 1 BatchNorm2d layer;

[0023] Step 17: Establish image global feature extraction subnetwork N global2 And initialize it; the image global feature extraction subnetwork N global2 Contains 1 set of pooling operations With 1 self-attention module Said Contains 1 layer of average pooling with a size of 2×2 and a stride of 2, and 1 layer of maximum pooling with a size of 2×2 and a stride of 2;

[0024] Step 18: Establish image global feature extraction subnetwork N global3 And initialize it; the image global feature extraction subnetwork N global3 Contains 1 set of pooling operations With 1 self-attention module Said Contains 1 layer of average pooling with a size of 2×2 and a stride of 2, and 1 layer of maximum pooling with a size of 2×2 and a stride of 2;

[0025] Step 19: Establish a local feature and global feature fusion sub-network N Merge1 And initialize it; the local feature and global feature fusion sub-network N Merge1 Contains a set of 128 convolution kernels of size 1×1, and each convolution kernel performs convolution operations with a step size of 1 pixel;

[0026] Step 110: Establish an image local feature extraction subnetwork N local3 And initialize it; the image local feature extraction subnetwork N local3 Contains 3 groups of convolution operations, namely Said Contains 1 set of 960 convolution kernels of size 1×1; It contains 1 group of 960 convolution kernels of size 3×3 and group number 960, and each convolution kernel performs convolution operation with a step size of 2 pixels, 1 BatchNorm2d data normalization layer, and 1 ReLU6 activation function layer; Contains 1 set of 128 convolution kernels of size 1×1 and 1 BatchNorm2d layer;

[0027] Step 111: Establish a local feature and global feature fusion sub-network N Merge2 And initialize it; the local feature and global feature fusion sub-network N Merge2 Contains a set of 256 convolution kernels of size 1×1, and each convolution kernel performs convolution operations with a step size of 1 pixel;

[0028] Step 112: Establish an image local feature extraction subnetwork N local4 And initialize it; the image local feature extraction subnetwork N local4 Contains 3 groups of convolution operations, namely Said Contains 1 set of 1920 convolution kernels of size 1×1; Contains 1 group of 1920 convolution kernels of size 3×3 and group number 1920, and each convolution kernel performs convolution operation with a step size of 2 pixels, 1 BatchNorm2d data normalization layer, and 1 ReLU6 activation function layer; Contains 1 set of 256 convolution kernels of size 1×1 and 1 BatchNorm2d layer;

[0029] Step 113: Establish a local feature and global feature fusion sub-network N Merge3 And initialize it; the local feature and global feature fusion sub-network N Merge3 Contains a set of 960 convolution kernels of size 1×1, and each convolution kernel performs convolution operations with a step size of 1 pixel;

[0030] Step 114: Establish channel attention module M ECA And initialize it; the channel attention module M ECA Contains 1 global average pooling layer, 1 adaptive one-dimensional convolution layer, 1 sigmoid activation function layer and 1 matrix Hadamard product layer;

[0031] Step 115: Create a classification head module M classifier And initialize it; the classification head module Mclassifier Contains 1 global average pooling layer and 1 classifier;

[0032] The global average pooling layer pools the feature map into a factor; the classifier includes 1 layer of Hardwish activation function, 1 layer of Dropout layer with a probability of 0.3, and 1 layer of fully connected layer with 960 input units and 4 output units.

[0033] Furthermore, the step 2 comprises the following steps:

[0034] Step 21: Take the indth image T from the training set ind and their corresponding annotated labels

[0035] Step 22: T ind Perform a horizontal flip operation with a probability of 0.4 and a 20° random rotation operation to achieve image T ind Data augmentation;

[0036] Step 23: Perceive sub-network N according to target area obj Processing image T ind , and get the feature map F obj ;

[0037] Step 24: Extract subnetwork N based on image fine-grained features stem Process image T to get feature map F stem ;

[0038] Step 25: Fusion sub-network N according to features fusion Calculate the feature map F fusion ;

[0039] Step 26: Extract subnetwork N based on local image features local1 Processing F fusion , and get the feature map F local1 ;

[0040] Step 27: Extract subnetwork N based on image global features global1 Processing F fusion , and get the feature map F global1 ;

[0041] Step 28: Extract subnetwork N based on local image features local2 Processing F local1 , and get the feature map F local2 ;

[0042] Step 29: Extract subnetwork N based on image global features global2 Processing F local1 , and get the feature map F global2 ;

[0043] Step 210: Extract subnetwork N based on global image features global3 Processing F local2 , and get the feature map F global3 ;

[0044] Step 211: F local2 and F global1 The connection operation is performed along the channel direction, and then the local feature and global feature fusion sub-network N is used. Merge1 Processing is performed to obtain the feature map F merge1 ;

[0045] Step 212: Extract subnetwork N based on local image features local3 Processing F merge1 , and get the feature map F local3 ;

[0046] Step 213: F local3 and F global2 The connection operation is performed along the channel direction, and then the local feature and global feature fusion sub-network N is used. Merge2 Processing is performed to obtain the feature map F merge2 ;

[0047] Step 214: Extract subnetwork N based on local image features local4 Processing F merge2 , and get the feature map F local4 ;

[0048] Step 215: Place F local4 and F global3 The connection operation is performed along the channel direction, and then the local feature and global feature fusion sub-network N is used. Merge3 Processing is performed to obtain the feature map F merge3 ;

[0049] Step 216: According to the channel attention module M ECA According to formula (1), the feature map F merge3 Calculate and get the feature map F ECA ;

[0050]

[0051] Among them, GAP(·) represents the global average pooling operation, Conv 1×1 (·) represents a one-dimensional convolution operation, δ(·) represents the Sigmoid function, represents the matrix Hadamard product;

[0052] Step 217: Using the classification head module M classifier FECA Process and calculate image T ind predicted probability of belonging to each category;

[0053] Step 218: Among the predicted probabilities of each category, select the category with the highest probability as image T ind The predicted category and its probability as image T ind The predicted probability of

[0054] Step 219: Let ind←ind+1, if ind>N testImg , then go to step 220, otherwise return to step 21;

[0055] Step 220: Calculate the prediction loss value L of the training set total :

[0056]

[0057] L total =ζ1×L crossEntropy +ζ2×L focal ;

[0058] The L focal represents the average focus loss value, L crossEntropy represents the average cross entropy loss value, α j and ξ are hyperparameters, k represents the number of disease types included in the training set, and p ij represents the probability that the model predicts that the i-th image in the training set belongs to the j-th disease category, y i represents the true label of the i-th image in the training set, Indicates the yth i The weight coefficient of the inverse frequency weighting of the class disease, Indicates that the model predicts that the i-th image in the training set belongs to the y-th image i The probability of the disease, N j represents the number of images belonging to the jth disease in the training set, N yi Indicates that the training set belongs to the yth i The number of images of the disease class, ζ1 and ζ2 represent the preset weight coefficients;

[0059] Step 221: Let iter←iter+1. If the number of iterations iter>Total_iter, the trained deep convolutional neural network N is obtained. apple , go to step 3, otherwise use the reverse error propagation algorithm based on stochastic gradient descent and the prediction loss L total Update N apple, set the counter ind←1, and go to step 21 to reprocess all images in the training set, where Total_iter represents the preset number of iterations.

[0060] Furthermore, the step 23 further comprises the following steps:

[0061] Step 231: Extract module M according to target area extraction For image T ind Calculate and get T′;

[0062] Step 232: Encode module M according to target area stride1 Convolution operation on T′, we get F T ;

[0063] Step 233: Let the feature map F obj ←T′+F T .

[0064] Further, the step 25 comprises the following steps:

[0065] Step 251: F obj and F stem Perform linear fusion processing to obtain feature F merge ;

[0066] F merge =σ(α1×F obj +α2×F stem );

[0067]

[0068] Among them, σ represents the Softmax function, α1 and α2 represent the normalized weight coefficients;

[0069] Step 252: F merge and F stem Perform residual connection to calculate F fusion .

[0070] Furthermore, the step 3 comprises the following steps:

[0071] Step 31: Perceive sub-network N according to target area obj Process image I to obtain feature map G obj ;

[0072] Step 32: Extract subnetwork N based on image fine-grained features stem Process image I to obtain feature map G stem ;

[0073] Step 33: Fusion sub-network N according to features fusionCalculate the feature map G fusion ;

[0074] Step 34: Extract subnetwork N based on local image features local1 Processing G fusion , get the feature map G local1 ;

[0075] Step 35: Extract subnetwork N based on image global features global1 Processing G fusion , get the feature map G global1 ;

[0076] Step 36: Extract subnetwork N based on local image features local2 Processing G local1 , get the feature map G local2 ;

[0077] Step 37: Extract subnetwork N based on image global features global2 Processing G local1 , get the feature map G global2 ;

[0078] Step 38: Extract subnetwork N based on image global features global3 Processing G local2 , get the feature map G global3 ;

[0079] Step 39: Place G local2 and G global1 The connection operation is performed along the channel direction, and then the local feature and global feature fusion sub-network N is used. Merge1 Processing is performed to obtain the feature map G merge1 ;

[0080] Step 310: Extract subnetwork N based on local image features local3 Processing G merge1 , get the feature map G local3 ;

[0081] Step 311: G local3 and G global2 The connection operation is performed along the channel direction, and then the local feature and global feature fusion sub-network N is used. Merge2 Processing is performed to obtain the feature map G merge2 ;

[0082] Step 312: Extract subnetwork N based on local image features local4 Processing G merge2 , get the feature map G local4 ;

[0083] Step 313: G local4 and Gglobal3 The connection operation is performed along the channel direction, and then the local feature and global feature fusion sub-network N is used. Merge3 Processing is performed to obtain the feature map G merge3 ;

[0084] Step 314: According to the channel attention module M ECA Calculate and get the feature map G ECA ;

[0085] Step 315: According to the classification header module M classifier For G ECA Process and calculate the predicted probability that image I belongs to each category;

[0086] Step 316: Among the predicted probabilities of each category, select the category with the highest probability as the predicted category C of image I I , and use its probability as the predicted probability of image I

[0087] Step 317: Output C I and

[0088] Furthermore, the step 31 includes the following steps:

[0089] Step 311: Extract module M according to target area extraction Calculate the image I and get G′;

[0090] Step 312: Encode module M according to target area stride1 Convolution operation is performed on G′ to obtain G T ;

[0091] Step 313: Let feature map G obj ←G′+G T .

[0092] Further, the step 33 includes the following steps:

[0093] Step 331: According to formula (2), obj and G stem Perform linear fusion processing to obtain feature G merge ;

[0094] G merge =σ(α1×G obj +α2×G stem ); (2)

[0095] Step 332: G merge and G stem Perform residual connection to calculate G fusion ;

[0096] Furthermore, the global average pooling layer of step 114 pools a feature map into 1 factor; the adaptive one-dimensional convolution layer uses a convolution kernel of size K×K, and the size of K is calculated according to formula (3);

[0097]

[0098] Among them, C represents the number of channels of the feature map, b and γ are hyperparameters, and |·| odd Represents a function that rounds to the nearest odd integer.

[0099] Compared with the prior art, the present invention has the following advantages:

[0100] First, in the target area perception subnetwork, the present invention uses adaptive contrast adjustment, maximum inter-class variance segmentation, and morphological operations to suppress the interference of low-frequency background and dynamic illumination, and then extracts fine-grained convolutional coding features of apple leaf images, which is conducive to fully combining the morphological prior and edge fractal information of apple leaves, and can also integrate the spatial features of the target area with the shallow features, so as to achieve more accurate adaptive perception of leaf areas and diseased areas;

[0101] Second, based on the shallow convolutional blocks of the network, the present invention introduces a lightweight Transformer block with a parallel cascade structure to extract the multi-scale texture and edge features of the apple leaf image in a lightweight horizontal manner, and realizes the adaptive selection of the optimal feature scale, the mining of long-distance dependencies and the fusion of cross-scale information in a skip connection manner, thereby more efficiently enhancing the network's information interaction and representation capabilities for multi-scale features, and improving the calculation speed and robustness of the network model;

[0102] Third, in the feature fusion subnetwork, the present invention introduces adaptive learning coefficients to perform adaptive feature weighting on the two encoding branches, thereby enhancing the network model's attention to the feature information of the apple leaf area, and then uses deep separable convolution and inverted residual mechanisms to further extract rich texture detail features of the apple leaf area, thereby improving the network model's accuracy in distinguishing different diseases.

[0103] Fourth, the present invention uses a weighted cross entropy loss function and a focal loss function in combination, which is conducive to quickly focusing the center of gravity on those difficult-to-distinguish samples, improving the tolerance and robustness of the loss function for the diversity of training samples, thereby solving the problem of uneven distribution of the number of disease categories in apple leaf images to a certain extent. Therefore, the present invention provides an apple leaf disease classification method that can effectively resist complex background and dynamic lighting interference, and has the characteristics of high accuracy, good robustness, strong generalization ability, and low computational complexity. BRIEF DESCRIPTION OF THE DRAWINGS

[0104] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0105] Figure 1 It is the confusion matrix obtained by the present invention on the test data set.

[0106] Figure 2 is the classification prediction result given by the present invention on the test images of different categories of diseases. Figure 2 (a) is the classification prediction result of black spot disease, Figure 2 (b) is the classification prediction result of black rot. Figure 2 (c) is the classification prediction result of cedar apple rust. Figure 2 (d) is the classification prediction result of healthy leaves.

[0107] Figure 3 It is a schematic diagram of the process of the present invention. DETAILED DESCRIPTION

[0108] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0109] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0110] like Figure 1-3As shown, the present invention provides an apple leaf disease classification method based on region perception and lightweight hybrid Transformer network, which is performed according to the following steps;

[0111] Step 1. Build and initialize a deep convolutional neural network N for apple leaf disease classification apple , including 1 target area perception subnetwork N obj ; 1 image fine-grained feature extraction subnetwork N stem ; 1 feature fusion sub-network N fusion ; 4 image local feature extraction sub-networks, denoted as N local1 、N local2 、N local3 、N local4 ; 3 image global feature extraction sub-networks, denoted as N global1 、N global2 、N global3 ; 3 local features and global features fusion sub-networks, respectively denoted as N Merge1 、N Merge2 With N Merge3 ; 1 channel attention module M ECA ; 1 classification head module M classifier ;

[0112] Step 1.1 Establish and initialize the target area perception subnetwork N obj , containing 1 target region extraction module M extraction and a target region encoding module M stride1 ;

[0113] The M extraction Contains 1 layer of adaptive contrast enhancement module M conenc , 1-layer image segmentation module M based on maximum inter-class variance seg , 1 layer of morphological closing operation module M based on an elliptical structure element of size (10,10) morph ;

[0114] The M stride1 Contains 3 groups of convolution operations, namely in, Contains 1 set of 18 convolution kernels of size 1×1, 1 BatchNorm2d data normalization layer, and 1 ReLU6 activation function layer; Contains 1 group of 18 convolution kernels of size 3×3 and group number 18, 1 BatchNorm2d data normalization layer, and 1 ReLU6 activation function layer; Contains 1 set of 32 convolution kernels of size 1×1 and 1 BatchNorm2d layer;

[0115] Step 1.2 Establish and initialize the image fine-grained feature extraction subnetwork N stem , contains 1 set of convolutional layers Conv stem , 1 BatchNorm2d data normalization layer, 1 ReLU6 activation function layer;

[0116] The Conv stem Contains 32 convolution kernels of size 3×3, each of which performs convolution operations with a step size of 1 pixel;

[0117] Step 1.3 Establish and initialize the feature fusion subnetwork N fusion , contains two learnable weight coefficients w1 and w2;

[0118] Step 1.4 Establish and initialize the image local feature extraction subnetwork N local1 , which contains 3 groups of convolution operations, namely

[0119] Said Contains 1 set of 192 convolution kernels of size 1×1; Contains 1 group of 192 convolution kernels of size 3×3 and group number 192, and each convolution kernel performs convolution operation with a step size of 2 pixels, 1 BatchNorm2d data normalization layer, and 1 ReLU6 activation function layer; Contains 1 group of 64 convolution kernels of size 1×1 and 1 BatchNorm2d layer;

[0120] Step 1.5 Establish and initialize the image global feature extraction subnetwork N global1 , including 1 set of pooling operations And 1 self-attention module

[0121] Said Contains 1 layer of average pooling operation with a size of 2×2 and a stride of 2, and 1 layer of maximum pooling operation with a size of 2×2 and a stride of 2;

[0122] Step 1.6 Establish and initialize the image local feature extraction subnetwork N local2 , which contains 3 groups of convolution operations, namely

[0123] Said Contains 1 set of 384 convolution kernels of size 1×1; Contains 1 group of 384 convolution kernels of size 3×3 and group number 384, and each convolution kernel performs convolution operation with a step size of 2 pixels, 1 BatchNorm2d data normalization layer, and 1 ReLU6 activation function layer; Contains 1 set of 128 convolution kernels of size 1×1 and 1 BatchNorm2d layer;

[0124] Step 1.7 Establish and initialize the image global feature extraction subnetwork N global2 , including 1 set of pooling operations With 1 self-attention module

[0125] Said Contains 1 layer of average pooling operation with a size of 2×2 and a stride of 2, and 1 layer of maximum pooling operation with a size of 2×2 and a stride of 2;

[0126] Step 1.8 Establish and initialize the image global feature extraction subnetwork N global3 , including 1 set of pooling operations With 1 self-attention module

[0127] Said Contains 1 layer of average pooling operation with a size of 2×2 and a stride of 2, and 1 layer of maximum pooling operation with a size of 2×2 and a stride of 2;

[0128] Step 1.9 Establish and initialize the local feature and global feature fusion sub-network N Merge1 , contains a set of 128 convolution kernels of size 1×1, and each convolution kernel performs convolution operation with a step size of 1 pixel;

[0129] Step 1.10 Establish and initialize the image local feature extraction subnetwork N local3 , which contains 3 groups of convolution operations, namely

[0130] Said Contains 1 set of 960 convolution kernels of size 1×1; It contains 1 group of 960 convolution kernels of size 3×3 and group number 960, and each convolution kernel performs convolution operation with a step size of 2 pixels, 1 BatchNorm2d data normalization layer, and 1 ReLU6 activation function layer; Contains 1 set of 128 convolution kernels of size 1×1 and 1 BatchNorm2d layer;

[0131] Step 1.11 Establish and initialize the local feature and global feature fusion sub-network N Merge2 , contains a set of 256 convolution kernels of size 1×1, and each convolution kernel performs convolution operation with a step size of 1 pixel;

[0132] Step 1.12 Establish and initialize the image local feature extraction subnetwork N local4 , which contains 3 groups of convolution operations, namely

[0133] Said Contains 1 set of 1920 convolution kernels of size 1×1; Contains 1 group of 1920 convolution kernels of size 3×3 and group number 1920, and each convolution kernel performs convolution operation with a step size of 2 pixels, 1 BatchNorm2d data normalization layer, and 1 ReLU6 activation function layer; Contains 1 set of 256 convolution kernels of size 1×1 and 1 BatchNorm2d layer;

[0134] Step 1.13 Establish and initialize the local feature and global feature fusion sub-network N Merge3 , contains a set of 960 convolution kernels of size 1×1, and each convolution kernel performs convolution operation with a step size of 1 pixel;

[0135] Step 1.14 Establish and initialize the channel attention module M ECA , including 1 global average pooling layer, 1 adaptive one-dimensional convolution layer, 1 Sigmoid activation function layer, and 1 matrix Hadamard product layer;

[0136] The global average pooling layer pools a feature map into 1 factor; the adaptive one-dimensional convolution layer uses a convolution kernel of size K×K, and the size of K is calculated according to formula (1);

[0137]

[0138] The C represents the number of channels of the feature map, b and γ are hyperparameters, |·| odd Represents a function that rounds to the nearest odd integer;

[0139] Step 1.15: Create and initialize the classification head module M classifier , including 1 global average pooling layer and 1 classifier;

[0140] The global average pooling layer pools the feature map into a factor; the classifier includes 1 layer of Hardwish activation function, 1 layer of Dropout layer with a probability of 0.3, and 1 layer of fully connected layer with 960 input units and 4 output units;

[0141] Step 2. Input the training set of apple leaf disease images and the manually annotated label set C label , for the deep convolutional neural network N apple For training, let the training set have N testImg images, and let the number of iterations iter←1, the counter ind←1;

[0142] Step 2.1 Take the indth image T from the training set ind and their corresponding annotated labels

[0143] Step 2.2: Transform image T ind Perform a horizontal flip operation with a probability of 0.4 and a 20° random rotation operation to achieve image T ind Data augmentation;

[0144] Step 2.3 Use the target area perception subnetwork N obj Processing image T ind , and get the feature map F obj ;

[0145] Step 2.3.1 Use the target region extraction module M extraction For image T ind Calculate and get T′;

[0146] Step 2.3.2 Using the target region encoding module M stride1 Convolution operation on T′, we get F T ;

[0147] Step 2.3.3 Let the feature map F obj ←T′+F T ;

[0148] Step 2.4: Use the image fine-grained feature extraction subnetwork N stem Process image T to get feature map F stem ;

[0149] Step 2.5 Using feature fusion subnetwork N fusion Calculate the feature map F fusion ;

[0150] Step 2.5.1 According to formula (2) to formula (4), obj and F stem Perform linear fusion processing to obtain feature F merge ;

[0151] F merge =σ(α1×F obj +α2×F stem ) (2)

[0152]

[0153] The σ represents the Softmax function, and α1 and α2 represent normalized weight coefficients;

[0154] Step 2.5.2 F merge and F stemPerform residual connection to calculate F fusion ;

[0155] Step 2.6 Use the local features of the image to extract the subnetwork N local1 Processing F fusion , and get the feature map F local1 ;

[0156] Step 2.7 Use the global feature extraction subnetwork N global1 Processing F fusion , and get the feature map F global1 ;

[0157] Step 2.8 Use the local features of the image to extract the subnetwork N local2 Processing F local1 , and get the feature map F local2 ;

[0158] Step 2.9: Use the global feature extraction subnetwork N global2 Processing F local1 , and get the feature map F global2 ;

[0159] Step 2.10 Use the global feature extraction subnetwork N global3 Processing F local2 , and get the feature map F global3 ;

[0160] Step 2.11 F local2 and F global1 The connection operation is performed along the channel direction, and then the local feature and global feature fusion sub-network N is used. Merge1 Processing is performed to obtain the feature map F merge1 ;

[0161] Step 2.12 Use the local features of the image to extract the subnetwork N local3 Processing F merge1 , and get the feature map F local3 ;

[0162] Step 2.13: F local3 and F global2 The connection operation is performed along the channel direction, and then the local feature and global feature fusion sub-network N is used. Merge2 Processing is performed to obtain the feature map F merge2 ;

[0163] Step 2.14 Use the local features of the image to extract the subnetwork N local4 Processing F merge2 , and get the feature map F local4 ;

[0164] Step 2.15: Flocal4 and F global3 The connection operation is performed along the channel direction, and then the local feature and global feature fusion sub-network N is used. Merge3 Processing is performed to obtain the feature map F merge3 ;

[0165] Step 2.16 Using the channel attention module M ECA According to formula (5), the feature map F merge3 Calculate and get the feature map F ECA ;

[0166]

[0167] The GAP(·) represents the global average pooling operation, Conv 1×1 (·) represents a one-dimensional convolution operation, δ(·) represents the Sigmoid function, represents the matrix Hadamard product;

[0168] Step 2.17 Using the classification head module M classifier F ECA Process and calculate image T ind predicted probability of belonging to each category;

[0169] Step 2.18 Among the predicted probabilities of each category, select the category with the highest probability as image T ind The predicted category and its probability as image T ind The predicted probability of

[0170] Step 2.19 Let ind←ind+1, if ind>N testImg , then go to step 2.20, otherwise return to step 2.1;

[0171] Step 2.20 Calculate the prediction loss value L of the training set according to formula (6) to formula (9) total ;

[0172]

[0173]

[0174] L total =ζ1×L crossEntropy +ζ2×L focal (9)

[0175] The L focal represents the average focus loss value, L crossEntropy represents the average cross entropy loss value, α j and ξ are hyperparameters, k represents the number of disease types included in the training set, and pij represents the probability that the model predicts that the i-th image in the training set belongs to the j-th disease category, y i represents the true label of the i-th image in the training set, Indicates the yth i The weight coefficient of the inverse frequency weighting of the class disease, Indicates that the model predicts that the i-th image in the training set belongs to the y-th image i The probability of the disease, N j represents the number of images belonging to the jth disease in the training set, N yi Indicates that the training set belongs to the yth i The number of images of the same disease type, ζ1 and ζ2 represent preset weight coefficients. In this embodiment, ζ1=0.5 and ζ2=0.5;

[0176] Step 2.21 Let iter←iter+1. If the number of iterations iter>Total_iter, the trained deep convolutional neural network N is obtained. apple , go to step 3, otherwise use the reverse error propagation algorithm based on stochastic gradient descent and the prediction loss L total Update N apple Parameters, set counter ind←1, go to step 2.1 to reprocess all images in the training set, the Total_iter represents the preset number of iterations, in this embodiment, Total_iter=300;

[0177] Step 3. Input the apple leaf disease image I to be processed, whose color space is RGB, and use the trained deep convolutional neural network N apple Predict the disease category to which I belongs;

[0178] Step 3.1 Use the target area perception subnetwork N obj Process image I to obtain feature map G obj ;

[0179] Step 3.1.1 Use the target region extraction module M extraction Calculate the image I and get G′;

[0180] Step 3.1.2 Using the target region encoding module M stride1 Convolution operation is performed on G′ to obtain G T ;

[0181] Step 3.1.3 Let the feature map G obj ←G′+G T ;

[0182] Step 3.2: Use the image fine-grained feature extraction subnetwork N stem Process image I to obtain feature map Gstem ;

[0183] Step 3.3 Using feature fusion subnetwork N fusion Calculate the feature map G fusion ;

[0184] Step 3.3.1 According to formula (10), obj and G stem Perform linear fusion processing to obtain feature G merge ;

[0185] G merge =σ(α1×G obj +α2×G stem ) (10)

[0186] Step 3.3.2 G merge and G stem Perform residual connection to calculate G fusion ;

[0187] Step 3.4 Use the local features of the image to extract the subnetwork N local1 Processing G fusion , get the feature map G local1 ;

[0188] Step 3.5 Use the global feature extraction subnetwork N global1 Processing G fusion , get the feature map G global1 ;

[0189] Step 3.6 Use the local features of the image to extract the subnetwork N local2 Processing G local1 , get the feature map G local2 ;

[0190] Step 3.7 Use the global feature extraction subnetwork N global2 Processing G local1 , get the feature map G global2 ;

[0191] Step 3.8 Use the global feature extraction subnetwork N global3 Processing G local2 , get the feature map G global3 ;

[0192] Step 3.9: G local2 and G global1 The connection operation is performed along the channel direction, and then the local feature and global feature fusion sub-network N is used. Merge1 Processing is performed to obtain the feature map G merge1 ;

[0193] Step 3.10 Use the local features of the image to extract the subnetwork N local3 Processing G merge1 , get the feature map G local3 ;

[0194] Step 3.11 G local3 and G global2 The connection operation is performed along the channel direction, and then the local feature and global feature fusion sub-network N is used. Merge2 Processing is performed to obtain the feature map G merge2 ;

[0195] Step 3.12: Use local image features to extract subnetwork N local4 Processing G merge2 , get the feature map G local4 ;

[0196] Step 3.13 G local4 and G global3 The connection operation is performed along the channel direction, and then the local feature and global feature fusion sub-network N is used. Merge3 Processing is performed to obtain the feature map G merge3 ;

[0197] Step 3.14 Use the channel attention module M ECA Calculate and get the feature map G ECA ;

[0198] Step 3.15 Using the classification head module M classifier For G ECA Process and calculate the predicted probability that image I belongs to each category;

[0199] Step 3.16: Among the predicted probabilities of each category, select the category with the highest probability as the predicted category C of image I. I , and use its probability as the predicted probability of image I

[0200] Step 3.17 Output C I and

[0201] Example:

[0202] To verify the effectiveness of the present invention, the PlantPathology Apple Dataset was selected for experiments. The download address is https: / / www.kaggle.com / lextoumbourou / plantvillageapplecolor. The dataset contains 3171 images of apple leaf diseases, involving four categories of conditions: apple scab, black rot, cedar apple rust, and healthy leaves. During the experiment, 80% of the images were randomly selected as the training set, and the remaining 20% ​​of the images were used as the test set.

[0203] Figure 1 Shown is the confusion matrix obtained by the present invention on the test data set. Figure 2 The figure shows the classification prediction results given by the present invention on the test images of different types of diseases, where: Figure 2 (a) is the classification prediction result of black spot disease, Figure 2 (b) is the classification prediction result of black rot. Figure 2 (c) is the classification prediction result of cedar apple rust. Figure 2 (d) is the classification prediction result of healthy leaves. It can be seen that the present invention achieves accurate classification of apple scab, black rot, cedar apple rust and healthy leaves.

[0204] At the same time, Table 1 gives a comparison of the classification performance of the method of the present invention with AlexNet, ResNet34, GoogleNet, VGG16, Alex+Inception, AICNN, and AMMFNet, and its objective evaluation indicators include accuracy, precision, sensitivity, specificity, and F1-score. As can be seen from Table 1, the method of the present invention has achieved the highest evaluation scores in all five objective indicators. Among them, due to the insufficient adaptive selection ability of AlexNet and VGG16 for the best multi-scale features, the classification accuracy of apple leaf diseases of the two is relatively low; since ResNet34 adds jump connections between different scales, ResNet34 can extract and nonlinearly fuse the best multi-scale features, and its classification accuracy is improved compared with AlexNet and VGG16; GoogleNet and Alex+Inception both use convolution kernels with parallel cascade structures, which can perform convolution at multiple scales simultaneously, thereby extracting independent features belonging to different scales in the same image at one time, and then aggregating the features with strong correlation, achieving a classification performance better than ResNet34; AM MFNet introduces a channel attention mechanism, which helps to explore the complementarity between RGB and multispectral image data, and further improves the classification accuracy; AICNN introduces a transfer learning strategy based on the VGG16 model, and jointly adopts a global average pooling layer, a batch normalization layer, and a fully connected layer, and its objective evaluation results reach more than 99%; the present invention effectively integrates regional perception coding and backbone network coding to enhance the degree of attention to the feature information of the leaf region, and sends the feature information with enhanced regional attention to the convolution block and the Transformer block, further extracts the local feature information and global feature information of the target region, and introduces an attention mechanism in the deep layer of the network to give the key feature map a higher weight coefficient, thereby achieving the highest classification efficiency. At the same time, the network parameter volume of the method of the present invention is only 4.14% of AlexNet, 10.65% of ResNet34, 41.82% of GoogleNet, 1.80% of VGG16, 42.59% of Alex+Inception, 1.19% of AMMFNet, and 16.43% of AICNN. Obviously, the network model volume and computational complexity of the method of the present invention are significantly lower than those of several other methods.

[0205] In summary, the present invention provides an apple leaf disease classification method that can effectively resist complex background and dynamic light interference, and has the characteristics of high accuracy, good robustness, strong generalization ability and low computational complexity.

[0206] Table 1 Comparison of classification results of different apple leaf disease classification methods

[0207]

[0208] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0209] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0210] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0211] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0212] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0213] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes.

[0214] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. The apple leaf disease classification method based on region perception and lightweight hybrid Transformer network is characterized by: The following steps are involved: Step 1: Build and initialize a deep convolutional neural network N for apple leaf disease classification apple ; The deep convolutional neural network N apple Includes: target area perception subnetwork N obj , image fine-grained feature extraction sub-network N stem , feature fusion sub-network N fusion , 4 image local feature extraction sub-networks, denoted as N local1 、N local2 、N local3 、N local4 , 3 image global feature extraction sub-networks, denoted as N global1 、N global2 、N global3 , 3 local features and global features fusion sub-networks, respectively denoted as N Merge1 、N Merge2 With N Merge3 , channel attention module M ECA And the classification head module M classifier ; Step 2: Input the training set of apple leaf disease images and the manually annotated label set C label , for the deep convolutional neural network N apple Perform training to obtain the trained deep convolutional neural network N apple ; The training set has a total of N testImg images, and let the number of iterations iter←1, the counter ind←1; Step 3: Input the apple leaf disease image I to be processed, wherein the color space of the apple leaf disease image I to be processed is RGB; according to the trained deep convolutional neural network N obtained in step 2 apple Predict the disease category of the apple leaf disease image I to be processed.

2. The apple leaf disease classification method based on region perception and lightweight hybrid Transformer network according to claim 1 is characterized in that: The step 1 comprises the following steps: Step 11: Establish the target area perception subnetwork N obj And initialize it; the target area perception subnetwork N obj Contains: 1 target area extraction module M extraction and a target region encoding module M stride1 ; The M extraction Contains: 1 layer of adaptive contrast enhancement module M conenc , 1-layer image segmentation module M based on maximum inter-class variance seg , 1 layer of morphological closing operation module M based on an elliptical structure element of size (10,10) morph ; The M stride1 Contains 3 groups of convolution operations, namely in, Contains 1 set of 18 convolution kernels of size 1×1, 1 BatchNorm2d data normalization layer, and 1 ReLU6 activation function layer; Contains 1 group of 18 convolution kernels of size 3×3 and group number 18, 1 BatchNorm2d data normalization layer, and 1 ReLU6 activation function layer; Contains 1 set of 32 convolution kernels of size 1×1 and 1 BatchNorm2d layer; Step 12: Establish image fine-grained feature extraction subnetwork N stem And initialize it; the image fine-grained feature extraction subnetwork N stem Includes: Convolution layer Conv stem , BatchNorm2d data normalization layer and ReLU6 activation function layer; the convolution layer Conv stem Includes: 32 convolution kernels of size 3×3, each convolution kernel performs convolution operation with a step size of 1 pixel; Step 13: Establish feature fusion sub-network N fusion And initialize it; the feature fusion sub-network N fusion Includes: 2 learnable weight coefficients w1 and w2; Step 14: Establish image local feature extraction subnetwork N local1 And initialize it; the image local feature extraction subnetwork N local1 It includes: 3 groups of convolution operations; the convolution operations are respectively Said Contains 1 set of 192 convolution kernels of size 1×1; Contains 1 group of 192 convolution kernels of size 3×3 and group number 192, and each convolution kernel performs convolution operation with a step size of 2 pixels, 1 BatchNorm2d data normalization layer, and 1 ReLU6 activation function layer; Contains 1 group of 64 convolution kernels of size 1×1 and 1 BatchNorm2d layer; Step 15: Establish image global feature extraction subnetwork N global1 And initialize it; the image global feature extraction subnetwork N global1 Contains 1 set of pooling operations And 1 self-attention module The pooling operation Contains 1 layer of average pooling with a size of 2×2 and a stride of 2, and 1 layer of maximum pooling with a size of 2×2 and a stride of 2; Step 16: Establish image local feature extraction subnetwork N local2 And initialize it; the image local feature extraction subnetwork N local2 Contains 3 groups of convolution operations, namely Said Contains 1 set of 384 convolution kernels of size 1×1; Contains 1 group of 384 convolution kernels of size 3×3 and group number 384, and each convolution kernel performs convolution operation with a step size of 2 pixels, 1 BatchNorm2d data normalization layer, and 1 ReLU6 activation function layer; Contains 1 set of 128 convolution kernels of size 1×1 and 1 BatchNorm2d layer; Step 17: Establish image global feature extraction subnetwork N global2 And initialize it; the image global feature extraction subnetwork N global2 Contains 1 set of pooling operations With 1 self-attention module Said Contains 1 layer of average pooling with a size of 2×2 and a stride of 2, and 1 layer of maximum pooling with a size of 2×2 and a stride of 2; Step 18: Establish image global feature extraction subnetwork N global3 And initialize it; the image global feature extraction subnetwork N global3 Contains 1 set of pooling operations With 1 self-attention module Said Contains 1 layer of average pooling with a size of 2×2 and a stride of 2, and 1 layer of maximum pooling with a size of 2×2 and a stride of 2; Step 19: Establish a local feature and global feature fusion sub-network N Merge1 And initialize it; the local feature and global feature fusion sub-network N Merge1 Contains a set of 128 convolution kernels of size 1×1, and each convolution kernel performs convolution operations with a step size of 1 pixel; Step 110: Establish an image local feature extraction subnetwork N local3 And initialize it; the image local feature extraction subnetwork N local3 Contains 3 groups of convolution operations, namely Said Contains 1 set of 960 convolution kernels of size 1×1; It contains 1 group of 960 convolution kernels of size 3×3 and group number 960, and each convolution kernel performs convolution operation with a step size of 2 pixels, 1 BatchNorm2d data normalization layer, and 1 ReLU6 activation function layer; Contains 1 set of 128 convolution kernels of size 1×1 and 1 BatchNorm2d layer; Step 111: Establish a local feature and global feature fusion sub-network N Merge2 And initialize it; the local feature and global feature fusion sub-network N Merge2 Contains a set of 256 convolution kernels of size 1×1, and each convolution kernel performs convolution operations with a step size of 1 pixel; Step 112: Establish an image local feature extraction subnetwork N local4 And initialize it; the image local feature extraction subnetwork N local4 Contains 3 groups of convolution operations, namely Said Contains 1 set of 1920 convolution kernels of size 1×1; Contains 1 group of 1920 convolution kernels of size 3×3 and group number 1920, and each convolution kernel performs convolution operation with a step size of 2 pixels, 1 BatchNorm2d data normalization layer, and 1 ReLU6 activation function layer; Contains 1 set of 256 convolution kernels of size 1×1 and 1 BatchNorm2d layer; Step 113: Establish a local feature and global feature fusion sub-network N Merge3 And initialize it; the local feature and global feature fusion sub-network N Merge3 Contains a set of 960 convolution kernels of size 1×1, and each convolution kernel performs convolution operations with a step size of 1 pixel; Step 114: Establish channel attention module M ECA And initialize it; the channel attention module M ECA Contains 1 global average pooling layer, 1 adaptive one-dimensional convolution layer, 1 sigmoid activation function layer and 1 matrix Hadamard product layer; Step 115: Create a classification head module M classifier And initialize it; the classification head module M classifier Contains 1 global average pooling layer and 1 classifier; The global average pooling layer pools the feature map into a factor; the classifier includes 1 layer of Hardwish activation function, 1 layer of Dropout layer with a probability of 0.3, and 1 layer of fully connected layer with 960 input units and 4 output units.

3. The apple leaf disease classification method based on region perception and lightweight hybrid Transformer network according to claim 1 is characterized in that: The step 2 comprises the following steps: Step 21: Take the indth image T from the training set ind and their corresponding annotated labels Step 22: T ind Perform a horizontal flip operation with a probability of 0.4 and a 20° random rotation operation to achieve image T ind Data augmentation; Step 23: Perceive sub-network N according to target area obj Processing image T ind , and get the feature map F obj ; Step 24: Extract subnetwork N based on image fine-grained features stem Process image T to get feature map F stem ; Step 25: Fusion sub-network N according to features fusion Calculate the feature map F fusion ; Step 26: Extract subnetwork N based on local image features local1 Processing F fusion , and get the feature map F local1 ; Step 27: Extract subnetwork N based on image global features global1 Processing F fusion , and get the feature map F global1 ; Step 28: Extract subnetwork N based on local image features local2 Processing F local1 , and get the feature map F local2 ; Step 29: Extract subnetwork N based on image global features global2 Processing F local1 , and get the feature map F global2 ; Step 210: Extract subnetwork N based on global image features global3 Processing F local2 , and get the feature map F global3 ; Step 211: F local2 and F global1 The connection operation is performed along the channel direction, and then the local feature and global feature fusion sub-network N is used. Merge1 Processing is performed to obtain the feature map F merge1 ; Step 212: Extract subnetwork N based on local image features local3 Processing F merge1 , and get the feature map F local3 ; Step 213: F local3 and F global2 The connection operation is performed along the channel direction, and then the local feature and global feature fusion sub-network N is used. Merge2 Processing is performed to obtain the feature map F merge2 ; Step 214: Extract subnetwork N based on local image features local4 Processing F merge2 , and get the feature map F local4 ; Step 215: Place F local4 and F global3 The connection operation is performed along the channel direction, and then the local feature and global feature fusion sub-network N is used. Merge3 Processing is performed to obtain the feature map F merge3 ; Step 216: According to the channel attention module M ECA According to formula (1), the feature map F merge3 Calculate and get the feature map F ECA ; Among them, GAP(·) represents the global average pooling operation, Conv 1×1 (·) represents a one-dimensional convolution operation, δ(·) represents the Sigmoid function, represents the matrix Hadamard product; Step 217: Using the classification head module M classifier F ECA Process and calculate image T ind predicted probability of belonging to each category; Step 218: Among the predicted probabilities of each category, select the category with the highest probability as image T ind The predicted category and its probability as image T ind The predicted probability of Step 219: Let ind←ind+1, if ind>N testImg , then go to step 220, otherwise return to step 21; Step 220: Calculate the prediction loss value L of the training set total : L total =ζ1×L crossEntropy +ζ2×L focal ; The L focal represents the average focus loss value, L crossEntropy represents the average cross entropy loss value, α j and ξ are hyperparameters, k represents the number of disease types included in the training set, and p ij represents the probability that the model predicts that the i-th image in the training set belongs to the j-th disease category, y i represents the true label of the i-th image in the training set, Indicates the yth i The weight coefficient of the inverse frequency weighting of the class disease, Indicates that the model predicts that the i-th image in the training set belongs to the y-th image i The probability of the disease, N j represents the number of images belonging to the jth disease in the training set, Indicates that the training set belongs to the yth i The number of images of the disease class, ζ1 and ζ2 represent the preset weight coefficients; Step 221: Let iter←iter+1. If the number of iterations iter>Total_iter, the trained deep convolutional neural network N is obtained. apple , go to step 3, otherwise use the reverse error propagation algorithm based on stochastic gradient descent and the prediction loss L total Update N apple , set the counter ind←1, and go to step 21 to reprocess all images in the training set, where Total_iter represents the preset number of iterations.

4. The apple leaf disease classification method based on region perception and lightweight hybrid Transformer network according to claim 3 is characterized in that: The step 23 further comprises the following steps: Step 231: Extract module M according to target area extraction For image T ind Calculate and get T′; Step 232: Encode module M according to target area stride1 Perform convolution operation on T′ to obtain F T ; Step 233: Let the feature map F obj ←T′+F T .

5. The apple leaf disease classification method based on region perception and lightweight hybrid Transformer network according to claim 3 is characterized in that: The step 25 comprises the following steps: Step 251: F obj and F stem Perform linear fusion processing to obtain feature F merge ; F merge =σ(α1×F obj +α2×F stem ); Among them, σ represents the Softmax function, α1 and α2 represent the normalized weight coefficients; Step 252: F merge and F stem Perform residual connection to calculate F fusion .

6. The apple leaf disease classification method based on region perception and lightweight hybrid Transformer network according to claim 1 is characterized in that: The step 3 comprises the following steps: Step 31: Perceive sub-network N according to target area obj Process image I to obtain feature map G obj ; Step 32: Extract subnetwork N based on image fine-grained features stem Process image I to obtain feature map G stem ; Step 33: Fusion sub-network N according to features fusion Calculate the feature map G fusion ; Step 34: Extract subnetwork N based on local image features local1 Processing G fusion , get the feature map G local1 ; Step 35: Extract subnetwork N based on image global features global1 Processing G fusion , get the feature map G global1 ; Step 36: Extract subnetwork N based on local image features local2 Processing G local1 , get the feature map G local2 ; Step 37: Extract subnetwork N based on image global features global2 Processing G local1 , get the feature map G global2 ; Step 38: Extract subnetwork N based on image global features global3 Processing G local2 , get the feature map G global3 ; Step 39: Place G local2 and G global1 The connection operation is performed along the channel direction, and then the local feature and global feature fusion sub-network N is used. Merge1 Processing is performed to obtain the feature map G merge1 ; Step 310: Extract subnetwork N based on local image features local3 Processing G merge1 , get the feature map G local3 ; Step 311: G local3 and G global2 The connection operation is performed along the channel direction, and then the local feature and global feature fusion sub-network N is used. Merge2 Processing is performed to obtain the feature map G merge2 ; Step 312: Extract subnetwork N based on local image features local4 Processing G merge2 , get the feature map G local4 ; Step 313: G local4 and G global3 The connection operation is performed along the channel direction, and then the local feature and global feature fusion sub-network N is used. Merge3 Processing is performed to obtain the feature map G merge3 ; Step 314: According to the channel attention module M ECA Calculate and get the feature map G ECA ; Step 315: According to the classification header module M classifier For G ECA Process and calculate the predicted probability that image I belongs to each category; Step 316: Among the predicted probabilities of each category, select the category with the highest probability as the predicted category C of image I I , and use its probability as the predicted probability p of image I CI ; Step 317: Output C I and p CI .

7. The apple leaf disease classification method based on region perception and lightweight hybrid Transformer network according to claim 6 is characterized in that: The step 31 comprises the following steps: Step 311: Extract module M according to target area extraction Calculate the image I and get G′; Step 312: Encode module M according to target area stride1 Convolution operation is performed on G′ to obtain G T ; Step 313: Let feature map G obj ←G′+G T .

8. The apple leaf disease classification method based on region perception and lightweight hybrid Transformer network according to claim 6 is characterized in that: The step 33 comprises the following steps: Step 331: According to formula (2), obj and G stem Perform linear fusion processing to obtain feature G merge ; G merge =σ(α1×G obj +α2×G stem ); (2) Step 332: G merge and G stem Perform residual connection to calculate G fusion。 9. The apple leaf disease classification method based on region perception and lightweight hybrid Transformer network according to claim 2 is characterized in that: The global average pooling layer of step 114 pools a feature map into 1 factor; The adaptive one-dimensional convolution layer uses a convolution kernel of size K×K, and the size of K is calculated according to formula (3); Among them, C represents the number of channels of the feature map, b and γ are hyperparameters, and |·| odd Represents a function that rounds to the nearest odd integer.

Images