A neural network construction method for crop leaf disease identification

Abstract

The present application relates to the technical field of crop leaf image disease classification and recognition, and particularly relates to a neural network construction method for crop leaf disease recognition. The method solves the problem of insufficient detail feature extraction of crop leaf diseases through a convolutional neural network. The technical scheme comprises the following steps: step one, constructing a pyramid multi-scale convolution feature extraction module; step two, constructing a channel and spatial dimension joint attention mechanism module; step three, constructing a multi-scale feature relationship modeling and adaptive feature selection module; step four, combining all the modules into a multi-scale adaptive feature extraction module; and step five, combining into a recognition model. The present application has the beneficial effect of using channel and spatial joint attention information to adaptively fine-tune and select useful multi-scale features, thereby significantly improving the recognition accuracy of neural networks for crop leaf diseases.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of crop leaf image disease classification and identification, and particularly relates to a neural network construction method for crop leaf disease identification. BACKGROUND

[0002] In China, the loss of grain caused by crop diseases and insect pests reaches 14 million tons every year, and crop leaf diseases are an important factor causing the loss. Leaf is an important place for plants to carry out photosynthesis, and pathogenic bacteria will affect the photosynthesis to produce nutrients after invading crop leaves, and then affect crop yield, and in severe cases, will lead to absolute loss. Accurate identification of crop leaf disease types will help to take preventive measures in time, avoid large-scale outbreak of diseases, reduce environmental pollution, crop pesticide damage, ecological environment destruction and other consequences caused by random use and misuse of pesticides, and pesticide residues in agricultural and livestock products through food chain enrichment, and then threaten human health. Therefore, rapid and accurate identification of crop leaf disease types to provide basis for taking effective prevention and control measures plays an important role in protecting crop production safety, ensuring the harmonious development of human and nature, and is also a current research hotspot.

[0003] For crop leaf disease identification problems, the traditional method is artificial identification by agricultural plant protection experts through naked eyes, which needs to consume a lot of manpower, and the identification efficiency and accuracy are relatively low. In the past decade, machine learning and pattern recognition have made significant progress in crop disease identification based on images. Although machine learning and pattern recognition technology has achieved better identification effect compared with artificial identification method, these methods still need to face some difficulties, including factors such as light conditions, image shooting angle, complex field conditions, climate conditions, color features, distribution area and the like. As an important branch of machine learning, deep learning has made remarkable achievements in image processing, natural language processing, recommendation system and other fields in recent years, and has made great progress in crop disease identification.

[0004] Through domestic and foreign literature and patent retrieval, it is found that the existing patent "Plant disease identification method and system based on CNN model and transfer learning" (Application No. CN201911101895.7) discloses a plant disease identification method based on CNN model and transfer learning. The invention obtains new training data set by data enhancement on original data, and sets bottleneck layer based on pre-trained model to obtain plant disease identification result in model construction process. And in the model training process, the picture is directly input into the network, avoiding the process of plant leaf lesion segmentation and complex background segmentation. However, the method and system use convolutional neural network to classify and identify the overall features of crop leaf image, which does not consider the details of the shape, texture, size and mutual relationship between lesions in the image, which affects the recognition accuracy.

[0005] With the development of deep learning technology, multi-scale feature utilization is considered an effective feature extraction method. The existing patent "Multi-scale plant disease identification method, device, storage medium and apparatus" (Application No. CN202310131683.3) discloses a multi-scale plant disease identification method, device, storage medium and apparatus. The invention uses the improved MSS-ResNet101 model to identify the disease characteristics of plant images obtained after data enhancement preprocessing, and replaces the original ResNet101 model's 7x7 convolution multi-scale structure Inception module and residual module with convolution kernel attention mechanism in MSS-ResNet101 model, improving the model recognition efficiency and accuracy. However, the method and system do not consider the mutual relationship and importance of the extracted multi-scale features, which may cause feature redundancy and reduce the model recognition performance.

[0006] In the existing paper "Field crop disease identification based on lightweight MIE_Net" (Jiangsu Agricultural Science, Vol. 51, No. 10, 2023), MobileNetV2 is used as the basic network structure, and the multi-scale feature extraction module is used to replace the original network's initial convolution layer to improve the network's feature extraction ability for different area lesions, increase the network's feature complexity, add ECA attention mechanism to improve the network's attention to leaf disease area, reduce the influence of complex background on small lesion feature extraction process, and use Swish activation function to increase the network's expression ability, making the network performance optimal. However, the model still lacks consideration of multi-scale relationship features and importance, which is not conducive to precise extraction of crop disease detail features.

[0007] In the existing paper "Crop Disease Recognition Model Based on Global Feature Extraction" (Transaction of the Chinese Agricultural Machinery Society, Volume 53, Issue 12, 2022), the network performance cannot be dynamically improved to meet the actual complex disease image recognition task by using training sample expansion, transformation, scaling and other methods. A deep learning network based on global feature extraction is designed by introducing a double-layer attention mechanism and a channel feature extraction mechanism into ResNet50. The network recognizes crop disease types based on global features. Although the attention mechanism is added, it has a certain inhibitory effect on irrelevant features, but the model lacks consideration of detailed features such as the shape, texture, size of the disease spot and the relationship between disease spots, resulting in insufficient generalization ability of the model.

[0008] In summary, the existing crop leaf disease recognition method based on deep learning method has the following shortcomings:

[0009] I. The existing method extracts global features of crop leaf diseases through convolutional neural networks, and the detail feature extraction is insufficient;

[0010] II. The relationship between crop leaf disease spots is not extracted;

[0011] III. The features extracted by a single scale convolution kernel cannot accurately describe the features of disease leaves of different sizes, resulting in low accuracy and robustness of the recognition model in recognizing crop leaf disease types. SUMMARY

[0012] The purpose of the present application is to provide a neural network construction method for crop leaf disease recognition.

[0013] In order to achieve the above-mentioned purpose of the application, the technical scheme adopted by the present application is as follows:

[0014] A neural network construction method for crop leaf disease recognition, comprising the following steps

[0015] Step 1: Construct a pyramid multi-scale convolution feature extraction module;

[0016] Step 2: Construct a channel and spatial dimension joint attention mechanism module;

[0017] Step 3: Construct a multi-scale feature relationship modeling and adaptive feature selection module;

[0018] Step 4: Combine the modules in steps 1, 2 and 3 (specifically: pyramid multi-scale convolution feature extraction module, channel and spatial dimension joint attention mechanism module, multi-scale feature relationship modeling and adaptive feature selection module) into a multi-scale adaptive feature extraction module;

[0019] Step five, the feature extraction module group is composed of superimposed multi-scale adaptive feature extraction modules, according to the different sizes of the output feature map, design several groups of convolution groups, combined into a recognition model;

[0020] Step one specifically includes:

[0021] The pyramid multi-scale convolution feature extraction module adopts a plurality of different sizes of convolution kernels according to the convolution group to which it belongs, from the input to the output direction of the superimposed feature extraction module group.

[0022] The "pyramid multi-scale convolution feature extraction module" in the first group of convolution groups is composed of 3x3 convolution kernel, 5x5 convolution kernel, 7x7 convolution kernel, 9x9 convolution kernel, a total of four types of convolution kernels;

[0023] The second group is composed of 3x3 convolution kernel, 5x5 convolution kernel, 7x7 convolution kernel, a total of three types of convolution kernels;

[0024] The third group is composed of 3x3 convolution kernel, 5x5 convolution kernel, a total of two types of convolution kernels;

[0025] The fourth group is composed of 3x3 convolution kernel, a total of one type of convolution kernel;

[0026] Each convolution group contains two pyramid multi-scale convolution feature extraction modules, and each pyramid multi-scale convolution feature extraction module can contain a plurality of convolution layers with different convolution kernel sizes.

[0027] Each convolution layer is followed by a regularization layer (BatchNomalization) (a commonly used layer in model design) and a ReLU activation function layer, respectively used to adjust the data distribution to avoid data drift and increase the nonlinear expression ability of the network (a commonly used nonlinear function in model design), thereby forming a pyramid feature extraction structure within the convolution module and between each convolution group;

[0028] The original input image size is 224x224 pixels, after a convolution kernel size of 7x7 pixels, a step of 2 (i.e. every two pixels are shifted to do convolution operation), the input image with a channel number of 3 is converted into a feature map with a size of 112x112 pixels and a channel number of 64, after a maximum pooling layer with a pooling window of 3x3 pixels, a step of 2 and a padding of 1, the output size is changed to 56x56 pixels, the feature map is input into the first convolution group, and four feature maps with a size of 56x56 pixels and a channel number of 256 are obtained after convolution operation by 3x3 convolution kernel, 5x5 convolution kernel, 7x7 convolution kernel and 9x9 convolution kernel, respectively, for the channel and spatial dimension joint attention mechanism and relationship modeling module to realize relationship modeling, the output feature map sizes of the second group, the third group and the fourth group are 28x28 pixels, 14x14 pixels and 7x7 pixels respectively, and the channel numbers are 512, 1024 and 2048 respectively.

[0029] Step two specifically includes:

[0030] The channel and spatial dimension joint attention mechanism and relationship modeling module includes a channel dimension feature extraction layer and a spatial dimension feature extraction layer, wherein the channel dimension feature extraction layer and the spatial dimension feature extraction layer are parallel operations, and the output results of the two extraction layers are added to form an output result;

[0031] The channel dimension feature extraction layer is sequentially connected by a global average pooling layer, a first convolution layer and a second convolution layer;

[0032] S21: The input size feature map has three dimensions of CxHxW, and after the global average pooling layer, the size is converted to Cx1x1. After the global average pooling layer, a plurality of feature maps with different sizes are combined in the last dimension of the feature matrix (the dimension of the width W of the feature map), that is, Cx1xM, and then the first and second dimensions of the feature matrix (C and W dimensions) are interchanged, so as to obtain a feature map with a size of 1xCxM;

[0033] Wherein C represents the number of channels of the channel dimension, H represents the height of the input feature map of the current module, W represents the width of the input feature map of the current module, and M represents the number of different scale features;

[0034] S22: In order to improve the feature extraction capability of the channel dimension feature extraction layer without increasing too many model parameters, a convolution layer with a convolution kernel size of 1x1 is selected in the channel dimension feature extraction layer, the input is subjected to the first convolution layer and then subjected to a ReLu activation function to improve the nonlinear expression capability, the feature vector is expanded in the channel dimension, thereby converting the feature map size to dxCxM, d is an inflation coefficient. S23: The feature map size is converted to (dXM)xCx1 after transformation, and then subjected to a second convolution layer for compression in the channel dimension, the feature map size becomes 1xCx1, and finally the first and second dimensions are interchanged to become Cx1x1;

[0035] The spatial dimension feature extraction layer is sequentially connected by a first convolution layer with a convolution kernel size of 1x3 and a convolution layer with a convolution kernel size of 3x1.

[0036] S24: In order to reduce the total number of parameters in the spatial dimension feature extraction layer, the input feature map is first subjected to a convolution layer with a convolution kernel size of 1x3 for learning the features of the input feature map in the vertical direction, and a regularization layer and a ReLu activation function layer are connected after the convolution layer.

[0037] S25: The output vector after the convolution operation of the output result is subjected to a convolution layer with a convolution kernel size of 3x1 for learning the features of the input feature map in the horizontal direction. The input feature map and the output feature map of the spatial dimension feature extraction layer have the same size, which is CxHxW.

[0038] S26: The output results of the channel dimension feature extraction layer and the spatial dimension feature extraction layer are added by corresponding position elements, the channel dimension feature extraction layer output result, i.e. the feature matrix value with a size of Cx1x1, is superimposed on the spatial dimension feature extraction layer result, thereby generating a three-dimensional feature weight vector with a size of CxHxW.

[0039] Step three specifically includes:

[0040] S31: In order to further fuse the features extracted in the channel dimension and the spatial dimension and enhance the nonlinear expression performance of the module, the three-dimensional feature vector is input to a convolution layer with a convolution kernel size of 1x1 and dimensionally compressed in the channel dimension, so that the output feature map size becomes C / rxHxW;

[0041] Wherein r represents a compression ratio;

[0042] S32: The output feature map is input to another convolution layer with a convolution kernel size of 1x1 to restore the feature map dimension to the original size, i.e. a three-dimensional feature weight with a size of CxHxW. Repeat the operation to generate a three-dimensional feature weight corresponding to the feature scale, i.e. M feature weight vectors with a size of CxHxW, and realize multi-scale feature relationship modeling through the above steps.

[0043] S33: To realize the reconstruction of multi-scale features, the obtained M CxHxW size feature weight vectors are multiplied with the input feature vectors of the corresponding scale respectively, the important features of the corresponding scale are stimulated, and the unimportant features are inhibited, so that the selection of crop leaf disease features is realized;

[0044] S34: To further realize the multi-scale feature selection, the output vector with the feature weight is mapped to the (0, 1) interval in the scale dimension by using the Softmax function, that is, the corresponding element point values of the three-dimensional feature weight vectors of each scale are added to 1, and finally the feature vectors after feature redistribution are added to obtain the final output vector, that is, the adaptive selection of features is realized.

[0045] Compared with the prior art, the beneficial effects of the present application are:

[0046] 1. The method of the present application achieves the highest average accuracy of 97.43% on the crop leaf disease data set. Through comparative experiments, it is found that the average accuracy of the single-scale method is less than 90%, and the accuracy is significantly improved to 95.34% after increasing the pyramid multi-scale feature, which shows that the pyramid multi-scale feature can effectively provide diversity features for model recognition. Similarly, by fine-tuning the features, the recognition accuracy of the model can be gradually improved by increasing the spatial dimension feature extraction layer, the channel dimension feature extraction layer, the multi-scale feature relationship modeling and the adaptive feature selection structure.

[0047] 2. The method proposed in the present application can accurately describe the leaf disease spot features of different sizes by constructing pyramid multi-scale features. At the same time, through the spatial and channel joint attention mechanism, the multi-scale feature relationship modeling and the adaptive feature selection structure, the crop disease leaf image feature information is effectively selected and fused.

[0048] 3. The method of the present application can utilize the multi-scale features of the input crop disease spots on the leaf, thereby strengthening the effective representation of different disease types. At the same time, the useful multi-scale features are adaptively fine-tuned and selected by using the channel and spatial joint attention information, which significantly improves the recognition accuracy of the neural network for crop leaf diseases. DETAILED DESCRIPTION

[0049] The accompanying drawings are used to provide a further understanding of the present application, and constitute a part of the specification, together with the embodiments of the present application, to explain the present application, and do not constitute a limitation of the present application.

[0050] Figure 1 It is a pyramid multi-scale crop disease type recognition network structure diagram.

[0051] Figure 2 The figure is a schematic diagram of the pyramid multi-scale feature extraction module of the present application.

[0052] Figure 3 The figure is a schematic diagram of the channel and spatial dimension joint attention module structure of the present application.

[0053] Figure 4 The figure is a schematic diagram of the multi-scale feature relationship modeling and adaptive feature selection module structure of the present application.

[0054] Figure 5 The figure is a flow chart of the crop leaf disease type identification method of the present application.

[0055] Figure 6 The figure is a schematic diagram of the average accuracy comparison results of the crop leaf disease identification method of the present application and other deep learning methods.

[0056] Figure 7 The figure is a schematic diagram of the robustness comparison results of the present application and other deep learning methods. DETAILED DESCRIPTION

[0057] In order to make the purpose, technical scheme and advantages of the present application more clear, the present application will be further described in detail below in combination with the drawings and examples. Of course, the specific examples described herein are only used to explain the present application and not to limit the present application.

[0058] Example 1

[0059] Figure 1 The figure shows a schematic diagram of a pyramid multi-scale crop disease type identification network structure. After the crop leaf disease image is input into the input module, it is sequentially input into four groups of multi-scale adaptive feature selection modules to extract leaf disease features. The first group contains four types of scale features, which are features extracted by 3x3, 5x5, 7x7 and 9x9 convolution kernels; the second group contains three types of scale features, which are features extracted by 3x3, 5x5 and 7x7 convolution kernels; the third group contains two types of scale features, which are features extracted by 3x3 and 5x5 convolution kernels; the fourth group contains one type of scale feature, which is a feature extracted by a 3x3 convolution kernel. In this way, a pyramid multi-scale feature is formed between the four groups of convolution groups.

[0060] Figure 2 The figure shows a schematic diagram of a pyramid multi-scale feature extraction module. Taking three-scale feature extraction as an example, the module contains three types of scale features, which are features extracted by 3x3, 5x5 and 7x7 convolution kernels, and the three features are output respectively, which are feature 1, feature 2 and feature 3, for subsequent use by the channel and spatial dimension joint attention mechanism and the relationship modeling module.

[0061] Figure 3 A schematic diagram of a channel and spatial dimension joint attention mechanism module structure is shown. This figure takes three-scale feature extraction as an example, and in this module, the channel and spatial dimension feature extraction layers are included to extract the channel and spatial features of the input multi-scale features, and the output results of the two feature extraction layers are added to form a three-dimensional feature vector for use by the subsequent multi-scale feature relationship modeling and adaptive feature selection module.

[0062] The technical scheme of the present application provides a neural network construction method for crop leaf disease identification, comprising the following steps

[0063] Step one, construct a pyramid multi-scale convolution feature extraction module;

[0064] Step two, construct a channel and spatial dimension joint attention mechanism module;

[0065] Step three, construct a multi-scale feature relationship modeling and adaptive feature selection module;

[0066] Step four, combine the modules in steps one, two and three (specifically: pyramid multi-scale convolution feature extraction module, channel and spatial dimension joint attention mechanism module, multi-scale feature relationship modeling and adaptive feature selection module) into a multi-scale adaptive feature extraction module;

[0067] Step five, stack the multi-scale adaptive feature extraction modules to form a feature extraction module group, and according to the different sizes of the output feature maps, design several convolution groups to combine into an identification model;

[0068] As shown in Figure 1 Step one specifically includes:

[0069] The pyramid multi-scale convolution feature extraction module uses multiple different size convolution kernels according to the convolution group to which it belongs, as shown in Figure 1 from the input to the output direction of the stacked feature extraction module group.

[0070] The "pyramid multi-scale convolution feature extraction module" in the first convolution group is composed of 3x3 convolution kernel, 5x5 convolution kernel, 7x7 convolution kernel, and 9x9 convolution kernel, a total of four types of convolution kernels;

[0071] The second group is composed of 3x3 convolution kernel, 5x5 convolution kernel, and 7x7 convolution kernel, a total of three types of convolution kernels;

[0072] The third group is composed of 3x3 convolution kernel and 5x5 convolution kernel, a total of two types of convolution kernels;

[0073] The fourth group is composed of 3x3 convolution kernel, a total of one type of convolution kernel;

[0074] Figure 2 The schematic diagram of the pyramid multi-scale convolution feature extraction module with three types of convolution kernels is shown. Each convolution group contains two pyramid multi-scale convolution feature extraction modules, and each pyramid multi-scale convolution feature extraction module can contain multiple convolution layers with different convolution kernel sizes.

[0075] Each convolution layer is followed by a regularization layer (BatchNomalization) and a ReLU activation function layer, which are commonly used in model design. The regularization layer is used to adjust the data distribution to avoid data drift, and the ReLU activation function layer is used to increase the non-linear representation ability of the network. Thus, a pyramid feature extraction structure is formed within the convolution module and between the convolution groups.

[0076] The original input image size is 224x224 pixels. After a convolution layer with a 7x7 pixel kernel size and a step size of 2 (i.e., moving two pixels at a time for convolution operation), the input image with 3 channels is converted to a feature map with 64 channels and a size of 112x112 pixels. After a max pooling layer with a 3x3 pixel pooling window, a step size of 2, and a padding of 1, the output size becomes 56x56 pixels. The feature map is then input into the first convolution group, which consists of 3x3, 5x5, 7x7, and 9x9 convolution kernels. After convolution operation, four feature maps with a size of 56x56 pixels and 256 channels are obtained, which are used for relationship modeling by the channel and spatial dimension joint attention mechanism and relationship modeling module. The output feature map sizes of the second, third, and fourth groups are 28x28 pixels, 14x14 pixels, and 7x7 pixels, respectively, and the channel numbers are 512, 1024, and 2048, respectively.

[0077] Step two specifically includes:

[0078] The channel and spatial dimension joint attention mechanism and relationship modeling module includes a channel dimension feature extraction layer and a spatial dimension feature extraction layer, as shown in Figure 3 The channel dimension feature extraction layer and the spatial dimension feature extraction layer are operated in parallel, and the output results of the two extraction layers are summed to form the output result.

[0079] The channel dimension feature extraction layer is composed of a global average pooling layer, a first convolution layer, and a second convolution layer connected in sequence.

[0080] S21: the input size feature map is CxHxW, which is converted to Cx1x1 after global average pooling layer. The feature maps of different sizes are combined in the last dimension of the feature matrix (the dimension of the width W of the feature map), that is, Cx1xM. Then the first and second dimensions of the feature matrix (C and W dimensions) are interchanged to obtain a feature map with a size of 1xCxM;

[0081] wherein C represents the number of channels in the channel dimension, H represents the height of the input current module feature map, W represents the width of the input current module feature map, and M represents the number of different scale features.

[0082] S22: In order to improve the feature extraction ability of the channel dimension feature extraction layer without increasing too many model parameters, a convolution layer with a convolution kernel size of 1x1 is selected in the channel dimension feature extraction layer. The input is processed by the first convolution layer and then by a ReLu activation function to improve the nonlinear expression ability. The feature vector is expanded in the channel dimension, so that the feature map size is converted to dxCxM, and d is the expansion coefficient. S23: The feature map size is converted to (dXM)xCx1 after transformation, and then compressed in the channel dimension after the second convolution layer, so that the feature map size becomes 1xCx1. Finally, the first and second dimensions are interchanged to become Cx1x1.

[0083] The spatial dimension feature extraction layer is sequentially connected by a first convolution layer with a convolution kernel size of 1x3 and a convolution layer with a convolution kernel size of 3x1.

[0084] S24: In order to reduce the total number of parameters in the spatial dimension feature extraction layer, the input feature map is first processed by a convolution layer with a convolution kernel size of 1x3 to learn the features of the input feature map in the vertical direction. A regularization layer and a ReLu activation function layer are connected after the convolution layer.

[0085] S25: The output vector after convolution operation is processed by a convolution layer with a convolution kernel size of 3x1 to learn the features of the input feature map in the horizontal direction. The input and output feature maps of the spatial dimension feature extraction layer have the same size, which is CxHxW.

[0086] S26: The output results of the channel dimension feature extraction layer and the spatial dimension feature extraction layer are added by corresponding position elements. The channel dimension feature extraction layer output result, that is, the feature matrix value with a size of Cx1x1, is added to the spatial dimension feature extraction layer result, so as to generate a three-dimensional feature weight vector with a size of CxHxW.

[0087] Step three specifically includes:

[0088] S31: To further fuse the channel dimension and the spatial dimension of the extracted features, enhance the nonlinear expression performance of the module, input the three-dimensional feature vector into a convolution layer with a convolution kernel size of 1x1 and perform dimension compression in the channel dimension, so that the size of the output feature map becomes C / rxHxW;

[0089] wherein r represents the compression ratio;

[0090] S32: Input the output feature map into another convolution layer with a convolution kernel size of 1x1 to restore the feature map dimension to the original size, i.e., a three-dimensional feature weight with a size of CxHxW; repeat the operation to generate a three-dimensional feature weight corresponding to each feature scale, i.e., M CxHxW size feature weight vectors, and realize multi-scale feature relationship modeling through the above steps;

[0091] S33: To realize the reconstruction of multi-scale features, multiply the obtained M CxHxW size feature weight vectors with the input feature vectors of the corresponding scale respectively, to stimulate important features of the corresponding scale while suppressing unimportant features, thereby realizing the selection of crop leaf disease features;

[0092] S34: To further realize multi-scale feature selection, use the Softmax function in the scale dimension to map each corresponding position element to the (0, 1) interval, i.e., the sum of the corresponding element points of the three-dimensional feature weight vectors of each scale is 1, and finally add the feature vectors after feature redistribution to obtain the final output vector, i.e., realize adaptive selection of features.

[0093] Embodiment 2

[0094] Based on Embodiment 1, Figure 4 The structure diagram of the multi-scale feature relationship modeling and adaptive feature selection module is shown. Taking three-scale feature extraction as an example, in the module, the feature fusion module generates three-scale feature weights from the input three-dimensional features after compression, and then inputs them into the adaptive feature selection module, which normalizes the multi-scale feature weights by the softmax function to obtain the corresponding scale weights and multiplies them with the corresponding scale features input by the pyramid multi-scale feature extraction module. Finally, the results are added and output to obtain the fine-tuned crop leaf disease features of the fused multi-scale features.

[0095] Figure 5The crop leaf disease type recognition process is shown, the input is a crop disease leaf image, the multi-scale features of the diseased leaf are extracted through a pyramid multi-scale feature extraction network, a three-dimensional attention weight vector is generated through a channel and space joint attention mechanism module, and then the key features of each scale are selected through a multi-scale feature relationship modeling and adaptive feature selection network module to generate a corresponding scale weight vector, and then the redistributed feature vector is fused into an output vector, so as to repeatedly combine the modules in steps 1), 2) and 3) into a multi-scale adaptive feature extraction module, and then a crop leaf disease recognition model is constructed by stacking the multi-scale adaptive feature extraction modules, and finally the crop leaf disease recognition result is output.

[0096] Figure 6 The average accuracy comparison results of the crop leaf disease recognition method and other deep learning methods are shown.

[0097] Figure 7 The robustness comparison results are shown, and the robustness of the method is better than that of other deep models.

[0098] As Figure 4 , the experimental data are as follows: 12 types of soybean, apple and rice leaf diseases, including soybean brown spot, soybean spot, soybean downy mildew, apple spot and leaf fall, apple brown spot, apple gray spot, apple mosaic disease, apple rust, rice white leaf wilt, rice blast, rice East Gulu disease and rice brown spot, 400 images of each disease type. The experimental process is as follows: 50% of all pictures in the data set are used for training, and 50% are used for testing. 5 comparison methods: single scale, pyramidal multiscale (PM), pyramidal multiscale + channel dimension feature extraction layer (Channel Attention, CA), pyramidal multiscale + channel dimension feature extraction layer (Channel Attention, CA) + spatial dimension feature extraction layer (Spatial Attention, SA) and the method proposed in the application: pyramidal multiscale + channel dimension feature extraction layer (Channel Attention, CA) + spatial dimension feature extraction layer (Spatial Attention, SA) + adaptive feature selection network (Adaptive Feature Selection, AFS). The initial value of the network model is randomly set during training, the average recognition accuracy of 5 training and test results is compared, and the experimental results are shown in Figure 7 , by comparison, it can be found that the accuracy performance of the method can be obtained more stably than other methods, thereby proving the robustness of the method.

[0099] From the experimental results, it can be seen that the average accuracy of the method of the present application is the highest, which is 97.43%, as shown in the formula (I). Figure 6 The accuracy of the single-scale method is lower than 90%, and the accuracy reaches 95.34% after adding the pyramid multi-scale feature, which shows that the pyramid multi-scale feature can provide diversity features for model identification. After adding the spatial dimension feature extraction layer, the channel dimension feature extraction layer and the adaptive feature selection structure, the identification accuracy can be gradually improved through fine-tuning features. Experiments prove that the method proposed in the present application can realize accurate description of different size leaf spot features through the construction of pyramid multi-scale features. At the same time, the spatial and channel joint attention mechanism and the adaptive feature selection structure can effectively fuse the feature information of the crop diseased leaf image.

[0100] The above only describes the preferred embodiments of the present application and is not intended to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for constructing a neural network for identifying foliar diseases in crops, characterized in that, Includes the following steps Step 1: Construct a pyramid-shaped multi-scale convolutional feature extraction module; Step 2: Construct a joint attention mechanism module for channel and spatial dimensions; Step two specifically includes: The joint attention mechanism module for channel and spatial dimensions includes a channel dimension feature extraction layer and a spatial dimension feature extraction layer. The channel dimension feature extraction layer and the spatial dimension feature extraction layer are parallel operations. The output results of the two extraction layers are then summed to form the output result. The channel dimension feature extraction layer consists of a global average pooling layer, a first convolutional layer, and a second convolutional layer connected sequentially. S21: Input size feature map is There are three dimensions in total, and the size is converted after passing through a global average pooling layer. Multiple feature maps of different sizes are combined in the last dimension of the feature matrix after passing through a global average pooling layer. Then, the first and second dimensions of the feature matrix are interchanged, resulting in a matrix of size 1. Size feature map; in The number of channels, representing the channel dimension. This indicates the height of the feature map of the current input module. This indicates the width of the feature map of the current input module. Represents the number of features at different scales; S22: In the channel-dimensional feature extraction layers, 1×1 convolutional layers are used. After the input passes through the first convolutional layer, it passes through a ReLU activation function to expand the feature vector in the channel dimension, thereby transforming the feature map size. Where: d is the expansion coefficient; S23: Convert the feature map dimensions to... After being compressed in the channel dimension by the second convolutional layer, the feature map size becomes Finally, after swapping the first and second dimensions, it becomes ; The spatial dimension feature extraction layer consists of a first convolutional layer with a kernel size of 1×3 and a convolutional layer with a kernel size of 3×1 connected sequentially; S24: The input feature map first passes through a convolutional layer with a kernel size of 1×3 to learn the features of the input feature map in the vertical direction. After each convolutional layer, a regularization layer and a ReLU activation function layer are connected. S25: The output vector, after undergoing convolution, is then passed through a 3×1 convolutional layer to learn the horizontal features of the input feature map; the input and output feature maps of the spatial dimension feature extraction layer have the same size. ; S26: The outputs of the channel dimension feature extraction layer and the spatial dimension feature extraction layer are summed element-wise. The output of the channel dimension feature extraction layer is then calculated. The feature matrix values ​​of size are superimposed on the spatial dimension feature extraction layer result to generate a three-dimensional feature weight vector with size . ; Step 3: Construct a multi-scale feature relationship modeling and adaptive feature selection module; Step three specifically includes: S31: Input the 3D feature vector into a convolutional layer with a kernel size of 1×1 and perform dimensionality compression in the channel dimension, so that the output feature map size becomes ; in Indicates the compression ratio; S32: Input the output feature map into another convolutional layer with a kernel size of 1×1 to restore the feature map's dimensions to their original size. Calculate the three-dimensional feature weights by size; repeat this step to generate three-dimensional feature weights that correspond one-to-one with the feature scale, i.e. indivual The feature weight vectors of varying sizes are used to model multi-scale feature relationships through the above steps. S33: To achieve the reconstruction of multi-scale features, the obtained... indivual The feature weight vectors of different sizes are multiplied by the input feature vectors of the corresponding scales, which in turn activate important features at the corresponding scales while suppressing unimportant features, thereby enabling the selection of features for crop leaf diseases. S34: To further realize multi-scale feature selection, the output vector with feature weights is mapped to the (0,1) interval by the Softmax function in the scale dimension. That is, the sum of the corresponding element values ​​of the three-dimensional feature weight vectors of each scale is 1. Finally, the feature vectors after feature redistribution are added to obtain the final output vector, that is, the adaptive selection of features is realized. Step 4: Combine the modules from Step 1, Step 2, and Step 3 into a multi-scale adaptive feature extraction module; Step 5: Assemble a feature extraction module group by superimposing multi-scale adaptive feature extraction modules. Design several convolution groups according to the different sizes of the output feature maps and combine them into a recognition model.

2. The neural network construction method for identifying crop leaf diseases according to claim 1, characterized in that, Step one specifically includes: The pyramid multi-scale convolutional feature extraction module uses various convolutional kernels of different sizes according to the convolutional group to which it belongs, from the input to the output direction of the superimposed feature extraction module group; The "Pyramid Multi-Scale Convolution Feature Extraction Module" in the first convolutional group consists of four types of convolutional kernels: 3×3 convolutional kernel, 5×5 convolutional kernel, 7×7 convolutional kernel, and 9×9 convolutional kernel. The second group consists of three types of convolution kernels: 3×3 convolution kernel, 5×5 convolution kernel, and 7×7 convolution kernel. The third group consists of two types of convolution kernels: 3×3 convolution kernels and 5×5 convolution kernels. The fourth group consists of 3×3 convolution kernels, with only one type of convolution kernel. Each convolutional layer is followed by a regularization layer and a ReLU activation function layer, thus forming a pyramid feature extraction structure both within the convolutional module and between each convolutional group; The original input image is 224×224 pixels. After passing through a convolutional layer with a kernel size of 7×7 pixels and a stride of 2, the input image with 3 channels is transformed into a feature map with 64 channels and a size of 112×112 pixels. After passing through a max pooling layer with a pooling window of 3×3 pixels, a stride of 2, and padding of 1, the output size becomes 56×56 pixels; The feature map is input into the first convolutional group, where it undergoes convolution operations with 3×3, 5×5, 7×7, and 9×9 kernels respectively to obtain four feature maps with a size of 56×56 pixels and 256 channels. These feature maps are then used by the channel and spatial dimension joint attention mechanism and relationship modeling module to implement relationship modeling. The output feature maps of the second, third, and fourth groups have sizes of 28×28 pixels, 14×14 pixels, and 7×7 pixels, respectively, and have 512, 1024, and 2048 channels, respectively.

3. A method for constructing a neural network for identifying foliar diseases in crops according to any one of claims 1-2, characterized in that, Each convolutional group contains two pyramid multi-scale convolutional feature extraction modules, and each pyramid multi-scale convolutional feature extraction module may contain multiple convolutional layers with different kernel sizes.

Images