Novel field crop leaf disease recognition system based on style and multi-scale feature extraction

Abstract

The invention provides a novel field crop leaf disease recognition system based on style and multi-scale feature extraction, and relates to the technical field of crop leaf disease recognition. Comprises: an image acquisition module for acquiring an original image of a leaf in a field; the data preprocessing module completes size normalization, illumination enhancement and noise reduction; the style re-calibration module SRMB is used for extracting and weighting calibration features; the multi-scale feature fusion module EMSF is used for aggregating information of different scales; and the classification output module is used for carrying out global pooling, full connection and Softmax on the enhanced feature map and outputting a disease category. According to the invention, the problem of insufficient field crop leaf disease identification accuracy in the prior art is solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of crop leaf disease recognition, and particularly relates to a novel field crop leaf disease recognition system based on style and multi-scale feature extraction. BACKGROUND

[0002] With the continuous growth of global population and the continuous development of agricultural production, timely identification and prevention of crop diseases become increasingly important. Pests and diseases not only reduce the yield and quality of crops, but also have an impact on human health and the ecological environment. Early prevention and control of diseases can recover part of the economic loss of agriculture, improve food safety and promote agricultural production. Crop disease symptoms usually first appear on the leaves, and the type of infected crop disease can be effectively determined by observing the characteristics of the leaves. Accurate and rapid identification of crop disease types is crucial for timely disaster relief measures and reducing yield and quality losses.

[0003] Research on crop diseases mainly focuses on detection, identification and classification, which requires accurate capture of disease characteristics, comparison with other types of diseases and classification. Traditional machine learning methods usually use image processing techniques and classifiers to identify plant diseases. Tian et al. proposed a method for identifying brown mottle disease of eggplant based on spot features in 2016. The H component of the HSI color space was used to extract the feature parameters of the spot area, and the feature parameters were selected to form a classification feature vector. The Fischer discriminant function was used for classification, and good experimental results were obtained. In addition, widely used machine learning methods include Bayesian model (BM), k-nearest neighbor (KNN), support vector machine (SVM), decision tree (DT), random forest tree (RF), etc. Machine learning has made great progress in the field of plant pest and disease identification. However, manually designed features require expensive resource conditions and professional knowledge, and are easily influenced by subjectivity. In addition, when there are many types of diseases and some symptoms are similar, these methods cannot meet the demand of modern agriculture for accurate identification of crop diseases.

[0004] In the past decade, with the continuous development and progress of machine learning, especially the progress of deep learning technology, the accuracy of crop leaf disease identification has been continuously improved, which has paved the way for more efficient and real-time disease detection. Lankarani et al. proposed two fast-converging models using fine-tuning and transfer learning strategies of AlexNet and VGG16, and achieved 97.29% and 97.49% recognition rates on the PlantVillage tomato dataset. Dogra et al. proposed CNN-VGG19 using two-stage fine-tuning and transfer learning, and achieved 93.0% recognition accuracy for rice leaf brown spot diseases. Reddy, Varma and Davuluri proposed using ResNet50 to extract color and texture features of plant leaf images, and then using a biological optimization method based on MRDOA to select the best features and design a simple classification convolutional neural network DLCNN for classifying selected plant leaf disease features in PlantVillage and rice datasets, with classification accuracies of 99.73% and 99.68%, respectively. Thakur proposed a lightweight hybrid model combining VisionTransformer and CNN, which achieved 98.86% accuracy and 98.9% precision on the PlantVillage dataset. Sunil used multi-level feature fusion with adaptive channel spatial and pixel attention mechanisms for tomato plant disease classification, achieving 99.88% training accuracy, 99.88% validation accuracy and 99.83% external test accuracy. The above studies prove the feasibility of deep learning in crop leaf disease identification, but mainly focus on single background disease identification scenarios, which are not suitable for leaf disease identification in field environments with complex background interference.

[0005] Deep learning models have excellent algorithm performance and the ability to directly extract deep features from images, which also show good applicability in crop disease recognition. Higher people proposed an apple leaf disease recognition model (BAM-Net), which uses an aggregated coordinate attention mechanism to enhance the network's attention to disease features, and introduces a multi-scale feature refinement module to improve the network's ability to distinguish similar disease features, achieving an accuracy of 95.64% on the test set. Li et al. based on ConvNeXt, proposed a pepper leaf disease classification and recognition model MCCM, in which the multi-scale feature fusion module (MSFFM) captures disease features of various sizes and locations in the image, and the mixed channel spatial attention mechanism (MCSAM) enhances the correlation between non-local channels and spatial features, enhancing the model's extraction of basic features of pepper leaf diseases. The accuracy on the test set reached 93.5%. Hu et al. proposed a lightweight model LFMNet for high-similarity corn leaf disease recognition, which replaced the adaptive pool kernel according to the size of the input feature map on the original PPA and reshaped the convolution layer. Instead of different pooling kernels, different scales of features are obtained based on GMDC, and a feature weighting matrix is generated to enhance the feature extraction of corn leaves in complex backgrounds, with an accuracy of 94.12% on the test dataset. Dai proposed a new DFN-PSAN model, which contains a multi-level deep information feature fusion network (DFN) that can effectively extract and fuse relevant features from different network layers, improve the positioning of infected plant disease areas, use PyramidSqueeze Attention (PSA) to fuse context information of different scales and produce better pixel-level attention, and use t-SNE and SHAP methods to improve the transparency of the model in feature clustering and multi-class disease attention discrimination. The above studies show that in the constantly changing natural environment, accurately identifying disease features is still a problem that has not been solved.

[0006] Inspired by the above research and a series of experimental explorations, the present study addresses the following key issues: (1) In the actual field environment, strong light causes the local color of the picture to be too bright or reflective, making it difficult to distinguish texture and color features, while weak light causes the image to be unclear and details to be lost; (2) In a high color complexity and high structural complexity background, more textures or colors similar to disease spots will be introduced, increasing the difficulty of model recognition; (3) When disease spots of different sizes coexist, the model's multi-scale feature extraction capability is insufficient. SUMMARY

[0007] In order to overcome the shortcomings of the prior art, the purpose of the present application is to provide a new field crop leaf disease recognition system based on style and multi-scale feature extraction, which solves the problem of insufficient accuracy of field crop leaf disease recognition in the prior art.

[0008] To achieve the above object, the present application provides the following scheme:

[0009] A novel field crop leaf disease identification system based on style and multi-scale feature extraction, comprising:

[0010] An image acquisition module is configured to acquire original images of field crop leaves in a field environment.

[0011] A data preprocessing module is connected to the image acquisition module and configured to perform size normalization, illumination enhancement, and noise suppression processing on the original images to output preprocessed images.

[0012] A style recalibration module SRMB is connected to the data preprocessing module and configured to perform feature extraction and calibration on the preprocessed images to obtain a second feature map.

[0013] A multi-scale feature fusion module EMSF is connected to the style recalibration module SRMB and configured to perform feature enhancement on the second feature map to obtain an enhanced feature map.

[0014] A classification output module is connected to the multi-scale feature fusion module EMSF and configured to perform global average pooling, full connection mapping, and Softmax processing on the enhanced feature map to output disease category data of the target leaf.

[0015] Preferably, the style recalibration module SRMB comprises:

[0016] A first extraction submodule is configured to extract a first feature map through a depth separable convolution.

[0017] A style vector submodule is configured to calculate the mean and standard deviation of each channel of the first feature map respectively to form a style vector through style pooling.

[0018] A style weight calculation submodule is configured to input the style vector into a channel full connection layer, a batch normalization layer, and a Sigmoid activation function to obtain a channel-level style weight.

[0019] A second extraction submodule is configured to generate a second feature map recalibrated by style by weighting the first feature map using the channel-level style weight.

[0020] Preferably, the multi-scale feature fusion module EMSF comprises:

[0021] A third extraction submodule is configured to extract grouped features through two 1x1 convolution branches and one 3x3 convolution branch for each sub-feature group of the second feature map to obtain a first convolution result and a second convolution result.

[0022] A convolution processing submodule is configured to multiply and aggregate the first convolution result and to expand the context of the second convolution result, to obtain a first processing result and a second processing result.

[0023] A splicing submodule is configured to aggregate and splice the first processing result and the second processing result, to obtain an enhanced feature map.

[0024] The present application discloses the following technical effects:

[0025] The present application provides a novel field crop leaf disease recognition system based on style and multi-scale feature extraction, comprising: an image acquisition module, configured to acquire an original image of a field crop leaf in a field environment; a data preprocessing module connected with the image acquisition module, configured to perform size normalization, illumination enhancement and noise suppression processing on the original image, and output a preprocessed image; a style recalibration module SRMB connected with the data preprocessing module, configured to perform feature extraction and calibration on the preprocessed image, to obtain a second feature map; a multi-scale feature fusion module EMSF connected with the style recalibration module SRMB, configured to perform feature enhancement on the second feature map, to obtain an enhanced feature map; and a classification output module connected with the multi-scale feature fusion module EMSF, configured to perform global average pooling, full connection mapping and Softmax processing on the enhanced feature map, and output disease category data of a target leaf. The present application introduces style pooling to respectively map overall intensity information and style changes such as hue, brightness and contrast through mean and standard deviation, then recalibrates the channel weight using style features, reweights each channel of the input feature map, suppresses background information and expands the representation of small target lesions; different scales of features are processed in parallel through multi-branch grouping, global and local information of the leaf is dynamically captured and fused, the ability of the network to focus on the distribution of the lesion on the whole leaf and small regional information such as specific morphology and edge features of the lesion is strengthened, and finally the relationship and dependence between clustered small lesions and the whole leaf features are captured through cross-space information aggregation, thereby improving the field crop leaf disease recognition accuracy. BRIEF DESCRIPTION OF DRAWINGS

[0026] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0027] Figure 1 A structure schematic diagram of a novel field crop leaf disease recognition system based on style and multi-scale feature extraction is provided for the embodiments of the present application.

[0028] Figure 2 A network structure detail schematic diagram of a new field crop leaf disease recognition system based on style and multi-scale feature extraction is provided for an embodiment of the present application.

[0029] Figure 3 An SRMB structure schematic diagram is provided for an embodiment of the present application.

[0030] Figure 4 An EMSF structure schematic diagram is provided for an embodiment of the present application.

[0031] Explanation of reference signs:

[0032] 1-image acquisition module, 2-data preprocessing module, 3-style recalibration module SRMB, 4-multi-scale feature fusion module EMSF, 5-classification output module. DETAILED DESCRIPTION

[0033] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0034] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the present application will be further described in detail below with reference to the drawings and specific embodiments.

[0035] As shown in the drawings, the present application provides a new field crop leaf disease recognition system based on style and multi-scale feature extraction, comprising: Figure 1 An image acquisition module 1 is used to acquire original images of field crop leaves in a field environment;

[0036] A data preprocessing module 2 is connected with the image acquisition module 1 and is used to perform size normalization, illumination enhancement and noise suppression processing on the original images, and output preprocessed images;

[0037] A style recalibration module SRMB 3 is connected with the data preprocessing module 2 and is used to perform feature extraction and calibration on the preprocessed images to obtain a second feature map;

[0038] A multi-scale feature fusion module EMSF 4 is connected with the style recalibration module SRMB 3 and is used to perform feature enhancement on the second feature map to obtain an enhanced feature map;

[0039]

[0040] ​The classification output module 5 is connected with the multi-scale feature fusion module EMSF 4, and is used for global average pooling, full connection mapping and Softmax processing on the enhanced feature map, and outputs disease category data of the target leaf.

[0041] Specifically, as shown in Figure 2 The application is applied to identification of field crop leaf diseases in a complex background. By introducing a style-based recalibration module (Style-based Recalibration MBconv, SRMB) and a multi-scale feature fusion module (Extended Multi-scale Fused-Mbconv, EMSF), the recognition ability of the model for small disease spots in a complex background is improved.

[0042] In the processing process, first, the basic features of the image are extracted through a 3*3 convolution layer, then the SRMB module is proposed to improve the SE module in the Mobile Inverted Bottleneck Convolution (MBConv), and the style feature calibration channel weight is introduced to enhance the model's ability to perceive the color and texture of small disease spots in a complex background. Then, the EMSF module is proposed to improve the ability of Fused-MBconv to capture the subtle feature differences of small crop leaf diseases. Through multi-scale convolution kernel parallel processing of features, the features of different size disease spots are captured and the information of different scales is fused, further improving the adaptability of the model to the size change of the disease spots. After multiple MBConv processing, the feature map extracts global information through the global average pooling (GAP) layer, enters the full connection layer for feature mapping, and finally outputs the classification probability through the Softmax layer, for accurate classification of field crop leaf diseases. The whole process can effectively distinguish the small disease spot features in a complex background while ensuring the computing efficiency, and improve the precision of disease detection.

[0043] Further, the style recalibration module SRMB 3 comprises:

[0044] The first extraction submodule is configured to extract a first feature map through a depth separable convolution;

[0045] The style vector submodule is configured to calculate the mean and standard deviation of each channel of the first feature map respectively to form a style vector by using style pooling;

[0046] The style weight calculation submodule is configured to input the style vector into a channel full connection layer, a batch normalization layer and a Sigmoid activation function to obtain a channel-level style weight;

[0047] A second extraction submodule is configured to weight the first feature map using the channel-level style weight to generate a second feature map that is style-reweighted.

[0048] Specifically, in the face of complex images containing high color complexity and high structural complexity background: feature dilution: the global average pooling in the SE (Squeeze-and-Excitation) module dilutes the small area but information-rich lesion features, especially when the lesion coexists with extensive complex background. This results in small lesions obtaining lower weights in the reweighting process, thereby affecting the recognition ability of the model. Background interference: although the SE module can enhance specific features by adjusting channel weights, the SE module lacks spatial differentiation ability, so there are still some complex background features interfering with the ability of SE in the local region.

[0049] In order to overcome the limitations of the above-mentioned SE, a new SRMB module is proposed herein, the structure of which is as shown in Figure 3 First, a depth separable convolution is performed on each input channel, and then two consecutive submodules are used: style pooling and style integration. Style pooling extracts style information (mean and standard deviation) from each channel of the feature map to obtain a style feature matrix T ∈ R C×2 , where C represents the number of channels of the feature map, and each row corresponds to the mean and standard deviation of a channel.

[0050] In the context of agricultural disease recognition, style features refer to a set of features that describe the overall visual appearance of a crop leaf disease image, including but not limited to texture, color, shape, and spatial layout. These features reflect the overall "style" of the image, helping to distinguish different types of diseases and healthy states. Specifically, it includes: texture features: describe the visual patterns and structures on the leaf surface, including the texture of leaf veins, spots and lesions, which help to identify pathological changes on the leaf. Color patterns: diseases change the color of the leaf, including the distribution, intensity and change of color. Shape features: geometric features of target objects (leaves and lesions), including size, contour and overall structure of shape. Spatial layout: distribution pattern of lesions, i.e. the position and relative arrangement of object elements in the image.

[0051] Assume that the input feature map is X ∈ R C×H×W , where C is the number of channels, H is the height, and W is the width. Calculate the mean μ c and standard deviation σ c for each channel c:

[0052] (1);

[0053] (2);

[0054] Style vector, t c ∈ R2 As a summary description of the style information of each channel c:

[0055] t c = [μ c , σ c ] (3);

[0056] Style integration, style features are converted to channel style weights by style integration operators. A simple combination of channel-level fully connected (CFC) layers, batch normalization (BN) layers and sigmoid activation functions is used to encode T, given that style representation T ∈ R C×2 As input, the style integration operator uses a learnable parameter W ∈ R C×2 Perform channel encoding:

[0057] z c = W c · t c (4);

[0058] Where Z ∈ R C is the encoded style feature. Then, the channel weight G is generated by the batch normalization (BN) layer and the sigmoid activation function:

[0059] (5);

[0060] (6);

[0061] (7);

[0062] (8);

[0063] Where γ and β ∈ RC are affine transformation parameters, and G ∈ RC represents the channel-level style weight.

[0064] BN uses fixed mean and variance approximations, which allows the BN layer to be merged into the preceding CFC layer. Therefore, style integration for each channel can be reduced to a single CFC layer followed by an activation function Finally, the original input X is recalibrated by the weight G to obtain the output:

[0065] (9);

[0066] Finally, adjust the number of channels using 1x1 PointwiseConvolution.

[0067] SRMB extracts style information from feature maps by specific style pooling, mean, variance, gradient distribution. Comprehensive consideration of the statistical properties and local details of the image, capturing local texture and color changes, helps to understand the image content and thus helps to identify small or initial subtle changes in disease spots. According to the extracted style features, weights are generated to adjust the importance of the feature channels. Different diseases and plant combinations show different visual style features, and each group of crop disease images has one or more fixed styles (texture, color distribution, and other structural information of disease spots). SRMB optimizes the model by learning the style differences. The generated weights are used to adjust each channel of the original feature map, thereby enhancing the disease spot-related features and suppressing irrelevant or interfering background features.

[0068] Further, the multi-scale feature fusion module EMSF4 comprises:

[0069] A third extraction submodule is configured to extract grouped features from each sub-feature group of the second feature map through two 1x1 convolution branches and one 3x3 convolution branch, respectively, to obtain a first convolution result and a second convolution result.

[0070] A convolution processing submodule is configured to perform multiplication aggregation on the first convolution result and context expansion on the second convolution result, to obtain a first processing result and a second processing result.

[0071] A splicing submodule is configured to aggregate and splice the first processing result and the second processing result to obtain an enhanced feature map.

[0072] Specifically, diseases have complex symptoms and morphological characteristics at different growth stages and scales. Diseases have different characteristics at the same scale, and the combination of global and local information is needed to determine the disease type. On the one hand, in order to capture the scattered disease spots in the leaf, we need to observe the large-scale and coarse-grained features of the leaf. On the other hand, in order to distinguish the subtle color, edge morphology and texture change differences between the disease spots and the healthy leaves under uneven lighting and leaf shading, we need to check the small-scale and fine-grained features of the leaf. Therefore, the multi-scale information of the leaf disease features in the image is crucial for accurately identifying the leaf disease type.

[0073] The EMSF module is a multi-scale feature extraction module improved on the basis of Fused_MBconv. A parallel multi-branch structure is used to achieve multi-scale feature extraction without multiple convolution layers or pooling layers. As shown in FIG. 4, the EMSF module is composed of a feature extraction submodule, a feature fusion submodule, and a feature splicing submodule. Figure 4As shown, the EMSF first extracts local features through fusion depth separable convolution to generate a feature map B with a size of CxHxW. The EMSF divides B into G (where G << C) sub-features in the cross-channel dimension to learn different semantics. Second, attention weight descriptors of the grouped feature maps are extracted through two parallel routes of 1x1 branches and one 3x3 branch. On the one hand, in order to achieve different cross-channel interaction features between the two parallel paths in the 1x1 branch, the two channel attention maps within each group are aggregated by simple multiplication to obtain the first convolution result. On the other hand, the 3x3 branch captures local cross-channel interaction through 3x3 convolution to expand the feature space and obtain the second convolution result. In this way, the EMSF extracts features at different scales, which can capture both fine-grained local lesion changes and large-scale global lesion distribution. In this way, the model can better adapt to lesions of different sizes and shapes. Then the EMSF uses a cross-space information aggregation method in different spatial dimensions to achieve more rich feature aggregation and generate an enhanced feature map B. Finally, the enhanced feature map B is further integrated and compressed through 1x1 convolution to generate the final output feature map (feature enhancement map).

[0074] The EMSF extracts features at different scales through feature grouping, parallel multi-scale processing, feature fusion and enhancement of the final output feature map, enhances the multi-scale feature extraction capability of the model, allows the module to capture multi-level features from fine to coarse, and adapts to the diversity and complexity of leaf disease features. Cross-channel and cross-space feature fusion enables the EMSF to independently learn and extract features in each group by dividing the feature map B in the channel dimension into multiple feature groups, which increases the sensitivity of the model to different lesion features. The parallel branches of 1x1 and 3x3 convolution kernels can extract detailed and contextual information respectively without increasing excessive computational burden, and effectively integrate local and global information.

[0075] The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0076] The principles and implementation modes of the present application are described by using specific examples in this specification, and the above description of the embodiments is only used to help understand the method of the present application and its core idea; at the same time, for those skilled in the art, according to the idea of the present application, the specific implementation mode and application range will be changed. In view of the above, the content of the specification should not be understood as a limitation of the present application.

Claims

1. A novel field crop leaf disease identification system based on style and multi-scale feature extraction, characterized in that, The application relates to a leaf disease diagnosis method based on style re-calibration and multi-scale feature fusion. An image acquisition module is used for acquiring an original image of a field crop leaf in a field environment; A data preprocessing module is connected with the image acquisition module and is used for performing size normalization, illumination enhancement and noise suppression processing on the original image to output a pretreated image; A style re-calibration module SRMB is connected with the data preprocessing module and is used for performing feature extraction and calibration on the pretreated image to obtain a second feature map; A multi-scale feature fusion module EMSF is connected with the style re-calibration module SRMB and is used for performing feature enhancement on the second feature map to obtain an enhanced feature map; A classification output module is connected with the multi-scale feature fusion module EMSF and is used for performing global average pooling, full connection mapping and Softmax processing on the enhanced feature map to output disease category data of a target leaf.

2. The novel field crop leaf disease identification system based on style and multi-scale feature extraction according to claim 1, characterized in that, The style re-calibration module SRMB comprises: A first extraction submodule is used for extracting a first feature map through a depth separable convolution; A style vector submodule is used for respectively calculating the mean and standard deviation of each channel of the first feature map to form a style vector through style pooling; A style weight calculation submodule is used for inputting the style vector into a channel full connection layer, a batch normalization layer and a Sigmoid activation function to obtain channel-level style weights; A second extraction submodule is used for weighting the first feature map through the channel-level style weights to generate a second feature map re-calibrated through a style.

3. The novel system for identifying leaf diseases of field crops based on style and multi-scale feature extraction according to claim 1, characterized in that, The multi-scale feature fusion module EMSF comprises: A third extraction submodule is used for extracting grouped features through two 1x1 convolution branches and one 3x3 convolution branch for each sub-feature group of the second feature map to obtain a first convolution result and a second convolution result; A convolution processing submodule is used for performing multiplication aggregation on the first convolution result and context expansion on the second convolution result to obtain a first processing result and a second processing result; A splicing submodule is used for aggregating and splicing the first processing result and the second processing result to obtain an enhanced feature map.

4. The novel field crop leaf disease identification system based on style and multi-scale feature extraction according to claim 1, characterized in that, The calculation expression of the mean is: ； wherein the input feature map is X e R C×H×W , C is the number of channels, H is the height, and W is the width.

5. The novel system for identification of leaf diseases in field crops based on style and multi-scale feature extraction as claimed in claim 1 wherein, The calculation expression of the standard deviation is: ； wherein the input feature map is X e R C×H×W , C is the number of channels, H is the height, W is the width, and c is the mean value of channel c.

Images