Multi-scale plant disease recognition method, device, storage medium and apparatus

Abstract

The application discloses a multi-scale plant disease identification method and device, equipment, a storage medium and the like, and the improved MSS-ResNet101 model is used for identifying the plant image obtained after data enhancement preprocessing, the plant disease image is determined according to the feature recognition result, the improved MSS-ResNet101 model comprises a multi-scale structure Inception module replacing the 7*7 convolution in the original ResNet101 model and a residual module integrating the convolution kernel attention mechanism, and the disease type is determined based on the plant disease image. Since the improved MSS-ResNet101 model is used for determining the disease type, the improved MSS-ResNet101 model is used for disease identification, the recognition efficiency of various types of plant disease images in a natural scene is improved, and the recognition accuracy is ensured.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a multi-scale plant disease identification method, device, storage medium and apparatus. Background Technology

[0002] With the development of agricultural economy, more and more plants are being planted and cultivated using mechanized methods. However, various factors can affect plant growth during the actual planting process. Among them, diseases are a major factor affecting plant growth. Every year, a large number of crops are attacked by different plant diseases, causing significant losses. Therefore, in order to ensure crop yield and plant lifespan, it is necessary to accurately detect and identify plant diseases.

[0003] Although existing technologies have computational methods for identifying plant diseases, current plant disease identification models suffer from poor recognition performance in natural scenes due to factors such as lighting conditions, shooting angle, and weather. The backgrounds are often complex, and disease features may appear anywhere in the image, leading to low identification efficiency and data errors that result in poor accuracy.

[0004] The above content is only used to help understand the technical solution of the present invention and does not represent an admission that the above content is prior art. Summary of the Invention

[0005] The main objective of this invention is to provide a multi-scale plant disease identification method, device, storage medium, and apparatus, aiming to solve the technical problem that in the prior art, the identification effect of the network model is poor when identifying disease images in natural scenes due to various factors, where disease features may appear at arbitrary locations in the image.

[0006] To achieve the above objectives, the present invention provides a multi-scale plant disease identification method, which includes the following steps:

[0007] Data augmentation preprocessing is performed on the target plant image to obtain the processed plant image;

[0008] The improved MSS-ResNet101 model uses a multi-scale Inception module to identify plant disease features in processed plant images. Based on the feature identification results, the model determines the plant disease image. The improved MSS-ResNet101 model includes a multi-scale Inception module that replaces the 7×7 convolution in the original ResNet101 model and a residual module that incorporates a convolutional kernel attention mechanism. The multi-scale Inception module is connected to the residual module, which in turn is connected to a global average pooling layer. The multi-scale Inception module extracts features from the input image and inputs the obtained multi-scale feature map to the residual module. The residual module fuses the multi-scale feature map according to the convolutional kernel attention mechanism and inputs the fused feature information to a global average pooling layer and a Softmax layer for classification and identification.

[0009] The type of disease is determined based on the plant disease images.

[0010] Optionally, the data augmentation preprocessing includes four preprocessing steps: random cropping, rotation, color brightening, and Gaussian blur noise reduction. The step of performing data augmentation preprocessing on the target plant image to obtain the processed plant image includes:

[0011] The target plant image is subjected to four preprocessing steps: random cropping, rotation, color brightening, and Gaussian blur noise addition, to obtain the processed plant image.

[0012] Optionally, before the step of performing disease feature recognition on the processed plant image based on the multi-scale Inception module of the improved MSS-ResNet101 model, and determining the plant disease image based on the feature recognition results, the method further includes:

[0013] The ResNet101 network model was trained using a dataset of target plant images in natural environments as the model training dataset to obtain a pre-trained model.

[0014] Based on the pre-trained model, combined with the multi-scale Inception module and the convolutional kernel attention mechanism SKNet, an improved pre-trained model is obtained.

[0015] An improved MSS-ResNet101 model was constructed based on the pre-trained model.

[0016] Optionally, the step of constructing an improved MSS-ResNet101 model based on the pre-trained model includes:

[0017] The improved pre-trained model was tested using a test set of target plant images in natural environments without image preprocessing, and the test results were obtained.

[0018] The test results are subjected to confusion matrix analysis, and the improved pre-trained model is output as the improved MSS-ResNet101 model based on the analysis results.

[0019] Optionally, the step of using the multi-scale Inception module in the improved MSS-ResNet101 model to identify disease features in the processed plant images and determining the plant disease images based on the feature identification results includes:

[0020] The Inception module of the improved MSS-ResNet101 model is used to extract multi-scale features from the processed plant images to obtain multi-scale structural feature maps.

[0021] Based on the SKNet convolutional kernel attention mechanism in the residual structure of the improved MSS-ResNet101 model, feature fusion is performed on the multi-scale structural feature map to obtain target feature maps of different scales.

[0022] Disease features are identified in the target feature map, and plant disease images are determined based on the feature identification results.

[0023] Optionally, the convolutional kernel attention mechanism SKNet consists of three operations: Split, Fuse, and Select; the step of fusing features of the multi-scale structural feature map based on the convolutional kernel attention mechanism SKNet in the residual structure of the improved MSS-ResNet101 model to obtain target feature maps of different scales includes:

[0024] Based on the Split operation corresponding to the convolution kernel attention mechanism in the residual structure of the improved MSS-ResNet101 model, the multi-scale structure feature map is subjected to multi-path convolution to obtain the convolutional feature map.

[0025] Based on the Fuse operation, a new feature map is obtained by summing the convolved feature map element by element.

[0026] The new feature map is fused according to the Select operation to obtain target feature maps of different scales.

[0027] Optionally, the step of determining the disease type based on the plant disease image includes:

[0028] The plant disease images are compared with a lesion feature database corresponding to preset disease types to obtain comparison results.

[0029] The disease type is determined based on the comparison results and the preset similarity rate.

[0030] Furthermore, to achieve the above objectives, the present invention also proposes a multi-scale plant disease identification device, which includes a memory, a processor, and a multi-scale plant disease identification program stored in the memory and executable on the processor. The multi-scale plant disease identification program is configured to implement the steps of multi-scale plant disease identification as described above.

[0031] Furthermore, to achieve the above objectives, the present invention also proposes a storage medium storing a multi-scale plant disease identification program, wherein the multi-scale plant disease identification program, when executed by a processor, implements the steps of the multi-scale plant disease identification method as described above.

[0032] Furthermore, to achieve the above objectives, the present invention also proposes a multi-scale plant disease identification device, the multi-scale plant disease identification device comprising:

[0033] The image preprocessing module is used to perform data augmentation preprocessing on the target plant image to obtain the processed plant image;

[0034] The disease image recognition module is used to identify disease features in processed plant images based on an improved MSS-ResNet101 model, and to determine plant disease images based on the feature recognition results. The improved MSS-ResNet101 model is a model obtained by training the MSS-ResNet101 model under high temperature conditions by combining the soft labels of the corresponding teacher model with the hard labels of the actual Solanaceae dataset through knowledge distillation. The loss function of the MSS-ResNet101 model is a hybrid loss function.

[0035] The disease type determination module is used to determine the disease type based on the plant disease image.

[0036] This invention obtains processed plant images by performing data augmentation preprocessing on target plant images; it then uses the multi-scale Inception module in an improved MSS-ResNet101 model to identify disease features in the processed plant images, and determines the plant disease image based on the feature identification results. The improved MSS-ResNet101 model includes a multi-scale Inception module that replaces the 7×7 convolution in the original ResNet101 model and a residual module that incorporates a convolutional kernel attention mechanism. The multi-scale Inception module is connected to the residual module, and the residual module is connected to a global average pooling layer. The multi-scale Inception module is used to extract features from the input image and input the obtained multi-scale structure feature map to the residual module. The residual module is used to fuse the multi-scale structure feature map according to the convolutional kernel attention mechanism, and input the fused feature information to the global average pooling layer and the Softmax layer for classification and identification; finally, the disease type is determined based on the plant disease image. This invention uses an improved MSS-ResNet101 model to identify disease features in plant images obtained through data augmentation preprocessing, and determines plant disease images based on the feature identification results, thereby further identifying the disease type. Compared to existing technologies where disease features may appear at arbitrary locations in natural scenes due to various factors, resulting in poor recognition performance of network models, this invention achieves disease identification through an improved MSS-ResNet101 model, improving the efficiency of identifying various types of plant disease images in natural scenes while ensuring recognition accuracy. Attached Figure Description

[0037] Figure 1 This is a schematic diagram of the structure of a multi-scale plant disease identification device in the hardware operating environment involved in the embodiments of the present invention;

[0038] Figure 2 This is a flowchart illustrating the first embodiment of the multi-scale plant disease identification method of the present invention.

[0039] Figure 3 This is a schematic diagram of the overall structure of the MSS-ResNet101 model in the first embodiment of the multi-scale plant disease identification method of the present invention.

[0040] Figure 4 This is a comparison of the average recognition rates of different models in the first embodiment of the multi-scale plant disease identification method of the present invention.

[0041] Figure 5 This is a schematic diagram of the loss value variation curves of different models in the first embodiment of the multi-scale plant disease identification method of the present invention;

[0042] Figure 6 This is a flowchart illustrating the second embodiment of the multi-scale plant disease identification method of the present invention.

[0043] Figure 7 This is a schematic diagram of the improved residual structure of the second embodiment of the multi-scale plant disease identification method of the present invention;

[0044] Figure 8 This is a schematic diagram of the SKNet structure in the second embodiment of the multi-scale plant disease identification method of the present invention;

[0045] Figure 9 This is a structural block diagram of the first embodiment of the multi-scale plant disease identification device of the present invention.

[0046] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0047] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0048] Reference Figure 1 , Figure 1 This is a schematic diagram of the structure of a multi-scale plant disease identification device in the hardware operating environment of an embodiment of the present invention.

[0049] like Figure 1 As shown, the multi-scale plant disease identification device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen, and optionally, it may also include a standard wired interface or a wireless interface. In this invention, the wired interface of the user interface 1003 may be a USB interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wireless-Fidelity (Wi-Fi) interface). The memory 1005 may be a high-speed random access memory (RAM) or a non-volatile memory (NVM), such as a disk storage device. The memory 1005 may also optionally be a storage device independent of the aforementioned processor 1001.

[0050] Those skilled in the art will understand that Figure 1The structure shown does not constitute a limitation on multi-scale plant disease identification devices and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0051] like Figure 1 As shown, the memory 1005, which is identified as a computer storage medium, may include an operating system, a network communication module, a user interface module, and a multi-scale plant disease identification program.

[0052] exist Figure 1 In the multi-scale plant disease identification device shown, the network interface 1004 is mainly used to connect to the backend server and communicate data with the backend server; the user interface 1003 is mainly used to connect to the user device; the multi-scale plant disease identification device calls the multi-scale plant disease identification program stored in the memory 1005 through the processor 1001 and executes the multi-scale plant disease identification method provided in the embodiment of the present invention.

[0053] Based on the above hardware structure, an embodiment of the multi-scale plant disease identification method of the present invention is proposed.

[0054] Reference Figure 2 , Figure 2 This is a flowchart illustrating the first embodiment of the multi-scale plant disease identification method of the present invention, which presents the first embodiment of the multi-scale plant disease identification method of the present invention.

[0055] In this embodiment, the multi-scale plant disease identification method includes the following steps:

[0056] Step S10: Perform data augmentation preprocessing on the target plant image to obtain the processed plant image.

[0057] It should be noted that the executing entity in this embodiment can be a device with plant disease identification function in natural scenes, such as a computer, laptop, PC, or tablet, or other plant disease identification devices that can achieve the same or similar functions. This embodiment does not limit this. This embodiment and the following embodiments will be described using the aforementioned computer as an example.

[0058] Understandably, the target plant image can refer to an image that needs to be identified for disease identification. The image includes images of leaves with lesions corresponding to the plant to be identified, as well as images of normal leaves. In order to accurately identify leaf images with lesions, disease images need to be screened out from the target plant images before disease identification. Therefore, the plant images need to be preprocessed to accurately identify disease images.

[0059] It should be understood that the plants to be identified include, but are not limited to, tomatoes, eggplants, peppers, and potatoes. These plants are all widely cultivated crops with high economic value and can also be plants commonly cultivated. This embodiment does not impose specific limitations on them.

[0060] Furthermore, the data augmentation preprocessing includes four preprocessing steps: random cropping, rotation, color brightening, and Gaussian blur noise addition. Step S10 includes performing the four preprocessing steps of random cropping, rotation, color brightening, and Gaussian blur noise addition on the target plant image to obtain the processed plant image.

[0061] It should be noted that for image recognition tasks, preprocessing significantly improves the quality of feature extraction, thereby enhancing the model's recognition performance. Therefore, image preprocessing is a crucial operation. Image preprocessing can modify sample images according to the requirements of the network model, reducing the interference of negative information and highlighting positive information to meet the actual data input requirements.

[0062] Understandably, real-world agricultural scenarios are complex and varied. Preprocessing the dataset using appropriate methods can effectively address the above issues and improve the model's disease recognition performance in natural scenes. Since data distribution can be uneven, it can negatively impact the model's recognition accuracy. Therefore, data augmentation preprocessing methods can be used to process the images to be recognized. The data augmentation methods employed are as follows: 1) Random cropping: Randomly selecting image pixel positions and orientations for cropping. 2) Rotation: Randomly rotating the image by 90, 180, and 270 degrees. 3) Color brightening: Adjusting hue (H), saturation (S), and brightness (V) to achieve image brightening. 4) Gaussian blur noise addition: Due to the varying image sharpness in actual shooting scenarios, Gaussian noise is added to the input image to fit the scene the model is suited for recognition.

[0063] In practice, the above-mentioned data augmentation methods can be used to highlight various features in the input plant image, such as disease features, color features, and texture features, so that the Solanaceae disease identification model can accurately extract disease features from the processed plant image for identification.

[0064] Step S20: Based on the multi-scale Inception module in the improved MSS-ResNet101 model, disease features are identified in the processed plant images. The plant disease images are determined according to the feature identification results. The improved MSS-ResNet101 model includes a multi-scale Inception module that replaces the 7×7 convolution in the original ResNet101 model and a residual module that incorporates a convolutional kernel attention mechanism. The multi-scale Inception module is connected to the residual module, and the residual module is connected to the global average pooling layer. The multi-scale Inception module is used to extract features from the input image and input the obtained multi-scale structure feature map to the residual module. The residual module is used to perform feature fusion on the multi-scale structure feature map according to the convolutional kernel attention mechanism and input the fused feature information to the global average pooling layer and the Softmax layer for classification and recognition.

[0065] It should be noted that the improved MSS-ResNet101 model is an improved version of the ResNet101 model. The ResNet101 model consists of four Bottlenecks, each block of which contains four residual modules. At the very front and very back of the network, there is a 7x7 convolutional layer, a maxpool layer, and an average pooling layer, respectively.

[0066] Understandably, the improved MSS-ResNet101 model includes a multi-scale Inception module that replaces the 7×7 convolution in the original ResNet101 model and a residual module that incorporates a convolutional kernel attention mechanism. The multi-scale Inception module is connected to the residual module, and the residual module is connected to a global average pooling layer. The multi-scale Inception module is used to extract features from the input image and input the obtained multi-scale structure feature map to the residual module. The residual module is used to perform feature fusion on the multi-scale structure feature map according to the convolutional kernel attention mechanism and input the fused feature information to the global average pooling layer and the Softmax layer for classification and recognition.

[0067] In the specific implementation, the processed plant image is used to extract features through the multi-scale structure Inception module in the improved MSS-ResNet101 model to obtain a multi-scale structure feature map. The multi-scale structure feature map is then fused according to the convolutional kernel attention mechanism in the residual module. The fused feature information is then input into the global average pooling layer and the Softmax layer for disease feature recognition. The plant disease image is determined based on the feature recognition results.

[0068] Furthermore, before step S20, the method further includes: using a dataset of target plant images in a natural environment as a model training dataset to train the ResNet101 network model to obtain a pre-trained model; combining the pre-trained model with the multi-scale Inception module and the convolutional kernel attention mechanism SKNet to obtain an improved pre-trained model; and constructing an improved MSS-ResNet101 model based on the pre-trained model.

[0069] It should be noted that this approach first trains the ResNet101 network model using a dataset of target plant images from natural environments as the model training dataset, obtaining a pre-trained model. However, plant images from natural environments are subject to interference from environmental factors, which become noise in the model's recognition process. This noise information is propagated during model training, and its weight increases with the depth of the network layers, ultimately affecting the model's recognition performance. Therefore, further improvements to the ResNet101 pre-trained model are needed. Thus, the SKNet convolutional kernel attention mechanism is introduced into the ResNet101 residual structure. SKNet effectively helps the model capture semantic information useful for the recognition task, suppresses the influence of noise and other interference factors, highlights key regions, and enhances the model's expressive power. Since the receptive field of a single-scale convolutional kernel is fixed, the extracted feature information is limited. Therefore, the Inception multi-scale structure Inception module is embedded into ResNet101 for feature extraction, enhancing the richness of feature channels and thus achieving better recognition results and improving recognition accuracy. These improvements result in an improved pre-trained model. An improved MSS-ResNet101 model was constructed based on the improved pre-trained model.

[0070] Understandably, the improved MSS-ResNet101 model is based on the ResNet101 network model, combined with the multi-scale Inception module and the convolutional kernel attention mechanism SKNet, resulting in an improved multi-scale network model that integrates convolutional kernel attention mechanisms (Multi-Scale-SK-ResNet101, hereinafter referred to as MSS-ResNet101). A schematic diagram of the overall structure of the MSS-ResNet101 model is shown below. Figure 3 As shown, the Inception module replaces the 7×7 convolutions in the original ResNet101, and a convolutional kernel attention mechanism is incorporated into each residual module. Finally, the global average pooling layer and the Softmax layer are retained.

[0071] It should be understood that in constructing traditional convolutional neural networks, a single type of convolutional kernel is typically used in each layer, and the size of the kernel is fixed, as is the size of its receptive field. However, the effect of fusing different convolutional kernels is not considered. Therefore, this proposal suggests a selective convolutional kernel mechanism, SKNet, which uses a non-linear method to fuse feature information from different convolutional kernels, thereby enabling neurons to adaptively adjust the size of their receptive fields according to the input.

[0072] In the specific implementation, to verify the recognition performance of the proposed model in natural scenes, experiments were conducted to compare the model trained on the Dataset of Tomato Leaves in a natural environment. VGG16, InceptionV3, and ResNet101 were selected as the original comparison models, and SENet attention mechanism and SKNet selective convolutional kernel attention mechanism were respectively introduced into ResNet101. Model validation was performed under the same experimental conditions. (See Table 1 and...) Figure 4 The comparison of average recognition rates of different models shows that ResNet101 demonstrates better performance in natural scenes compared to VGG16 and InceptionV3, with an average recognition rate of 83.65%. The ResNet101 model incorporating SKNet has an average recognition rate 2.44% higher than the ResNet101 model incorporating SENet, indicating that the ResNet101 model incorporating SKNet can better enhance lesion extraction and improve the recognition effect of tomato diseases in natural scenes. By incorporating the Inception multi-scale structure module on the basis of SKNet, the final recognition rate reaches 92.51%, verifying that the ResNet101 model incorporating the Inception multi-scale structure module can effectively improve the feature richness of tomato disease images with complex backgrounds and enhance the model's recognition performance in natural scenes. To further verify the effectiveness of the improved model in reducing interference from complex backgrounds, a test case of yellow leaf curl disease was used. Grad-CAM weighted class activation maps were used to present the lesion regions of the disease image as a heatmap. Feature visualization was performed on the original ResNet101 model, the ResNet101 model fused with SKNet, and the ResNet101 model that simultaneously fused multi-scale structures and the Softmax layer from SKNet. The fused images show that the SKNet-fused ResNet101 model highlights key local regions and enhances useful features compared to the original model. Further incorporating the Inception multi-scale structure into the ResNet101 model can, to some extent, reduce interference from complex backgrounds and highlight disease features. The improved ResNet101 model in this scheme can effectively extract tomato disease features and can be used to classify and identify different types of plant diseases in natural scenes.

[0073] Table 1 Comparison of recognition performance of different models

[0074]

[0075] Understandably, to determine the impact of SKNet and the Inception multi-scale structure's Inception module on the convergence of the plant disease identification model, loss function curves for different models were plotted, and the comparison results are as follows: Figure 5 The diagram shows the loss function curves of different models. As can be seen from the figure, ResNet101 has a slightly lower loss function value compared to VGG16 and InceptionV3 models. The ResNet101 model, which incorporates attention mechanisms SENet and SKNet, converges slightly faster and has a lower loss function value than the original model. Furthermore, SKNet can further remove interference information from complex backgrounds compared to SENet, while reducing the model's oversensitivity to the location of disease features, which is more conducive to model convergence. Meanwhile, the fusion of Inception modules in the multi-scale Inception structure improves the model's ability to fit disease features at different scales. Therefore, the model can converge quickly and achieve a very low loss function value for tomato disease image recognition in natural scenes.

[0076] Step S30: Determine the disease type based on the plant disease image.

[0077] It should be noted that disease types include the specific diseases affecting different plants. Different plants have different disease types and their symptoms will also differ. Disease types are usually determined by the characteristics of lesions on the leaves. For example, if the plant is a tomato, the disease type can be determined by identifying the lesion characteristics on the tomato leaves. These disease types may include bacterial leaf spot, early blight, late blight, leaf mold, leaf spot, two-spotted spider mite, ring spot, mosaic virus, yellow leaf curl, etc.

[0078] Understandably, by identifying lesions in plant disease images that contain disease characteristics, the type of disease corresponding to the disease image can be determined.

[0079] Furthermore, step S30 also includes: performing feature comparison between the plant disease image and a preset disease type lesion feature library to obtain a comparison result; and determining the disease type based on the comparison result and a preset similarity rate.

[0080] It should be noted that the preset disease type can be a pre-set disease type used to identify the disease type corresponding to the lesion; the lesion feature library includes lesion features corresponding to each type of plant.

[0081] It should be understood that by comparing the lesion features contained in plant disease images with the lesion features in plant disease images in the lesion feature database, the disease type corresponding to the lesion features contained in the plant disease image can be determined. This allows for the identification of the corresponding plant disease type, ensuring accurate pest control operations in the later stages, thereby increasing yield and reducing plant losses caused by diseases.

[0082] This embodiment obtains processed plant images by performing data augmentation preprocessing on target plant images. Based on the improved MSS-ResNet101 model's multi-scale Inception module, disease features are identified in the processed plant images. The plant disease image is determined based on the feature recognition results. The improved MSS-ResNet101 model includes a multi-scale Inception module that replaces the 7×7 convolution in the original ResNet101 model and a residual module incorporating a convolutional kernel attention mechanism. The multi-scale Inception module is connected to the residual module, which is connected to a global average pooling layer. The multi-scale Inception module extracts features from the input image and inputs the obtained multi-scale structure feature map to the residual module. The residual module fuses the multi-scale structure feature map according to the convolutional kernel attention mechanism and inputs the fused feature information to the global average pooling layer and the Softmax layer for classification. The disease type is determined based on the plant disease image. This embodiment uses an improved MSS-ResNet101 model to identify plant disease features in preprocessed data-enhanced images, and determines plant disease images based on the feature identification results, thereby further identifying the disease type. Compared to existing technologies where disease features may appear at any location in natural scenes due to various factors, resulting in poor recognition performance of the network model, this embodiment achieves disease identification using an improved MSS-ResNet101 model, improving the efficiency of identifying various types of plant disease images in natural scenes while ensuring recognition accuracy.

[0083] Reference Figure 6 , Figure 6 This is a flowchart illustrating the second embodiment of the multi-scale plant disease identification method of the present invention, based on the above. Figure 2 The first embodiment shown presents a second embodiment of the multi-scale plant disease identification method of the present invention.

[0084] In this embodiment, the step of constructing an improved MSS-ResNet101 model based on the pre-trained model includes: testing the improved pre-trained model with a test set of target plant images in a natural environment without image preprocessing, and obtaining test results; performing confusion matrix analysis on the test results, and outputting the improved pre-trained model as the improved MSS-ResNet101 model according to the analysis results.

[0085] It should be noted that the test set of target plant images in natural environments without image preprocessing can be a test set of target plant images composed of single leaves, multiple leaves, simple backgrounds and complex backgrounds.

[0086] Understandably, the pre-trained model (improved MSS-ResNet101 model) obtained through the above method needs further verification of its performance on the test set. This can be achieved by inputting a test set of plant images from natural environments without image preprocessing into the improved MSS-ResNet101 model for testing, obtaining test results, performing confusion matrix analysis on the test results, visualizing the confusion matrix, and determining whether the model's recognition effect meets the preset indicators based on the confusion matrix recognition results. The trained MSS-ResNet101 model is then output as the improved MSS-ResNet101 model. Specifically, to determine the accuracy and misclassification rate of the improved model for each disease category, the trained model is used to identify the test set, obtaining a confusion matrix for plant disease recognition. The confusion matrix serves as one of the evaluation indicators for classification models. The values ​​at the intersection of rows and columns represent the probability that the category is predicted as the corresponding column label, and the values ​​at the diagonal represent the probability of being correctly predicted. The darker the color of the diagonal, the better the model's performance.

[0087] In practice, the above analysis shows that, under the premise of ensuring a high recognition rate, the model with the best recognition performance will be used as the improved MSS-ResNet101 model for later practical use.

[0088] Further, step S20 includes:

[0089] Step S201: Based on the multi-scale structure Inception module in the improved MSS-ResNet101 model, multi-scale feature extraction is performed on the processed plant image to obtain a multi-scale structure feature map.

[0090] It should be noted that the multi-scale structure Inception module in the improved MSS-ResNet101 model performs multi-scale feature extraction on the processed plant images to obtain multi-scale structure feature maps.

[0091] Understandably, the Inception module extends the convolutional operations between different network layers and uses convolutional kernels of different sizes for feature extraction to obtain different receptive fields. Its network structure is as follows: Figure 3 As shown, the input is first subjected to parallel convolution operations: a 1×1 convolution, a concatenation of a 1×1 convolution and a 3×3 convolution, a concatenation of a 1×1 convolution and a 5×5 convolution, and a concatenation of a 3×3 max-pooling layer and a 1×1 convolution. Finally, the features extracted from the four parts are concatenated along the channel dimension to obtain a multi-scale structural feature map. Convolution kernels of different sizes can obtain a variety of local features. Fusing features obtained from different receptive fields can greatly enrich the feature information extracted by the model, thereby improving recognition performance.

[0092] Step S202: Based on the convolutional kernel attention mechanism SKNet in the residual structure of the improved MSS-ResNet101 model, feature fusion is performed on the multi-scale structural feature map to obtain target feature maps of different scales.

[0093] It should be noted that this scheme incorporates a selective convolutional kernel attention mechanism into each residual structure of the ResNet101 network. The operation involves performing a Split operation after the Conv convolution, followed by depthwise separable convolutions and dilated convolutions with two holes. The features output from the Split are then fused, and the fused features are sequentially subjected to fully connected operations, ReLU operations, and another fully connected operation. The weights of the Split operation are then regressed based on the Softmax operation. Finally, a Select operation is performed to aggregate the feature information, resulting in the improved residual structure. (See diagram below.) Figure 7 As shown, the residual structure incorporated into SKNet can adaptively adjust the size of the receptive field and select useful feature channels, thereby improving the accuracy of classification tasks.

[0094] Furthermore, the convolutional kernel attention mechanism SKNet consists of three operations: Split, Fuse, and Select. Step S202 further includes: performing multi-path convolution on the multi-scale structural feature map according to the Split operation corresponding to the convolutional kernel attention mechanism SKNet in the residual structure of the improved MSS-ResNet101 model to obtain the convolved feature map; summing the convolved feature map element-wise according to the Fuse operation to obtain a new feature map; and performing feature fusion on the new feature map according to the Select operation to obtain target feature maps of different scales.

[0095] It should be noted that the SKNet consists of three operations: Split, Fuse, and Select. A schematic diagram of the SKNet structure is shown below. Figure 8As shown. The three operations, Split, Fuse, and Select, include:

[0096] (1) Split operation: Multi-path convolution is used to convolve the input feature map. Taking two-path convolution as an example, the input feature map X is first subjected to 3×3 and 5×5 convolution operations respectively. In practical applications, a dilated convolution with a dilation of 2 and a kernel of 3×3 is often used to replace the original 5×5 convolution to further improve the receptive field, and finally two feature maps U% and U% are obtained.

[0097] (2) Fuse operation: The results of the split convolution branches are fused by element-wise summation to obtain a global representation of the selection weights. The two feature maps obtained in the previous step are summed to obtain a new feature map U: Then, global average pooling is used to obtain global information, and finally, channel statistics S∈R are generated. d×1 As shown in the following formula.

[0098]

[0099] Where C represents the dimension of the feature map S.

[0100] Then, a fully connected layer is applied to the output S to find the weight Z of each channel, ensuring the accuracy and adaptability of the convolution kernel. This is illustrated in the following equation:

[0101] z = F fc (s)=δ(B(W s ));

[0102] Where δ represents the ReLU function and B represents batch normalization (BN).

[0103] The number of channels in Z is d, as shown in the following formula:

[0104] d = max(C / r, L);

[0105] Where C represents the number of channels, r represents the attenuation rate, and L is the minimum value of d (L can be taken as 32).

[0106] (3) Select operation: Aggregate feature maps of different kernel sizes according to the selection weights. First, output matrix a using the weight information between the softmax regression channel and the convolution kernel. c b c In the case of two-way convolution, b c For a redundant matrix, b c =1-a c As shown in the following formula:

[0107]

[0108] Where A, B∈R c×d A c ∈R 1×d Represents the c-th row of A, a c Let c represent the c-th element of a. The same logic applies to B.

[0109] Then a c b c The features U% and U% after convolution with Split respectively Perform the multiplication and summation operation as shown in the following formula:

[0110]

[0111] In specific implementation, the multi-scale structural feature map is multi-convolved according to the Split operation corresponding to the convolution kernel attention mechanism SKNet in the residual structure of the improved MSS-ResNet101 model to obtain the convolved feature map; the convolved feature map is summed element-wise according to the Fuse operation to obtain a new feature map; and the new feature map is fused according to the Select operation to obtain target feature maps of different scales.

[0112] Step S203: Perform disease feature recognition on the target feature map, and determine the plant disease image based on the feature recognition results.

[0113] It should be noted that the target feature maps of different scales obtained through the above fusion are used to input the fused feature information into the global average pooling layer and the Softmax layer for disease feature recognition, and the plant disease image is determined based on the feature recognition results.

[0114] This embodiment obtains processed plant images by performing data augmentation preprocessing on the target plant images; multi-scale feature extraction is performed on the processed plant images based on the multi-scale structure Inception module in the improved MSS-ResNet101 model to obtain multi-scale structure feature maps; feature fusion is performed on the multi-scale structure feature maps according to the convolution kernel attention mechanism SKNet in the residual structure of the improved MSS-ResNet101 model to obtain target feature maps at different scales; disease feature identification is performed on the target feature maps, and plant disease images are determined based on the feature identification results. The improved MSS-ResNet101 model includes a replacement of the original ResNet101 model. The Net101 model employs a 7×7 convolutional multi-scale Inception module and a residual module incorporating a convolutional kernel attention mechanism. The multi-scale Inception module is connected to the residual module, which in turn is connected to a global average pooling layer. The multi-scale Inception module extracts features from the input image and inputs the obtained multi-scale feature map to the residual module. The residual module then fuses the multi-scale feature map using the convolutional kernel attention mechanism and inputs the fused feature information to a global average pooling layer and a Softmax layer for classification and identification. Based on the plant disease image, the model determines the disease type. This embodiment uses an improved MSS-ResNet101 model to identify plant disease features in preprocessed data-enhanced images, and determines plant disease images based on the feature identification results, thereby further identifying the disease type. Compared to existing technologies where disease features may appear at any location in natural scenes due to various factors, resulting in poor recognition performance of the network model, this embodiment achieves disease identification using an improved MSS-ResNet101 model, improving the efficiency of identifying various types of plant disease images in natural scenes while ensuring recognition accuracy.

[0115] Furthermore, to achieve the above objectives, the present invention also proposes a multi-scale plant disease identification device, which includes a memory, a processor, and a multi-scale plant disease identification program stored in the memory and executable on the processor. The multi-scale plant disease identification program is configured to implement the steps of multi-scale plant disease identification as described above.

[0116] Furthermore, to achieve the above objectives, the present invention also proposes a storage medium storing a multi-scale plant disease identification program, wherein the multi-scale plant disease identification program, when executed by a processor, implements the steps of the multi-scale plant disease identification method as described above.

[0117] Reference Figure 9 , Figure 9 This is a structural block diagram of the first embodiment of the multi-scale plant disease identification device of the present invention.

[0118] like Figure 9 As shown, the multi-scale plant disease identification device proposed in this embodiment of the invention includes:

[0119] Image preprocessing module 10 is used to perform data augmentation preprocessing on the target plant image to obtain the processed plant image;

[0120] The disease image recognition module 20 is used to identify disease features in processed plant images based on the multi-scale structure Inception module in the improved MSS-ResNet101 model, and to determine the plant disease image based on the feature recognition results. The improved MSS-ResNet101 model includes a multi-scale structure Inception module that replaces the 7×7 convolution in the original ResNet101 model and a residual module that incorporates a convolutional kernel attention mechanism. The multi-scale structure Inception module is connected to the residual module, and the residual module is connected to a global average pooling layer. The multi-scale structure Inception module is used to extract features from the input image and input the obtained multi-scale structure feature map to the residual module. The residual module is used to perform feature fusion on the multi-scale structure feature map according to the convolutional kernel attention mechanism, and input the fused feature information to the global average pooling layer and the Softmax layer for classification and recognition.

[0121] Disease type determination module 30 is used to determine the disease type based on the plant disease image.

[0122] This embodiment obtains processed plant images by performing data augmentation preprocessing on target plant images. Based on the improved MSS-ResNet101 model's multi-scale Inception module, disease features are identified in the processed plant images. The plant disease image is determined based on the feature recognition results. The improved MSS-ResNet101 model includes a multi-scale Inception module that replaces the 7×7 convolution in the original ResNet101 model and a residual module incorporating a convolutional kernel attention mechanism. The multi-scale Inception module is connected to the residual module, which is connected to a global average pooling layer. The multi-scale Inception module extracts features from the input image and inputs the obtained multi-scale structure feature map to the residual module. The residual module fuses the multi-scale structure feature map according to the convolutional kernel attention mechanism and inputs the fused feature information to the global average pooling layer and the Softmax layer for classification. The disease type is determined based on the plant disease image. This embodiment uses an improved MSS-ResNet101 model to identify plant disease features in preprocessed data-enhanced images, and determines plant disease images based on the feature identification results, thereby further identifying the disease type. Compared to existing technologies where disease features may appear at any location in natural scenes due to various factors, resulting in poor recognition performance of the network model, this embodiment achieves disease identification using an improved MSS-ResNet101 model, improving the efficiency of identifying various types of plant disease images in natural scenes while ensuring recognition accuracy.

[0123] Furthermore, the image preprocessing module 10 is also used to perform four preprocessing operations on the target plant image: random cropping, rotation, color brightening, and Gaussian blur noise addition, to obtain the processed plant image.

[0124] Furthermore, the multi-scale plant disease identification device also includes a model training module, which is used to train the ResNet101 network model with a dataset of target plant images in natural environments as the model training dataset to obtain a pre-trained model; based on the pre-trained model, combined with the multi-scale structure Inception module and the convolutional kernel attention mechanism SKNet, an improved pre-trained model is obtained; and based on the pre-trained model, an improved MSS-ResNet101 model is constructed.

[0125] Furthermore, the model training module is also used to test the improved pre-trained model with a test set of target plant images in a natural environment without image preprocessing, and obtain test results; perform confusion matrix analysis on the test results, and output the improved pre-trained model as an improved MSS-ResNet101 model based on the analysis results.

[0126] Furthermore, the disease image recognition module 20 is also used to extract multi-scale features from the processed plant image based on the multi-scale structure Inception module in the improved MSS-ResNet101 model to obtain a multi-scale structure feature map; to perform feature fusion on the multi-scale structure feature map according to the convolution kernel attention mechanism SKNet in the residual structure of the improved MSS-ResNet101 model to obtain target feature maps of different scales; to perform disease feature recognition on the target feature map, and to determine the plant disease image based on the feature recognition results.

[0127] Furthermore, the disease image recognition module 20 is also used to perform multi-path convolution on the multi-scale structural feature map according to the Split operation corresponding to the convolution kernel attention mechanism SKNet in the residual structure of the improved MSS-ResNet101 model, to obtain the convolved feature map; to obtain a new feature map by summing the convolved feature map element by element according to the Fuse operation; and to perform feature fusion on the new feature map according to the Select operation to obtain target feature maps of different scales.

[0128] Furthermore, the disease type determination module 30 is also used to perform feature comparison between the plant disease image and the lesion feature library corresponding to the preset disease type to obtain the comparison result; and to determine the disease type based on the comparison result and the preset similarity rate.

[0129] It should be understood that the above are merely illustrative examples and do not constitute any limitation on the technical solutions of the present invention. In specific applications, those skilled in the art can make settings as needed, and the present invention does not impose any restrictions on this.

[0130] It should be noted that the workflow described above is merely illustrative and does not limit the scope of protection of this invention. In practical applications, those skilled in the art can select some or all of the workflow to achieve the purpose of this embodiment according to actual needs, and no restrictions are imposed here.

[0131] In addition, for technical details not described in detail in this embodiment, please refer to the multi-scale plant disease identification method provided in any embodiment of the present invention, which will not be repeated here.

[0132] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0133] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments. In the unit claims listing several devices, several of these devices may be embodied by the same hardware item. The use of the terms first, second, and third, etc., does not indicate any order and can be interpreted as names.

[0134] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as a read-only memory image (ROM) / random access memory (RAM), magnetic disk, optical disk), and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0135] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A multi-scale plant disease recognition method, characterized in that, The multi-scale plant disease identification method comprises: performing data enhancement preprocessing on the target plant image to obtain a processed plant image; performing disease feature identification on the processed plant image based on a multi-scale structure Inception module in an improved MSS-ResNet101 model, and determining a plant disease image according to a feature identification result, wherein the improved MSS-ResNet101 model comprises a multi-scale structure Inception module replacing a 7*7 convolution in an original ResNet101 model and a residual module integrating a convolution kernel attention mechanism, the multi-scale structure Inception module is connected with the residual module, and the residual module is connected with a global average pooling layer; the multi-scale structure Inception module is used for feature extraction on an input image, and multi-scale structure feature maps obtained are input into the residual module; the residual module is used for feature fusion on the multi-scale structure feature maps according to the convolution kernel attention mechanism, and fused feature information is input into the global average pooling layer and a Softmax layer for classification and identification, and the convolution kernel attention mechanism is an SKNet attention mechanism; determining a disease type based on the plant disease image.

2. The multi-scale plant disease recognition method of claim 1, wherein, The data enhancement preprocessing comprises random cropping, rotation, color brightening, and Gaussian blur noise addition, and the step of performing data enhancement preprocessing on the target plant image to obtain the processed plant image comprises: performing random cropping, rotation, color brightening, and Gaussian blur noise addition on the target plant image to obtain the processed plant image.

3. The multi-scale plant disease recognition method of claim 1, wherein, Before the step of performing disease feature identification on the processed plant image based on the multi-scale structure Inception module in the improved MSS-ResNet101 model and determining a plant disease image according to a feature identification result, the method further comprises: training a ResNet101 network model by taking a target plant image dataset in a natural environment as a model training dataset to obtain a pre-training model; obtaining an improved pre-training model by combining the multi-scale structure Inception module and the convolution kernel attention mechanism SKNet based on the pre-training model; constructing an improved MSS-ResNet101 model based on the pre-training model.

4. The multi-scale plant disease recognition method of claim 3, wherein, The step of constructing the improved MSS-ResNet101 model based on the pre-training model comprises: testing the improved pre-training model by taking a target plant image test set in a natural environment without image preprocessing to obtain a test result; performing confusion matrix analysis on the test result, and outputting the improved pre-training model as the improved MSS-ResNet101 model according to an analysis result.

5. The multi-scale plant disease recognition method of claim 1, wherein, The step of performing disease feature identification on the processed plant image based on the multi-scale structure Inception module in the improved MSS-ResNet101 model and determining a plant disease image according to a feature identification result comprises: The improved MSS-ResNet101 model is used for multi-scale feature extraction of the processed plant image based on the multi-scale structure Inception module, to obtain a multi-scale structure feature map; The multi-scale structure feature map is fused based on the convolution kernel attention mechanism SKNet in the residual structure of the improved MSS-ResNet101 model, to obtain a target feature map of different scales; The target feature map is subjected to disease feature recognition, and a plant disease image is determined according to a feature recognition result.

6. The multi-scale plant disease recognition method of claim 5, wherein, The convolution kernel attention mechanism SKNet is composed of three operations of Split, Fuse and Select; the step of fusing the multi-scale structure feature map based on the convolution kernel attention mechanism SKNet in the residual structure of the improved MSS-ResNet101 model to obtain a target feature map of different scales comprises: The multi-scale structure feature map is subjected to multi-path convolution based on the Split operation corresponding to the convolution kernel attention mechanism SKNet in the residual structure of the improved MSS-ResNet101 model, to obtain a feature map after convolution; The feature map after convolution is subjected to element-wise summation based on the Fuse operation, to obtain a new feature map; The new feature map is subjected to feature fusion based on the Select operation, to obtain a target feature map of different scales.

7. The multi-scale plant disease recognition method of claim 1, wherein, The step of determining a disease type based on the plant disease image comprises: Feature comparison is performed between the plant disease image and a lesion feature library corresponding to a preset disease type, to obtain a comparison result; A disease type is determined according to the comparison result and a preset similarity rate.

8. A multi-scale plant disease recognition apparatus, characterized by, The multi-scale plant disease identification device comprises a memory, a processor and a multi-scale plant disease identification program stored on the memory and executable on the processor, and the multi-scale plant disease identification program realizes the multi-scale plant disease identification method of any one of claims 1 to 7 when executed by the processor.

9. A storage medium, characterized by The storage medium stores a multi-scale plant disease identification program, and the multi-scale plant disease identification program realizes the multi-scale plant disease identification method of any one of claims 1 to 7 when executed by the processor.

10. A multi-scale plant disease recognition apparatus, characterized by, The multi-scale plant disease identification device comprises: An image preprocessing module is configured to perform data enhancement preprocessing on a target plant image, to obtain a processed plant image; The disease image recognition module is configured to recognize disease characteristics of the processed plant image based on a multi-scale structure Inception module in an improved MSS-ResNet101 model, and determine a plant disease image according to a characteristic recognition result. The improved MSS-ResNet101 model comprises a multi-scale structure Inception module replacing a 7*7 convolution in an original ResNet101 model and a residual module integrating a convolution kernel attention mechanism. The multi-scale structure Inception module is connected with the residual module, and the residual module is connected with a global average pooling layer. The multi-scale structure Inception module is configured to extract features of an input image, and input a multi-scale structure feature map obtained to the residual module. The residual module is configured to perform feature fusion on the multi-scale structure feature map according to the convolution kernel attention mechanism, and input fused feature information to the global average pooling layer and a Softmax layer for classification recognition. The convolution kernel attention mechanism is an SKNet attention mechanism. The disease type determination module is configured to determine a disease type based on the plant disease image.

Images