Crop disease and pest identification method and system based on multi-task learning

Abstract

The invention discloses a crop disease and pest identification method and system based on multi-task learning. The method comprises the following steps: firstly, constructing a crop disease and pest identification model comprising a multi-scale feature information fusion module, a disease detection branch and a severity classification branch; the method comprises the following steps: preprocessing an input leaf image, and extracting and fusing a multi-scale feature map by a multi-scale feature information fusion module; the disease detection branch generates disease and insect pest candidate regions by using a region proposal network, and outputs disease and insect pest positions and categories through disease detection in combination with a multi-scale fusion feature map; meanwhile, the fusion feature map with the maximum size is input into a severity classification branch to realize four-stage evaluation; a weighted loss function design thought is provided, and multi-task network branches can be guided to carry out joint training. According to the method, end-to-end multi-task cooperative processing is realized, disease and pest positioning, classification and severity evaluation are synchronously completed by sharing a multi-scale fusion feature map, the recognition efficiency is greatly improved, and the real-time monitoring requirement of an agricultural scene is met.

Description

Technical Field

[0001] The present invention belongs to the technical field of crop disease and insect pest identification, and in particular relates to a crop disease and insect pest identification method and system based on multi-task learning. Background Art

[0002] In recent years, deep learning, with its powerful feature extraction and pattern recognition capabilities, has led some researchers to apply the autoencoder network framework to banana leaf disease classification. Other researchers have compared the performance of various CNN (Convolutional Neural Network) models in rice disease classification. Compared to classification networks, detection networks can identify a wider range of pests and diseases. With the introduction of frameworks such as SSD (Single Shot MultiBox Detector), Faster R-CNN (Regions with Convolutional Neural Networks), and YOLO (You Only Look Once), detection concepts have gradually been introduced into the field of agricultural pests and diseases. Researchers have used SSD with a dual-backbone network to detect citrus pests and diseases, achieving a mean average precision (mAP) of 72.54% on a citrus test set. Researchers have proposed a cloud computing system for crop pest classification based on Fast R-CNN, which can more quickly and conveniently detect pests within a certain range. Compared to networks like RCNN and SSD, RCNN can achieve an accuracy of 98.9%. Some researchers have introduced adaptive feature fusion into feature pyramids to improve target localization accuracy. Others have demonstrated that improved YOLO and SSD algorithms not only increase speed but also accuracy, particularly for small objects such as insects. Others used YOLOv5 (You Only Look Once version 5) to identify thistle caterpillars, achieving a maximum mAP accuracy of 59% on a thistle caterpillar dataset. Furthermore, some researchers have integrated the attention mechanism with object detection. Attention mechanisms are primarily used in natural language processing to capture the most important tasks in neural networks and allocate resources appropriately. By integrating the attention mechanism, adaptive spatial feature fusion (ASFF) is used to identify pests at different scales. Multitask learning is also increasingly being applied to the field of agricultural pests and diseases. Some researchers have proposed a multitask model to address the location, stage, and severity of blueberry disease, demonstrating the superiority of multitask learning. Some researchers have used a multitask learning framework to share underlying information features of rice leaves, improving the accuracy of identifying the type and severity of rice leaf diseases. Researchers have proposed a method for independent crop and disease identification for multi-task classification of crops and diseases under field conditions. Using homoscedastic uncertainty optimization for multi-task learning, the accuracy of crops and diseases under field conditions reached 84.71% and 75.06%, respectively, improving performance across various tasks.Some scholars and others used a multi-task Bayesian method to detect the location of aphids and estimate their size, and conducted tests on a variety of crops to demonstrate the robustness of the method.

[0003] However, the above-mentioned identification of the types and severity of crop pests and diseases belongs to classification tasks and can only identify one category of pests and diseases. The present invention proposes a crop pest and disease identification method and system based on multi-task learning, which can identify multiple pests and diseases and can perform an overall assessment of the invasion status of multiple pests and diseases. Summary of the Invention

[0004] In response to the above technical problems, the present invention provides a crop disease and pest identification method and system based on multi-task learning.

[0005] The technical solution adopted by the present invention to solve the technical problem is: A method for identifying crop diseases and insect pests based on multi-task learning, the method comprising the following steps: S100: Build a crop disease and pest identification model, including a multi-scale feature information fusion module, a disease detection branch, and a severity classification branch; obtain input images of crop leaves, preprocess the input images, and obtain a preprocessed dataset; S200: Input the pre-processed input image into the multi-scale feature information fusion module, and obtain a multi-scale fusion feature map through feature extraction, multi-scale construction, attention enhancement and adaptive fusion processing; S300: Generate pest and disease candidate regions through the region proposal network on the multi-scale fusion feature map, and combine the pest and disease candidate regions with the multi-scale fusion feature map. Figure 1 Input the data into the disease detection branch for disease detection, and output the detection results of the location and category of the pests and diseases; S400: Input the largest feature map in the multi-scale fusion feature map into the severity classification branch to perform severity classification, and output the severity level of the pests and diseases; S500: The pest and disease identification model is trained according to the pest and disease location and category detection results, the pest and disease severity level and the preset loss function. When the preset training end condition is reached, the trained pest and disease identification model is obtained, and crop pest and disease identification is completed in real time based on the trained pest and disease identification model.

[0006] Preferably, the multi-scale feature information fusion module in S200 includes a ResNet-50 feature extraction layer, an FPN multi-scale feature construction layer, a CBAM attention enhancement layer, and an ASFF adaptive feature fusion layer. S200 includes: S210: Use the ResNet-50 feature extraction layer to extract and output a four-level intermediate feature map from the preprocessed input image; S220: The four-level intermediate feature map is input into the FPN multi-scale feature construction layer, and the convolution compression channel is performed on each level of features, and fused from top to bottom to obtain the multi-scale feature layer {P2, P3, P4, P5}; S230: The multi-scale feature layer {P2, P3, P4, P5} is input to the CBAM attention enhancement layer, and the channel and spatial dual attention operations are applied to each level of features in parallel to output four levels of enhanced features; S240: The four-level enhanced features are input to the ASFF adaptive feature fusion layer, normalized weight parameters are generated for each level, the features of different layers are upsampled or downsampled to the target level size, and then weighted summation is performed to output four sets of fused feature maps.

[0007] Preferably, S240 includes: S241: The four-level enhanced features are input to the ASFF adaptive feature fusion layer to generate normalized weight parameters for each level; S242: Use the Softmax function to process the weight parameters to ensure that the sum of the four weight parameters is 1. Then multiply each weight parameter by the corresponding feature layer to obtain ASFF - 1, ASFF - 2, ASFF - 3, and ASFF - 4. S243: For ASFF-1, 1×1 convolution is used to upsample the second, third and fourth layers by 2, 4 and 8 times respectively, and expand them to the same feature layer size as the first layer, and then the information of the four layers is added; for ASFF-2, 1×1 convolution is used to downsample the first layer by 2, and upsample the third and fourth layers by 2 and 4 times respectively, and transform them to the same feature layer size as the second layer, and then the information of the four layers is added; for ASFF-3, 1×1 convolution is used to downsample the first and second layers by 4 and 2 times respectively, and upsample the fourth layer by 2 times, and transform them to the same feature layer size as the third layer, and then the information of the four layers is added; for ASFF-4, 1×1 convolution is used to downsample the first, second and third layers by 8, 4 and 2 times respectively, and transform them to the same feature layer size as the fourth layer, and then the information of the four layers is added, and finally four fused feature maps of different scales are obtained.

[0008] Preferably, S243 is specifically: ; in, 、 、 and Represents a weight tensor and satisfies , and Represents the position index, indicating the spatial dimension or feature dimension, 、 、 、 Respectively represent the first feature layer, the second feature layer, the third feature layer and the fourth feature layer of the fusion, 、 、 、 Respectively represent unifying the scale of the first feature layer to the first feature layer, the second feature layer, the third feature layer, and the fourth feature layer, 、 、 、 Respectively, they represent unifying the scale of the second feature layer to the first, second, third, and fourth feature layers. 、 、 、 They represent unifying the scale of the third feature layer to the first, second, third, and fourth feature layers, respectively. 、 、 、 They respectively represent unifying the scale of the fourth feature layer to the first feature layer, the second feature layer, the third feature layer, and the fourth feature layer.

[0009] Preferably, S300 includes: S310: The multi-scale fusion feature maps are transformed through a 7×7 convolutional layer. The transformed features are then fed into two parallel 1×1 convolutional layers. These two convolutional layers are responsible for generating candidate bounding boxes and proposed extraction regions, respectively. Candidate pest and disease regions are generated based on the candidate bounding boxes and the proposed extraction regions. S320: The candidate pest and disease area and the multi-scale fusion feature map are passed through the ROI pooling layer for feature unification and size normalization. The pooled features are then passed through two fully connected layers. Finally, this branch simultaneously outputs the finely adjusted pest and disease location regression frame and its corresponding specific classification category of the pest and disease.

[0010] Preferably, S400 includes: S410: Extract the feature map with the largest spatial size from the four output feature maps, whose length, width, and number of channels are 256×256×256 respectively; S420: The adjusted image enters the feature extraction stage, which includes two groups of convolution blocks with the same structure. Each group of convolution blocks includes two consecutive convolution layers. Each convolution layer is sequentially connected to a batch normalization layer and a ReLU activation function. After each group of convolution blocks is processed, it passes through a maximum pooling layer for spatial downsampling. S430: After feature extraction and downsampling, the obtained features are sent to the fully connected layer. Finally, this branch outputs the classification results of the severity level of crop diseases and insect pests, which are divided into four categories: healthy, mild, moderate, and severe.

[0011] Preferably, the preset loss function is specifically: ; ; ; ; ; ; ;

[0012] in, represents the total number of samples, Indicates the total number of categories, Represents a symbolic function, sample The true category is equal to hour Equal to 1, otherwise is equal to 0, Represents an observation sample Belong to category The predicted probability of To detect the classification loss and positioning loss of the network branch, represents the weighted sum of the two-class loss and the bounding box regression loss, represents the weight factor, Indicates the number of samples of candidate boxes, Indicates the The probability that a candidate box is predicted to be a target, Indicates whether there is a target sample in the box, the value is 1 if it exists, and 0 if it does not exist. Indicates that the predicted value and the true value Perform binary cross entropy loss to obtain classification loss value; Indicates the number of bounding box position parameters, Indicates the prediction The regression parameters of the bounding box are There are four parameters , represents the center point coordinates and width and height , similarly, Indicates sample The true regression parameters of the bounding box, where There are four parameters , represents the center point coordinates and width and height , Indicates that the predicted value and the true value Perform binary cross entropy loss to obtain the bounding box loss value; represents the weighted sum of the crop pest and disease multi-classification loss and the bounding box regression loss, Indicates the true categories of various pests and diseases and predicted categories Cross entropy loss function, represents the bounding box regression loss, Indicates the corresponding category of the bounding box regressor The regression parameters of , Represents the bounding box regression parameters of the true target .

[0013] A crop pest and disease identification system based on multi-task learning, including: The model building and input image acquisition module is used to build a crop disease and pest identification model, including a multi-scale feature information fusion module, a disease detection branch, and a severity classification branch; it obtains input images of crop leaves and preprocesses the input images to obtain a preprocessed dataset; The multi-scale feature information fusion module is used to perform feature extraction, multi-scale construction, attention enhancement and adaptive fusion processing on the pre-processed input image to obtain a multi-scale fusion feature map; The disease detection branch is used to generate candidate pest regions through a region proposal network on a multi-scale fusion feature map, perform disease detection based on the candidate pest regions and the multi-scale fusion feature map, and output the location and category detection results of the pests and diseases; The severity classification branch is used to receive the largest feature map in the multi-scale fusion feature map for severity classification and output the severity level of the pests and diseases; The model training module is used to train the pest and disease identification model based on the pest and disease location and category detection results, the pest and disease severity level and the preset loss function. When the preset training end conditions are met, the trained pest and disease identification model is obtained, and crop pest and disease identification is completed in real time based on the trained pest and disease identification model.

[0014] A computer device includes a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of a method for identifying crop diseases and insect pests based on multi-task learning when executing the computer program.

[0015] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of a method for identifying crop diseases and insect pests based on multi-task learning.

[0016] The above-mentioned crop disease and pest identification method and system based on multi-task learning realizes end-to-end multi-task collaborative processing. By sharing multi-scale fusion feature maps, it simultaneously completes the location, classification and severity assessment of pests and diseases, greatly improving the recognition efficiency and meeting the real-time monitoring needs of agricultural scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 This is a flow chart of a method for identifying crop diseases and insect pests based on multi-task learning in one embodiment of the present invention; Figure 2 This is a framework diagram of crop disease and insect pest identification based on multi-task learning in one embodiment of the present invention; Figure 3 2 is a framework diagram of a multi-scale feature information fusion module according to an embodiment of the present invention; Figure 4 This is a diagram of a dual-branch network structure for pest and disease detection and severity classification in one embodiment of the present invention. DETAILED DESCRIPTION

[0018] In order to enable those skilled in the art to better understand the technical solution of the present invention, the present invention is further described in detail below with reference to the accompanying drawings.

[0019] In one embodiment, Figure 1 As shown, a method for identifying crop diseases and insect pests based on multi-task learning comprises the following steps: S100: Build a crop disease and pest identification model, including a multi-scale feature information fusion module, a disease detection branch, and a severity classification branch. Obtain input images of crop leaves and perform preprocessing on the input images, such as resizing and pixel normalization, to obtain a preprocessed dataset. S200: Input the pre-processed input image into the multi-scale feature information fusion module, and obtain a multi-scale fusion feature map through feature extraction, multi-scale construction, attention enhancement and adaptive fusion processing; S300: Generate pest and disease candidate regions through the region proposal network on the multi-scale fusion feature map, and combine the pest and disease candidate regions with the multi-scale fusion feature map. Figure 1 Input the data into the disease detection branch for disease detection, and output the detection results of the location and category of the pests and diseases; S400: Input the largest feature map in the multi-scale fusion feature map into the severity classification branch to perform severity classification, and output the severity level of the pests and diseases; S500: The pest and disease identification model is trained according to the pest and disease location and category detection results, the pest and disease severity level and the preset loss function. When the preset training end condition is reached, the trained pest and disease identification model is obtained, and crop pest and disease identification is completed in real time based on the trained pest and disease identification model.

[0020] Specifically, the multi-scale feature information fusion module is used to extract information at different scales of crops and share the extracted feature maps. The detection of the dual-branch network of disease detection and severity classification can identify multiple diseases and pests on a single leaf of crops, and the severity classification branch conducts an overall assessment of the degree of invasion of diseases and pests.

[0021] The above-mentioned crop disease and pest identification method based on multi-task learning realizes end-to-end multi-task collaborative processing. By sharing multi-scale fusion feature maps, it simultaneously completes the location, classification and severity assessment of pests and diseases, greatly improving the recognition efficiency and meeting the real-time monitoring needs of agricultural scenarios.

[0022] In one embodiment, the multi-scale feature information fusion module in S200 includes a ResNet-50 (Residual Network with 50 layers) feature extraction layer, an FPN (Feature Pyramid Network) multi-scale feature construction layer, a CBAM (Convolutional Block Attention Module) attention enhancement layer, and an ASFF (Adaptive Spatial Feature Fusion) adaptive feature fusion layer. S200 includes: S210: Use the ResNet-50 feature extraction layer to extract and output a four-level intermediate feature map from the preprocessed input image; S220: The four-level intermediate feature map is input into the FPN multi-scale feature construction layer, and the convolution compression channel is performed on each level of features, and fused from top to bottom to obtain the multi-scale feature layer {P2, P3, P4, P5}; S230: The multi-scale feature layer {P2, P3, P4, P5} is input to the CBAM attention enhancement layer, and the channel and spatial dual attention operations are applied to each level of features in parallel to output four levels of enhanced features; S240: The four-level enhanced features are input to the ASFF adaptive feature fusion layer, normalized weight parameters are generated for each level, the features of different layers are upsampled or downsampled to the target level size, and then weighted summation is performed to output four sets of fused feature maps.

[0023] Specifically, the framework of the multi-scale feature information fusion module is as follows: Figure 3As shown, a ResNet-50 model is used for feature extraction. After obtaining intermediate feature maps, the FCA (FPN-CBAM-ASFF) architecture is used to generate multi-scale feature maps. These feature maps are then shared with both the disease detection and severity classification branches, facilitating the extraction of the required information for each task. While ResNet-50 can capture deeper object features in feature extraction, it often struggles to accurately locate objects of varying scales. This is particularly true when dealing with small objects, where ResNet-50 may struggle to capture important information. To address this issue, the attention mechanism has been introduced into computer vision. Spatial attention focuses on the relative position of input data, deeply exploring the important information of each pixel, guiding the model to focus on task-critical areas and improving overall performance. Channel attention models identify important channels and assign them greater weights, allowing the model to prioritize these important channels in subsequent computations. In order to improve the detection accuracy of small targets, the present invention integrates the ASFF feature fusion module into the ResNet50 network, uses the feature extraction capability of ResNet50 to capture the deep features of the target, and enhances the focus on the feature information of small targets through the ASFF module. Figure 2 The connection between each module is explained. The four-level feature fusion architecture of ResNet-50 feature extraction layer, FPN multi-scale feature construction layer, CBAM attention enhancement layer, and ASFF adaptive feature fusion layer significantly enhances the multi-scale pest and disease feature expression capability and improves the detection accuracy of pests and diseases.

[0024] In one embodiment, S240 includes: S241: The four-level enhanced features are input to the ASFF adaptive feature fusion layer to generate normalized weight parameters for each level; S242: Use the Softmax (normalized exponential) function to process the weight parameters to ensure that the sum of the four weight parameters is 1. Then multiply each weight parameter by the corresponding feature layer to obtain ASFF-1, ASFF-2, ASFF-3, and ASFF-4. S243: For ASFF-1, 1×1 convolution is used to upsample the second, third and fourth layers by 2, 4 and 8 times respectively, and expand them to the same feature layer size as the first layer, and then the information of the four layers is added; for ASFF-2, 1×1 convolution is used to downsample the first layer by 2, and upsample the third and fourth layers by 2 and 4 times respectively, and transform them to the same feature layer size as the second layer, and then the information of the four layers is added; for ASFF-3, 1×1 convolution is used to downsample the first and second layers by 4 and 2 times respectively, and upsample the fourth layer by 2 times, and transform them to the same feature layer size as the third layer, and then the information of the four layers is added; for ASFF-4, 1×1 convolution is used to downsample the first, second and third layers by 8, 4 and 2 times respectively, and transform them to the same feature layer size as the fourth layer, and then the information of the four layers is added, and finally four fused feature maps of different scales are obtained.

[0025] Specifically, through the adaptive weighted sampling mechanism of ASFF, pest and disease identification targets of different scales can be solved at the fusion layer, thereby improving the recognition accuracy.

[0026] In one embodiment, S243 is specifically: ; in, 、 、 and Represents a weight tensor and satisfies , and Represents the position index, indicating the spatial dimension or feature dimension, 、 、 、 Respectively represent the first feature layer, the second feature layer, the third feature layer and the fourth feature layer of the fusion, 、 、 、 Respectively represent unifying the scale of the first feature layer to the first feature layer, the second feature layer, the third feature layer, and the fourth feature layer, 、 、 、 Respectively, they represent unifying the scale of the second feature layer to the first, second, third, and fourth feature layers. 、 、 、 They represent unifying the scale of the third feature layer to the first, second, third, and fourth feature layers, respectively. 、 、 、 They respectively represent unifying the scale of the fourth feature layer to the first feature layer, the second feature layer, the third feature layer, and the fourth feature layer.

[0027] In one embodiment, S300 includes: S310: The multi-scale fusion feature maps are transformed through a 7×7 convolutional layer. The transformed features are then fed into two parallel 1×1 convolutional layers. These two convolutional layers are responsible for generating candidate bounding boxes and proposed extraction regions, respectively. Candidate pest and disease regions are generated based on the candidate bounding boxes and the proposed extraction regions. S320: The candidate pest and disease area and the multi-scale fusion feature map are passed through the ROI (Region of Interest) pooling layer for feature unification and size normalization. The pooled features are further processed by two fully connected layers. This branch also outputs the finely adjusted pest and disease location regression frame and its corresponding specific classification category.

[0028] Specifically, the branch network structure diagram of pest and disease detection is as follows: Figure 4 As shown in the figure, the detection network can treat the pest-infested leaf area as a scene and then identify this area, greatly reducing the impact of irrelevant information interference and improving recognition efficiency. Furthermore, when multiple pest and disease categories exist on a single leaf, the detection network typically generates multiple labels, dividing the leaf into different areas and then classifying the affected areas into different categories, further improving detection efficiency.

[0029] In one embodiment, S400 includes: S410: Extract the feature map with the largest spatial size from the four output feature maps, whose length, width, and number of channels are 256×256×256 respectively; S420: The adjusted image enters the feature extraction stage, which includes two groups of convolution blocks with the same structure. Each group of convolution blocks includes two consecutive convolution layers. Each convolution layer is sequentially connected to a batch normalization layer and a ReLU (Rectified Linear Unit) activation function. After each group of convolution blocks is processed, it passes through a maximum pooling layer for spatial downsampling. S430: After feature extraction and downsampling, the obtained features are sent to the fully connected layer. Finally, this branch outputs the classification results of the severity level of crop diseases and insect pests, which are divided into four categories: healthy, mild, moderate, and severe.

[0030] Specifically, the severity level classification branch network structure diagram is as follows Figure 4As shown in the figure, a custom-designed convolutional block is designed for severity classification to effectively extract pathological texture features; the four-level classification meets the needs of precise agronomic decision-making and reduces the severity misjudgment rate.

[0031] Furthermore, the present invention involves loss functions for different tasks, which are composed of two parts: one is the loss function for disease detection, and the other is the loss function for severity. （total） Guided network joint training, mutually reinforcing each other, achieve optimal results for both classification and detection tasks. Directly summing the loss functions of different tasks as the total loss function for the multi-task learning model can guide proper joint training of the model. However, the time and number of training cycles required to reach optimal accuracy for each task vary. Experiments have shown that the loss function of the classification network converges faster than that of the detection network. After multiple iterations of training, the classification network is prone to overfitting, while the loss function of the detection network has not yet converged, resulting in an imbalance in joint training.

[0032] The classification network loss function is specifically: ; in, represents the total number of samples, Indicates the total number of categories, Represents a symbolic function, sample The true category is equal to hour Equal to 1, otherwise is equal to 0, Represents an observation sample Belong to category The predicted probability of The binary classification loss and bounding box regression loss functions are specifically: ; in, represents the weight factor, Indicates the number of samples of candidate boxes, Indicates the probability that the i-th candidate box is predicted to be the target, Indicates whether there is a target sample in the box, the value is 1 if it exists, and 0 if it does not exist. Indicates that the predicted value and the true value Perform binary cross entropy loss to obtain classification loss value; Indicates the number of bounding box position parameters, Indicates the prediction The regression parameters of the bounding box are There are four parameters , represents the center point coordinates and width and height , similarly, Indicates sample The true regression parameters of the bounding box, where There are four parameters , represents the center point coordinates and width and height , Indicates that the predicted value and the true value Perform binary cross entropy loss to obtain the bounding box loss value; The multi-classification loss and bounding box regression loss functions of crop pests and diseases are specifically: ; ; ; in, Represents the weight factor. Indicates the true categories of various pests and diseases and predicted categories Cross entropy loss function, Indicates the corresponding category of the bounding box regressor The regression parameters of , Represents the bounding box regression parameters of the true target .

[0033] The loss function of the crop pest and disease detection network is given by and composition: ; Finally, the total loss function is obtained by the loss of the severity assessment classification network and the crop pest detection network loss function. The present invention introduces a weight factor before the loss function of each task. The main purpose of this weight factor is to adjust the proportion of each task in each task. Therefore, the weighted cross entropy loss function designed by the present invention is obtained by expanding multiple cross entropy loss functions. The weighted loss function expression is: .

[0034] The end-to-end multi-task learning framework proposed in this invention significantly improves the recognition efficiency of complex pest and disease scenarios on crop leaves by deeply integrating the dual-branch structure of pest and disease detection and severity classification. Its core advantages are: First, the innovatively designed attention-based multi-scale feature fusion mechanism CA module (CBAM-ASFF) breaks through the limitations of traditional single-scale feature extraction, and adopts convolution and channel attention collaborative operations. Based on the four-level features extracted by the ResNet-50 backbone network, it dynamically aggregates pathological feature information of different receptive fields to adapt to the identification of pests and diseases of different scales and fully mine features. Secondly, the dual-branch collaborative architecture realizes efficient parallelization of multiple tasks: the disease detection branch introduces an improved RPN (Region Proposal Network) network to obtain pooling operations of candidate boxes and adaptive regions of interest, supporting the simultaneous identification of multiple types of pests and diseases on a single leaf; the severity classification branch constructs a four-level gradient evaluation system to effectively reduce the misjudgment rate and provide a quantitative basis for precise pesticide application. Finally, based on the multi-task dynamic weighted loss function, the weighted pest and disease detection loss and severity classification loss function are used to jointly guide the joint training of the network during the backpropagation process, solving the model bias problem caused by sample imbalance, effectively improving the training convergence speed, and improving the recognition accuracy of pests and diseases.

[0035] In one embodiment, a crop disease and insect pest identification system based on multi-task learning is also provided, including: The model building and input image acquisition module is used to build a crop disease and pest identification model, including a multi-scale feature information fusion module, a disease detection branch, and a severity classification branch; it obtains input images of crop leaves and preprocesses the input images to obtain a preprocessed dataset; The multi-scale feature information fusion module is used to perform feature extraction, multi-scale construction, attention enhancement and adaptive fusion processing on the pre-processed input image to obtain a multi-scale fusion feature map; The disease detection branch is used to generate candidate pest regions through a region proposal network on a multi-scale fusion feature map, perform disease detection based on the candidate pest regions and the multi-scale fusion feature map, and output the location and category detection results of the pests and diseases; The severity classification branch is used to receive the largest feature map in the multi-scale fusion feature map for severity classification and output the severity level of the pests and diseases; The model training module is used to train the pest and disease identification model based on the pest and disease location and category detection results, the pest and disease severity level and the preset loss function. When the preset training end conditions are met, the trained pest and disease identification model is obtained, and crop pest and disease identification is completed in real time based on the trained pest and disease identification model.

[0036] For the specific definition of a crop disease and pest identification system based on multi-task learning, please refer to the definition of a crop disease and pest identification method based on multi-task learning above, which will not be repeated here. The various modules in the above-mentioned crop disease and pest identification system based on multi-task learning can be implemented in whole or in part through software, hardware, and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in hardware form, or can be stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to the above modules.

[0037] A computer device includes a memory and a processor, wherein the memory stores a computer program, and is characterized in that when the processor executes the computer program, it implements the steps of a crop disease and insect pest identification method based on multi-task learning.

[0038] A computer-readable storage medium having a computer program stored thereon, characterized in that when the computer program is executed by a processor, the computer program implements the steps of a method for identifying crop diseases and pests based on multi-task learning.

[0039] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the above-described method embodiments. Any reference to memory, storage, database, or other media used in the embodiments provided herein may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical storage. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0040] The above is a detailed introduction to the crop disease and insect pest identification method and system based on multi-task learning provided by the present invention. This article uses specific examples to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the core ideas of the present invention. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present invention, several improvements and modifications can be made to the present invention, and these improvements and modifications also fall within the scope of protection of the claims of the present invention.

Claims

1. A method for identifying crop diseases and insect pests based on multi-task learning, characterized in that: The method comprises the following steps: S100: Build a crop disease and pest identification model, including a multi-scale feature information fusion module, a disease detection branch, and a severity classification branch; obtain input images of crop leaves, preprocess the input images, and obtain a preprocessed dataset; S200: Input the pre-processed input image into the multi-scale feature information fusion module, and obtain a multi-scale fusion feature map through feature extraction, multi-scale construction, attention enhancement and adaptive fusion processing; S300: Generate candidate pest regions through the region proposal network on the multi-scale fusion feature map, input the candidate pest regions and the multi-scale fusion feature map into the disease detection branch for disease detection, and output the pest location and category detection results; S400: Input the largest feature map in the multi-scale fusion feature map into the severity classification branch to perform severity classification, and output the severity level of the pests and diseases; S500: The pest and disease identification model is trained according to the pest and disease location and category detection results, the pest and disease severity level and the preset loss function. When the preset training end condition is reached, the trained pest and disease identification model is obtained, and crop pest and disease identification is completed in real time based on the trained pest and disease identification model.

2. The method according to claim 1, characterized in that The multi-scale feature information fusion module in S200 includes the ResNet-50 feature extraction layer, the FPN multi-scale feature construction layer, the CBAM attention enhancement layer, and the ASFF adaptive feature fusion layer. S200 includes: S210: Use the ResNet-50 feature extraction layer to extract and output a four-level intermediate feature map from the preprocessed input image; S220: The four-level intermediate feature map is input into the FPN multi-scale feature construction layer, and the convolution compression channel is performed on each level of features, and fused from top to bottom to obtain the multi-scale feature layer {P2, P3, P4, P5}; S230: The multi-scale feature layer {P2, P3, P4, P5} is input to the CBAM attention enhancement layer, and the channel and spatial dual attention operations are applied to each level of features in parallel to output four levels of enhanced features; S240: The four-level enhanced features are input to the ASFF adaptive feature fusion layer, normalized weight parameters are generated for each level, the features of different layers are upsampled or downsampled to the target level size, and then weighted summation is performed to output four sets of fused feature maps.

3. The method according to claim 2, characterized in that The S240 includes: S241: The four-level enhanced features are input to the ASFF adaptive feature fusion layer to generate normalized weight parameters for each level; S242: Use the Softmax function to process the weight parameters to ensure that the sum of the four weight parameters is 1. Then multiply each weight parameter by the corresponding feature layer to obtain ASFF - 1, ASFF - 2, ASFF - 3, and ASFF - 4. S243: For ASFF-1, 1×1 convolution is used to upsample the second, third and fourth layers by 2, 4 and 8 times respectively, and expand them to the same feature layer size as the first layer, and then the information of the four layers is added; for ASFF-2, 1×1 convolution is used to downsample the first layer by 2 times, and upsample the third and fourth layers by 2 and 4 times respectively, and transform them to the same feature layer size as the second layer, and then the information of the four layers is added; for ASFF-3, 1×1 convolution is used to downsample the first and second layers by 4 and 2 times respectively, and upsample the fourth layer by 2 times, and transform them to the same feature layer size as the third layer, and then the information of the four layers is added; for ASFF-4, 1×1 convolution is used to downsample the first, second and third layers by 8, 4 and 2 times respectively, and transform them to the same feature layer size as the fourth layer, and then the information of the four layers is added, and finally four fused feature maps of different scales are obtained.

4. The method according to claim 3, characterized in that S243 is specifically: ； in, 、 、 and Represents a weight tensor and satisfies , and Represents the position index, indicating the spatial dimension or feature dimension, 、 、 、 Respectively represent the first feature layer, the second feature layer, the third feature layer and the fourth feature layer of the fusion, 、 、 、 Respectively represent unifying the scale of the first feature layer to the first feature layer, the second feature layer, the third feature layer, and the fourth feature layer, 、 、 、 Respectively, they represent unifying the scale of the second feature layer to the first, second, third, and fourth feature layers. 、 、 、 They represent unifying the scale of the third feature layer to the first, second, third, and fourth feature layers, respectively. 、 、 、 They respectively represent unifying the scale of the fourth feature layer to the first feature layer, the second feature layer, the third feature layer, and the fourth feature layer.

5. The method according to claim 4, characterized in that S300 includes: S310: The multi-scale fusion feature maps are transformed through a 7×7 convolutional layer. The transformed features are then fed into two parallel 1×1 convolutional layers. These two convolutional layers are responsible for generating candidate bounding boxes and proposed extraction regions, respectively. Candidate pest and disease regions are generated based on the candidate bounding boxes and the proposed extraction regions. S320: The candidate pest and disease area and the multi-scale fusion feature map are passed through the ROI pooling layer for feature unification and size normalization. The pooled features are then passed through two fully connected layers. Finally, this branch simultaneously outputs the pest and disease location regression frame and its corresponding specific classification category.

6. The method according to claim 5, characterized in that S400 includes: S410: Extract the feature map with the largest spatial size from the four output feature maps, whose length, width, and number of channels are 256×256×256 respectively; S420: The adjusted image enters the feature extraction stage, which includes two groups of convolution blocks with the same structure. Each group of convolution blocks includes two consecutive convolution layers. Each convolution layer is sequentially connected to a batch normalization layer and a ReLU activation function. After each group of convolution blocks is processed, it passes through a maximum pooling layer for spatial downsampling. S430: After feature extraction and downsampling, the obtained features are sent to the fully connected layer. Finally, this branch outputs the classification results of the severity level of crop diseases and insect pests, which are divided into four categories: healthy, mild, moderate, and severe.

7. The method according to claim 6, characterized in that The preset loss function is: ； ； ； ； ； ； ； in, represents the classification network loss function, represents the total number of samples, Indicates the total number of categories, Represents a symbolic function, sample The true category is equal to hour Equal to 1, otherwise is equal to 0, Represents an observation sample Belong to category The predicted probability of To detect the classification loss and positioning loss of the network branch, represents the weighted sum of the two-class loss and the bounding box regression loss, represents the weight factor, Indicates the number of samples of candidate boxes, Indicates the The probability that a candidate box is predicted to be a target, Indicates whether there is a target sample in the box, the value is 1 if it exists, and 0 if it does not exist. Indicates that the predicted value and the true value Perform binary cross entropy loss to obtain classification loss value; Indicates the number of bounding box position parameters, Indicates the prediction The regression parameters of the bounding box are There are four parameters , represents the center point coordinates and width and height , similarly, Indicates sample The true regression parameters of the bounding box, where There are four parameters , represents the center point coordinates and width and height , Indicates that the predicted value and the true value Perform binary cross entropy loss to obtain the bounding box loss value; represents the weighted sum of the crop pest and disease multi-classification loss and the bounding box regression loss, Indicates the true categories of various pests and diseases and predicted categories Cross entropy loss function, represents the bounding box regression loss, Indicates the corresponding category of the bounding box regressor The regression parameters of , Represents the bounding box regression parameters of the true target .

8. A crop disease and insect pest identification system based on multi-task learning, characterized by: include: The model building and input image acquisition module is used to build a crop disease and pest identification model, including a multi-scale feature information fusion module, a disease detection branch, and a severity classification branch; Acquire an input image of a crop leaf, preprocess the input image, and obtain a preprocessed data set; The multi-scale feature information fusion module is used to perform feature extraction, multi-scale construction, attention enhancement and adaptive fusion processing on the pre-processed input image to obtain a multi-scale fusion feature map; The disease detection branch is used to generate candidate pest regions through a region proposal network on a multi-scale fusion feature map, perform disease detection based on the candidate pest regions and the multi-scale fusion feature map, and output the location and category detection results of the pests and diseases; The severity classification branch is used to receive the largest feature map in the multi-scale fusion feature map for severity classification and output the severity level of the pests and diseases; The model training module is used to train the pest and disease identification model based on the pest and disease location and category detection results, the pest and disease severity level and the preset loss function. When the preset training end conditions are met, the trained pest and disease identification model is obtained, and crop pest and disease identification is completed in real time based on the trained pest and disease identification model.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images