A crop disease and pest identification method and system based on multi-task learning

Abstract

The application discloses a crop disease and pest identification method and system based on multi-task learning, first constructs a crop disease and pest identification model containing a multi-scale feature information fusion module, a disease detection branch and a severity classification branch; through preprocessing of an input leaf image, multi-scale feature maps are extracted and fused by the multi-scale feature information fusion module; the disease detection branch generates a disease and pest candidate region by using a region proposal network, and combines the multi-scale fused feature maps to output a disease and pest position and category through disease detection; meanwhile, the largest size fused feature map is input into the severity classification branch to realize four-level evaluation; a weighted loss function design idea is proposed, which can guide the joint training of the multi-task network branch. End-to-end multi-task collaborative processing is realized, disease and pest positioning, classification and severity evaluation are synchronously completed through shared multi-scale fused feature maps, the identification efficiency is greatly improved, and the real-time monitoring demand of an agricultural scene is met.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of crop disease and pest identification, and particularly relates to a crop disease and pest identification method and system based on multi-task learning. BACKGROUND

[0002] In recent years, deep learning has been used in banana leaf disease classification due to its powerful feature extraction and pattern recognition capabilities. Some scholars have compared the performance of various CNNs in rice disease classification. Detection networks can identify multiple pests and diseases compared to classification networks. With the introduction of SSD, Faster R-CNN, and YOLO series frameworks, the detection idea has gradually been introduced into the field of agricultural pests and diseases. Some scholars have used a dual-main network SSD to detect citrus pests and diseases, achieving an mAP precision of 72.54% on the citrus test set. Some scholars have proposed a cloud computing system based on a fast R-CNN crop pest classification, which can more quickly and conveniently detect pests within a certain range. RCNN can achieve an accuracy of 98.9% compared to networks such as RCNN and SSD. Some scholars have introduced adaptive feature fusion into the feature pyramid, improving the accuracy of target positioning. Some scholars have proven that the improved YOLO and SSD algorithms not only improve speed but also improve accuracy, especially for small objects such as insects. Some scholars have used YOLOv5 to identify the Cabbage Moth, achieving a maximum mAP precision of 59% on the Cabbage Moth dataset. In addition, some scholars have integrated attention mechanisms with object detection, which are mainly used in natural language processing to capture important tasks in neural networks and allocate resources reasonably. By integrating attention mechanisms with adaptive spatial feature fusion (ASFF), different scales of pests can be identified. Multi-task learning has also been gradually applied in the field of agricultural pests and diseases. Some scholars have proposed a multi-task model to solve the problem of blueberry disease location, stage, and severity, and have proven the superiority of multi-task learning. Some scholars have used a multi-task learning framework to share underlying information features of rice leaves, improving the accuracy of rice leaf disease classification and severity recognition. Some scholars have proposed a method for independent recognition of crops and diseases for multi-task classification of crops and diseases under field conditions. After optimizing multi-task learning using homoscedastic uncertainty, the accuracy of crops and diseases under field conditions was 84.71% and 75.06%, respectively, improving the performance of various tasks.Some scholars and others use multi-task Bayesian methods to detect the location and estimate the size of aphids and demonstrate the robustness of the method by detecting in various crops.

[0003] However, the crop pest and disease type and severity recognition described above are all classification tasks, which can only identify one type of pest and disease, and the present application proposes a crop pest and disease recognition method and system based on multi-task learning, which can identify multiple pests and diseases and can overall evaluate the invasion of multiple pests and diseases. SUMMARY

[0004] To solve the above technical problems, the present application provides a crop pest and disease recognition method and system based on multi-task learning.

[0005] The technical solution adopted by the present application to solve its technical problems is:

[0006] A crop pest and disease recognition method based on multi-task learning, the method comprising the following steps:

[0007] S100: Construct a crop pest and disease recognition model, including a multi-scale feature information fusion module, a disease detection branch and a severity classification branch; obtain an input image of a crop leaf, pre-process the input image to obtain a pre-processed data set;

[0008] S200: input the pre-processed input image into the multi-scale feature information fusion module, and obtain a multi-scale fusion feature map through feature extraction, multi-scale construction, attention enhancement and adaptive fusion processing;

[0009] S300: generate a pest and disease candidate region on the multi-scale fusion feature map through a region proposal network, input the pest and disease candidate region and the multi-scale fusion feature Figure One to the disease detection branch for disease detection, and output the pest and disease location and category detection result;

[0010] S400: input the largest feature map in the multi-scale fusion feature map into the severity classification branch for severity classification, and output the severity level of the pest and disease;

[0011] S500: train the pest and disease recognition model according to the pest and disease location and category detection result, the pest and disease severity level and the preset loss function, obtain the trained pest and disease recognition model when the preset training end condition is reached, and complete the real-time crop pest and disease recognition based on the trained pest and disease recognition model.

[0012] Preferably, the multi-scale feature information fusion module in S200 comprises a ResNet-50 feature extraction layer, an FPN multi-scale feature construction layer, a CBAM attention enhancement layer, and an ASFF adaptive feature fusion layer, and S200 comprises:

[0013] S210: extracting output four-level intermediate feature maps from the preprocessed input image using the ResNet-50 feature extraction layer;

[0014] S220: inputting the four-level intermediate feature maps into the FPN multi-scale feature construction layer, performing convolution compression channels on each level of features, and fusing from top to bottom to obtain a multi-scale feature layer {P2, P3, P4, P5};

[0015] S230: inputting the multi-scale feature layer {P2, P3, P4, P5} into the CBAM attention enhancement layer, applying channel and spatial double attention operations to each level of features in parallel, and outputting four-level enhanced features;

[0016] S240: inputting the four-level enhanced features into the ASFF adaptive feature fusion layer, generating normalized weight parameters for each level, performing up-sampling or down-sampling on different layer features to the target level size, and then performing weighted summation to output four groups of fusion feature maps.

[0017] Preferably, S240 comprises:

[0018] S241: inputting the four-level enhanced features into the ASFF adaptive feature fusion layer to generate normalized weight parameters for each level;

[0019] S242: processing the weight parameters using a Softmax function to ensure that the sum of the four weight parameters is 1, and then multiplying each weight parameter with the corresponding feature layer to obtain ASFF-1, ASFF-2, ASFF-3, and ASFF-4;

[0020] S243: For ASFF-1, a 1×1 convolution is used to upsample the second, third, and fourth layers by 2, 4, and 8 times respectively, expanding them to the same feature layer size as the first layer, and then the information from the four layers is added together. For ASFF-2, a 1×1 convolution is used to downsample the first layer by 2 times, and upsample the third and fourth layers by 2 and 4 times respectively, transforming them to the same feature layer size as the second layer, and then the information from the four layers is added together. For ASFF-3, a 1×1 convolution is used to downsample the first and second layers by 4 and 2 times respectively, and upsample the fourth layer by 2 times, transforming it to the same feature layer size as the third layer, and then the information from the four layers is added together. For ASFF-4, a 1×1 convolution is used to downsample the first, second, and third layers by 8, 4, and 2 times respectively, transforming them to the same feature layer size as the fourth layer, and then the information from the four layers is added together, finally obtaining four fused feature maps at different scales.

[0021] Preferably, S243 specifically includes:

[0022] ;

[0023] in, , , and Let the weight tensor be a finite element that satisfies the following conditions: , and This represents the location index, or the spatial dimension or feature dimension. , , , These represent the first, second, third, and fourth feature layers of the fusion process, respectively. , , , These represent unifying the scale of the first feature layer to the scale of the first, second, third, and fourth feature layers, respectively. , , , These represent unifying the scale of the second feature layer to that of the first, second, third, and fourth feature layers, respectively. , , , These represent unifying the scale of the third feature layer to that of the first, second, third, and fourth feature layers, respectively. , , , respectively represent that the scale of the fourth feature layer is unified to the first feature layer, the second feature layer, the third feature layer and the fourth feature layer.

[0024] Preferably, S300 comprises:

[0025] S310: The multi-scale fused feature map respectively passes through a 7x7 convolution layer for feature transformation, and the transformed feature enters two parallel 1x1 convolution layers, which are respectively responsible for generating candidate boundary boxes and proposal extraction regions, and the disease and pest candidate regions are generated based on the candidate boundary boxes and the proposal extraction regions;

[0026] S320: The disease and pest candidate regions and the multi-scale fused feature map are subjected to feature unification and size normalization through an ROI pooling layer, and the pooled features are further subjected to two fully connected layers, and finally the branch outputs the fine adjusted disease and pest position regression frame and the corresponding specific classification category of the disease and pest.

[0027] Preferably, S400 comprises:

[0028] S410: The feature map with the largest spatial size is extracted from the output four feature maps, and the length, width and channel number thereof are 256x256x256 respectively;

[0029] S420: The adjusted image enters the feature extraction stage, which includes two groups of convolution blocks with the same structure, each group of convolution blocks includes two consecutive convolution layers, and a batch normalization layer and a ReLU activation function are sequentially connected after each convolution layer, and after each group of convolution blocks is processed, a maximum pooling layer is used for spatial down-sampling;

[0030] S430: After the feature extraction and down-sampling are completed, the obtained feature is sent to a fully connected layer, and finally the branch outputs the severity grade classification result of the crop disease and pest, which is divided into four categories: healthy, mild, moderate and severe.

[0031] Preferably, the preset loss function is specifically:

[0032] ;

[0033] ;

[0034] ;

[0035] ;

[0036] ;

[0037] ;

[0038] ;

[0039] wherein, denotes the total number of samples, denotes the total number of classes, denotes the indicator function, the true class of sample is equal to when is equal to 1, otherwise is equal to 0, denotes the predicted probability that observation sample belongs to class , is the classification loss and localization loss of the detection network branch, denotes the weighted sum of the binary classification loss and the bounding box regression loss, denotes the weight factor, denotes the number of samples of the candidate box, denotes the probability that the th candidate box is predicted as the target, denotes whether there is a target sample in the box, 1 for existence and 0 for non-existence, denotes the binary cross-entropy loss of the predicted value and the true value ; denotes the number of position parameters of the bounding box, denotes the regression parameter of the predicted th bounding box, wherein there are four parameters , denoting the center point coordinates and the width and height , and similarly, denotes the regression parameter of the true th bounding box of the sample, wherein there are four parameters , denoting the center point coordinates and the width and height , denotes the binary cross-entropy loss of the predicted value and the true value ; denotes the weighted sum of the crop disease and pest multi-classification loss and the bounding box regression loss, denotes the cross-entropy loss function of the real class and the predicted class of each type of disease and pest, denotes the bounding box regression loss, denotes the regression parameter of the bounding box regressor corresponding to the class , Bounding box regression parameters representing real targets .

[0040] A crop disease and pest identification system based on multi-task learning comprises:

[0041] A model construction and input image acquisition module is configured to construct a crop disease and pest identification model, comprising a multi-scale feature information fusion module, a disease detection branch and a severity classification branch; the input image of the crop leaf is acquired, and the input image is preprocessed to obtain a preprocessed dataset;

[0042] The multi-scale feature information fusion module is configured to perform feature extraction, multi-scale construction, attention enhancement and adaptive fusion processing on the preprocessed input image to obtain a multi-scale fusion feature map;

[0043] The disease detection branch is configured to generate a disease and pest candidate region on the multi-scale fusion feature map through a region proposal network, perform disease detection based on the disease and pest candidate region and the multi-scale fusion feature map, and output a disease and pest position and category detection result;

[0044] The severity classification branch is configured to receive the largest feature map in the multi-scale fusion feature map to perform severity classification and output a disease and pest severity level;

[0045] The model training module is configured to train the disease and pest identification model according to the disease and pest position and category detection result, the disease and pest severity level and a preset loss function, obtain a trained disease and pest identification model when a preset training end condition is reached, and complete real-time crop disease and pest identification based on the trained disease and pest identification model.

[0046] A computer device comprises a memory and a processor, the memory stores a computer program, and the processor implements the steps of a crop disease and pest identification method based on multi-task learning when executing the computer program.

[0047] A computer readable storage medium stores a computer program, and the computer program implements the steps of a crop disease and pest identification method based on multi-task learning when executed by a processor.

[0048] The crop disease and pest identification method and system based on multi-task learning realize end-to-end multi-task cooperative processing, complete disease and pest positioning, classification and severity evaluation synchronously through shared multi-scale fusion feature maps, greatly improve the identification efficiency, and meet the real-time monitoring needs of agricultural scenes. BRIEF DESCRIPTION OF DRAWINGS

[0049] Figure 1 The above is a flowchart of a crop disease and pest identification method based on multi-task learning in an embodiment of the present application.

[0050] Figure 2 A framework diagram of a crop disease and pest recognition method based on multi-task learning in an embodiment of the present application;

[0051] Figure 3 A framework diagram of a multi-scale feature information fusion module in an embodiment of the present application;

[0052] Figure 4 A disease and pest detection and severity classification double-branch network structure diagram in an embodiment of the present application. DETAILED DESCRIPTION

[0053] In order to enable personnel in the technical field to better understand the technical solutions of the present application, the present application will be further described in detail below with reference to the accompanying drawings.

[0054] In one embodiment, as shown in Figure 1 A crop disease and pest recognition method based on multi-task learning, the method comprising the following steps:

[0055] S100: Construct a crop disease and pest recognition model, including a multi-scale feature information fusion module, a disease detection branch, and a severity classification branch; obtain an input image of a crop leaf, and perform preprocessing such as size adjustment and pixel normalization on the input image to obtain a preprocessed data set;

[0056] S200: Input the preprocessed input image into the multi-scale feature information fusion module, and obtain a multi-scale fusion feature map through feature extraction, multi-scale construction, attention enhancement, and adaptive fusion processing;

[0057] S300: Generate a disease and pest candidate region on the multi-scale fusion feature map through a region proposal network, input the disease and pest candidate region and the multi-scale fusion feature Figure One into the disease detection branch for disease detection, and output a disease and pest position and category detection result;

[0058] S400: Input the largest feature map in the multi-scale fusion feature map into the severity classification branch for severity classification, and output a disease and pest severity level;

[0059] S500: Train the disease and pest recognition model according to the disease and pest position and category detection result, the disease and pest severity level, and a preset loss function, obtain a trained disease and pest recognition model when a preset training end condition is reached, and complete real-time crop disease and pest recognition based on the trained disease and pest recognition model.

[0060] Specifically, the multi-scale feature information fusion module is used to extract information of different scales of crops, and the shared extracted feature maps are used for disease detection and severity classification of the double-branch network. The detection can identify multiple diseases and pests of single leaf of crops, and the severity classification branch can overall evaluate the degree of invasion of diseases and pests.

[0061] The above-mentioned crop disease and pest identification method based on multi-task learning realizes end-to-end multi-task cooperative processing, synchronously completes disease and pest positioning, classification and severity evaluation through sharing of multi-scale fusion feature maps, greatly improves identification efficiency, and meets the real-time monitoring needs of agricultural scenes.

[0062] In one embodiment, the multi-scale feature information fusion module in S200 includes a ResNet-50 (Residual Network with 50 layers) feature extraction layer, an FPN (Feature Pyramid Network) multi-scale feature construction layer, a CBAM (Convolutional Block Attention Module) attention enhancement layer, and an ASFF (Adaptive Spatial Feature Fusion) adaptive feature fusion layer. S200 includes:

[0063] S210: using the ResNet-50 feature extraction layer to extract output four-level intermediate feature maps from the preprocessed input image;

[0064] S220: inputting the four-level intermediate feature maps into the FPN multi-scale feature construction layer, performing convolution compression channels on each level of features, and fusing from top to bottom to obtain a multi-scale feature layer {P2, P3, P4, P5};

[0065] S230: inputting the multi-scale feature layer {P2, P3, P4, P5} into the CBAM attention enhancement layer, applying channel and spatial double-attention operations to each level of features in parallel, and outputting four-level enhanced features;

[0066] S240: inputting the four-level enhanced features into the ASFF adaptive feature fusion layer, generating normalized weight parameters for each level, performing up-sampling or down-sampling on different layer features to the target level size, and then performing weighted summation to output four groups of fusion feature maps.

[0067] Specifically, the framework of the multi-scale feature information fusion module is as follows: Figure 3As shown, the ResNet-50 model is used for feature extraction, and after obtaining the intermediate feature map, the FCA (FPN-CBAM-ASFF) structure is used to generate a multi-scale feature map, and then the feature map is shared to the disease detection and severity classification branches, which is beneficial to the extraction of the required information of each task. ResNet50 can capture deeper target features in feature extraction, but it is often difficult to accurately locate the positions of these targets when dealing with targets of different scales. In particular, when dealing with small targets, ResNet50 may have difficulty capturing important information about them. In order to solve this problem, attention mechanisms are introduced into the field of computer vision. Spatial attention focuses on the relative position of input data, deeply mining the important information of each pixel point, guiding the model to focus on the areas that are crucial to the task, and improving the overall performance of the model. The channel attention model finds out which channels are important and gives them more weight, so that the model pays more attention to these important channels in subsequent calculations. In order to improve the detection accuracy of small targets, the ASFF feature fusion module is integrated into the ResNet50 network, which uses the feature extraction capability of ResNet50 to capture the deep features of the target, and enhances the attention to small target feature information through the ASFF module, Figure 2 The connection relationship between each module is illustrated. Through the four-level feature fusion architecture of the ResNet-50 feature extraction layer, the FPN multi-scale feature construction layer, the CBAM attention enhancement layer and the ASFF adaptive feature fusion layer, the multi-scale disease and pest feature expression capability is significantly enhanced, and the detection accuracy of diseases and pests is improved.

[0068] In one embodiment, S240 includes:

[0069] S241: The four-level enhanced features are input to the ASFF adaptive feature fusion layer to generate normalized weight parameters for each level;

[0070] S242: The weight parameters are processed using the Softmax (normalized exponential) function to ensure that the sum of the four weight parameters is 1, and then each weight parameter is multiplied by the corresponding feature layer to obtain ASFF-1, ASFF-2, ASFF-3, and ASFF-4;

[0071] S243: for ASFF-1, using 1x1 convolution, the second layer, the third layer and the fourth layer are respectively up-sampled by 2 times, 4 times and 8 times to expand to the same feature layer size as the first layer, and then the four layers of information are added; for ASFF-2, using 1x1 convolution, the first layer is down-sampled by 2 times, the third layer and the fourth layer are respectively up-sampled by 2 times and 4 times to transform to the same feature layer size as the second layer, and then the four layers of information are added; for ASFF-3, using 1x1 convolution, the first layer and the second layer are respectively down-sampled by 4 times and 2 times, and the fourth layer is up-sampled by 2 times to transform to the same feature layer size as the third layer, and then the four layers of information are added; for ASFF-4, using 1x1 convolution, the first layer, the second layer and the third layer are respectively down-sampled by 8 times, 4 times and 2 times to transform to the same feature layer size as the fourth layer, and then the four layers of information are added, and finally four fusion feature maps of different scales are obtained.

[0072] Specifically, through the adaptive weighted sampling mechanism of the ASFF, different scale pest and disease recognition targets can be solved on the fusion layer, thereby improving the recognition accuracy.

[0073] In one embodiment, S243 is specifically:

[0074] ;

[0075] wherein, , , and represent weight tensors, and satisfy , and represent position indexes, represent spatial dimensions or feature dimensions, , , , represent the first feature layer, the second feature layer, the third feature layer and the fourth feature layer of the fusion respectively, , , , represent the scale of the first feature layer unified to the first feature layer, the second feature layer, the third feature layer and the fourth feature layer respectively, , , , represent the scale of the second feature layer unified to the first feature layer, the second feature layer, the third feature layer and the fourth feature layer respectively, , , , respectively represent the scale of the third feature layer is unified to the first feature layer, the second feature layer, the third feature layer and the fourth feature layer, 、 、 、 respectively represent the scale of the fourth feature layer is unified to the first feature layer, the second feature layer, the third feature layer and the fourth feature layer.

[0076] In one embodiment, S300 includes:

[0077] S310: The multi-scale fusion feature map respectively passes through a 7x7 convolutional layer for feature transformation, and the transformed feature enters two parallel 1x1 convolutional layers, which are respectively responsible for generating candidate boundary boxes and proposal extraction regions, and the candidate boundary boxes and the proposal extraction regions are used to generate pest candidate regions;

[0078] S320: The pest candidate region and the multi-scale fusion feature map are subjected to feature unification and size normalization through an ROI (Region of Interest) pooling layer, and the pooled features are further processed through two fully connected layers. The branch simultaneously outputs a fine-tuned pest position regression frame and its corresponding pest specific classification category.

[0079] Specifically, the pest detection branch network structure diagram is as shown in Figure 4 The detection network can regard the pest leaf region as a scene, then identify this part of the region, greatly reduce the influence of irrelevant information interference, and improve the identification efficiency. At the same time, when there are multiple pest categories on a single leaf, the detection network usually gives multiple labels, divides the leaf with multiple pests into different regions, and then divides the diseased regions into different categories, thereby improving the detection efficiency.

[0080] In one embodiment, S400 includes:

[0081] S410: Extract the feature map with the largest spatial size from the output four feature maps, and the length, width and channel number are 256x256x256 respectively;

[0082] S420: The adjusted image enters the feature extraction stage, which includes two groups of convolutional blocks with the same structure. Each group of convolutional blocks includes two consecutive convolutional layers, and a batch normalization layer and a ReLU (Rectified Linear Unit) activation function are connected in sequence after each convolutional layer. After each group of convolutional blocks is processed, a maximum pooling layer is used for spatial down-sampling;

[0083] S430: After feature extraction and downsampling are completed, the obtained features are sent to a full connection layer, finally, the branch outputs the severity level classification result of crop diseases and insect pests, and the result is classified into four categories of healthy, mild, moderate and severe.

[0084] Specifically, the severity level classification branch network structure diagram is as shown in the figure. Figure 4 The convolution block is designed for severity classification, which effectively extracts pathological texture features; four-level classification meets the demand of accurate agricultural decision-making and reduces the misjudgment rate of severity.

[0085] Further, the present application relates to the loss functions of different tasks, which are composed of two parts, one part is the loss function of disease detection, and the other part is the loss function of severity. Through the loss function Loss （total） The guided network is jointly trained and promotes each other, so that the classification task and the detection task can achieve the best result. If the loss functions of different tasks are directly added as the total loss function of the multi-task learning model, the normal joint training of the model can be guided. However, the time and the number of training times to reach the optimal accuracy of each task are different. Through experiments, it is proved that the classification network loss function converges faster than the detection network. Through multiple iteration training, the classification network is prone to overfitting, while the loss function of the detection network has not converged, causing the imbalance of joint training.

[0086] The classification network loss function is specifically:

[0087] ;

[0088] Among them, indicates the total number of samples, indicates the total number of categories, indicates the symbol function, the true category of the sample is equal to when is equal to 1, otherwise is equal to 0, indicates the predicted probability that the observed sample belongs to the category ;

[0089] The binary classification loss and the bounding box regression loss function are specifically:

[0090] ;

[0091] Among them, indicates a weight factor, indicates the sample number of the candidate frame, indicates the probability that the i-th candidate frame is predicted as a target, This indicates whether the target sample exists within the bounding box; a value of 1 indicates presence, and a value of 0 indicates absence. This indicates the predicted value and the true value Apply binary cross-entropy loss to obtain the classification loss value; Indicates the number of bounding box position parameters. Indicates the prediction of the first The regression parameters of the bounding boxes, where There are four parameters , representing the coordinates of the center point and width and height Similarly, Indicates the sample number The true regression parameters of the bounding box, among which There are four parameters , representing the coordinates of the center point and width and height , This indicates the predicted value and the true value Apply binary cross-entropy loss to obtain the bounding box loss value;

[0092] The specific loss functions for multi-class classification of crop diseases and pests and bounding box regression are as follows:

[0093] ;

[0094] ;

[0095] ;

[0096] in, This represents the weighting factor. Indicates the true categories of various diseases and pests and prediction categories Cross-entropy loss function, Indicates the category corresponding to the bounding box regressor regression parameters , Regression parameters of the bounding box representing the true target .

[0097] The loss function of the crop disease and pest detection network is... and composition:

[0098] ;

[0099] Finally, the total loss function is obtained from the loss function of the severity assessment classification network and the loss function of the crop disease and pest detection network. This invention introduces a weighting factor before the loss function for each task. The weight factor mainly aims to adjust the proportion of each task in each task, and therefore, the designed weighted cross-entropy loss function of the application is obtained by expansion of multiple cross-entropy loss functions. The expression of the weighted loss function is as follows:

[0100] .

[0101] The end-to-end multi-task learning framework proposed in the application significantly improves the recognition efficiency of complex crop leaf disease and pest scenes by deeply fusing the double-branch structure of disease and pest detection and severity classification. The core advantages are as follows: first, the innovative attention-based multi-scale feature fusion mechanism CA module (CBAM-ASFF) breaks through the limitation of traditional single-scale feature extraction, cooperatively operates convolution and channel attention, dynamically aggregates pathological feature information of different receptive fields on the basis of four-level features extracted by the ResNet-50 backbone network, and adapts to recognition of diseases and pests of different scales to fully exploit features. Second, the double-branch collaborative architecture realizes efficient parallelism of multi-task: the disease detection branch introduces an improved RPN (‌Region Proposal Network) network to obtain a candidate frame and a self-adaptive region of interest pooling operation, supports simultaneous recognition of multiple diseases and pests for a single leaf, and the severity classification branch constructs a four-level gradient evaluation system to effectively reduce the misjudgment rate and provide a quantitative basis for precise pesticide application. Finally, based on the multi-task dynamic weighted loss function, the weighted disease and pest detection loss and severity classification loss function are used to jointly guide the network training in the process of back propagation, solve the model bias problem caused by sample imbalance, effectively improve the training convergence speed, and improve the recognition accuracy of diseases and pests.

[0102] In one embodiment, a crop disease and pest recognition system based on multi-task learning is also provided, comprising:

[0103] A model construction and input image acquisition module is configured to construct a crop disease and pest recognition model, including a multi-scale feature information fusion module, a disease detection branch, and a severity classification branch; and acquire an input image of a crop leaf, pre-process the input image, and obtain a pre-processed data set.

[0104] The multi-scale feature information fusion module is configured to perform feature extraction, multi-scale construction, attention enhancement, and adaptive fusion processing on the pre-processed input image to obtain a multi-scale fusion feature map.

[0105] The disease detection branch is configured to generate a disease and pest candidate region on the multi-scale fusion feature map through a region proposal network, perform disease detection based on the disease and pest candidate region and the multi-scale fusion feature map, and output a disease and pest position and class detection result.

[0106] The severity classification branch is configured to receive a feature map with the largest size in the multi-scale fusion feature map to perform severity classification, and output a disease and pest severity level.

[0107] The model training module is configured to train the disease and pest recognition model according to the disease and pest location and category detection result, the disease and pest severity level, and a preset loss function, obtain a trained disease and pest recognition model when a preset training end condition is reached, and complete real-time disease and pest recognition of the crops based on the trained disease and pest recognition model.

[0108] For specific limitations of the crop disease and pest recognition system based on multi-task learning, refer to the limitations of the crop disease and pest recognition method based on multi-task learning in the above, which will not be repeated here. Each module in the crop disease and pest recognition system based on multi-task learning can be realized by software, hardware, and combinations thereof, in whole or in part. The above modules can be embedded in or independent of the processor in the computer device in hardware form, or stored in the memory in the computer device in software form, so as to be called and executed by the processor to perform the operations corresponding to each module.

[0109] A computer device includes a memory and a processor, and the memory stores a computer program. When the processor executes the computer program, the steps of the crop disease and pest recognition method based on multi-task learning are implemented.

[0110] A computer readable storage medium stores a computer program. When the computer program is executed by a processor, the steps of the crop disease and pest recognition method based on multi-task learning are implemented.

[0111] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when the computer program is executed, the processes of the above-mentioned embodiments of each method can be included. Any reference to memory, storage, database or other medium used in each embodiment provided by the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0112] The above describes in detail the crop disease and pest identification method and system based on multi-task learning provided by the present application. The principles and implementation modes of the present application are described by applying specific examples in this paper. The above description of the embodiments is only used to help understand the core idea of the present application. It should be pointed out that for those skilled in the art, without departing from the principles of the present application, the present application can be improved and modified in several ways, and these improvements and modifications also fall within the protection scope of the claims of the present application.

Claims

1. A crop disease and pest identification method based on multi-task learning, characterized in that, The method comprises the following steps: S100: Construct a crop disease and pest identification model, including a multi-scale feature information fusion module, a disease detection branch, and a severity classification branch; obtain an input image of a crop leaf, and pre-process the input image to obtain a pre-processed dataset; S200: input the pre-processed input image into the multi-scale feature information fusion module, and obtain a multi-scale fusion feature map through feature extraction, multi-scale construction, attention enhancement, and adaptive fusion processing; S300: generate a disease and pest candidate region on the multi-scale fusion feature map through a region proposal network, input the disease and pest candidate region and the multi-scale fusion feature map into the disease detection branch for disease detection, and output a disease and pest position and category detection result; S400: input the largest feature map in the multi-scale fusion feature map into the severity classification branch for severity classification, and output a disease and pest severity level; S500: train the disease and pest identification model according to the disease and pest position and category detection result, the disease and pest severity level, and a preset loss function, obtain a trained disease and pest identification model when a preset training end condition is reached, and complete real-time crop disease and pest identification based on the trained disease and pest identification model; The multi-scale feature information fusion module in S200 comprises a ResNet-50 feature extraction layer, an FPN multi-scale feature construction layer, a CBAM attention enhancement layer, and an ASFF adaptive feature fusion layer, and S200 comprises: S210: use the ResNet-50 feature extraction layer to extract an output four-level intermediate feature map from the pre-processed input image; S220: input the four-level intermediate feature map into the FPN multi-scale feature construction layer, perform convolution compression channels on each level feature, and perform top-down fusion to obtain a multi-scale feature layer {P2, P3, P4, P5}; S230: input the multi-scale feature layer {P2, P3, P4, P5} into the CBAM attention enhancement layer, apply channel and spatial double attention operations to each level feature in parallel, and output four-level enhanced features; S240: input the four-level enhanced features into the ASFF adaptive feature fusion layer, generate normalized weight parameters for each level, perform up-sampling or down-sampling on different layer features to the target level size, and then perform weighted summation to output four groups of fusion feature maps; S240 comprises: S241: input the four-level enhanced features into the ASFF adaptive feature fusion layer to generate normalized weight parameters for each level; S242: process the weight parameters using a Softmax function to ensure that the sum of the four weight parameters is 1, and then multiply each weight parameter with the corresponding feature layer to obtain ASFF-1, ASFF-2, ASFF-3, and ASFF-4; S243: for ASFF-1, using 1x1 convolution, the second layer, the third layer and the fourth layer are respectively up-sampled by 2 times, 4 times and 8 times to expand to the same feature layer size as the first layer, and then the four layers of information are added; for ASFF-2, using 1x1 convolution, the first layer is down-sampled by 2 times, and the third layer and the fourth layer are up-sampled by 2 times and 4 times respectively, which are transformed to the same feature layer size as the second layer, and then the four layers of information are added; for ASFF-3, using 1x1 convolution, the first layer and the second layer are respectively down-sampled by 4 times and 2 times, and the fourth layer is up-sampled by 2 times, which are transformed to the same feature layer size as the third layer, and then the four layers of information are added; for ASFF-4, using 1x1 convolution, the first layer, the second layer and the third layer are respectively down-sampled by 8 times, 4 times and 2 times, which are transformed to the same feature layer size as the fourth layer, and then the four layers of information are added, finally obtaining four fusion feature maps of different scales.

2. The method of claim 1, wherein, S243 specifically is: ； wherein , , and denote weight tensors and satisfy , and denote position indices, denote spatial dimensions or feature dimensions, , , , denote a fused first feature layer, a second feature layer, a third feature layer and a fourth feature layer, respectively, , , , denote a scale unification of the first feature layer to the first feature layer, the second feature layer, the third feature layer and the fourth feature layer, respectively, , , , denote a scale unification of the second feature layer to the first feature layer, the second feature layer, the third feature layer and the fourth feature layer, respectively, , , , denote a scale unification of the third feature layer to the first feature layer, the second feature layer, the third feature layer and the fourth feature layer, respectively, , , , denote a scale unification of the fourth feature layer to the first feature layer, the second feature layer, the third feature layer and the fourth feature layer, respectively.

3. The method of claim 2, wherein, S300 includes: S310: the multi-scale fusion feature maps respectively pass through a 7x7 convolution layer for feature transformation, and the transformed features enter two parallel 1x1 convolution layers, which are respectively responsible for generating candidate bounding boxes and proposal extraction regions, and the disease and pest candidate regions are generated based on the candidate bounding boxes and the proposal extraction regions; S320: the disease and pest candidate regions and the multi-scale fusion feature maps are unified in feature and normalized in size through an ROI pooling layer, and the pooled features pass through two fully connected layers, finally the branch outputs the disease and pest position regression frame and the corresponding specific classification category of the disease and pest.

4. The method of claim 3, wherein, S400 includes: S410: extracting the feature map with the largest spatial size from the four output feature maps, which has a length, a width and a channel number of 256x256x256; S420: the adjusted image enters the feature extraction stage, which includes two groups of convolution blocks with the same structure, each group of convolution blocks includes two consecutive convolution layers, and a batch normalization layer and a ReLU activation function are connected in sequence after each convolution layer, and after each group of convolution blocks is processed, a maximum pooling layer is used for spatial down-sampling; S430: after feature extraction and down-sampling are completed, the obtained features are sent to a fully connected layer, finally, the branch outputs the severity classification result of the crop disease and pest, which is divided into four categories: healthy, mild, moderate and severe.

5. The method of claim 4, wherein, The preset loss function is specifically: ； ； ； ； ； ； ； wherein, represents a classification network loss function, represents the total number of samples, represents the total number of classes, represents a symbol function, sample The real class is equal to when is equal to 1, otherwise is equal to 0, represents the predicted probability that the observed sample belongs to class , is the classification loss and localization loss of the detection network branch, represents the weighted sum of the binary classification loss and the bounding box regression loss, represents a weight factor, represents the number of samples of the candidate box, represents the probability that the th candidate box is predicted to be the target, represents whether there is a target sample in the box, and the value is 1 if there is, and 0 if there is not, represents the binary cross-entropy loss of the predicted value and the true value , and the classification loss value is obtained; represents the number of bounding box position parameters, represents the regression parameter of the predicted th bounding box, wherein there are four parameters , representing the center point coordinates and the width and height , and similarly, represents the real regression parameter of the th bounding box of the sample, wherein there are four parameters , representing the center point coordinates and the width and height , represents the binary cross-entropy loss of the predicted value and the true value , and the bounding box loss value is obtained; represents the weighted sum of the crop disease and pest multi-classification loss and the bounding box regression loss, represents the cross-entropy loss function of the real class and the predicted class of each class of disease and pest, represents the bounding box regression loss, represents the regression parameter of the corresponding class of the bounding box regressor , Bounding box regression parameters representing a real target .

6. A crop disease and pest identification system based on multi-task learning of the method according to any one of claims 1 to 5. It includes: a model construction and input image acquisition module for constructing a crop disease and pest recognition model, including a multi-scale feature information fusion module, a disease detection branch and a severity classification branch; acquiring an input image of a crop leaf, and preprocessing the input image to obtain a preprocessed data set; a multi-scale feature information fusion module for feature extraction, multi-scale construction, attention enhancement and adaptive fusion processing on the preprocessed input image to obtain a multi-scale fusion feature map; The disease detection branch is configured to generate disease and pest candidate regions by a region proposal network on the multi-scale fusion feature map, perform disease detection based on the disease and pest candidate regions and the multi-scale fusion feature map, and output disease and pest position and category detection results. The severity classification branch is configured to receive the largest feature map in the multi-scale fusion feature map to perform severity classification, and output a disease and pest severity level. The model training module is configured to train the disease and pest recognition model according to the disease and pest position and category detection results, the disease and pest severity level, and a preset loss function, obtain the trained disease and pest recognition model when a preset training end condition is reached, and complete real-time disease and pest recognition of crops based on the trained disease and pest recognition model. 7.A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer device is configured to perform the method according to any one of claims 1-6 when the computer program is executed by the processor. The computer program, when executed by the processor, implements the steps of the method of any one of claims 1 to 5.

8. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program, when executed by the processor, implements the steps of the method of any one of claims 1 to 5.

Images