A mulberry leaf pest and disease detection method based on small sample learning

By constructing the RP-DCNet model, combining the relative position coding module and the intensive relationship distillation module, the problems of low accuracy and low efficiency caused by the small number of samples in mulberry leaf pest detection are solved, and high-precision identification of rare pests and diseases are achieved, reducing the Testing cost.

CN114741548BActive Publication Date: 2024-11-08ANHUI AGRICULTURAL UNIVERSITY
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
CN202210458831.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-04-27
Publication Date
2024-11-08
Estimated Expiration
2042-04-27

AI Technical Summary

Technical Problem

In the detection of mulberry leaf pests and diseases, the existing technology has low sample size, which leads to low model accuracy, low efficiency and high cost, especially the poor accuracy of identification of rare pests and diseases.

Method used

Using a small sample learning method, by constructing the RP-DCNet model, using the pest and pest image data sets of other crop leaves as the base data set, combining the pest and pest image data sets for meta-learning and meta-fine-tuning, and adding the relative position coding module. Extract the intensive relationship distillation module and the context-aware polymerization module to perform feature extraction and error adjustment to improve detection accuracy.

Benefits of technology

In the case of small samples, it can effectively identify the types of mulberry leaf diseases and pests, maintain high detection accuracy, and help mulberry leaf growers take timely prevention and control measures to reduce crop losses.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114741548B_ABST
    Figure CN114741548B_ABST
Patent Text Reader

Abstract

The present invention discloses a mulberry leaf pest and disease detection method based on small sample learning, comprising the following steps: step 1, data set acquisition: obtaining a base class data set and a new class data set; step 2, data set preprocessing; step 3, building an RP-DCNet model; step 4, training the RP-DCNet model with the base class data set in a meta-learning stage; step 5, training the RP-DCNet model with a new class data set and a base class data set equivalent to the new class data set in a meta-fine-tuning stage; step 6, adjusting the parameters of the RP-DCNet model after training to obtain optimal configuration parameters; step 7, using the model under the optimal configuration parameters for mulberry leaf pest and disease detection. The present invention can still maintain a high detection accuracy when the number of samples is small.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to a pest and disease image detection method, a mulberry leaf pest and disease detection method based on small sample learning. Background Art

[0002] In the process of mulberry planting, mulberry diseases are caused by pathogenic microorganisms or unsuitable environmental conditions, resulting in poor growth and development, reduced mulberry leaf yield and deterioration of quality. Therefore, the prevention and control of mulberry leaf diseases and pests has always been a major problem. If they are not prevented and controlled in time, mulberry leaf growers will reduce their income. Therefore, timely prevention and control of diseases and pests is the key to fundamentally reducing losses. Common diseases and pests of mulberry leaves include: mulberry shriveling disease, mulberry blight, mulberry brown spot disease, etc. Parasitic diseases include mulberry root rot, mulberry dry disease, mulberry plaster disease, mulberry powdery mildew, mulberry leaf stain disease and mulberry sclerotinia disease. With the increasing computing power of computers, the application of computer fields in agriculture has become more and more extensive. Artificial intelligence algorithms can be effectively applied to specific agricultural scenarios to help agricultural workers improve product quality and yield more scientifically.

[0003] Because mulberry leaves have some relatively rare pests and diseases, the number of samples collected from the Internet or in the field is extremely small. In the field of mulberry leaf pest and disease identification, there is currently no large pest and disease dataset for machine learning model training. Some pest and disease types often have only a few pictures, and conventional target detection frameworks often have poor accuracy for such small sample numbers.

[0004] At present, a large number of researchers have applied computer vision methods to the identification of crop pests and diseases. For example, in tomato leaf disease images and rice pest and disease images. However, these technologies are not mature enough for crop disease images, especially mulberry leaf disease images, and conventional deep learning often needs to be supported by a large number of data sets to more accurately determine the category of pest and disease detection. In addition, due to the small number of people paying attention to these fields, high collection costs and rare types of diseases, there are often no relevant pest and disease type data sets for researchers to refer to, resulting in low model accuracy, low efficiency and high cost. Summary of the invention

[0005] The purpose of the present invention is to provide a mulberry leaf disease and pest detection method based on small sample learning, so as to solve the problems of low model accuracy, low efficiency and high cost when the sample size is small in the prior art using machine learning for mulberry leaf disease and pest detection and identification.

[0006] In order to achieve the above object, the technical solution adopted by the present invention is:

[0007] A mulberry leaf pest detection method based on small sample learning comprises the following steps:

[0008] Step 1: Data collection:

[0009] Obtain existing image datasets of pests and diseases of leaves of other crops as base class datasets, and obtain image datasets of pests and diseases of mulberry leaves as new class datasets;

[0010] Step 2: Dataset preprocessing:

[0011] The base class data set and the new class data set obtained in step 1 are divided into a training set and a test set respectively, and the training set and the test set of the base class data set and the new class data set are preprocessed respectively;

[0012] Step 3: Build the RP-DCNet model based on the DCNet model:

[0013] The DCNet model includes a feature extractor, a dense relation distillation module, and a context-aware aggregation module. The query feature extractor in the DCNet model takes the query image as input to obtain a query feature map, and the query feature map is then processed by the key encoder and value encoder of the dense relation distillation module to obtain the query key feature map and value feature map. The support feature extractor in the DCNet model takes the support image and its corresponding binary mask image as input to obtain the support feature map, and the support feature map is then processed by the key encoder and value encoder of the dense relation distillation module to obtain the supported key feature map and value feature map;

[0014] A relative position encoding module is added to the dense relation distillation module in the DCNet model, thereby obtaining an RP-DCNet model; the key encoder and the value encoder in the dense relation distillation module generate corresponding key feature graphs and value feature graphs on the query feature graph and the support feature graph, and the relative position encoding module encodes based on the dimensions of the key feature graphs generated by the query feature graph and the support feature graph, thereby establishing relative position encoding on the key feature graph;

[0015] Step 4: The training of the DCNet model consists of two stages: meta-learning and meta-fine-tuning. The training set in the base class dataset is input into the RP-DCNet model for training in the meta-learning stage, and the output results of the RP-DCNet model during training are compared with the test set in the base class dataset for error calculation. The parameters of the RP-DCNet model are adjusted to the optimal parameters based on the error calculation results.

[0016] Step 5: Input the training set in the new class data set and the training set in the base class data set with the same number as the training set in the new class data set into the RP-DCNet model for training in the meta-fine-tuning stage, and perform error calculation between the output result of the RP-DCNet model during training and the test set in the new class data set, and adjust the parameters of the RP-DCNet model to the optimal parameters again based on the error calculation result;

[0017] Step 6: Use the RP-DCNet model adjusted to the optimal parameters in steps 4 and 5 to detect the mulberry leaf pest and disease images to obtain the pest and disease detection results.

[0018] Furthermore, the preprocessing described in step 2 includes mosaic data augmentation, random flipping, random cropping, and scaling.

[0019] Furthermore, during preprocessing in step 2, Mosaic data enhancement is used, and random flipping is performed with a set probability. One of several scales is randomly selected to scale the data in the training set, and a part of the image is randomly cropped as a new image.

[0020] Furthermore, in step 3, the key feature map and the value feature map are extracted from the query feature map and the support feature map respectively. The relative position encoding module is embedded after the extracted key feature map is matrix multiplied and enters the output of the Softmax function;

[0021] The relative position encoding module takes the last two dimensions of the key feature map as a reference, calculates the relative coordinates of the current position and other positions, and thereby establishes a relative position encoding in the last two dimensions of the key feature map;

[0022] Then the relative position encoding module performs matrix addition on the formed relative position encoding and the output of the Softmax function in the relational dense distillation module to obtain the output result.

[0023] Furthermore, in the meta-learning phase in step 4, the base class dataset is input into the RP-DCNet model, and the query feature extractor and the support feature extractor are jointly trained in this phase. Similarly, in the meta-fine-tuning phase, the dense relation distillation module, context-aware aggregation module and other basic model components in the RP-DCNet model are also learned during the training.

[0024] Furthermore, the error calculation in steps 4 and 5 includes classification error calculation and regression error calculation.

[0025] Furthermore, in steps 4 and 5, the number of training rounds in the meta-fine-tuning phase is less than that in the meta-learning phase.

[0026] Compared with the prior art, the advantages of the present invention are:

[0027] The present invention proposes a small-sample mulberry leaf disease and insect pest detection method. Even when the number of samples is small, the types of mulberry leaf diseases and insect pests can still be effectively identified, helping mulberry leaf growers to accurately identify rare diseases and insect pests, so as to take corresponding prevention and control measures, avoiding crop losses due to incorrect identification of the types of diseases and insect pests, and even when the number of samples is only a few, a high detection accuracy can still be maintained. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 It is a flowchart of the method of the present invention.

[0029] Figure 2 It is the RP-DCNet framework structure diagram proposed by the method of the present invention.

[0030] Figure 3 It is a flow chart of relative position coding construction proposed by the method of the present invention.

[0031] Figure 4 This is a specific structural diagram of the relative position coding proposed by the method of the present invention in RP-DCNet.

[0032] Figure 5 It is a flow chart of the training steps of the method of the present invention. DETAILED DESCRIPTION

[0033] The present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0034] like Figure 1 As shown, the mulberry leaf pest detection method based on small sample learning in the embodiment of the present invention includes the following steps:

[0035] 1. Find a large number of existing image datasets of pests and diseases of leaves of other crops on the Internet as the base class dataset of the model, and collect image datasets of mulberry leaf pests and diseases from the Internet and the field as new class datasets of the model.

[0036] 2. Divide the data in the base class dataset into training set, test set and validation set in a ratio of 7:2:1, and divide the data in the new class dataset into training set, test set and validation set in a ratio of 7:2:1.

[0037] 3. Perform Mosaic data augmentation, random flipping, random cropping, and scaling on the training set and test set of the base class dataset and the new class dataset respectively.

[0038] Mosaic data enhancement involves randomly cropping four images, splicing them into a new image as new data, randomly flipping them with a probability of 0.5, randomly cropping a part of the image as a new image, and randomly selecting one of several scales to scale the data in the training set.

[0039] 4. Build the RP-DCNet model. The complete structure of the RP-DCNet model is as follows: Figure 2 As shown in the figure, the RP-DCNet model is improved based on the DCNet model. The DCNet model includes a feature extractor, a dense relation distillation module, and a context-aware aggregation module. The feature extractor in the DCNet model takes the query image and the support image and their corresponding binary mask images as input, and obtains the query feature map and the support feature map through the 3*3 convolution layer with shared weights in the feature extractor. The query feature map and the support feature map are respectively input to the key encoder and the value encoder in the dense relation distillation module to obtain the key feature map and the value feature map corresponding to the query feature map and the support feature map, respectively. The value feature is used to quantify the feature similarity between the query image and the support image set, thereby activating the area in the corresponding value feature map.

[0040] A relative position encoding module is added to the dense relation distillation module in the DCNet model, thereby obtaining the RP-DCNet model. The query feature map and the support feature map extract the key feature map and the value feature map respectively. The relative position encoding module is embedded in the position where the extracted key feature map is matrix multiplied into the dimension of the Softmax function output, thereby establishing a relative position encoding on the key feature map.

[0041] For the query feature graph, let the key feature graph after key encoding be k q , where k q ∈R C / 8×H×W For the support feature map, let the key feature map after key encoding be k s , where k s ∈R N×C / 8×H×W After matrix multiplication and Softmax function operation, the output dimension is k q,s ∈R N×H×W×H×W Therefore, a relative position encoding r is established on the key feature map in the support feature map, where r∈R N×H×W×H×W .

[0042] The relative position encoding in the RP-DCNet model is obtained by calculating the relative coordinates of the current position and other positions through mathematical operations. The relative position encoding is calculated on the last two dimensions of the key feature map. Therefore, when there is a position with calculated coordinates (p, q), the calculation formula for the corresponding position (i, j) is as follows:

[0043] P i,j,p,q =(pi,ji),

[0044] P i,j,p,qIt is the relative position difference between the pth row and jth column and the ith row and jth column. Where (p,i={0,1…,W-1}q,j={0,1…,H-1}), H represents the height of the key feature map that supports feature map generation, and W represents the width of the key feature map that supports feature map generation. Add W-1 to both the row and column labels so that the values ​​of the row and column labels are not less than 0, then multiply the row label by 2H-1, add the row and column labels, and then concatenate W*H matrices, thus obtaining a relative position code with a dimension of WH×WH. The value range of the relative position code is 0 to (2H-1)×(2H-1)+2(WH).

[0045] The generation of relative position coding requires the guidance of a relative position coding table, which is obtained by a randomly initialized learnable parameter matrix. The dimension of the relative position coding table is (2H-1)×(2H-1)+2(WH). Then the value of the corresponding position is queried through the relative position coding table as the final relative position coding. Because there are N classes in the support matrix, N relative position coding tables are required. The dimension of the final relative position coding table is WH×WH×N. The generation structure of the relative position coding is as follows: Figure 3 As shown in the figure, the present invention is illustrated by taking W=2 and H=2 as an example.

[0046] A relative position encoding module is added on the basis of DCNet. The relative position encoding is embedded in the position where the key feature of the query feature and the key feature of the support feature are matrix-multiplied. In this way, the pixel-level feature similarity of the query feature and the support feature is obtained. The calculation formula is as follows:

[0047] F(k qi ,k sj )=φ(k qi ) T φ′(k sj )

[0048] Where: i and j are the position indices of the query and support features, φ and φ′ represent different linear operations, and F(k qi ,k sj ) represents the similarity calculation function, φ(k qi ) T =ak qi +b,φ′(k sj ) means ck qi +d,k qi represents the i-th value of the query key feature map, k sj represents the jth value of the support key feature map, and the linear operation φ(k qi ) T and φ′(k sj) can be continuously learned by the model during training. After Softmax normalization, the final similarity weight W is output. ij . Where W ij The calculation formula is as follows:

[0049]

[0050] The relative position code r with the dimension of WH×WH×N is matrix-added with the similarity weight W and output to the next module. Figure 4 shown.

[0051] 7. In the meta-learning phase, a large number of base class datasets constructed from leaves of other crop pests and diseases are input into the model. In this phase, the query feature extractor and the support feature extractor are jointly trained. Similarly, the dense relation distillation module, context-aware aggregation module and other basic model components in the RP-DCNet model are also learned during the training.

[0052] In the meta-fine-tuning stage, because there is less data on the mulberry leaf pest and disease category, while there are more data sets on other crop leaf pest and disease data sets, in order to balance the sample difference between the two data sets, the present invention selects samples in the base class data set that have the same number of marked detection frames as the new class data set. The new class data set and the selected base class data set are input into the model for training. Similar to the meta-learning stage, during the meta-fine-tuning stage training, the basic modules in the model will continuously learn and update parameters to detect new categories of mulberry leaf pest and disease. In order to avoid overfitting, the number of training rounds in the meta-fine-tuning stage should be less than that in the meta-learning stage. The method of model meta-learning and meta-fine-tuning training is as follows: Figure 5 shown.

[0053] 8. After the RP-DCNet model generates a prediction target, the classification error and regression error are calculated in the meta-learning stage and the meta-fine-tuning stage respectively, and then the error results are reflected in the various parameters of the RP-DCNet model, and the network parameters in the RP-DCNet model are updated to generate the expected output; the validation set is used as input to verify the model and test the robustness of the model as the final detection model.

[0054] 9. Finally, the RP-DCNet model with optimal parameters is adjusted to detect the mulberry leaf pest and disease images to obtain the pest and disease detection results.

[0055] The embodiments described in the present invention are merely descriptions of the preferred implementation modes of the present invention, and are not intended to limit the concept and scope of the present invention. Without departing from the design concept of the present invention, various modifications and improvements made to the technical solutions of the present invention by engineers and technicians in this field should fall within the protection scope of the present invention. The technical contents for which protection is sought in the present invention have all been recorded in the claims.

Claims

1. A mulberry leaf pest detection method based on small sample learning, characterized in that: The following steps are involved: Step 1: Data collection: Obtain existing image datasets of pests and diseases of leaves of other crops as base class datasets, and obtain image datasets of pests and diseases of mulberry leaves as new class datasets; Step 2: Dataset preprocessing: The base class data set and the new class data set obtained in step 1 are divided into a training set and a test set respectively, and the training set and the test set of the base class data set and the new class data set are preprocessed respectively; Step 3: Build the RP-DCNet model: The RP-DCNet model is improved based on the DCNet model. The DCNet model includes a feature extractor, a dense relation distillation module, and a context-aware aggregation module. The feature extractor in the DCNet model takes the query image and the support image and their corresponding binary mask images as input. The query feature map and the support feature map are obtained through the 3*3 convolutional layer with shared weights in the feature extractor. The query feature map and the support feature map are respectively input into the key encoder and the value encoder in the dense relation distillation module to obtain the key feature map and the value feature map corresponding to the query feature map and the support feature map. The value feature is used to quantify the feature similarity between the query image and the support image set, thereby activating the area in the corresponding value feature map. A relative position encoding module is added to the dense relation distillation module in the DCNet model, thereby obtaining the RP-DCNet model, querying the feature map and the support feature map to extract the key feature map and the value feature map respectively, and the relative position encoding module is embedded in a position based on the dimension after the extracted key feature map is matrix multiplied into the output of the Softmax function, thereby establishing a relative position encoding on the key feature map; For the query feature graph, let the key feature graph after key encoding be k q , where k q ∈R C / 8×H×W , for the support feature map, let the key feature map after key encoding be k s , where k s ∈R N×C / 8×H×W After matrix multiplication and Softmax function operation, the output dimension is k q,s ∈R N×H×W×H×W , so a relative position encoding r is established on the key feature map in the support feature map, where r∈R N×H×W×H×W ; The relative position encoding in the RP-DCNet model is obtained by calculating the relative coordinates of the current position and other positions through mathematical operations. The relative position encoding is calculated on the last two dimensions of the key feature map. Therefore, when there is a position with calculated coordinates (p, q), the calculation formula for the corresponding position (i, j) is as follows: P i,j,p,q =(p-i,j-i), P i,j,p,q is the relative position difference between the p-th row and j-th column and the i-th row and j-th column, where (p,i={0,1…,W-1}q,j={0,1…,H-1}), H represents the height of the key feature map that supports feature map generation, W represents the width of the key feature map that supports feature map generation, W-1 is added to the row and column labels so that the values ​​of the row and column labels are not less than 0, then the row label is multiplied by 2H-1, and then the row and column labels are added, and then W*H matrices are concatenated, thereby obtaining a relative position code of dimension WH×WH, and the value range of the relative position code is 0 to (2H-1)×(2H-1)+2(WH); The generation of relative position coding requires the guidance of a relative position coding table. The relative position coding table is obtained by a randomly initialized learnable parameter matrix. The dimension of the relative position coding table is (2H-1)×(2H-1)+2(WH). The value of the corresponding position is queried through the relative position coding table as the final relative position coding. Because there are N classes in the support matrix, N relative position coding tables are required. The dimension of the final relative position coding table is WH×WH×N. A relative position encoding module is added on the basis of DCNet. The relative position encoding is embedded in the position where the key feature of the query feature and the key feature of the support feature are matrix-multiplied to obtain the pixel-level feature similarity of the query feature and the support feature. The calculation formula is as follows: F(k qi ,k sj )=φ(k qi ) T φ'(k sj ) Where: i and j are the position indices of the query and support features, φ and φ' represent different linear operations, and F(k qi ,k sj ) represents the similarity calculation function, φ(k qi ) T =ak qi +b,φ'(k sj ) means ck qi +d,k qi represents the i-th value of the query key feature map, k sj represents the jth value of the support key feature map, and the linear operation φ(k qi ) T and φ'(k sj ) can be continuously learned by the model during training, and then processed by Softmax normalization to output the final similarity weight W ij , where W ij The calculation formula is as follows: The relative position code r with the dimension of WH×WH×N is matrix-added with the similarity weight W and output to the next module; Step 4: Meta-learning phase: a large number of base class datasets constructed from leaves of other crop pests and diseases are input into the model. In this phase, the query feature extractor and the support feature extractor are jointly trained. Similarly, the dense relation distillation module, context-aware aggregation module and other basic model components in the RP-DCNet model are also learned during the training phase. In the meta-fine-tuning stage, because there is less data on mulberry leaf pests and diseases, while there are more data sets on other crop leaf pests and diseases, in order to balance the sample difference between the two data sets, samples with the same number of marked detection boxes as those in the new class data set are selected from the base class data set, and the new class data set and the selected base class data set are input into the model for training. As in the meta-learning stage, during the meta-fine-tuning stage training, the basic modules in the model will continuously learn and update parameters to detect new mulberry leaf pest and disease categories. In order to avoid overfitting, the number of training rounds in the meta-fine-tuning stage is less than that in the meta-learning stage. Step 5: After the RP-DCNet model generates a prediction target, the classification error and regression error are calculated in the meta-learning stage and the meta-fine-tuning stage respectively, and then the error results are reflected in the various parameters of the RP-DCNet model, and the network parameters in the RP-DCNet model are updated to generate the expected output; the validation set is used as input to validate the model and test the robustness of the model as the final detection model; Step 6: Finally, the RP-DCNet model with the optimal parameters is adjusted to detect the mulberry leaf pest and disease images to obtain the pest and disease detection results.

2. A mulberry leaf pest detection method based on small sample learning according to claim 1, characterized in that: The preprocessing described in step 2 includes mosaic data augmentation, random flipping, random cropping, and scaling.

3. A mulberry leaf pest detection method based on small sample learning according to claim 2, characterized in that: During preprocessing in step 2, Mosaic data enhancement is used, and random flipping is performed with a set probability. One of several scales is randomly selected to scale the data in the training set, and a part of the image is randomly cropped as a new image.

Citation Information

Patent Citations

  • Crop disease and pest identification method based on deep learning

    CN113609941A

  • Image processing system

    US20210142097A1

  • Selective sensing: a data-driven nonuniform subsampling approach for computation-free on-sensor data dimensionality reduction

    US20210349945A1

  • Method for constructing pest detection model

    WO2021203505A1

  • KR20210153270A