Mung bean leaf spot disease detection method based on chlorophyll fluorescence imaging and DB-YOLOv7

Abstract

This invention discloses a method for detecting mung bean leaf spot disease based on chlorophyll fluorescence imaging and DB-YOLOv7. The method involves pre-acquiring mung bean leaf spot disease images, creating a mung bean leaf spot disease dataset through manual annotation, and dividing it into training, validation, and test sets. A mung bean leaf spot disease detection model based on DB-YOLOv7 is constructed. The SPPCSPC module is reconstructed on the YOLOv7 model to more effectively capture relevant information and enhance the extraction ability of small lesions. Secondly, a GFPN module is introduced and improved to shorten the distance between high-level and low-level images, solving the feature loss problem caused by excessively long paths. Additionally, a CA attention mechanism module is introduced to improve the model's robustness. The DB-YOLOv7 model is trained and evaluated based on the training and validation sets. The test set images are input into the trained DB-YOLOv7 model for recognition, achieving mung bean leaf spot disease detection. This invention can effectively improve the accuracy of mung bean leaf spot disease detection and has broad application prospects.

Description

Technical Field

[0001] The invention belongs to the field of image processing and plant leaf disease detection, and specifically relates to a mung bean leaf spot disease detection method based on chlorophyll fluorescence imaging and DB-YOLOv7. Background Art

[0002] Mung bean is one of the main edible beans and is widely cultivated around the world. Mung bean has strong adaptability and stress resistance, is drought-resistant and barren-resistant, has a short growth period, a long sowing period, and has the ability to fix nitrogen. It is a suitable crop for intercropping with cotton, potatoes, and cereals. It has irreplaceable advantages over other crops in the adjustment of agricultural planting structure and the development of high-yield, high-efficiency, and high-quality agriculture. In terms of sown area, the sown area of ​​mung beans accounts for 78% of the national sown area. China is one of the exporters of mung beans, but due to the increasing domestic demand for mung beans in recent years, the changes in the supply and demand relationship in the mung bean market, and the low price of imported mung beans, the import volume of mung beans has increased year by year. At present, my country has changed from a major exporter of mung beans to a major importer of mung beans.

[0003] In terms of ensuring crop yields, it is extremely important to quickly detect crop diseases and pests during crop growth. During crop growth, many crop leaf diseases will directly affect crop yields. Among them, leaf spot is one of the main diseases of mung beans, which mainly damages the leaves, causing them to wither and fall off. In the later stages of the disease, it will cause premature aging of the crop, and it may occur at any stage of mung bean growth.

[0004] Based on this, in the traditional breeding process, people will manually identify leaf diseases and grade leaf diseases, and select seeds with strong disease resistance as the next generation of breeding objects based on the type of leaf disease and the degree of grading. However, relying solely on manual methods to identify diseases requires a lot of manpower and time. At the same time, due to the seasonality of crop growth, the data used to manually classify and grade crop diseases is usually small, and it is impossible to accurately and comprehensively reflect the phenotypic parameters of a batch of crops. Therefore, there is an urgent need for a method of automated detection of crop diseases to replace manual tasks. Therefore, based on the existing target detection algorithm, some improvements have been made to the characteristics of mung bean leaf spot disease, and a mung bean leaf spot disease detection method based on chlorophyll fluorescence imaging and DB-YOLOv7 is proposed. Summary of the invention

[0005] Purpose of the invention: The invention proposes a mung bean leaf spot disease detection method based on chlorophyll fluorescence imaging and DB-YOLOv7, which solves the problem that small spots of mung bean leaf spot disease are easily ignored after multi-level convolution operations due to the different sizes of mung bean leaf spot disease spots, thereby improving the accuracy of mung bean leaf spot disease detection.

[0006] Technical solution: The present invention proposes a mung bean leaf spot disease detection method based on chlorophyll fluorescence imaging and DB-YOLOv7, which specifically includes the following steps:

[0007] (1) Chlorophyll fluorescence images of mung bean leaves were obtained in advance, and a mung bean leaf spot disease dataset was created through manual annotation, which was divided into a training set, a validation set, and a test set;

[0008] (2) Construct a mung bean leaf spot detection model based on DB-YOLOv7; including the dynamic spatial convolutional pooling pyramid module DSPP for feature extraction and the network module BcFPN for enhanced feature extraction, and introduce the CA attention mechanism module;

[0009] (3) Train and evaluate the mung bean leaf spot disease detection model based on DB-YOLOv7 based on the training set and validation set;

[0010] (4) The test set images are input into the trained mung bean leaf spot disease detection model based on DB-YOLOv7 for recognition to realize mung bean leaf spot disease detection.

[0011] Furthermore, the implementation process of step (1) is as follows:

[0012] FluorCam chlorophyll fluorescence imaging technology was used to collect chlorophyll fluorescence imaging data of mung bean leaves after disease. Resistance was divided according to the disease index. Labelimg software was used to annotate the disease resistance level of leaves. The mung bean leaf spot disease dataset was divided into training set, validation set and test set in a ratio of 8:1:1.

[0013] Furthermore, the dynamic spatial convolutional pooling pyramid module DSPP in step (2) includes 7 ODC convolutions and 6 ordinary convolutions with a convolution kernel size of 3×3, wherein the ODC convolution is optimized based on the full-dimensional dynamic convolution ODConv; specifically, the ReLU activation function is replaced by the GeLU activation function to solve the gradient disappearance problem; the ODC convolution includes a global average pooling layer GAP, a fully connected layer FC, a GeLU activation function, a sigmoid activation function, a softmax function, and a convolution kernel W. i The spatial position weight α si , input channel weight α ci , output channel weight α fi , kernel weight α wi , a multi-dimensional attention mechanism is introduced through a parallel strategy to learn more flexible attention on the four dimensions of the convolution kernel space.

[0014] Furthermore, the working process of the dynamic spatial convolutional pooling pyramid module DSPP in step (2) is as follows:

[0015] Input feature map F in , whose length, width and number of channels are H, W and C respectively; F in First, three ODC convolutions and one ODC convolution are performed to obtain the feature maps F1 and F2 of H×W×C / 2. Then, F1 undergoes 1, 2, and 3 ordinary convolutions with a kernel size of 3×3 and an unprocessed connection to obtain four feature maps of H×W×C / 2, namely F 11 、F 12 、F 13 、F 14 , the calculation process is as follows:

[0016] F in =H×W×C (1)

[0017] F1=ODC(ODC(ODC(F in ))) (2)

[0018] F2=ODC(F in ) (3)

[0019] F 11 =Conv(F1) (4)

[0020] F 12 =Conv(Conv(F1)) (5)

[0021] F 13 =Conv(Conv(Conv(F1))) (6)

[0022] F 14 =F1 (7)

[0023] Then F 11 、F 12 、F 13 、F 14 Perform channel concatenation to obtain the feature map F3 of H×W×2C, then perform two ODC convolutions on the feature map F3 to obtain the feature map of H×W×C / 2, then perform channel concatenation on F2 and F3 to obtain the feature map F4 of H×W×C, and finally perform one ODC convolution on the feature map F4 to obtain the feature map Fo of H×W×C / 2. ut , the calculation process is as follows:

[0024] F3=cat(F 11 ; F 12 ; F 13 ; F 14 ) (8)

[0025] F4=cat(F2;F3) (9)

[0026] Fo ut =ODC(F4) (10).

[0027] Furthermore, the network module BcFPN for enhanced feature extraction in step (2) adopts a generalized feature pyramid network structure, and a cross-scale feature fusion method is used to fuse feature information; the cross-scale feature fusion method includes three modes: nodes of each scale simultaneously perform feature fusion on the three nodes above, obliquely above, and obliquely below; nodes of each scale simultaneously perform feature fusion on the obliquely below nodes across one layer; features of the previous layer and the current layer are fused and a skip-layer connection is established.

[0028] Furthermore, the CA attention mechanism module in step (2) is introduced after the backbone feature extraction network, which is used for embedding position information and generating coordinate information feature maps, thereby enhancing important channel and spatial feature information in the features.

[0029] Furthermore, the implementation process of step (3) is as follows:

[0030] The mung bean leaf spot detection model based on DB-YOLOv7 was trained and evaluated using the divided training set and validation set. The best weights obtained from the training were retained as the final weights of the network to obtain a network model that can accurately detect the disease resistance level of mung bean leaves.

[0031] Beneficial effects: Compared with the prior art, the present invention has the following beneficial effects:

[0032] 1. Use chlorophyll fluorescence imaging technology to collect images. Chlorophyll fluorescence imaging can be used to measure the chlorophyll fluorescence characteristics that appear on leaves when plants are subjected to biological or abiotic stresses, which can better monitor the current growth status of plants;

[0033] 2. The present invention proposes a DB-YOLOv7 detection algorithm. In view of the fact that lesions vary in size and small lesions are easily ignored, the SPPCSPC module of the original network is reconstructed, which increases the receptive field and makes it more conducive to feature extraction and identification of small lesions;

[0034] 3. Aiming at the problem that some low-level features are lost due to the long path between high-level features and low-level features, the feature pyramid network structure of the YOLOv7 network is improved to enhance the ability to aggregate multi-scale feature information, so that it can more effectively fuse feature information of different resolutions, improve the multi-scale feature fusion efficiency of the model, and further improve the accuracy of target detection;

[0035] 4. To address the problem that small disease spots are easily ignored, an attention mechanism is introduced after Backbone, which enables the model to more accurately locate and identify the semantic information of small targets, thereby improving the robustness of the model;

[0036] 5. Compared with the existing methods, the present invention has low cost, effectively improves the detection accuracy, has good generalization performance and good robustness, and has a very broad application prospect. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 is a flow chart of the present invention;

[0038] Figure 2 This is an example of an image collected using the FluorCam chlorophyll fluorescence imaging system;

[0039] Figure 3 It is a schematic diagram of the DB-YOLOv7 network structure proposed in the present invention;

[0040] Figure 4 It is a schematic diagram of the DSPP module structure proposed by the present invention;

[0041] Figure 5 It is a schematic diagram of the improved ODC convolution structure of the present invention;

[0042] Figure 6 It is a schematic diagram of the structure of the BcFPN module proposed in the present invention;

[0043] Figure 7 It is a schematic diagram of the CA attention mechanism module structure adopted in the present invention. DETAILED DESCRIPTION

[0044] The present invention will be further described in detail below in conjunction with the accompanying drawings.

[0045] like Figure 1 As shown, the present invention proposes a mung bean leaf spot disease detection method based on chlorophyll fluorescence imaging and DB-YOLOv7, comprising the following steps:

[0046] Step 1: Obtain chlorophyll fluorescence images of mung bean leaves in advance and create a mung bean leaf spot disease dataset through manual annotation.

[0047] The collection of mung bean leaf images was carried out using the FluorCam chlorophyll fluorescence imaging system to collect chlorophyll fluorescence imaging data of mung bean leaves seven days after the onset of the disease. Figure 2 shown.

[0048] According to the disease index, there are 5 types of resistance. Among them, the disease index between 0-2.0 is divided into high resistance (HR), between 2.1-15.0 is divided into resistance (R), between 15.1-40.0 is divided into moderate resistance (MR), between 40.1-60.0 is divided into moderate sensitivity (MS), and between 60.1-80.0 is divided into sensitivity (S).

[0049] The disease resistance level of leaves was labeled using labelimg software. The mung bean leaf spot disease dataset was divided into training set, validation set, and test set in a ratio of 8:1:1.

[0050] Step 2: Build a mung bean leaf spot disease detection model based on DB-YOLOv7; including the dynamic spatial convolutional pooling pyramid module DSPP (Dynamic Spatial Pyramid Pooling) for feature extraction and the network module BcFPN (Broadly connected Feature Pyramid Networks) for enhanced feature extraction, and introduce the CA attention mechanism module. The network structure is as follows: Figure 3 As shown, the specific steps are as follows:

[0051] The obtained feature map P3 is connected to the dynamic spatial convolutional pooling pyramid module DSPP for processing to obtain P 3 ', then the feature maps of three sizes P1, P2, and P3' obtained by the backbone feature extraction network are first connected to the CA attention mechanism module to extract key features, and then respectively connected to the enhanced feature extraction network module BcFPN to further extract features, and finally respectively input into the detection head for detection to obtain the final prediction result.

[0052] Dynamic spatial convolutional pooling pyramid module DSPP, the specific structure is as follows Figure 4 As shown in the figure, it includes 7 Omni-Dimensional Dynamic Convolution (ODC) convolutions and 6 ordinary convolutions with a kernel size of 3×3. The ODC convolution is optimized based on the full-dimensional dynamic convolution ODConv. Specifically, the ReLU activation function is replaced by the GeLU activation function. The specific structure is as follows: Figure 5 As shown in Figure 1, the gradient disappearance problem can be effectively solved. The ODC includes a global average pooling layer GAP, a fully connected layer FC, a GeLU activation function, a sigmoid activation function, a softmax function, and a convolution kernel W. i The spatial position weight α si , input channel weight α ci , output channel weight α fi , kernel weight α wi , a multi-dimensional attention mechanism is introduced through a parallel strategy to learn more flexible attention on the four dimensions of the convolution kernel space. In this DSPP module, the input feature map F in , whose length, width and number of channels are H, W and C respectively; F inFirst, three ODC convolutions and one ODC convolution are performed to obtain the feature maps F1 and F2 of H×W×C / 2, and then F1 is subjected to 1, 2, and 3 ordinary convolutions with a kernel size of 3×3 and an unprocessed connection to obtain four feature maps F of H×W×C / 2. 11 、F 12 、F 13 、F 14 :

[0053] F in =H×W×C (1)

[0054] F1=ODC(ODC(ODC(F in ))) (2)

[0055] F2=ODC(F in ) (3)

[0056] F 11 =Conv(F1) (4)

[0057] F 12 =Conv(Conv(F1)) (5)

[0058] F 13 =Conv(Conv(Conv(F1))) (6)

[0059] F 14 =F1 (7)

[0060] Then F 11 、F 12 、F 13 、F 14 Perform channel concatenation operation (cat) to obtain the feature map F3 of H×W×2C, then pass the feature map F3 through two ODC convolutions to obtain the feature map of H×W×C / 2, then perform channel concatenation operation on F2 and F3 to obtain the feature map F4 of H×W×C, and finally pass the feature map F4 through an ODC convolution to obtain the feature map Fo of H×W×C / 2 ut The calculation process is shown in formulas (8)-(10).

[0061] F3=cat(F 11 ; F 12 ; F 13 ; F 14 ) (8)

[0062] F4=cat(F2;F3) (9)

[0063] Fo ut =ODC(F4) (10)

[0064] The network module BcFPN for enhanced feature extraction has the following structure: Figure 6 As shown in the figure, a generalized feature pyramid network structure is used to fuse feature information using a cross-scale feature fusion method, wherein the cross-scale feature fusion method includes three modes: nodes of each scale simultaneously fuse features of the three nodes above, obliquely above, and obliquely below, nodes of each scale simultaneously fuse features of the obliquely below nodes across one layer, and fuse features of the previous layer and the current layer and establish a skip-layer connection. While achieving feature fusion, intra-block connection is achieved, achieving more efficient information transmission, and also solving the problem of partial loss of low-level features due to the long path between high-level features and low-level features, so that feature information of different resolutions can be more effectively fused, and the multi-scale feature fusion efficiency of the model is improved.

[0065] The CA attention mechanism module has a specific structure as follows: Figure 7 As shown in the figure, in the process of feature extraction by the backbone network, due to multiple downsampling, a lot of detail information is easily lost, and small target features are easily ignored. Therefore, a CA attention mechanism module is introduced after the backbone feature extraction network. This module first averages the input feature map in the horizontal and vertical directions, and then encodes the spatial information through dimension shifting, dimension splicing, dimension reduction and other operations. Finally, the spatial information and channel features are weighted and fused to consider the channel information and position information at the same time, which is used for position information embedding and coordinate information feature map generation, and enhances the important channel and spatial feature information in the features.

[0066] Step 3: Based on the training set and the validation set, the mung bean leaf spot disease detection model based on DB-YOLOv7 is trained and evaluated as follows:

[0067] The DB-YOLOv7 network model was trained and evaluated using the divided training set and validation set, and the best weights obtained from the training were retained as the final weights of the network. Then, a network model that can accurately detect the disease resistance level of mung bean leaves was obtained.

[0068] Step 4: Input the test set images into the trained mung bean leaf spot disease detection model based on DB-YOLOv7 for recognition to realize mung bean leaf spot disease detection, and send the images divided into the test set into the trained DB-YOLOv7 network model for detection and classification to obtain the mung bean leaf spot disease resistance level detection results.

[0069] Table 1 Comparison of experimental results of the method of the present invention and two existing methods

[0070] Model Name YOLOv5 YOLOv7 DB-YOLOv7 mAP 0.737 0.786 0.811

[0071] Among them, the labeled dataset image is used as input, and the input image scale is 640*640*3. The initial learning rate is initially set to 0.01, the batch size of the algorithm training is 9, momentum is used for optimization, and the learning rate decay weight is set to 0.0005, and the training batch is 300 epochs. The experiment was carried out on the deep learning framework of Python3.6 and Pytorch1.10, the operating system used was Centos8.0, and the GPU was an Nvidia TeslaV100 with a video memory of 16GB. After experimental comparison and analysis, compared with other existing methods, the DB-YOLOv7 model proposed in the present invention has a higher detection accuracy for mung bean leaf spot, and its evaluation index (mean Average Precision, mAP) is shown in Table 1. It can be seen that the method of the present invention has higher detection accuracy.

Claims

1. A method for detecting mung bean leaf spot disease based on chlorophyll fluorescence imaging and DB-YOLOv7, characterized in that: The following steps are involved: (1) Chlorophyll fluorescence images of mung bean leaves were obtained in advance, and a mung bean leaf spot disease dataset was created through manual annotation, which was divided into a training set, a validation set, and a test set; (2) Construct a mung bean leaf spot detection model based on DB-YOLOv7; including the dynamic spatial convolutional pooling pyramid module DSPP for feature extraction and the network module BcFPN for enhanced feature extraction, and introduce the CA attention mechanism module; (3) Train and evaluate the mung bean leaf spot disease detection model based on DB-YOLOv7 based on the training set and validation set; (4) Input the test set images into the trained mung bean leaf spot disease detection model based on DB-YOLOv7 for recognition to achieve mung bean leaf spot disease detection; The dynamic spatial convolutional pooling pyramid module DSPP in step (2) includes 7 ODC convolutions and 6 ordinary convolutions with a convolution kernel size of 3×3, wherein the ODC convolution is optimized based on the full-dimensional dynamic convolution ODConv; specifically, the ReLU activation function is replaced by the GeLU activation function to solve the gradient disappearance problem; the ODC convolution includes a global average pooling layer GAP, a fully connected layer FC, a GeLU activation function, a sigmoid activation function, a softmax function, and a convolution kernel W. i The spatial position weight α si , input channel weight α ci , output channel weight α fi , kernel weight α wi , a multi-dimensional attention mechanism is introduced through a parallel strategy to learn more flexible attention on the four dimensions of the convolution kernel space; The working process of the dynamic spatial convolutional pooling pyramid module DSPP in step (2) is as follows: Input feature map F in , whose length, width and number of channels are H, W and C respectively; F in First, three ODC convolutions and one ODC convolution are performed to obtain the feature maps F1 and F2 of H×W×C / 2, and then F1 is subjected to 1, 2, and 3 ordinary convolutions with a kernel size of 3×3 and an unprocessed connection to obtain four feature maps of H×W×C / 2, namely F 11 、F 12 、F 13 、F 14 , the calculation process is as follows: F in =H×W×C (1) F1=ODC(ODC(ODC(F in ))) (2) <h2 style=";text-align:left;direction:ltr">F2=ODC(F<h2 style=";text-align:left;direction:ltr"> in <h2 style=";text-align:left;direction:ltr"> ) (3) F 11 =Conv(F1) (4) F 12 =Conv(Conv(F1)) (5) F 13 =Conv(Conv(Conv(F1))) (6) F 14 =F1 (7) Then F 11 、F 12 、F 13 、F 14 Perform channel concatenation to obtain the feature map F3 of H×W×2C, then perform two ODC convolutions on the feature map F3 to obtain the feature map of H×W×C / 2, then perform channel concatenation on F2 and F3 to obtain the feature map F4 of H×W×C, and finally perform one ODC convolution on the feature map F4 to obtain the feature map F of H×W×C / 2 out , the calculation process is as follows: F3=cat(F 11 ;F 12 ;F 13 ;F 14 ) (8) F4=cat(F2;F3) (9) F out =ODC(F4) (10); The network module BcFPN for enhanced feature extraction in step (2) adopts a generalized feature pyramid network structure, and a cross-scale feature fusion method is used to fuse feature information; the cross-scale feature fusion method includes three modes: the nodes of each scale simultaneously fuse the features of the three nodes above, obliquely above, and obliquely below; the nodes of each scale simultaneously fuse the features of the obliquely below nodes across one layer; the features of the previous layer and the current layer are fused and a skip-layer connection is established; The CA attention mechanism module described in step (2) is introduced after the backbone feature extraction network, and is used to embed position information and generate coordinate information feature maps, thereby enhancing important channel and spatial feature information in the features.

2. The method for detecting mung bean leaf spot based on chlorophyll fluorescence imaging and DB-YOLOv7 according to claim 1, characterized in that: The implementation process of step (1) is as follows: FluorCam chlorophyll fluorescence imaging technology was used to collect chlorophyll fluorescence imaging data of mung bean leaves after disease. Resistance was divided according to the disease index. Labelimg software was used to annotate the disease resistance level of leaves. The mung bean leaf spot disease dataset was divided into training set, validation set and test set in a ratio of 8:1:

1.

3. The method for detecting mung bean leaf spot based on chlorophyll fluorescence imaging and DB-YOLOv7 according to claim 1, characterized in that: The implementation process of step (3) is as follows: The mung bean leaf spot detection model based on DB-YOLOv7 was trained and evaluated using the divided training set and validation set. The best weights obtained from the training were retained as the final weights of the network to obtain a network model that can accurately detect the disease resistance level of mung bean leaves.

Images