A method for detecting the extent of the forest disease of the stemborer based on improved YOLO

Abstract

The present application relates to the technical field of computer vision, and particularly relates to a kind of based on improved YOLO to detect the method of forest disease degree of Ramothrips, by unmanned aerial vehicle obtains self-made Ramothrips image data set pre-processing and classification, then the training set and verification set after classification are input into YOLOv7 network model to train and verify, add CloFormer attention enhancement local feature module, increase classification contrast loss function Focal Loss and small target detection loss function NWD (NormalizedWasserstein Distance) Replace the loss function in present YOLOv7 network model, obtain improved network model, based on training set and verification set, model training is carried out to improved network model, and the final YOLO network model after optimization is obtained;The final YOLO network model is used to detect Ramothrips data set.Foraging, the present application can maintain high-precision detection under various illumination conditions, so that YOLOv7 model reaches higher accuracy in identifying Ramothrips bored trees, and effectively solves the problem of missed detection.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision, in particular to a method for detecting the degree of forest disease of the pine shoot beetle based on an improved YOLO. BACKGROUND

[0002] Pest problems are one of the main disasters of forests, which can affect the normal growth of trees, destroy the ecological system, and cause the death of some organisms, resulting in huge losses to the forest economy. Strengthening pest control work not only maintains the stability of the forest ecosystem, but also ensures the smooth development of ecological construction.

[0003] Through computer image recognition technology combined with the identification of forestry pests, the traditional monitoring of human resources can be overcome, and the disadvantages of not being timely are overcome, but the traditional computer image recognition technology faces many problems, such as the need for a large number of high-quality forest disease image datasets under different seasons and light conditions, the complexity of the image background affecting the accuracy of the detection effect, and the deep learning model usually requiring a large amount of computing resources for training and reasoning, which may become a bottleneck in resource-limited practical application scenarios. SUMMARY

[0004] The purpose of the present application is to provide a method for detecting the degree of forest disease of the pine shoot beetle based on an improved YOLO, which uses self-made pine shoot beetle data sets to determine the color degree change of insect damage by means of an improved YOLO model, and improves the detection accuracy of the degree of forest disease.

[0005] To achieve the above purpose, the present application provides a method for detecting the degree of forest disease of the pine shoot beetle based on an improved YOLO, comprising the following steps:

[0006] Step 1: 4684 self-made pine shoot beetle data sets are obtained using a drone, image data is obtained using Labelme for image classification labeling and cross-validation, labeled image data is obtained, and the pine shoot beetle data sets are randomly divided into a training set, a validation set and a test set according to a preset ratio of 8:1:1;

[0007] Step 2: the training set and the validation set are input into a YOLOv7 network model for training and verification, the training weight file of the YOLOV7 network model is obtained, and the training weight file is iteratively optimized based on the loss function through forward and backward propagation, and the trained YOLOv7 network model is obtained;

[0008] Step 3: add CloFormer attention enhanced local feature module, increase classification contrast loss function Focalloss and small target detection loss function NWD to replace the loss function in the present YOLOV7 network model, and obtain an improved network model;

[0009] Step 4: Model training is performed on the improved network model based on the training set and the validation set to obtain an optimized final YOLO network model; and the final YOLO network model is used to detect the data set of the shoot borer.

[0010] Optionally, the Cloformer attention enhanced local feature module comprises a Clo block, each Clo block comprising a local branch and a global branch;

[0011] The key vector K and the value vector V are down-sampled in the global branch to reduce the amount of calculation, and then the Q, K and V are subjected to standard attention operation, and the formula is as follows:

[0012] X global =Attntion(Q g, Pool(K g )Pool(V g ))

[0013] The local branch AttnConv first extracts a local representation using a depthwise convolution DWconv having shared weights, and then uses context-aware weights to enhance the local features.

[0014] Optionally, the classification contrast loss function Focal Loss optimizes the training by adjusting the weights of positive and negative samples and easy and difficult classification samples, and the expression is as follows:

[0015] FL(p t )=-α t (1-p t ) γ log(p t )

[0016] Wherein, P t is the prediction probability of the model for the target class; α t is a balance factor for adjusting the influence between positive and negative samples; and γ is a focal factor for adjusting the weight of easy and difficult samples.

[0017] Optionally, when the focal factor γ = 0, the classification contrast loss function focalloss is a standard cross-entropy loss, and when γ increases, the modulation coefficient also increases.

[0018] Optionally, the NWD regression classification loss function models the frame as a Gaussian distribution, and then uses the Wasserstein distance to measure the similarity between the two distributions, and the expression is as follows:

[0019]

[0020] 1)N a and Nb denotes the distance between two points, W2 2 a b is the Wasserstein distance

[0021] 2) is the square root of the Wasserstein distance, so that the result is the standard Wasserstein distance W2 2 a b

[0022] 3) C: This is a normalization constant or scaling factor, which is used to adjust the dimension of the Wasserstein distance, ensuring that the result is suitable for specific application scenarios. The choice of this constant may depend on the specific application, the scale of the distance, or numerical stability, etc.

[0023] 4) is an exponential function exp(x) used to map the Wasserstein distance to a range more suitable for distance measurement. Since the exponential function is monotonically decreasing, as the Wasserstein distance W2(N a b ) increases, the value of NWD(N a, b ) will gradually decrease. The NWD value produced by this formula is usually between 0 and 1, indicating the similarity between two distributions. The closer the value is to 1, the more similar the two are; the closer the value is to 0, the greater the difference between them.

[0024] The application provides a method for detecting the degree of forest disease caused by the Ramotha borer based on an improved YOLO. The method comprises the following steps: obtaining self-made Ramotha borer boring image dataset preprocessing and classification through a UAV; inputting the classified training set and verification set into a YOLOv7 network model for training and verification; adding a CloFormer attention enhancement local feature module, increasing a classification contrast loss function FocalLoss and a small target detection loss function NWD loss function to replace the loss function in the existing YOLOv7 network model; obtaining an improved network model; training the improved network model based on the training set and the verification set to obtain an optimized final YOLO network model; and detecting the Ramotha borer boring dataset by using the final YOLO network model. Experiments prove that the application can maintain high-precision detection under various lighting conditions, so that the YOLOv7 model achieves higher accuracy in identifying Ramotha borer boring trees, and effectively solves the problem of missed detection. BRIEF DESCRIPTION OF DRAWINGS

[0025] ​​​​​​​In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the accompanying drawings required by the embodiments or prior art description will be briefly introduced below. Obviously, the accompanying drawings in the following description only constitute some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor.

[0026] Figure 1 is a logic schematic block diagram of a method for detecting the extent of the forest disease of the pine shoot moth based on the improved YOLO of the present application.

[0027] Figure 2 is a detection diagram of the color extent damage of the pine shoot moth in the forest disease of the specific embodiment of the present application.

[0028] Figure 3 is a YOLOv7P-R curve diagram of the specific embodiment of the present application.

[0029] Figure 4 is a YOLOv7 detection result diagram in the specific embodiment of the present application. DETAILED DESCRIPTION

[0030] The embodiments of the present application will be described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals represent the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the accompanying drawings are exemplary and are intended to explain the present application, and cannot be understood as a limitation of the present application.

[0031] The present application provides a method for detecting the extent of the forest disease of the pine shoot moth based on the improved YOLO, comprising the following steps:

[0032] Step 1: 4684 self-made pine shoot moth data sets are obtained by using a drone, image data is obtained by using Labelme for image classification labeling and cross-validation, labeled image data is obtained, and the pine shoot moth data set is randomly divided into a training set, a validation set and a test set according to a preset ratio of 8:1:1;

[0033] Step 2: the training set and the validation set are input into the YOLOv7 network model for training and verification, the training weight file of the YOLOv7 network model is obtained, and the training weight file is iteratively optimized based on the loss function through forward and backward propagation, and the trained YOLOv7 network model is obtained;

[0034] Step 3: add CloFormer attention enhanced local feature module, increase classification contrast loss function focalloss and small target detection loss function NWD to replace the loss function in the existing YOLOV7 network model, and obtain the improved network model;

[0035] Step 4: Train the improved network model based on the training and validation sets to obtain the optimized final YOLO network model; use the final YOLO network model to detect the twig borer dataset.

[0036] like Figure 1 The diagram shows a flowchart of the method for detecting the severity of forest diseases caused by the leafminer moth based on the improved YOLO method described in this invention.

[0037] The following will further explain the improvements made in the execution steps:

[0038] 1. The CloFormer attention-enhanced local feature module of this invention introduces a local branch, AttnConv, which combines attention mechanisms and convolution operations to capture high-frequency local information. Compared to traditional convolution operations, AttnConv uses shared weights and context-aware weights, enabling better handling of relationships between different locations in an image. To incorporate local information into the Feed-Forward Network (FFN) process, this invention replaces the traditional FFN with ConvFFN. The main difference between ConvFFN and commonly used FFNs is that ConvFFN uses depthwise convolution (DWconv) after the GELU activation function, allowing it to aggregate local information. Due to the presence of DWconv, ConvFFN can directly perform downsampling internally without introducing a PatchMerge module. CloFormer uses two types of ConvFFN: one is ConvFFN used within the same stage, which directly utilizes skip connections; the other is ConvFFN connecting two stages. For skip connections in this type of ConvFFN, DWconv and fully connected layers are used for downsampling and dimensionality increase; the Clo block plays a key role in CloFormer. Each block consists of a local branch and a global branch. In the global branch, K and V are first downsampled, and then standard attention operations are performed on Q, K, and V to extract low-frequency global information.

[0039] The Clo block in CloFormer is a crucial component. Each Clo block consists of a local branch and a global branch.

[0040] 1) Global branch: The global branch uses the traditional attention mechanism, but downsamples K and V to reduce computation and thus capture low-frequency global information.

[0041] The formula is as follows: X global =Attntion(Q g, Pool(K g ) Pool(V g )).

[0042] 2) Local branch (AttnConv). In the local branch, a well-designed AttnConv, a simple and effective convolution operator, is introduced, which adopts the style of attention mechanism. The proposed AttnConv effectively fuses shared weights and context-aware weights to aggregate high-frequency local information.

[0043] Specifically, AttnConv first extracts local representation using a depthwise convolution (DWconv) with shared weights. Then, it uses context-aware weights to enhance local features. Unlike the methods of generating context-aware weights such as Non-Local, AttnConv generates context-aware weights using a gating mechanism, which introduces stronger nonlinearity than the commonly used attention mechanism. In addition, AttnConv applies convolution operators to Query and Key to aggregate local information, then computes the Hadamard product of Q and K, and performs a series of linear or nonlinear transformations on the result to generate context-aware weights ranging between [-1, 1]. It is worth noting that AttnConv inherits the translational invariance of convolution, as all its operations are based on convolution.

[0044] The formula is as follows:

[0045] Q, K, V = FC(X in )

[0046] V s = DWconv(V)

[0047] Q l = DWconv(Q)

[0048] K l = DWconv(K)

[0049]

[0050] Finally, the global features and local features are combined, and an MLP is used to obtain the final output.

[0051] The formula is as follows:

[0052]

[0053] 2. Classification contrast loss function Focal Loss

[0054] Focal Loss is a loss function used to solve the problem of class imbalance, which optimizes training by adjusting the weights of positive and negative samples and easy and difficult classification samples. The loss function controls the contribution of high confidence samples and the contribution of low confidence samples through alpha and gamma parameters in binary classification problems, so as to improve the performance of the model on minority class samples.

[0055] Formula: FL(p t )=-α t (1-p t ) γ log(p t )

[0056] p t is the predicted probability of the model for the target class;

[0057] α t is a balance factor for adjusting the influence between positive and negative samples;

[0058] γ focal factor, used to adjust the weight of easy and difficult samples.

[0059] 1) About modulation coefficient

[0060] When the probability score p t of a sample is very small, the modulation factor (1-p t ) is larger and close to 1, and the modulation coefficient also tends to 1, and the weight of the loss function is basically not reduced;

[0061] When the probability score p t of a sample is very large and close to 1, the modulation factor (1-p t ) is close to 0, and the modulation coefficient also tends to 0, resulting in the weight of the loss function being reduced to close to 0. Then the loss weight of the sample that is better (high probability score) is adjusted.

[0062] Principle: the weight of normal cross-entropy loss is 1, while Focal Loss controls the weight of loss through modulation coefficient. The weight of difficult samples (low score) tends to 1, while the weight of simple samples (high score) is scaled to be very small. The loss weight of difficult samples remains unchanged, and the loss weight of simple samples is reduced, which is equivalent to increasing the weight of difficult samples.

[0063] 2) The value of parameter γ

[0064] When γ = 0, Focal Loss is the traditional cross-entropy loss, and when γ increases, the modulation coefficient also increases. The focus parameter γ smoothly adjusts the proportion of easy samples to reduce the weight.

[0065] Increasing γ enhances the effect of the modulation factor; experiments show that γ = 2 is optimal. Intuitively, the modulation factor reduces the loss contribution of easily separable samples, broadening the range of low-loss sample reception.

[0066] When γ is constant, for example, equal to 2, the loss of the easy example (pt = 0.9) is 100+ times smaller than the standard cross-entropy loss. t When the loss is 0.968, it is more than 1000 times smaller, but for hard examples (pt < 0.5), the loss is at most 4 times smaller. This significantly increases the weight of hard examples, thereby increasing the importance of misclassified samples.

[0067] 3. NWD Small Object Detection Loss Function

[0068] The NWD (Normalized Gaussian Wasserstein) loss function for small object detection can be expressed as NWD Loss, which stands for Normalized Gaussian Wasserstein. The formula is as follows:

[0069]

[0070] Non-uniform bounding box (NWD) is a novel method for calculating the similarity between bounding boxes. It models the boxes as Gaussian distributions and then uses the Wasserstein distance to measure the similarity between these two distributions, replacing the IoU (Intersection over Union) method. The advantage of this distance method is that it can still measure similarity even if two boxes are completely different or have very little overlap. Furthermore, NWD is not sensitive to the scale of the target and is more stable for small targets. Compared to IoU, NWD has the following advantages:

[0071] 1. Scale invariance.

[0072] 2. The changes in position are gradual.

[0073] 3. It has the ability to measure the similarity of boxes that do not want to overlap.

[0074] The IoU (Intersection over Union, IoU) based indicators, such as IoU itself and its extensions, are very sensitive to the position deviation of small targets, and when used in anchor-based detectors, the detection performance is severely reduced. In order to solve this problem, the present application proposes a new small target detection evaluation method based on Wasserstein distance. Specifically, first, the BBox is modeled as a two-dimensional Gaussian distribution, and then a new metric called Normalized Wasserstein Distance (NWD) is proposed, which calculates the similarity between them through their corresponding Gaussian distributions. The proposed NWD metric can be easily embedded into the Assignment, non-maximum suppression and loss function of any anchor-based detector to replace the commonly used IoU metric.

[0075] Further, the present application also proposes specific embodiments for auxiliary description, and the following are the execution steps of the specific embodiments:

[0076] 1. Prepare the data set: 4684 self-made data sets of pine shoot borer data are obtained using a drone, and Labelme is used for image classification labeling as red, yellow and gray corresponding to the damage degree of trees.

[0077] 2. Try to ensure the accuracy and integrity of the labeling, in order to improve the labeling quality, adopt the way of multi-person cooperation to label, and cross-verify and correct the labeling results to obtain labeled image data.

[0078] 3. According to the preset proportion 8:1:1, the pine shoot borer data set is randomly divided into training set, verification set and test set

[0079] 4. Configure the deep learning environment: the notebook computer system is Windows10, download the code of YOLOv7, manually download zip or git clone remote repository, the code folder downloaded by the present application is YOLOv7x version code, there is a requirements.txt file in the code folder, which describes the required installation package. The pytorch version installed this time is 1.8.1, the torchvision version is 0.9.1, the python is 3.7.10, and the other dependent libraries can be installed according to the requirements.txt file.

[0080] 5. Add CloFormer attention enhanced local feature module, increase classification contrast loss function Focal loss and small target detection loss function NWD (Normalized Wasserstein Distance) to replace the loss function in the present YOLOv7 network model, and get the improved network model for model training.

[0081] 6. Finally, use the separately divided validation set and test set pictures for model final test, and get the final target model.

[0082] For details, please refer to Figures 2 to 4 .

[0083] Figure 2 The experimental detection chart of the color degree of damage of the forest disease caused by the tip moth is shown in the figure. The upper left corner of the figure is the case under insufficient lighting conditions, and the rest is the case under relatively sufficient lighting conditions. The corresponding color description is as follows: first yellow, then red, gray is the most serious, indicating easy to fall off, yellow is light (early stage), red is medium, and gray is late (basically dry).

[0084] Figure 3 YOLOv7 P-R curve diagram:

[0085] Specifically, precision: the relationship between accuracy and confidence:

[0086]

【Confidence: used to determine whether the object in the bounding box is a positive sample or a negative sample. Greater than the confidence threshold is determined as a positive sample, and less than the confidence threshold is determined as a negative sample, i.e. background.

[0087] Recall: recall rate. Obviously, the lower the confidence, the higher the recall rate, i.e. it is not easy to miss pictures with objects.

[0088] The relationship between precious and recall, the area surrounded by the PR curve is called AP, and the average of all class APs is mAP.

[0089] There is no doubt that the lower R is better, and the higher P is better, but this is contradictory, the former requires the confidence threshold to be reduced, and the latter requires the confidence threshold to be increased. They are a pair of contradictory performance measurement indicators.

[0090] Therefore, there is a PR_curve, precision and recall curve, P here is mAP (mean Average Precision), i.e. average precision. If P and R can both reach 1, i.e. the curve forms the largest square box, and the area is close to 1.

[0091] The balance point (BEP) is the value when P = R (accuracy = recall), i.e. the slope is 1, and the larger the F1 value, the better the performance of the learner can be considered.

[0092] Figure 4 YOLOv7 detection result diagram:

[0093] The corresponding symbols in the figure are as follows:

[0094] Box: YOLO v7 uses GIOU Loss as the loss of bounding box, Box is the average of GIoU loss function, the smaller the box is, the more accurate it is;

[0095] Objectness: It is estimated that the average of the target detection loss is smaller, the more accurate the target detection is;

[0096] Classification: It is estimated that the average of the classification loss is smaller, the more accurate the classification is Val: is the validation set part.

[0097] Precision: Accuracy or accuracy (the proportion of positive samples in all correct classifications); that is, "the proportion of positive samples that the detection model considers to be positive and is indeed positive" measures the probability that the positive class classified by a classifier is indeed positive. Two extreme cases are, if the precision is 100%, it means that all the positive classes classified by the classifier are indeed positive. If the precision is 0%, it means that none of the positive classes classified by the classifier is positive. Just precision cannot measure the goodness of the classifier, for example, 50 positive samples and 50 negative samples, my classifier classifies 49 positive samples and 50 negative samples as negative samples, and the remaining one positive sample as positive sample, so my precision is also 100%.

[0098] Recall: Recall rate, also known as recall rate (how many positive samples are found); that is, "the proportion of positive samples that the detection model considers to be positive and is indeed positive" measures the ability of a classification to find all positive classes. Two extreme cases, if the recall rate is 100%, it means that all positive classes are classified as positive classes by the classifier. If the recall rate is 0%, it means that none of the positive classes is classified as positive. For example, the machine needs to detect ten airplanes in the picture, but the machine only labels five of them, although the five are correctly labeled, the precision is 100%, but the recall is still only 50%. Because the machine missed half of them.

[0099] val Box: bounding box loss of the validation set

[0100] val Objectness: average of target detection loss of the validation set

[0101] val classification: average of classification loss of the validation set

[0102] mAP@0.5: average mAP of threshold greater than 0.5.

where, mAP refers to the average value of the area AP under the PR curve, and the number after @ represents the threshold value of judging IoU as positive and negative samples

[0103] mAP@0.5:0.95: indicates the average mAP at different IoU thresholds (from 0.5 to 0.95, step 0.05) (0.5, 0.55, 0.6, 0.65, 0.7, 0.75, 0.8, 0.85, 0.9, 0.95)

[0104] In summary, the present application has the following beneficial effects:

[0105] The improved YOLOv7 can identify the position of the tree damaged by the shoot borer in the picture. In the case of sufficient light, the recognition accuracy of the tree damaged by the shoot borer can reach more than 85% accuracy. In the case of insufficient light, the model recognition accuracy can also be more than 80% by using two improved ways.

[0106] The improved YOLOv7 model can accurately identify the position of the tree damaged by the shoot borer. In the case of sufficient light, the recognition accuracy of the model can reach more than 85%, which can effectively capture target features and cope with interference environment. After adding the cloformer module, the improved YOLOv7 model improves the detection accuracy to 80% in the case of insufficient light, which proves its advantage in complex background. In addition, through multi-scale feature aggregation, the performance of the model on multiple targets is further enhanced, and the robustness and accuracy under different light conditions are improved. Experiments prove that the YOLOv7 model can maintain high-precision detection under various light conditions, making the YOLOv7 model achieve higher accuracy in identifying the tree damaged by the shoot borer, effectively solving the problem of missed detection.

[0107] The above disclosure is only one preferred embodiment of the present application, of course, cannot be limited by the scope of the present application, and those skilled in the art can understand that the above-mentioned embodiment can be implemented by all or part of the process, and the equivalent changes made according to the claims of the present application still belong to the scope covered by the present application.

Claims

1. A method for detecting the severity of forest diseases caused by the leafminer moth based on an improved YOLO algorithm, characterized in that, Includes the following steps: Step 1: Use a drone to acquire 4684 images of the self-made twig borer dataset. Use Labelme to classify and label the images and perform cross-validation. The labeled image data is then randomly divided into training, validation and test sets according to a preset ratio of 8:1:

1. Step 2: Input the training set and validation set into the YOLOv7 network model for training and validation, obtain the training weight file of the YOLOv7 network model, and iteratively optimize the training weight file through forward and backward propagation based on the loss function to obtain the trained YOLOv7 network model. Step 3: Add the CloFormer attention-enhancing local feature module, and replace the loss functions in the current YOLOv7 network model with the classification contrast loss function Focal Loss and the small object detection loss function NWD to obtain the improved network model; The CloFormer attention-enhanced local feature module includes Clo blocks, each of which consists of a local branch and a global branch. In the global branch, the key vector K and value vector V are downsampled to reduce computation, and then standard attention operations are performed on Q, K, and V, as shown in the following formula: X global =Attntion(Q g, Pool(K g )Pool(V g )) The local branch AttnConv first uses depthwise convolution (DWconv) to extract local representations, where DWconv has shared weights, and then uses context-aware weights to enhance local features. Step 4: Train the improved network model based on the training and validation sets to obtain the optimized final YOLO network model; use the final YOLO network model to detect the twig borer dataset.

2. The method for detecting the severity of forest diseases caused by the leafminer moth based on the improved YOLO as described in claim 1, characterized in that, The classification contrastive loss function, Focal Loss, optimizes training by adjusting the weights of positive and negative samples and samples that are easy or difficult to classify. Its expression is: FL(p t )=-a t (1-p t ) γ log(p t ) Where, p t It is the model's predicted probability of the target class; α t γ is the balancing factor, used to adjust the influence between positive and negative samples; γ is the focus factor, used to adjust the weights of easy and difficult samples.

3. The method for detecting the severity of forest diseases caused by the leafminer moth based on the improved YOLO as described in claim 2, characterized in that, When the focus factor γ = 0, the classification contrast loss function Focal Loss is the standard cross-entropy loss. As γ increases, the modulation coefficient also increases.

4. The method for detecting the severity of forest diseases caused by the leafminer moth based on the improved YOLO as described in claim 3, characterized in that, The NWD small object detection loss function models the bounding boxes as Gaussian distributions and then uses the Wasserstein distance to measure the similarity between these two distributions. The expression is: Where, N a and N b This represents the distance between two points. is the Wasserstein distance; C is a normalization constant or scaling factor used to adjust the dimensions of the Wasserstein distance; the exponential function exp(x) is used to map the Wasserstein distance to a range more suitable for distance metrics.

Images