Hedgerow robust detection method based on improved YOLO model

Abstract

The application discloses a kind of based on the hedge robust detection method of improved YOLO model, comprising: step 1, the original picture of hedge is obtained, and hedge dataset is collected;Step 2, part of the collected picture is screened out, and is pasted on patch, and the hedge in all pictures is labeled, and training set and test set are divided;Step 3, the YOLOv8 model is improved, and CFR module and threshold judgment mechanism are added;Step 4, the model is trained, and the robust YOLOv8 model is obtained;Step 5, the effect of training model is tested on validation set.The method of the application, CFR module and threshold judgment mechanism are innovatively added on the original YOLOv8 model.The model can use clean image to reconstruct the consistency features of input image, promote the model to learn robust features, improve the recognition accuracy of the model to clean image and adversarial image.Add threshold judgment mechanism, the mechanism can make CFR module more stable when selecting random vector.Improve the recognition accuracy and robustness of the model to hedge.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision processing, in particular to a hedge robust detection method based on an improved YOLO model. BACKGROUND

[0002] Hedges are widely used in landscape design, which can weaken noise, beautify the environment, enclose the site, divide space, act as a barrier or guide the line of sight to the focal point of the scene, and can also serve as the background of statues, fountains, small garden facilities, etc.

[0003] The widespread use of hedges has brought new development opportunities to the landscape maintenance industry, but it has also brought problems such as rapid increase in labor costs, tedious and inefficient pruning work, and low level of intelligence, etc., which urgently need automated hedge pruning robots to solve this problem. Pruning hedges using a hedge pruning robot usually requires collecting pictures of the hedges to be pruned, and identifying and locating the hedges in the pictures.

[0004] However, due to the complexity of the working environment, the detection performance of the model can be significantly affected by adversarial attacks, which can cause the pruning robot to malfunction. Therefore, there is an urgent need for a robust detection method to improve the ability of the hedge pruning robot to resist interference.

[0005] The information disclosed in this BACKGROUND section is only for the purpose of increasing the understanding of the background of the present application and should not be taken as an acknowledgment or any form of suggestion that this information forms prior art that is publicly known. SUMMARY

[0006] The purpose of the present application is to provide a hedge robust detection method based on an improved YOLO model, thereby overcoming the poor ability of the hedge pruning robot to resist interference.

[0007] To achieve the above-mentioned purpose, the present application provides a hedge robust detection method based on an improved YOLO model, comprising:

[0008] Step 1, obtaining the original picture of the hedge, collecting the hedge dataset;

[0009] Step 2, selecting part of the collected pictures, pasting patches, and labeling the hedges in all pictures, dividing the training set and the test set;

[0010] Step 3, improving the YOLOv8 model, adding a CFR (Consistency Feature Reconstruction) module and a threshold judgment mechanism;

[0011] Step 4, training the model to obtain a robust YOLOv8 model;

[0012] Step 5: Test the performance of the trained model on the validation set.

[0013] Preferably, in the above technical solution, step 1, obtaining the original image of the hedge, involves taking photos of different types of hedges at different times of the same day, at different distances and angles, to obtain the original image of the hedge.

[0014] Preferably, in the above technical solution, step 2, applying the patch, includes:

[0015] (1) Before applying the patch, the patch is preprocessed, including inverting, scaling, cropping and rotating, to obtain a patch set;

[0016] (2) Select several hedge images with obvious differences and randomly paste different patches on different positions on the hedge images.

[0017] Preferably, in the above technical solution, the annotation in step 2 includes: using LabelImg to annotate the hedges in the image, requiring that the predicted bounding box of each hedge be larger than the outer contour of the hedge, allowing overlap between the predicted bounding boxes, and finally saving the labels in YOLO format and txt format, and the image in jpg format.

[0018] Preferably, in the above technical solution, step 2, dividing the training set and the test set, includes: exporting the labeled images and labels, dividing the training set and the test set according to the ratio, wherein all images with patches are included in the test set.

[0019] Preferably, in the above technical solution, the ratio of the training set to the test set is 7.5:2.5.

[0020] Preferably, in the above technical solution, step 3, improving the YOLOv8 model, includes:

[0021] (1) The original YOLOv8 model includes a backbone network, a Neck structure, and a Head prediction part. A consistency reconstruction module is added before the Upsample operation and the last concat operation in the Neck structure; the reconstruction loss function is:

[0022] L re =||G(z)-x|| 2 ,z~N(μ,∑)

[0023] Among them, ||.|| 2 Let G(z) denote the L2 norm, G(z) denote the reconstructed image, and x denote the clean input image.

[0024] The purpose of the CFR module is to reconstruct clean images using CFR image reconstruction constraints, thereby promoting the model's learning of robust features.

[0025] (2) Add a threshold judgment, for L re Set a threshold. If the result is less than the threshold, output the result. If the result is greater than the threshold, continue iterating.

[0026] Preferably, in the above technical solution, step 4, training the model, is to train the model using the TRADES adversarial training method, and during the training process, WIoU v1 is selected as the loss function for target box regression to obtain a robust YOLOv8 model.

[0027] Preferably, in the above technical solution, step 4, training the model, includes:

[0028] (1) Randomly initialize the network f θ Alternatively, use a pre-trained network;

[0029] (2) Read the dataset B = {x1,...,x} in small batches. m}, where each batch contains m samples;

[0030] (3) Based on the sample x in each mini-batch i Initialize the corresponding adversarial sample x i The adversarial example generation method is: x i '←x i +0.001N(0,1), where N(0,1) is a standard normal distribution;

[0031] (4) Iterate K times for each adversarial example. The specific iteration method is as follows:

[0032]

[0033] Where ∏ represents the projection operator;

[0034] (5) Update the network parameter θ, specifically as follows:

[0035]

[0036] Specifically, the original GIoU target box regression loss function of the YOLOv8 model is replaced with the WIoU v1 loss function. The purpose of introducing the WIoU v1 loss function is to reduce the penalty for geometric factors when the anchor box and target box overlap well, thereby reducing training intervention and allowing the model to achieve better generalization ability. The WIoU v1 loss function is defined as follows:

[0037] L WIoUv1 =R WIoU LIoU

[0038]

[0039]

[0040] S u =wh+w gt h gt -W i H i

[0041] Where (x, y, h, w) represents the center coordinates, height, and width of the anchor box; (x gt ,y gt ,h gt ,w gt (H) represents the center coordinates, height, and width of the target box; i W i (H) represents the height and width of the overlapping area between the anchor box and the target box. g W g ) represents the height and frame of the minimum bounding box.

[0042] Preferably, in the above technical solution, step 5, testing the effect of the trained model on the validation set, includes inputting the test set into the trained model, and the model outputting an image with target prediction boxes and confidence scores.

[0043] Compared with the prior art, the present invention has the following beneficial effects:

[0044] (1) This invention presents a robust hedge detection method based on an improved YOLO model. It innovatively adds a CFR module and a threshold judgment mechanism to the original YOLOv8 model. After the improvement, the model can reconstruct the consistency features of the input image using clean images, promoting the model's learning of robust features and improving the model's recognition accuracy for both clean and adversarial images. The added threshold judgment mechanism makes the CFR module more stable when selecting random vectors. All of these selections aim to improve the model's recognition accuracy and robustness for hedges.

[0045] (2) This invention is based on an improved YOLO model for robust hedge detection. The YOLOv8 model is trained using the TRADES adversarial training method, which balances adversarial robustness and accuracy. The WIoU v1 loss function is used as the YOLOv8 target box regression loss function. The purpose is to reduce the penalty for geometric factors when the anchor box and target box overlap well, thereby reducing training intervention and allowing the model to achieve better generalization ability. Attached Figure Description

[0046] Figure 1 This is a flowchart of the robust hedge detection method based on the improved YOLO model according to the present invention.

[0047] Figure 2 This is the image after the patch has been applied;

[0048] Figure 3 This is a diagram of the improved YOLOv8 model framework;

[0049] Figure 4 This is a CFR module structure diagram;

[0050] Figure 5 This is a schematic diagram of the threshold judgment mechanism;

[0051] Figure 6 It is a clean image detection result;

[0052] Figure 7 This is the detection result with the patch image attached. Detailed Implementation

[0053] The specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings, but it should be understood that the scope of protection of the present invention is not limited to the specific embodiments.

[0054] Unless otherwise expressly stated, throughout the specification and claims, the term "comprising" or its variations such as "including" or "comprises" shall be understood to include the stated elements or components without excluding other elements or other components.

[0055] like Figures 1 to 7 As shown, a robust hedge detection method based on an improved YOLO model according to a specific embodiment of the present invention includes the following steps:

[0056] Step 1: Collect the hedge dataset

[0057] On the same day, during the morning and afternoon periods, different types of hedges (circular, cylindrical, etc.) were photographed at different distances (0.5m, 1m, 2m, 5m) and at different angles (side, front, top) to obtain raw RGB images.

[0058] Step 2: Select a portion of the collected images and apply patches.

[0059] (1) Before applying the patch, the patch is preprocessed, including inverting, scaling, cropping and rotating, to obtain a patch set;

[0060] (2) Select several hedge images with significant differences, and randomly apply different patches to different positions on the hedge images. The result after applying the patches is shown below. Figure 2 As shown.

[0061] Step 3: Label the images and divide them into training and testing sets.

[0062] (1) Labeling: LabelImg is used to label the hedges in the image. Each hedge prediction box must be larger than the outer contour of the hedge. Overlapping of prediction boxes is allowed. Finally, the labels are saved in YOLO format and txt format, and the image is saved in jpg format.

[0063] (2) Export the images and labels generated in step (1) and divide them into training set and test set according to the ratio of 7.5:2.5 to prepare for subsequent training and testing of the model. All images with patches are included in the test set.

[0064] Step 4: Improve the YOLOv8 model

[0065] Based on the YOLOv8 model, improvements were made. YOLOv8 is currently the most advanced model in the YOLO series, offering higher recognition accuracy compared to previous YOLO models. An innovative CFR (Constraint for Image Reconstruction) module was added to the YOLOv8 model. Through CFR image reconstruction constraints, clean images are reconstructed, promoting the model's learning of robust features. Furthermore, an innovative threshold judgment mechanism was designed, making the CFR module more stable when selecting random vectors. All these choices aim to improve the model's accuracy and robustness in identifying hedges. The improvement steps are as follows:

[0066] (1) As Figure 4 As shown, a CFR module is added to the original YOLOv8 model. The original YOLOv8 model includes a backbone network, a Neck structure, and a Head prediction part. The CFR (Consistency Features Reconstruction) module is added before the Upsample operation and before the last concat operation in the Neck structure. The reconstruction loss function is:

[0067] L re =||G(z)-x|| 2 ,z~N(μ,∑)

[0068] Among them, ||.|| 2 Let G(z) denote the L2 norm, G(z) denote the reconstructed image, and x denote the clean input image.

[0069] The purpose of the CFR module is to reconstruct clean images using CFR image reconstruction constraints, thereby promoting the model's learning of robust features.

[0070] (2) Figure 5 As shown, a threshold judgment mechanism is added, for L. re Set a threshold. If the result is less than the threshold, output the result. If the result is greater than the threshold, continue iterating. Figure Five The diagram shown illustrates the principle of threshold determination.

[0071] Step 5: Train the model

[0072] The model was trained using the TRADES adversarial training method, and WIoU v1 was selected as the loss function for bounding box regression during the training process to obtain a robust YOLOv8 model.

[0073] The purpose of introducing adversarial training is to improve the robustness of the model, ensuring that it maintains a certain level of accuracy when recognizing adversarial images. However, the model's accuracy in recognizing clean images decreases after adversarial training, and the TRADES adversarial training method can alleviate this problem to some extent. The specific model training method is as follows:

[0074] (1) Randomly initialize the network f θ Alternatively, use a pre-trained network;

[0075] (2) Read the dataset B = {x1,...,x} in small batches. m}, where each batch contains m samples;

[0076] (3) Based on the sample x in each mini-batch i Initialize the corresponding adversarial sample x i The adversarial example generation method is: x i '←x i +0.001·N(0,1), where N(0,1) is a standard normal distribution;

[0077] (4) Iterate K times for each adversarial example. The specific iteration method is as follows:

[0078]

[0079] Where Π represents the projection operator;

[0080] (5) Update the network parameter θ, specifically as follows:

[0081]

[0082] Due to the limited dataset and the presence of low-quality samples, learning from these samples can lead to a decrease in the model's generalization ability, consequently reducing its accuracy in recognizing different hedges in complex real-world environments. To address this issue, the original GIoU bounding box regression loss function in the YOLOv8 model is replaced with the latest WIoUv1 loss function. The purpose of introducing the WIoU v1 loss function is to reduce the penalty imposed by geometric factors when the anchor box and target box overlap well, thereby reducing training intervention and allowing the model to achieve better generalization ability. The WIoU v1 loss function is defined as follows:

[0083] L WIoUv1 =R WIoU L IoU

[0084]

[0085]

[0086] S u =wh+w gt h gt -W i H i

[0087] Where (x, y, h, w) represents the center coordinates, height, and width of the anchor box; (x gt ,y gt ,h gt ,w gt (H) represents the center coordinates, height, and width of the target box; i W i (H) represents the height and width of the overlapping area between the anchor box and the target box. g W g ) represents the height and frame of the minimum bounding box.

[0088] Step 6: Test the performance of the trained model on the validation set.

[0089] The test set is input into the trained model, and the model outputs an image with the predicted bounding box and confidence score. The detection results are as follows: Figure 6 and 7 As shown, 6 represents the clean image detection result. Figure 7 This indicates the image detection results after the patch has been applied.

[0090] The foregoing description of specific exemplary embodiments of the invention is for illustrative and explanatory purposes. These descriptions are not intended to limit the invention to the precise forms disclosed, and it will be apparent that many changes and variations can be made in accordance with the foregoing teachings. The exemplary embodiments were chosen and described in order to explain the specific principles of the invention and its practical application, thereby enabling those skilled in the art to implement and utilize various different exemplary embodiments of the invention, as well as various different choices and variations. The scope of the invention is intended to be defined by the claims and their equivalents.

Claims

1. A robust hedge detection method based on an improved YOLO model, characterized in that, include: Step 1: Obtain the original images of the hedges and collect the hedge dataset; Step 2: Select some of the collected images, patch them, and label the hedges in all images to divide them into training and test sets; Step 3: Improve the YOLOv8 model by adding a consistency feature reconstruction module and a threshold judgment mechanism. The original YOLOv8 model includes a backbone network, a Neck structure, and a Head prediction part. A consistency reconstruction module is added before the upsampling operation and the last connection operation in the Neck structure. Step 4: Train the model to obtain a robust YOLOv8 model; use the TRADES adversarial training method to train the model, and select WIoU v1 as the loss function for bounding box regression during the training process to obtain a robust YOLOv8 model. Step 5: Test the performance of the trained model on the validation set.

2. The robust hedge detection method based on the improved YOLO model according to claim 1, characterized in that, Step 1 involves obtaining original images of the hedges by taking photos of different types of hedges at different times of the same day, from different distances and angles.

3. The robust hedge detection method based on the improved YOLO model according to claim 1, characterized in that, Step 2, which involves applying the patch, includes: (1) Before applying the patch, the patch is preprocessed, including inversion, scaling, clipping and rotation, to obtain a patch set; (2) Select several hedge images with obvious differences and randomly paste different patches on different positions on the hedge images.

4. The robust hedge detection method based on the improved YOLO model according to claim 1, characterized in that, The annotation in step 2 includes: using LabelImg to annotate the hedges in the image, requiring that the predicted bounding box of each hedge be larger than the outer contour of the hedge, allowing overlap between the predicted bounding boxes, and finally saving the labels in YOLO format and txt format, and the image in jpg format.

5. The robust hedge detection method based on the improved YOLO model according to claim 1, characterized in that, Step 2, dividing the training and test sets, includes: exporting the labeled images and labels, and dividing them into training and test sets according to the proportions. All images with patches are included in the test set.

6. The robust hedge detection method based on the improved YOLO model according to claim 5, characterized in that, The ratio of training set to test set is 7.5:2.

5.

7. The robust hedge detection method based on the improved YOLO model according to claim 1, characterized in that, Step 3 involves improving the YOLOv8 model, including: (1) The original YOLOv8 model includes a backbone network, a Neck structure, and a Head prediction part. A consistency reconstruction module is added before the upsampling operation and before the last connection operation in the Neck structure; the reconstruction loss function is: ； in Let G(z) denote the L2 norm, G(z) denote the reconstructed image, and x denote the clean input image; (2) Add threshold judgment, for Set a threshold. If the result is less than the threshold, output the result. If the result is greater than the threshold, continue iterating.

8. The robust hedge detection method based on the improved YOLO model according to claim 1, characterized in that, Step 4 involves training the model using the TRADES adversarial training method. During training, WIoU v1 is selected as the loss function for bounding box regression to obtain a robust YOLOv8 model.

9. The robust hedge detection method based on the improved YOLO model according to claim 8, characterized in that, Step 4, training the model, includes: (1) Randomly initialize the network Alternatively, use a pre-trained network; (2) Reading datasets in small batches Each batch contains m samples; (3) Based on the samples in each mini-batch Initialize the corresponding adversarial sample The adversarial examples are generated as follows: ,in It follows a standard normal distribution; (4) Iterate K times for each adversarial example, with the specific iteration method as follows: ； in Represents the projection operator; (5) Network parameters To update, the specific steps are as follows: ； Specifically, the original GIoU bounding box regression loss function of the YOLOv8 model is replaced with the WIoU v1 loss function, which is defined as follows: ； ； ； ； in Indicates the coordinates of the center point of the anchor frame, as well as its height and width; This indicates the coordinates of the center point of the target bounding box, as well as its height and width. This represents the height and width of the overlapping area between the anchor box and the target box. This represents the height and width of the smallest bounding box.

10. The robust hedge detection method based on the improved YOLO model according to claim 1, characterized in that, Step 5, testing the effectiveness of the trained model on the validation set, involves inputting the test set into the trained model, and the model outputting an image with target prediction boxes and confidence scores.

Images