A detection method based on a target detection algorithm and an image classification algorithm

Abstract

This application relates to a detection method based on object detection and image classification algorithms, comprising: collecting images of the object to be detected and background images; labeling the collected images containing the object to be detected to obtain a dataset; dividing the dataset into training, validation, and test sets; clustering the bounding boxes in the dataset, randomly cropping background images as background categories, and adding the cropped images from the dataset to obtain the object to be detected and background category datasets. The dataset is used to train an image classification network; a YOLOv3 object detection network is trained using the dataset; the detection network detects the images to be detected; if an object is found, the image dataset is cropped based on the object's coordinate information, the shorter side of the image is padded, and the image is reshaped to a preset size before being fed into the image classification network for detection and analysis; this method achieves high recall and high precision.

Description

Technical Field

[0001] This application relates to the field of deep learning technology, and in particular to a detection method based on object detection algorithm and image classification algorithm. Background Technology

[0002] In recent years, deep neural networks have been widely used in object detection tasks, giving rise to some classic object detection networks. These include one-stage algorithms such as the YOLO series, SSD algorithm, and RetinaNet algorithm, and two-stage algorithms such as the R-CNN series. Among the YOLO series algorithms, YOLOv3 and YOLOv4 are widely used in industry due to their fast detection speed and high accuracy. However, in practical detection, they are also prone to false detections, such as mistaking the background for the target, leading to missed detections and false positives. This poses a challenge to fault detection algorithms, making it difficult to simultaneously improve precision and recall. In fact, increasing recall inevitably reduces precision, and vice versa. For fault identification, a high recall rate can easily lead to misidentifying the background as a faulty target, resulting in false alarms, and a decrease in precision can also cause false positives. Conversely, a low recall rate may result in some faults going undetected, leading to missed detections. Missed detections, incorrect detections, and erroneous detections can all lead to safety hazards and accidents. Summary of the Invention

[0003] To address the problems existing in the aforementioned background technology, this invention proposes a method based on object detection algorithms and image classification algorithms to improve detection performance. This method not only achieves a high recall rate but also improves the precision rate, fundamentally reducing false negatives and missed detections.

[0004] The detection method based on object detection algorithm and image classification algorithm provided in this application adopts the following technical solution:

[0005] A detection method based on object detection algorithm and image classification algorithm includes the following steps:

[0006] Step 1: Collect images of the object to be detected and background images that do not contain the object to be detected;

[0007] Step 2: Use the image annotation tool labelImg to annotate the collected images containing the objects to be detected, and obtain the dataset;

[0008] Step 3: Divide the dataset from Step 2 into a training set, a validation set, and a test set in a ratio of 7:2:1;

[0009] Step 4: Perform K-Means clustering on each rectangle in the dataset described in Step 2 to obtain nine anchor boxes;

[0010] Step 5: Based on the nine anchor boxes obtained in Step 4, randomly crop the background image from Step 1 as the background category, and add it to the image cropped from the anchor boxes obtained in Step 2 to obtain the dataset of the object to be detected and the background category.

[0011] Step 6: Use the dataset obtained in Step 5 to train the image classification network.

[0012] Step 7: Train the YOLOv3 object detection network using the dataset from Step 3;

[0013] Step 8: Use the detection network obtained in Step 7 to detect the image to be detected. If the detection results show a target, cut these targets according to their coordinate information to obtain an image dataset. Perform padding on the short side of the image, reshape it to a preset size, and send it into the classification network obtained in Step 6 for detection. Analyze the detected targets.

[0014] Furthermore, step two includes obtaining the label file: using the labelImg annotation tool to annotate the image and generate a label file conforming to the VOC2007 format. The file format is XML, and the file content includes the target category and the target's location information, including the coordinates of the actual target's xmin, ymin, xmax, and ymax.

[0015] Furthermore, before performing the reshape operation on the image obtained in step five, a padding operation is performed on the short side of the image to make the length and width of the image equal; after reshaping, the length and width of the image are equal, and the pixel size of the image is the average pixel size of all images after the padding operation.

[0016] Furthermore, step four uses the k-Means clustering algorithm to cluster the real target boxes, obtaining nine anchor boxes. The specific steps are as follows:

[0017] S100. The real target boxes in the dataset of step two are used as the original data for the k-Means clustering algorithm. The length and width values ​​of each real target box are a sample.

[0018] S200. Select the length and width of the nine boxes as the cluster centers for the k-Means clustering algorithm;

[0019] S300. Calculate the distance between each box and the cluster center according to the formula d = 1 – IOU, where IOU is the intersection-union ratio of the ground truth box and the cluster center. Find the cluster center with the closest distance between the ground truth box and the nine cluster centers, and assign the ground truth box to that cluster center.

[0020] S400. Perform step S300 on all ground truth boxes until all ground truth boxes are assigned to cluster centers. Then, calculate a new cluster center for the length and width of the ground truth boxes in each cluster (the ground truth boxes in each cluster center). The new cluster center is the average length and width of all ground truth boxes under that cluster center.

[0021] S500. Repeat steps S100 to S400 until the length and width of the cluster centers no longer change. The length and width of the nine cluster centers are the size of the anchor.

[0022] Furthermore, the image classification network in step six is ​​a multi-layer convolutional network, comprising convolutional layers, pooling layers, activation layers, fully connected layers, and a softmax layer. CrossEntropyLoss is used as the classification loss function, and the loss function formula is as follows:

[0023] loss = −∑t i lny i ;

[0024] Where ti represents the true value, which is a one-hot encoded vector, and yi represents the calculated Softmax value, which is:

[0025] y i =e i / ∑ j e j .

[0026] In summary, this application includes at least one of the following beneficial technical effects: it can not only achieve a high recall rate, but also greatly improve the precision rate; and it plays a fundamental role in improving the rates of missed detections, false detections, and misdetections. Attached Figure Description

[0027] Figure 1 This is a flowchart illustrating an embodiment of the present invention;

[0028] Figure 2 This is a graph showing the results of YOLOv3 network detection;

[0029] Figure 3 This is a detection result image of YOLOv3 with an improved image classification algorithm. Detailed Implementation

[0030] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to represent selected embodiments of the invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0031] The following is in conjunction with the appendix Figure 1-3 The embodiments will provide a more detailed description of the method provided by the present invention for improving detection performance based on object detection algorithm and image classification algorithm.

[0032] Figure 1 This is a flowchart of the method for improving detection performance based on object detection algorithm and image classification algorithm proposed in this invention. The method for improving detection performance based on object detection algorithm and image classification algorithm includes:

[0033] S1. Collect images of the object to be detected and background images that do not contain the object to be detected, including 1500 images of the object to be detected and 200 background images.

[0034] S2. Use the image annotation tool labelImg to annotate the collected images containing the objects to be detected, and obtain the dataset. The dataset format is VOC2007 data format, and an XML file will be generated for each annotated image.

[0035] S3. Divide the dataset from step S2 into a training set, a validation set, and a test set in a ratio of 7:2:1.

[0036] S4. Perform K-Means clustering on each rectangle in the dataset described in step S2 to obtain nine anchor boxes;

[0037] The specific operating steps are as follows:

[0038] S41. The real target boxes in the dataset of step two are used as the original data for the k-Means clustering algorithm. The length and width values ​​of each real target box are a sample.

[0039] S42. Select the length and width of the nine boxes as the cluster centers for the k-Means clustering algorithm.

[0040] S43. Calculate the distance between each box and the cluster center according to the formula d = 1 – IOU, where IOU is the intersection-union ratio of the ground truth box and the cluster center. Find the cluster center with the closest distance between the ground truth box and the nine cluster centers, and assign the ground truth box to that cluster center.

[0041] S44. Perform step S43 on all ground truth boxes until all ground truth boxes are assigned to cluster centers. Then, calculate a new cluster center for the length and width of the ground truth boxes in each cluster (the ground truth boxes in each cluster center). The new cluster center is the average length and width of all ground truth boxes under that cluster center.

[0042] S45. Repeat steps S41 to S44 until the length and width of the cluster centers no longer change. The length and width of the nine cluster centers are the size of the anchor.

[0043] S5. Based on the nine anchor boxes obtained in step S4, randomly crop the background image from step one as the background category, and add it to the image cropped from the anchor boxes obtained in step S2 to obtain the dataset of the object to be detected and the background category.

[0044] S6. The dataset obtained in step S5 is used to train the image classification network.

[0045] S7. Use the dataset from step S3 to train the YOLOv3 object detection network to obtain the detection network;

[0046] S8. Use the detection network obtained in step S7 to detect the image to be detected. If the detection result shows a target, cut these targets according to the coordinate information of the target to obtain the image dataset. Perform padding operation on the short side of the image, and send the image to the classification network obtained in step S6 for detection. Further analyze the detected targets and output the results.

[0047] Figure 2 The image obtained by the YOLOv3 object detection network in step S7 shows that there are cases of missed detections, false detections, and misdetection of the background.

[0048] Figure 3 To improve the classification algorithm obtained in step S6 Figure 2 As shown in the image, the improved predictions are completely accurate.

[0049] The above description represents a preferred embodiment of the present invention. Therefore, any equivalent changes or modifications made to the structure, features, and principles described in the claims of this patent application are included within the scope of this patent application.

Claims

1. A detection method based on object detection algorithm and image classification algorithm, characterized in that: The method includes the following steps: Step 1: Collect images of the object to be detected and background images that do not contain the object to be detected; Step 2: Use an image annotation tool to annotate the collected images containing the objects to be detected, and obtain the dataset; Step 3: Divide the dataset from Step 2 into a training set, a validation set, and a test set according to the specified proportions; Step 4: Cluster the labeled content of the dataset described in Step 2 to obtain the anchor boxes; Step 5: Based on the anchor box, randomly crop the background image as the background category, and add the image cropped from the anchor box to obtain the dataset of the object to be detected and the background category; Step 6: Use the dataset obtained in Step 5 to train the image classification network; Step 7: Train the YOLOv3 object detection network using the dataset from Step 3; Step 8: Use the detection network obtained in Step 7 to detect the image to be detected. If the detection results show a target, cut these targets according to their coordinate information to obtain an image dataset. Perform padding on the short side of the image, reshape it to a preset size, and send it into the classification network obtained in Step 6 for detection, analysis, and output of results. Step four further includes: using the k-Means clustering algorithm to cluster the labeled real target boxes to obtain a preset number of anchor boxes. Specific steps include: S100. The real target boxes in the dataset of step two are used as the original data for the k-Means clustering algorithm, and the length and width values ​​of each real target box are used as a sample. S200: Select the length and width of a preset number of anchor boxes as the cluster centers for the k-Means clustering algorithm; S300. Calculate the distance between each anchor box and the cluster center according to the formula d = 1 – IOU, where IOU is the intersection-union ratio of the ground truth box and the cluster center. Find the cluster center with the closest distance between the ground truth box and a preset number of cluster centers, and assign the ground truth box to the cluster center with the closest distance. S400: Perform the operation described in S300 on all ground truth boxes until all ground truth boxes are assigned to the nearest cluster center. Then, calculate a new cluster center for each cluster center based on the length and width of the ground truth box. The new cluster center is the average of the length and width of all ground truth boxes under the nearest cluster center. S500: Repeat steps S200 to S400 until the length and width of the cluster centers calculated in S400 no longer change. The length and width of the preset number of cluster centers are the size of the anchor box. Step two also includes generating a tag file, the generation process of which includes: Use the labelImg annotation tool to annotate the image and generate a label file that conforms to the preset format. The label file contains the category of the target and the location information of the target. The location information includes the xmin, ymin, xmax and ymax coordinates of the real target. In step six, the image classification network is a multi-layer convolutional network, including convolutional layers, pooling layers, activation layers, fully connected layers, and softmax layers. CrossEntropyLoss is used as the classification loss function, and the loss function formula is: loss = -∑ t i lny i ;in t i The true value is a one-hot encoded vector. y i The calculated Softmax value is: y i =e i / ∑ j e j .

2. The detection method based on target detection algorithm and image classification algorithm according to claim 1, characterized in that: Increase the recall of the YOLOv3 object detection network, decrease the accuracy of the YOLOv3 object detection network, and then execute step six.

3. The detection method based on target detection algorithm and image classification algorithm according to claim 1, characterized in that: Before performing the reshape operation on the obtained image in step eight, the padding operation on the short side of the image also includes: making the length and width of the image equal, so that the length and width of the reshaped image are equal, and the pixel size of the image is the average pixel size of all images after the padding operation.

Images