Target detection pre-labeling method based on fast algorithm

Abstract

The invention discloses a target detection pre-labeling method based on a fast algorithm, and relates to the technical field of computer vision and geometric processing, and the target detection pre-labeling method based on the fast algorithm comprises the following steps: S1, data acquisition and preprocessing; and the collected image data is transmitted to a cloud data center in real time through wireless transmission. According to the target detection pre-labeling method based on the fast algorithm, image information can be locked and recognized more accurately through operation of two times of pre-labeling, so that the collected image information can be recognized and screened more accurately, a certain workload can be reduced for subsequent manual auditing and correction, and the accuracy of target detection and pre-labeling is improved. And meanwhile, it is also ensured that the pre-labeling model can carry out accurate labeling processing on a relatively fuzzy or shielded image, and through operation of a YOLOv5 algorithm, the pre-labeling model can be more efficient and faster when carrying out image recognition.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision and geometry processing, in particular to a target detection pre-labeling method based on a fast algorithm. BACKGROUND

[0002] Target detection pre-labeling is a step in the field of machine learning and computer vision to locate and classify objects in images. To improve the efficiency and quality of labeling, pre-labeling is usually performed before manual labeling, that is, using a trained model to automatically perform preliminary target detection and labeling. Specifically, the target class to be detected is determined, the collected image data is analyzed and recognized by the model, and representative samples are selected. Then, the target detection model is used to pre-label the selected data. However, the current target detection pre-labeling still has the following shortcomings:

[0003] The existing target detection pre-labeling uses single feature information for detection and labeling, resulting in a large amount of image information after pre-labeling and an inability to accurately lock the image information, thereby causing certain interference to subsequent manual review and correction and reducing the actual application effect of the target detection pre-labeling method, thereby having certain use defects. SUMMARY

[0004] The present application aims to provide a target detection pre-labeling method based on a fast algorithm to solve the problems raised in the background.

[0005] To achieve the above-mentioned purpose, the present application provides the following technical solution: a target detection pre-labeling method based on a fast algorithm, comprising the following steps:

[0006] S1, data acquisition and preprocessing:

[0007] S11, data collection: using a camera device to collect raw image data and transmitting the collected image data to a cloud data center in real time through wireless transmission;

[0008] S12, data preprocessing: when the cloud data center receives the image data, the received image data is uniformly processed to form a uniform image size and format, so that subsequent data screening is more convenient, and for high-resolution images such as 4K / 8K, they need to be scaled to 640x640 mainstream model input size. After reasoning, the target frame is mapped back to the original image, and the detection area of simple background images is extracted, that is, the background within the target range is completely extracted and recognized, so as to ignore the non-target area (non-detection target) and reduce the calculation range.

[0009] S13, data screening: screening the image data to be pre-labeled for target detection to automatically remove images with resolution lower than or higher than the preset value, remove images with blur or noise exceeding the preset value, and find and process visually similar images using an image similarity detection tool when detecting and deleting visually similar images. The image similarity detection tool includes phash. For continuous frame image information, the most clear 1 frame is selected in every 10-20 frames for fast model detection (key frame), and the remaining frames predict the target position by optical flow method or tracking algorithm (such as DeepSORT);

[0010] S2, pre-labeling model running:

[0011] S21, image classification: using YOLOv5 algorithm to quickly identify and process the images after preliminary screening, and automatically classifying according to the extracted feature information. If the image contains multiple feature information, it is recorded in the corresponding classification. The image that cannot be accurately identified due to occlusion is classified into a new file for subsequent pre-labeling model secondary running for identification processing, and outputs the target frame (x1, y1, x2, y2), class, confidence;

[0012] S22, result filtering: removing low confidence targets (such as confidence <0.3) to reduce invalid labeling in the future. The detailed process includes: 1. Extracting key features such as boundary box coordinates (x1, y1, x2, y2), class label: class and corresponding class probability pclass, confidence score: conf (model confidence that "the box is a real target", usually the product of class probability and boundary box confidence), and auxiliary features such as target area, aspect ratio, etc. 2. Confidence threshold filtering (basic filtering), which directly removes low confidence prediction boxes (high probability of false detection) by setting a confidence threshold, reducing invalid labeling. 3. Remove repeated labeling (NMS filtering), which retains the optimal box through non-maximum suppression (NMS) to remove multiple overlapping boundary boxes (repeated labeling) of the same target. 4. Boundary box quality filtering (geometric rationality check), which filters "physically reasonable" labels by the geometric properties (size, position, shape) of the boundary box and removes obviously abnormal boxes. 5. Class rationality filtering (semantic check), which removes labels with "class and scene mismatch" or "class probability ambiguity" by combining class semantics and scene logic.

[0013] S23, target detection: on the basis of feature extraction, detection is performed in the image target corresponding to the classification to identify the target in the image or scene and determine its approximate position. If the detected feature information is blocked by other objects, edge detection can be performed to find the image area of the suspected target category, or detection can be directly performed according to other feature information, and the image information result after detection is converted into a label tool compatible format;

[0014] S24, pre-labeling: according to the result of target detection, automatically generate the labeling information of the target, and sort the image complexity (target quantity, background confusion degree) during the labeling process, preferentially process simple samples (such as images with few targets), and improve the overall labeling efficiency;

[0015] S3, secondary operation of pre-labeling model:

[0016] S31, target locking: integrate and store the pre-labeled image data into a new file in table form, then connect the feature points on the pre-labeled image data plane to form feature point trajectories, and perform three-dimensional rotation and equal proportion scaling on the feature point trajectories of suspected target categories. During the rotation and scaling process, the feature point trajectories are compared with the feature point trajectories of each part of the target category. If they coincide, it is preliminarily determined that the suspected target category is the required target category;

[0017] S32, secondary pre-labeling: when multiple feature information in the selected image information coincide, automatically perform secondary labeling on the target, and when the YOLOv5x algorithm performs secondary pre-labeling on the target, dynamically adjust the confidence threshold of pre-labeling according to the corresponding relationship between the confidence of the algorithm and the actual accuracy of the artificial feedback, so as to reduce invalid labeling and further improve the pre-labeling speed of the model;

[0018] S4, result output: save and output the labeling results in a specified format for subsequent manual review and correction.

[0019] Further, in step S11, the camera device includes common camera devices such as unmanned aerial vehicle image capture, security monitoring, and traffic scene, to achieve more comprehensive image data reception purposes.

[0020] Further, in step S13, for repeatedly appearing targets (such as fixed devices in monitoring, etc.) in continuous frame image information, cache their positions and categories, and directly reuse subsequent frames to avoid repeated calculation.

[0021] Further, in step S21, after the YOLOv5 algorithm finishes fast recognition and classification of the images, the image files are automatically read and converted into binary data, then the MD5 hash algorithm is used to calculate the hash values of the images, and the hash values of the images are stored in the database in table form.

[0022] Further, after all image processing is completed, if the hash values in the hash value table in the database are the same, it means that the images corresponding to the two hash values are the same, and thus any one of the images is automatically deleted.

[0023] Further, in step S21, the YOLOv5 algorithm only detects and recognizes single feature information during operation, and classifies the recognized targets in the image or scene, wherein the center coordinate prediction: the offset of the target center coordinate relative to the grid is constrained in the range of [0, 1] by a sigmoid function, and the formula is:

[0024] x=σ(t x )+c x

[0025] y=σ(t y )+c y

[0026] wherein: t x ,t y are the original offsets output by the network; σ(.) is a sigmoid function (mapping values to 0-1); c x ,c y are the coordinates of the upper left corner of the current grid (unit: grid size, for example, when the input image is divided into S×S grids, c x is the grid column index, and c y is the grid row index), width and height prediction: the width and height of the bounding box are scaled based on the anchor (Anchor), and the formula is:

[0027] ω=p ω .e tω

[0028] h=p h .e th

[0029] wherein: t ω ,t h are the width and height offsets output by the network; p ω ,p h are predefined anchor width and height (the anchor of YOLOv5s needs to be obtained according to the data set clustering), e tω , e th ensure that the width and height are positive (exponential function characteristics).

[0030] Further, in the step S32, the YOLOv5 algorithm is used for secondary running, and the YOLOv5 algorithm detects and identifies multiple feature information of the classified image during running, so as to improve the accuracy of pre-labeling. In the first pre-labeling, YOLOv5s is used for quickly generating a basic label, and in the second pre-labeling, YOLOv5x with higher accuracy is used to verify and identify low confidence targets (0.2<conf<0.3), and the confidence score formula is: Confidence=Pr(Object)×IoU(pred,truth), Pr(Object) is the probability of the existence of the target in the grid (0 or 1, which is determined by whether the real box falls into the grid during training), and IoU(pred,truth) is the intersection over union of the predicted box and the real box.

[0031] The application provides a target detection pre-labeling method based on a fast algorithm.

[0032] 1、The application can more accurately lock and identify image information through twice pre-labeling, so as to more accurately identify and screen collected image information, reduce the workload of subsequent manual auditing and correction, ensure that the pre-labeling model can accurately label and process relatively blurred or shielded images, greatly reduce the workload of manual auditing, and make the pre-labeling model more efficient and fast during image identification through the running of the YOLOv5 algorithm.

[0033] 2、The application can uniformly process collected image information by pre-processing received image data, so as to form uniform image size and format, provide convenience for subsequent data screening, automatically remove invalid and completely unidentifiable images in the image information through the running of the image screening, provide convenience for subsequent pre-labeling, and improve the identification and classification efficiency of the YOLOv5 algorithm during running to a certain extent. BRIEF DESCRIPTION OF DRAWINGS

[0034] Fig. 1 It is a use flowchart of the target detection pre-labeling method based on the fast algorithm.

[0035] Fig. 2 It is a data acquisition and pre-processing running flowchart of the target detection pre-labeling method based on the fast algorithm.

[0036] Fig. 3 It is a pre-labeling model running flowchart of the target detection pre-labeling method based on the fast algorithm. Detailed Implementation

[0037] The embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples. The following examples are for illustrative purposes only and should not be construed as limiting the scope of the invention.

[0038] like Figs. 1-3 As shown, a target detection pre-labeling method based on a fast algorithm includes the following steps:

[0039] S1. Data Acquisition and Preprocessing:

[0040] S11. Data collection: Use camera equipment to collect raw image data and transmit the collected image data to the cloud data center in real time via wireless transmission. In step S11, the camera equipment includes common shooting equipment such as drone images, security monitoring, and traffic scene images, in order to achieve a more comprehensive image data reception and acquisition purpose.

[0041] S12. Data Preprocessing: When the cloud data center receives image data, it processes the received image data in a unified manner to form a unified image size and format, which makes it easier to filter data later. For high-resolution images such as 4K / 8K, they need to be scaled to the mainstream model input size of 640×640. After inference, the target box is mapped back to the original image. At the same time, for images with simple backgrounds, the detection region is extracted, that is, the target and the background within the target range are completely extracted, thereby ignoring the non-target areas (non-detection targets) and reducing the calculation range.

[0042] S13. Data Filtering: The image data to be pre-labeled for target detection is filtered to automatically remove images with resolutions lower or higher than preset values, as well as blurry or noisy images. When detecting and deleting visually similar images, an image similarity detection tool is used to find and process visually similar images. Image similarity detection tools include Phash. For continuous frame image information, the clearest frame is selected from every 10-20 frames and detected using a fast model (key frame). The target position is predicted for the remaining frames using optical flow or tracking algorithms (such as DeepSORT). In step S13, the position and category of recurring targets (such as fixed equipment in monitoring) in continuous frame image information are cached and reused in subsequent frames to avoid redundant calculations.

[0043] S2, Running the pre-annotated model:

[0044] S21, image classification: using YOLOv5 algorithm to preliminarily screen the image and carry out automatic classification processing according to the extracted feature information, if the image contains multiple feature information, it is recorded in the corresponding classification, in step S21, when YOLOv5 algorithm runs to quickly identify and classify the image, automatically read the image file and convert the image into binary data, then use MD5 hash algorithm to calculate the hash value of the image, and store the hash value of each image in the database in the form of table, so as to provide convenience for subsequent image comparison to remove repeated images, when all image processing is completed, image comparison is carried out again, if the hash value in the hash value table in the database is completely the same, it means that the two images corresponding to the hash value are completely the same, so as to automatically delete any one image, so as to reduce the invalid running workload of subsequent fast algorithm, in step S21, YOLOv5 algorithm only detects and identifies single feature information when running, and classifies the target in the identified image or scene, wherein the center coordinate prediction: the offset of target center coordinate relative to the grid is constrained in the range of [0, 1] by sigmoid function, the formula is:

[0045] x = σ (t x ) + c x

[0046] y = σ (t y ) + c y

[0047] Wherein: t x , t y is the original offset of network output; σ (.) is sigmoid function (mapping value to 0-1); c x , c y is the coordinate of the left top corner of the current grid (unit: grid size, for example, when the input image is divided into S×S grid, c x is the grid column index, and c y is the grid row index), width and height prediction: the width and height of the boundary box are based on the scaling of anchor (anchor), the formula is:

[0048] ω = p ω e tω

[0049] h = p h e th

[0050] Wherein: t ω , t h is the width and height offset of network output; p ω , p h is the predefined anchor width and height (the anchor of YOLOv5s needs to be obtained according to the data set clustering), etω , e th Ensure that the width and height are positive (exponential function characteristics), so that the subsequent accurate identification of complex images is convenient, and the images that cannot be accurately identified due to occlusion are classified into new files for subsequent pre-labeling model secondary operation for identification processing, and output the target frame (x1, y1, x2, y2), category, confidence;

[0051] S22, result filtering: eliminate low-confidence targets (such as confidence <0.3), thereby reducing subsequent invalid labeling, and the detailed process includes: 1. Extract key features such as: bounding box coordinates (x1, y1, x2, y2), class label: class and corresponding class probability pclass, confidence score: conf (the confidence of the model that "the box is a real target", usually the product of the class probability and the bounding box confidence) and auxiliary features such as target area, aspect ratio, etc. 2. Confidence threshold filtering (basic filtering), by setting a confidence threshold, directly eliminating low-confidence prediction boxes (high probability of false detection), reducing invalid labeling, 3. Remove duplicate annotations (NMS filtering), the same target may be predicted by the model Multiple overlapping bounding boxes (duplicate annotations) need to be retained by non-maximum suppression (NMS), 4. Bounding box quality filtering (geometric rationality check), filter "physically reasonable" annotations by the geometric properties (size, position, shape) of the bounding box, and eliminate obviously abnormal boxes, 5. Class rationality filtering (semantic check), combined with class semantics and scene logic, eliminate annotations that "class and scene do not match" or "class probability is ambiguous";

[0052] S23, target detection: based on feature extraction, detect within the corresponding classified image target to identify the target in the image or scene and determine its approximate position. If the detected feature information is blocked by other objects, edge detection can be performed to find the image area of the suspected target category, or direct detection can be performed according to other feature information, and the detected image information result is converted into a label tool compatible format;

[0053] S24, pre-labeling: automatically generate target labeling information according to the results of target detection, and sort images by complexity (number of targets, background clutter) during labeling to process simple samples (such as images with few targets) first, improving overall labeling efficiency;

[0054] S3, secondary operation of pre-labeling model:

[0055] S31, target locking: the pre-labeled image data is integrated and stored in a new file in table form, then each feature point on the pre-labeled image data plane is connected to form a feature point track, the feature point track of the suspected target category is subjected to three-dimensional rotation and equal proportion scaling, and the feature point track is compared with the feature point track of each part of the target category during the rotation and scaling, if overlap occurs, the suspected target category is preliminarily determined as the required target category;

[0056] S32, secondary pre-labeling: when multiple feature information in the selected image information all overlap, the target is automatically labeled twice, and when the YOLOv5x algorithm labels the target twice, the confidence threshold of pre-labeling is dynamically adjusted according to the corresponding relationship between the confidence of the algorithm and the actual accuracy of artificial feedback, so as to reduce invalid labeling, thereby further improving the pre-labeling speed of the model, in step S32, the YOLOv5 algorithm is used for secondary operation, and the YOLOv5 algorithm detects and identifies multiple feature information of the classified image during operation, so as to improve the accuracy of pre-labeling, so that the pre-labeling model can adapt to the recognition and labeling processing of complex images, and YOLOv5s is used for fast generation of basic labeling in the first pre-labeling, and YOLOv5x with higher accuracy is used for verification and identification of low confidence targets (0.2<conf<0.3) in the second pre-labeling, so as to improve the speed of the model during operation and the labeling accuracy, and the confidence score formula is: Confidence = Pr (Object) x IoU (pred, truth), Pr (Object) is the probability of the existence of the target in the grid (0 or 1, which is determined by whether the real box falls into the grid during training), and IoU (pred, truth) is the intersection over union (Intersection over Union) of the predicted box and the real box;

[0057] S4, result output: the labeling result is saved and output in a specified format, so as to be audited and corrected by artificial in the future.

[0058] The embodiments of the present application are given for example and description, and are not exhaustive or limit the present application to the disclosed forms. Many modifications and changes are obvious to those skilled in the art. The embodiments are selected and described to better illustrate the principles and practical application of the present application, and to enable those skilled in the art to understand the present application so as to design various embodiments with various modifications suitable for specific purposes.

Claims

1. A fast algorithm based target detection pre-annotation method, characterized in that, The target detection pre-labeling method based on the fast algorithm comprises the following steps: S1, data acquisition and preprocessing: S11, data collection: collect original image data using a camera device, and transmit the collected image data to a cloud data center in real time through wireless transmission; S12, data preprocessing: when the cloud data center receives the image data, the accepted image data is uniformly processed to form a uniform image size and format, so that subsequent data screening is more convenient, and for high-resolution images such as 4K / 8K, they need to be scaled to 640x640 mainstream model input size, and after inference, the target frame is mapped back to the original image. The detection area is extracted for simple background images, that is, the background in the target range is completely extracted and recognized, so as to ignore the target-free area and reduce the calculation range; S13, data screening: screen the image data to be pre-labeled for target detection, so as to automatically remove images with resolution lower than or higher than the preset value, remove images with blur or noise exceeding the preset value, and use an image similarity detection tool to find and process visually similar images when detecting and deleting visually similar images. The image similarity detection tool includes phash, and for continuous frame image information, the clearest 1 frame is selected from every 10-20 frames for fast model detection, and the remaining frames are predicted for target position by optical flow method or tracking algorithm; S2, pre-labeling model running: S21, image classification: using YOLOv5 algorithm to quickly identify and process the images screened initially, and automatically classifying according to the extracted feature information, if the image contains multiple feature information, it is recorded in the corresponding classification, and the image that cannot be accurately identified due to shielding is classified into a new file, so that the subsequent pre-labeling model is run again for identification processing, and the target frame, category and confidence are output; S22, result filtering: remove low-confidence targets to reduce invalid labeling in the future, the detailed process includes: 1, extract key features such as boundary box coordinates, class label: class and corresponding class probability pclass, confidence score: conf and auxiliary features such as target area and aspect ratio, 2, confidence threshold filtering, directly remove low-confidence prediction boxes by setting a confidence threshold, reduce invalid labeling, 3, remove repeated labeling, the same target may be predicted by the model as multiple overlapping boundary boxes, which need to be retained by non-maximum suppression. The optimal frame, 4, boundary box quality filtering, filter "physically reasonable" labels by the geometric properties of the boundary box, and remove obviously abnormal boxes, 5, category rationality filtering, combined with category semantics and scene logic, remove labels that are "category and scene mismatch" or "category probability ambiguous". S23, target detection: on the basis of feature extraction, detection is carried out in the corresponding classified image target to identify the target in the image or scene and determine its approximate position. If the detected feature information is blocked by other objects, edge detection can be performed to find the image area of the suspected target category, or detection can be directly performed according to other feature information, and the detected image information result is converted into a label tool compatible format; S24, pre-labeling: according to the result of target detection, the target labeling information is automatically generated, and the image complexity is sorted during the labeling process, the simple samples are processed preferentially, and the overall labeling efficiency is improved; S3, secondary operation of pre-labeling model: S31, target locking: the pre-labeled image data is integrated and stored in a new file in table form, then the feature points on the pre-labeled image data plane are connected to form feature point trajectories, the feature point trajectories of suspected target categories are rotated and scaled in three dimensions, and the feature point trajectories are compared with the feature point trajectories of each part of the target category during rotation and scaling. If they coincide, it is preliminarily determined that the suspected target category is the required target category; S32, secondary pre-labeling: when multiple feature information in the selected image information coincide, the target is automatically labeled again, and when the YOLOv5x algorithm performs secondary pre-labeling on the target, the confidence threshold of pre-labeling is dynamically adjusted according to the corresponding relationship between the confidence of the algorithm and the actual accuracy of the artificial feedback, so as to reduce invalid labeling and further improve the pre-labeling speed of the model; S4, result output: save and output the labeling results in a specified format for subsequent manual review and correction.

2. The target detection pre-labeling method based on a fast algorithm according to claim 1, characterized in that, In step S11, the camera equipment includes unmanned aerial vehicle image shooting, security monitoring, and common shooting equipment in traffic scenes to achieve more comprehensive image data reception purposes.

3. The target detection pre-annotation method based on a fast algorithm according to claim 1, characterized in that, In step S13, the positions and categories of the repeatedly appearing targets in the continuous frame image information are cached, and the subsequent frames are directly reused to avoid repeated calculation.

4. The target detection pre-labeling method based on a fast algorithm of claim 1, wherein, In step S21, after the YOLOv5 algorithm runs to quickly identify and classify the image, the image file is automatically read and the image is converted into binary data, then the MD5 hash algorithm is used to calculate the hash value of the image, and the hash values of each image are stored in a table form database.

5. The method of claim 3, wherein the method further comprises: When all image processing is completed, if the hash values in the hash value table in the database are exactly the same, it means that the two images corresponding to the hash values are exactly the same, so any one image is automatically deleted.

6. The method of claim 1, wherein the method is based on a fast algorithm for object detection pre-annotation. In step S21, the YOLOv5 algorithm only detects and identifies single feature information during operation, and classifies the identified targets in the image or scene. The center coordinate prediction: the offset of the target center coordinate relative to the grid is constrained in the range of [0, 1] by the sigmoid function, and the formula is: x = σ(t x ) + c x y = σ(t y ) + c y Where: t x ,t y It is the original offset of the network output; σ(.) is the sigmoid function; c x ,c y This is the coordinate of the top-left corner of the current grid, in units of grid size. For example, if the input image is divided into an S×S grid, c x For grid column indexes, c y For grid row indexing, width and height prediction: the bounding box width and height are based on the scaling of the anchor box, using the following formula: ω = p ω .e tω h = p h .e th wherein: t ω , t h is the width-height offset of the network output; p ω , p h is the predefined anchor box width-height, e tω , e th ensures the width-height is positive.

7. The method of claim 1, wherein the method further comprises: In the step S32, the YOLOv5 algorithm is used for secondary running, and the YOLOv5 algorithm detects and identifies multiple feature information of the classified image during running, so as to improve the accuracy of the pre-labeling. In the first pre-labeling, YOLOv5s is used to quickly generate a basic label, and in the second pre-labeling, YOLOv5x with higher accuracy is used to verify and identify low-confidence targets. The confidence score formula is: Confidence=Pr×IoU, Pr is the probability of the existence of the target in the grid, and IoU is the intersection over union of the prediction box and the real box.

Images