A pre-labeling method for object detection that facilitates data screening

Abstract

The present invention discloses a target detection pre-labeling method for convenient data screening, which relates to the fields of computer vision and geometric processing technology. The target detection pre-labeling method for convenient data screening includes the following steps: S1, data cleaning; S2, application of pre-labeling model; S3, processing of suspected target categories; S4, secondary application of pre-labeling model. The target detection pre-labeling method for convenient data screening can quickly screen out images with high processing difficulty based on the target detection pre-labeling model, and after finding the area of ​​the suspected target category in such images, use Harris corner point detection to find feature points and connect them. The feature point trajectory formed after connecting is rotated and scaled to compare with the complete feature point trajectory of the required target category. Based on the comparison and overlap, the image is subjected to secondary detection, so that such image data with high processing difficulty can be quickly and automatically processed and pre-labeled, without the need for manual labeling.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision and geometric processing, and in particular to a target detection pre-labeling method that facilitates data screening. Background Art

[0002] Target detection pre-labeling is a step in the field of machine learning and computer vision to locate and classify objects in images. In order to improve the efficiency and quality of labeling, pre-labeling is usually performed before manual labeling, that is, using a trained model to automatically perform preliminary target detection and labeling. Specifically, the target category to be detected is first determined, and the model analyzes and identifies the collected image data, and selects representative samples, and then the target detection model is used to pre-label the screened data.

[0003] When existing target detection models screen image data, the target categories in the image data are occluded by other objects, making it difficult for the target detection models to analyze and identify the required target categories from the image data, and additional manual labeling is required to fill in the gaps.

[0004] Therefore, in view of this, the existing structure and deficiencies are studied and improved, and a target detection pre-labeling method that facilitates data screening is proposed. Summary of the invention

[0005] In view of the deficiencies in the prior art, the present invention provides a target detection pre-labeling method that facilitates data screening and solves the problems raised in the above-mentioned background technology.

[0006] To achieve the above objectives, the present invention is implemented by the following technical scheme: a target detection pre-labeling method for convenient data screening, the target detection pre-labeling method for convenient data screening comprises the following steps:

[0007] S1. Data cleaning:

[0008] Clean the image dataset that is pre-labeled for target detection, use scripts to automatically open each image file, clear files that cannot be opened, non-image files, and damaged images, and then identify and delete images with resolutions lower than or higher than a preset value, as well as images with blur or noise exceeding a preset value based on the opened image files, and then detect and delete identical and visually similar images;

[0009] S2. Apply the pre-annotated model:

[0010] Based on the image dataset after data cleaning, the trained target detection pre-annotation model is used to pre-annotate it. The pre-annotation results include the target's bounding box and category information. The target detection pre-annotation model uses image feature extraction technology to extract the target category in the image. For a type of image data where the target category is blocked by other objects in the image, edge detection is first performed to find the image area of ​​the suspected target category, and then Harris corner detection is performed to find the significant feature points in the image area;

[0011] S3. Processing of suspected target categories:

[0012] Based on the feature points found in the image area where the suspected target category is located, the feature points on the plane are connected to form a feature point trajectory, and the feature point trajectory of the suspected target category is three-dimensionally rotated and proportionally scaled. During the rotation and scaling process, the feature point trajectory is compared with the feature point trajectory of each part of the target category. If there is overlap, the suspected target category is preliminarily determined to be the required target category;

[0013] S4. Secondary application pre-annotation model:

[0014] When the feature point trajectory of the suspected target category overlaps with the feature point trajectory of the required target category, it is also necessary to use the image feature technology to extract the features of the image area where the feature point trajectory is located through the target detection pre-labeling model for auxiliary analysis. Only when all features are consistent with the features of the required target category can the suspected target category be officially judged to be the required target category.

[0015] Furthermore, in step S1, detecting and deleting identical images includes the following operations:

[0016] Read the image file and convert the image into binary data, calculate the hash value of the image using the MD5 hash algorithm, the MD5 hash algorithm converts the binary data of the image into a 128-bit hash value, and stores the hash value of each image in a database or file in a table form;

[0017] Then the images are compared. If two hash values ​​in the hash value table are exactly the same, it means that the images corresponding to the two hash values ​​are exactly the same, so either image is deleted.

[0018] Furthermore, in the step S1, when detecting and deleting visually similar images, an image similarity detection tool is used to find and process visually similar images, and the image similarity detection tool includes phash.

[0019] Furthermore, in the step S1, in the image data set after data cleaning, images containing target categories are divided into two categories, namely, a category in which the target category is completely displayed in the image, and a category in which the target category is blocked by other objects in the image.

[0020] Furthermore, in step S2, the number of feature points in the image region found by using Harris corner point detection is not 1.

[0021] Furthermore, in step S3, the feature points are connected in sequence from one direction, including but not limited to from left to right or from top to bottom.

[0022] Furthermore, in step S3, when the feature point trajectory of the suspected target category is compared with the feature point trajectory of each part of the target category, the specific part of the required target category corresponding to the feature point trajectory shape of the suspected target category is first determined based on the feature point trajectory shape of the suspected target category and the name of the required target category, and then the feature point trajectory shape of the suspected target category only needs to be compared with the feature point trajectory of the corresponding part of the required target category.

[0023] Furthermore, in step S3, a database specifically used to store the corresponding feature point trajectories is provided based on the required target categories.

[0024] Furthermore, the target detection pre-labeling method that facilitates data screening is suitable for detecting and pre-labeling target categories with fixed single shapes in images.

[0025] The present invention provides a target detection pre-labeling method that facilitates data screening, which has the following beneficial effects:

[0026] 1. The target detection pre-labeling method that facilitates data screening is based on the target detection pre-labeling model. The first detection operation cannot detect the image of the target category, so as to quickly screen out the images with higher processing difficulty. After finding the area of ​​the suspected target category in this type of image, Harris corner detection is used to find the feature points and connect them. The feature point trajectory formed after connecting is rotated and scaled to compare the complete feature point trajectory of the required target category. The image is subjected to secondary detection based on the comparison and overlap. When the extracted image features are consistent with the required target category, the suspected target category is the required target category. In this way, this type of image data with higher processing difficulty can be quickly and automatically processed and pre-labeled, so that manual labeling is not required. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 The specific operation flow of the target detection pre-labeling method for facilitating data screening of the present invention is schematically shown. DETAILED DESCRIPTION

[0028] The following embodiments of the present invention are described in further detail in conjunction with the accompanying drawings and examples. The following examples are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0029] like Figure 1 As shown, the present invention provides a technical solution: a target detection pre-labeling method that facilitates data screening, the target detection pre-labeling method that facilitates data screening includes the following steps:

[0030] S1. Data cleaning:

[0031] Clean the image dataset that is pre-labeled for target detection, use scripts to automatically open each image file, clear files that cannot be opened, non-image files, and damaged images, and then identify and delete images with resolutions lower than or higher than a preset value, as well as images with blur or noise exceeding a preset value based on the opened image files, and then detect and delete identical and visually similar images;

[0032] Among them, detecting and deleting the same image includes the following operations:

[0033] Read the image file and convert the image into binary data, calculate the hash value of the image using the MD5 hash algorithm, the MD5 hash algorithm converts the binary data of the image into a 128-bit hash value, and stores the hash value of each image in a database or file in a table form;

[0034] Specifically, first, pad the original data (binary data) so that its length (in bits) is modulo 512 with a remainder of 448, that is, ensure that the length of the data reaches a length that is 64 bits less than a multiple of 512. The padding always starts with 1 bit, followed by 0 bits, until the above length condition is met. A 64-bit representation of the original data length is appended to the padded message. At this time, the data length should be an integer multiple of 512.

[0035] Initialize 4 32-bit registers (usually denoted as A, B, C, and D) for calculating intermediate and final hash values, divide the padded and length-appended message into 512-bit blocks, and further divide each block into 16 32-bit sub-blocks. Process each 512-bit block. Each processing includes four rounds, and each round has 16 operations. These operations involve bit operations (including AND, OR, NOT, XOR) and addition, as well as operations using a fixed set of constants and circular left shifts.

[0036] Finally, the contents of the four registers A, B, C, and D are concatenated (low-order bits are output first) to form a 128-bit hash value. The function used in each round of operation is different, as follows:

[0037] The first round uses the F function: (F(B,C,D) = (B\land C)\lor(\neg B\land D))

[0038] The second round uses the G function: (G(B,C,D) = (B\land D)\lor(C\land\neg D))

[0039] The third round uses the H function: (H(B,C,D)=B\oplus C\oplus D)

[0040] The fourth round uses the I function: (I(B,C,D)=C\oplus(B\lor\neg D))

[0041] At each step, the contents of a sub-block, a constant, and possibly left-rotate the result are also added;

[0042] After that, the images are compared. If two hash values ​​in the hash value table are exactly the same, it means that the images corresponding to the two hash values ​​are exactly the same, so either image is deleted.

[0043] When detecting and deleting visually similar images, image similarity detection tools are used to find and process visually similar images. Image similarity detection tools include phash;

[0044] In the image dataset after data cleaning, images containing target categories are divided into two categories: one in which the target category is fully displayed in the image, and one in which the target category is blocked by other objects in the image. The former is easier to identify and pre-label because the target category is fully displayed, while the latter is incomplete because the target category is blocked, especially when a large area is blocked, which makes detection and pre-labeling more difficult;

[0045] S2. Apply the pre-annotated model:

[0046] Based on the image dataset after data cleaning, the trained target detection pre-annotation model is used to pre-annotate it. The pre-annotation result includes the bounding box and category information of the target. The target detection pre-annotation model uses image feature extraction technology to extract the target category in the image. For a type of image data where the target category is blocked by other objects in the image, edge detection is first performed to find the image area suspected of the target category, that is, the edge detection technology is used to find the suspected area in the image that is close to the edge contour of the required target category, and then Harris corner point detection is performed to find the significant feature points in the image area;

[0047] The number of feature points in the image area found by Harris corner detection is not 1;

[0048] S3. Processing of suspected target categories:

[0049] Based on the feature points found in the image area where the suspected target category is located, the feature points on the plane are connected to form a feature point trajectory, and the feature point trajectory of the suspected target category is three-dimensionally rotated and proportionally scaled. During the rotation and scaling process, the feature point trajectory is compared with the feature point trajectory of each part of the target category. If there is overlap, the suspected target category is preliminarily determined to be the required target category;

[0050] A database is provided based on the required target category to store the corresponding feature point trajectories;

[0051] When connecting the feature points, they are connected in sequence from one direction, including but not limited to from left to right or from top to bottom. When the feature point trajectory of the suspected target category is compared with the feature point trajectory of each part of the target category, the specific part of the required target category corresponding to the feature point trajectory shape of the suspected target category is first determined based on the feature point trajectory shape of the suspected target category and the name of the required target category, and then the feature point trajectory shape of the suspected target category only needs to be compared with the feature point trajectory of the corresponding part of the required target category;

[0052] S4. Secondary application pre-annotation model:

[0053] When the feature point trajectory of the suspected target category overlaps with the feature point trajectory of the required target category, it is also necessary to use the image feature technology to extract the features of the image area where the feature point trajectory is located through the target detection pre-labeling model for auxiliary analysis. Only when all features are consistent with the features of the required target category can the suspected target category be officially judged to be the required target category.

[0054] This object detection pre-labeling method that facilitates data screening is suitable for detecting and pre-labeling object categories with fixed single shapes in images.

[0055] In summary, when the target detection pre-labeling method that facilitates data screening is used, the image data set is first pre-processed to remove useless images, and the remaining image data are divided into a category with lower processing difficulty and a category with higher processing difficulty. Among them, the category with higher processing difficulty has target categories that are blocked and are difficult to detect when pre-labeling is performed through the target detection pre-labeling model. At this time, the area of ​​the suspected target category is first found through edge detection, and then the feature points are found by Harris corner point detection. The feature point trajectory formed by the feature point connection line is rotated and scaled to compare the complete feature point trajectory of the required target category, and the image data of this type is pre-labeled with secondary detection. On the basis of consistent image feature extraction results and correct trajectory comparison, this type of image data with higher processing difficulty can be quickly and automatically processed and pre-labeled, so that manual labeling is not required;

[0056] Therefore, based on the target detection pre-labeling model, the first detection operation cannot detect the target category of the image, so as to quickly screen out the images with higher processing difficulty. After finding the area of ​​the suspected target category in this type of image, Harris corner detection is used to find the feature points and connect them. The feature point trajectory formed after connecting the lines is rotated and scaled to compare the complete feature point trajectory of the required target category. The image is subjected to a second detection based on the overlap. When the extracted image features are consistent with the required target category, the suspected target category is the required target category, so that the target category of this type of image with higher processing difficulty is framed and pre-labeled.

[0057] The embodiments of the present invention are given for the purpose of illustration and description, and are not intended to be exhaustive or to limit the invention to the disclosed forms. Many modifications and variations will be apparent to those of ordinary skill in the art. The embodiments are selected and described in order to better illustrate the principles and practical applications of the present invention and to enable those of ordinary skill in the art to understand the present invention and thereby design various embodiments with various modifications suitable for specific uses.

Claims

1. A target detection pre-labeling method for convenient data screening, characterized by: The target detection pre-labeling method for convenient data screening comprises the following steps: S1. Data cleaning: Clean the image dataset that is pre-labeled for target detection, use scripts to automatically open each image file, clear files that cannot be opened, non-image files, and damaged images, and then identify and delete images with resolutions lower than or higher than a preset value, as well as images with blur or noise exceeding a preset value based on the opened image files, and then detect and delete identical and visually similar images; S2. Apply the pre-annotated model: Based on the image dataset after data cleaning, the trained target detection pre-annotation model is used to pre-annotate it. The pre-annotation results include the target's bounding box and category information. The target detection pre-annotation model uses image feature extraction technology to extract the target category in the image. For a type of image data where the target category is blocked by other objects in the image, edge detection is first performed to find the image area of ​​the suspected target category, and then Harris corner detection is performed to find the significant feature points in the image area; S3. Processing of suspected target categories: Based on the feature points found in the image area where the suspected target category is located, the feature points on the plane are connected to form a feature point trajectory, and the feature point trajectory of the suspected target category is three-dimensionally rotated and proportionally scaled. During the rotation and scaling process, the feature point trajectory is compared with the feature point trajectory of each part of the target category. If there is overlap, the suspected target category is preliminarily determined to be the required target category; S4. Secondary application pre-annotation model: When the feature point trajectory of the suspected target category coincides with the feature point trajectory of the required target category, it is also necessary to extract the features of the image area where the feature point trajectory is located using the image feature technology through the target detection pre-annotation model for auxiliary analysis. Only when all features are consistent with the features of the required target category can the suspected target category be officially determined to be the required target category. In step S1, detecting and deleting the same image includes the following operations: Read the image file and convert the image into binary data, calculate the hash value of the image using the MD5 hash algorithm, the MD5 hash algorithm converts the binary data of the image into a 128-bit hash value, and stores the hash value of each image in a database or file in a table form; Then the images are compared. If two hash values ​​in the hash value table are exactly the same, it means that the images corresponding to the two hash values ​​are exactly the same, so either image is deleted. In step S3, when comparing the feature point trajectory of the suspected target category with the feature point trajectory of each part of the target category, firstly determine the specific part of the required target category corresponding to the feature point trajectory shape of the suspected target category based on the feature point trajectory shape of the suspected target category and the name of the required target category, and then the feature point trajectory shape of the suspected target category only needs to be compared with the feature point trajectory of the corresponding part of the required target category; The target detection pre-labeling method that facilitates data screening is suitable for detecting and pre-labeling target categories with fixed single shapes in images.

2. The target detection pre-labeling method for convenient data screening according to claim 1, characterized in that: In the step S1, when detecting and deleting visually similar images, an image similarity detection tool is used to find and process visually similar images, and the image similarity detection tool includes phash.

3. The target detection pre-labeling method for convenient data screening according to claim 1, characterized in that: In the step S1, in the image data set after data cleaning, images containing target categories are divided into two categories, namely, a category in which the target category is completely displayed in the image, and a category in which the target category is blocked by other objects in the image.

4. The target detection pre-labeling method for convenient data screening according to claim 1, characterized in that: In step S2, the number of feature points in the image region found by Harris corner point detection is not 1.

5. The target detection pre-labeling method for convenient data screening according to claim 1, characterized in that: In step S3, the feature points are connected in sequence from one direction, including but not limited to from left to right or from top to bottom.

6. The target detection pre-labeling method for convenient data screening according to claim 1, characterized in that: In the step S3, a database specifically used to store the corresponding feature point trajectories is set based on the required target category.

Images