Automatic quality inspection method for annotation information, electronic equipment and storage medium

Abstract

The invention belongs to the technical field of machine learning, and discloses an automatic quality inspection method for annotation information, electronic equipment and a storage medium, and the method comprises the steps: obtaining annotation information on a to-be-trained image; wherein the annotation information comprises coordinates of each annotation frame; the to-be-trained image and the corresponding labeling information are input into a trained large model, the trained large model is used for extracting the feature information of the to-be-trained image to obtain a feature map of the to-be-trained image, and according to the coordinates of all the labeling boxes, the feature map is cut in an equal proportion to obtain picture features corresponding to all the labeling boxes; and carrying out clustering analysis on the extracted picture features corresponding to each labeling box to identify abnormal labels and realize automatic quality inspection of the labeling information. According to the method, the quality inspection efficiency and accuracy of the annotation information can be improved, the dependence on manual auditing can be reduced, and a large amount of labor cost and time cost are saved.

Description

Technical Field

[0001] The present invention belongs to the technical field of machine learning, and more specifically, relates to an automatic quality inspection method for labeled information, an electronic device, and a storage medium. Background Art

[0002] In the field of machine learning and artificial intelligence, the quality of data annotation directly affects the performance of the model. Therefore, it is crucial to obtain accurate and consistent data annotations.

[0003] Traditional data annotation is mostly done manually, and each annotation is manually checked to ensure accuracy and consistency. However, manual quality inspection of annotation information takes too long and may result in omissions. Summary of the invention

[0004] In view of the above defects or improvement needs of the prior art, the present invention provides an automatic quality inspection method for annotation information, an electronic device and a storage medium, which aims to improve the quality inspection efficiency and accuracy of annotation information.

[0005] To achieve the above object, according to a first aspect of the present invention, there is provided an automatic quality inspection method for annotation information, comprising:

[0006] Obtaining annotation information on the image to be trained; wherein the annotation information includes coordinates of the annotation box;

[0007] Input the image to be trained and its corresponding annotation information into the trained large model, the trained large model is used to extract the feature information of the image to be trained, obtain the feature map of the image to be trained, and crop the feature map in equal proportion according to the coordinates of each annotation box to obtain the image features corresponding to each annotation box; wherein the image features corresponding to each annotation box are used to reflect the label of the image in the annotation box;

[0008] Cluster analysis is performed on the image features corresponding to each extracted annotation box to identify abnormal labels and realize automatic quality inspection of annotation information.

[0009] Furthermore, cluster analysis is performed on the image features corresponding to each annotation box to identify abnormal labels, including:

[0010] S1. Randomly select K image features from the image features corresponding to each annotation box as the initial clustering centers of K clusters;

[0011] S2, calculating the first Euclidean distance between each image feature and the current cluster center, and assigning each image feature to the cluster to which the current cluster center is closest according to the first Euclidean distance, to obtain K new clusters;

[0012] S3, calculating the mean of all image features in each new cluster, and taking the mean as the current cluster center, and jumping to S2, until the update distance of the current cluster center is less than a preset threshold, and obtaining K new clusters after convergence;

[0013] S4. Calculate the second Euclidean distance between each image feature and the target cluster center of the cluster to which it belongs, and the image feature whose second Euclidean distance is greater than the preset distance is an abnormal label.

[0014] Furthermore, it also includes: taking the image features whose second Euclidean distance is less than or equal to a preset distance as normal labels.

[0015] Furthermore, the annotation information also includes labels corresponding to each annotation box;

[0016] The trained large model is used to extract feature information of the image to be trained, including: classifying each annotation frame according to the label in the annotation information to obtain the annotation frames with the same label; and extracting the image features corresponding to the annotation frames with the same label of the image to be trained using the trained large model;

[0017] Before clustering analysis is performed on the extracted image features corresponding to each annotation box, the method further includes: taking the image features corresponding to the annotation boxes with the same label as data of a category label to construct a database corresponding to different category labels;

[0018] When cluster analysis is performed on the extracted image features corresponding to each annotation box, the image features come from the database.

[0019] Furthermore, after identifying the abnormal label, it also includes: displaying the abnormal label, modifying the abnormal label, and / or deleting the abnormal label;

[0020] Among them, displaying the abnormal label includes: bolding the abnormal label, popping up prompt information, changing the label color or one or more thereof; modifying the abnormal label includes: modifying the abnormal label to a normal label.

[0021] Furthermore, the trained large model is a DINOv2 model.

[0022] According to a second aspect of the present invention, there is provided an electronic device comprising a computer-readable storage medium and a processor;

[0023] The computer-readable storage medium is used to store executable instructions;

[0024] The processor is used to read the executable instructions stored in the computer-readable storage medium to execute the automatic quality inspection method for labeling information described in any one of the first aspects.

[0025] According to a third aspect of the present invention, there is provided a computer-readable storage medium having a computer program stored thereon, wherein when the program is executed by a processor, the automatic quality inspection method for labeling information as described in any one of the first aspects is implemented.

[0026] According to a fourth aspect of the present invention, a computer program product is provided. When the computer program product is run on a computer, the computer executes the automatic quality inspection method for annotation information as described in any one of the first aspects.

[0027] In general, the above technical solutions conceived by the present invention can achieve the following beneficial effects:

[0028] (1) The automatic quality inspection method of the annotation information in the present invention obtains the annotation information on the image to be trained, inputs the image to be trained and its corresponding annotation information into the trained large model to obtain the image features corresponding to each annotation box, and clusters based on the image features corresponding to the annotation box using a cluster analysis algorithm to identify abnormal labels and realize automatic quality inspection of the annotation information. The method of the present invention does not require manual quality inspection of each label one by one, thereby improving the quality inspection efficiency of the annotation information, and uses the trained large model to automatically identify the image features in each annotation box without additional model training. At a relatively low cost, it can achieve the recognition ability of fine-grained objects. Using this advanced large model technology, it can more accurately identify annotation errors and inconsistencies, reduce human omissions, and enhance quality inspection accuracy. In addition, it can also reduce the reliance on manual review, saving a lot of manpower and time costs.

[0029] (2) As a preference, when identifying abnormal data points, a K-means clustering algorithm is used to generate clusters and outliers for multiple image features. Each new cluster represents features of the same category and belongs to normal data points. Outliers correspond to abnormal data points and are usually far away from normal data. Based on this, abnormal labels are screened out. The algorithm is simple in concept and has high accuracy.

[0030] (3) Furthermore, the labels of the annotation boxes can be obtained in advance, and the annotation boxes can be classified according to the labels to obtain the annotation boxes with the same labels, and the image features corresponding to the annotation boxes with the same labels are used as data of a category label to build a database corresponding to different category labels. When performing cluster analysis on the image features corresponding to each annotation box, the image features corresponding to each annotation box can be directly obtained from the database, thereby reducing the amount of calculation in the quality inspection process and further improving the quality inspection efficiency.

[0031] (4) As a preferred option, the trained large model is the DINOv2 model, which can learn rich and robust visual features and further improve the accuracy of fine-grained object feature recognition.

[0032] In summary, the present invention significantly reduces the time required for manual quality inspection through automated processing and cluster analysis, speeds up the data processing process, and improves quality inspection efficiency; and, by utilizing advanced large model technology, it can more accurately identify labeling errors and inconsistencies, reduce human omissions and enhance quality inspection accuracy; through standardized automated quality inspection processes, it ensures the consistency of data labeling and improves the learning effect of subsequent algorithms; and, it can also reduce reliance on manual review, saving a lot of manpower and time costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 The figure is a flow chart of an automatic quality inspection method for annotation information in an embodiment of the present invention.

[0034] Figure 2 is a schematic block diagram of an electronic device in an embodiment of the present invention. DETAILED DESCRIPTION

[0035] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0036] In the present invention, the terms "first", "second", etc. in the present invention and the accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0037] Embodiment 1

[0038] like Figure 1 As shown, an embodiment of the present invention provides an automatic quality inspection method for annotation information, which mainly includes:

[0039] Obtaining annotation information on the image to be trained, including the coordinates of the annotation box;

[0040] The image to be trained and its corresponding annotation information are input into the trained large model. The trained large model is used to extract the feature information of the image to be trained to obtain a feature map of the image to be trained, and the feature map is proportionally cropped according to the coordinates of each annotation box to obtain the image features corresponding to each annotation box; wherein the image features corresponding to each annotation box are used to reflect the label of the image; in the embodiment of the present invention, the feature information of the image to be trained includes the coordinates of the image to be trained.

[0041] The clustering analysis algorithm is used to perform cluster analysis on the image features corresponding to each extracted annotation box to identify abnormal labels and realize automatic quality inspection of the annotation information.

[0042] In the embodiment of the present invention, the annotation box refers to a rectangular area used to locate the target in the image. The annotation box is usually used in the target detection task to indicate the position of the target identified by the model in the image. The annotation box is usually represented by four coordinates, namely, the horizontal and vertical coordinates of the upper left corner and the horizontal and vertical coordinates of the lower right corner. These four coordinates define the position of the annotation box, so that the machine learning model can identify and locate specific objects in the image.

[0043] Labels are information attached to data samples, providing descriptive information for the data and telling the model which categories the data samples belong to or have certain characteristics.

[0044] In some embodiments, the large model is a semantic segmentation model. In an embodiment of the present invention, the trained large model is a DINOv2 model. DINOv2 (Learning Robust Visual Features without Supervision) can learn rich and robust visual features and has the following characteristics: 1. No supervised learning required: DINOv2 continues to use a learning method that does not require manual labels, which is particularly important for reducing dependence on large-scale annotated data sets. 2. Improved training strategy: Compared with the original DINO, DINOv2 has optimized the training strategy to improve learning efficiency and the generalization ability of the model. 3. More robust visual features: DINOv2 focuses on generating more robust visual features, which enables the model to perform better on a variety of visual tasks. 4. Application of visual Transformer: Continue to use visual Transformer as the infrastructure and use its powerful representation capabilities to capture complex patterns in images.

[0045] In the embodiment of the present invention, the cluster analysis algorithm is a K-means clustering algorithm. The idea of ​​the K-Means algorithm is simple. For a given sample set, the Euclidean distance is used as an indicator to measure the similarity between data objects. The similarity is inversely proportional to the distance between data objects. The greater the similarity, the smaller the distance. The algorithm is simple and has high accuracy. In other embodiments, a hierarchical clustering algorithm, a DBSCAN clustering algorithm, a mean shift clustering algorithm, a spectral clustering algorithm, etc. may also be used.

[0046] Specifically, in the embodiment of the present invention, a K-means clustering algorithm is used to perform cluster analysis on the extracted image features corresponding to each annotation box to identify abnormal labels, including:

[0047] S1. Randomly select K image features from the image features corresponding to each annotation box as the initial clustering centers of K clusters;

[0048] S2. Calculate the first Euclidean distance between each image feature and the current cluster center, and assign each image feature to the cluster to which the current cluster center is closest according to the first Euclidean distance, to obtain K new clusters.

[0049] S3, calculate the mean of all image features in each new cluster, and use the mean as the current cluster center, and jump to S2 until the update distance of the current cluster center is less than the preset threshold, and obtain K new clusters after convergence;

[0050] S4, calculating the second Euclidean distance between each image feature and the target cluster center of the cluster to which it belongs, and the image feature whose second Euclidean distance is greater than the preset distance is an abnormal label. The preset threshold and the preset distance are selected according to the actual situation.

[0051] In the embodiment of the present invention, when identifying abnormal data points, the K-means clustering algorithm is used to generate clusters and outliers for multiple image features. Outliers are points that are far away from the target cluster center. In addition, each new cluster represents features of the same category and belongs to normal data points; outliers correspond to abnormal data points, which are usually far away from normal data. Based on this, abnormal labels are screened out.

[0052] In some embodiments, the method further includes: taking the image features whose second Euclidean distance is less than or equal to the preset distance as normal labels.

[0053] In some embodiments, the labels corresponding to each annotation box can be obtained in advance, and each annotation box can be classified according to the labels in the annotation information to obtain the annotation boxes with the same labels. The image features corresponding to the annotation boxes with the same labels are obtained based on the large model, and the image features corresponding to the annotation boxes with the same labels are used as data of a category label to build a database corresponding to different category labels; for example, if the labels include "cat", "dog", "car", etc., the annotation boxes corresponding to the labels of "cat" are determined as the annotation boxes with the same labels, and the image features corresponding to the annotation boxes can be used as data of the category label of "cat". When clustering analysis is performed on the image features corresponding to each annotation box using a clustering analysis algorithm, the image features corresponding to each annotation box can be directly from the database to reduce the amount of calculation in the quality inspection process and further improve the quality inspection efficiency. And the image features obtained by the large model are directly used to construct the database, which greatly reduces the complexity of database construction.

[0054] In some embodiments, after identifying abnormal labels, it also includes generating modification suggestions for abnormal labels, displaying abnormal labels, modifying abnormal labels, etc. Among them, displaying abnormal labels specifically includes: bolding abnormal labels, popping up prompt information, changing the annotation color or one or more of them, so as to prompt the annotator, so that the annotator can intuitively see which annotation boxes and labels are abnormal.

[0055] Modifying the abnormal label includes: modifying the abnormal label to a normal label, thereby completing the modification operation after quality inspection. The modification can be performed by electronic equipment or by a labeler.

[0056] In some embodiments, if the labeler finds that the identified abnormal label is a normal label, the labeler can also cancel the identified abnormal label and convert the identified abnormal label into a normal label.

[0057] In some embodiments, the method further includes deleting the identified abnormal labels, or deleting the data points corresponding to the identified abnormal labels.

[0058] The automatic quality inspection method of the annotation information in the present invention obtains the annotation information on the image to be trained, inputs the image to be trained and its corresponding annotation information into the trained large model to obtain the image features corresponding to each annotation box, and clusters based on the image features corresponding to the annotation box using a cluster analysis algorithm to identify abnormal labels and realize automatic quality inspection of the annotation information. The method of the present invention does not require manual quality inspection of each label one by one, thereby improving the quality inspection efficiency of the annotation information, and uses the trained large model to automatically identify the image features in each annotation box without the need for additional model training. At a relatively low cost, it is possible to achieve the recognition capability of fine-grained objects. Using this advanced large model technology, it is possible to more accurately identify annotation errors and inconsistencies, reduce human omissions, and enhance quality inspection accuracy. Moreover, it can also reduce the reliance on manual review, saving a lot of manpower and time costs.

[0059] Embodiment 2

[0060] like Figure 2 As shown, an electronic device 1 is provided in an embodiment of the present invention, comprising: one or more processors 11; a memory 10, for storing one or more computer programs, and the one or more processors 11 are used to execute the one or more computer programs stored in the memory 10, so that the one or more processors 11 execute the automatic quality inspection method of the labeling information as in the embodiment.

[0061] The electronic device 1 may be a series of electronic devices such as a smart phone, a tablet computer, a wearable electronic device, a smart home electronic product, and an industrial device.

[0062] For the specific implementation method, please refer to the corresponding detailed description in Example 1, which will not be repeated here.

[0063] The electronic device 1 of the embodiment of the present invention significantly reduces the time required for manual quality inspection through automated processing and cluster analysis, speeds up the data processing process, and improves quality inspection efficiency; and, by utilizing advanced large model technology, can more accurately identify labeling errors and inconsistencies, reduce human omissions and enhance quality inspection accuracy; through standardized automated quality inspection processes, ensures the consistency of data labeling and improves the learning effect of subsequent algorithms; and, the present invention can also reduce reliance on manual review, saving a lot of manpower and time costs.

[0064] Embodiment 3

[0065] An embodiment of the present invention provides a computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are executed by an electronic device, the electronic device executes the automatic quality inspection method for labeling information in the embodiment.

[0066] For the related technical solutions, please refer to the corresponding description in Embodiment 1, which will not be repeated here.

[0067] Embodiment 4

[0068] An embodiment of the present invention provides a computer program product. When the computer program product runs on a computer, the computer executes the automatic quality inspection method for labeling information in the first embodiment.

[0069] For the related technical solutions, please refer to the corresponding description in Embodiment 1, which will not be repeated here.

[0070] Those skilled in the art can understand that all or part of the features / steps of the above-mentioned method embodiments can be implemented by methods, data processing systems or computer programs, and these features can be implemented without hardware, entirely by software, or by a combination of hardware and software. The aforementioned computer program can be stored in one or more computer-readable storage media, and the storage medium stores the computer program. When the computer program is executed (such as by a processor), the steps of the automatic quality inspection method embodiment including the above-mentioned annotation information are executed.

[0071] The aforementioned storage media that can store program codes include: static hard disks, solid-state hard disks, random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), optical storage devices, magnetic storage devices, flash memory, magnetic disks or optical disks and / or combinations of the above devices, that is, it can be implemented by any type of volatile or non-volatile storage device or a combination thereof.

[0072] It will be easily understood by those skilled in the art that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. An automatic quality inspection method for labeling information, characterized in that: include: Obtaining annotation information on the image to be trained; wherein the annotation information includes coordinates of the annotation box; Input the image to be trained and its corresponding annotation information into the trained large model, the trained large model is used to extract the feature information of the image to be trained, obtain the feature map of the image to be trained, and crop the feature map in equal proportion according to the coordinates of each annotation box to obtain the image features corresponding to each annotation box; wherein the image features corresponding to each annotation box are used to reflect the label of the image in the annotation box; Cluster analysis is performed on the image features corresponding to each extracted annotation box to identify abnormal labels and realize automatic quality inspection of annotation information.

2. The automatic quality inspection method for labeling information according to claim 1, characterized in that: Perform cluster analysis on the image features corresponding to each annotation box to identify abnormal labels, including: S1. Randomly select K image features from the image features corresponding to each annotation box as the initial clustering centers of K clusters; S2, calculating the first Euclidean distance between each image feature and the current cluster center, and assigning each image feature to the cluster to which the current cluster center is closest according to the first Euclidean distance, to obtain K new clusters; S3, calculating the mean of all image features in each new cluster, and taking the mean as the current cluster center, and jumping to S2, until the update distance of the current cluster center is less than a preset threshold, and obtaining K new clusters after convergence; S4. Calculate the second Euclidean distance between each image feature and the target cluster center of the cluster to which it belongs, and the image feature whose second Euclidean distance is greater than the preset distance is an abnormal label.

3. The automatic quality inspection method for labeling information according to claim 2, characterized in that: Also includes: The image features whose second Euclidean distance is less than or equal to the preset distance are taken as normal labels.

4. The automatic quality inspection method for labeling information according to claim 1, characterized in that: The annotation information also includes a label corresponding to each annotation frame; using the trained large model to extract feature information of the image to be trained, including: classifying each annotation frame according to the label in the annotation information to obtain annotation frames with the same label; using the trained large model to extract image features corresponding to the annotation frames with the same label of the image to be trained; Before clustering analysis is performed on the extracted image features corresponding to each annotation box, the method further includes: taking the image features corresponding to the annotation boxes with the same label as data of a category label to construct a database corresponding to different category labels; When cluster analysis is performed on the extracted image features corresponding to each annotation box, the image features come from the database.

5. The automatic quality inspection method for labeling information according to claim 1, characterized in that: After the abnormal label is identified, the method further includes: displaying the abnormal label, modifying the abnormal label, and / or deleting the abnormal label; Among them, displaying the abnormal label includes: bolding the abnormal label, popping up prompt information, changing the label color or one or more thereof; modifying the abnormal label includes: modifying the abnormal label to a normal label.

6. The automatic quality inspection method for labeling information according to claim 1, characterized in that: The trained large model is the DINOv2 model.

7. An electronic device, characterized in that: comprising a computer readable storage medium and a processor; The computer-readable storage medium is used to store executable instructions; The processor is used to read the executable instructions stored in the computer-readable storage medium to execute the automatic quality inspection method for labeling information according to any one of claims 1 to 6.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, an automatic quality inspection method for annotation information according to any one of claims 1 to 6 is implemented.

9. A computer program product, characterized in that When the computer program product runs on a computer, the computer is enabled to execute the automatic quality inspection method for annotation information according to any one of claims 1 to 6.

Images