An automatic quality inspection method for labeling information, an electronic device, and a storage medium

Abstract

The application belongs to the technical field of machine learning, and discloses an automatic quality inspection method for labeling information, an electronic device and a storage medium, which comprises the following steps: obtaining labeling information on a to-be-trained image; wherein the labeling information comprises coordinates of each labeling frame; inputting the to-be-trained image and the corresponding labeling information into a well-trained large model, wherein the well-trained large model is used to extract feature information of the to-be-trained image, obtain a feature map of the to-be-trained image, and obtain picture features corresponding to each labeling frame by equally cutting the feature map according to the coordinates of each labeling frame; and performing clustering analysis on the extracted picture features corresponding to each labeling frame to identify abnormal labels and realize automatic quality inspection of the labeling information. The application can improve the quality inspection efficiency and accuracy of the labeling information, and can also reduce the dependence on manual review, thereby saving a large amount of labor cost and time cost.

Description

Technical Field

[0001] This invention belongs to the field of machine learning technology, and more specifically, relates to an automatic quality inspection method, electronic device, and storage medium for labeled information. Background Technology

[0002] In the fields of machine learning and artificial intelligence, the quality of data labeling directly affects model performance. Therefore, obtaining accurate and consistent data labeling is crucial.

[0003] Traditional data annotation is mostly done manually, with each annotation undergoing quality control to ensure accuracy and consistency. However, manual quality control of annotation information is time-consuming and prone to oversights. Summary of the Invention

[0004] In view of the above-mentioned defects or improvement needs of the prior art, the present invention provides an automatic quality inspection method, electronic device and storage medium for labeled information, the purpose of which is to improve the quality inspection efficiency and accuracy of labeled information.

[0005] To achieve the above objectives, according to a first aspect of the present invention, an automatic quality inspection method for labeled information is provided, comprising:

[0006] Obtain annotation information on the image to be trained; wherein, the annotation information includes the coordinates of the bounding box;

[0007] The image to be trained and its corresponding annotation information are input into a pre-trained large model. The pre-trained large model is used to extract the feature information of the image to be trained to obtain the feature map of the image to be trained. The feature map is then cropped proportionally according to the coordinates of each annotation box to obtain the image features corresponding to each annotation box. The image features corresponding to each annotation box are used to reflect the label of the image within the annotation box.

[0008] Cluster analysis is performed on the image features corresponding to each extracted annotation box to identify abnormal labels and achieve automatic quality inspection of annotation information.

[0009] Furthermore, cluster analysis is performed on the image features corresponding to each extracted bounding box to identify anomalous labels, including:

[0010] S1. Randomly select K image features from the image features corresponding to each labeled box as the initial cluster centers of K clusters;

[0011] S2. Calculate the first Euclidean distance between each image feature and the current cluster center, and assign each image feature to the cluster to which the nearest current cluster center belongs based on the first Euclidean distance, to obtain K new clusters;

[0012] S3. Calculate the mean of all image features in each new cluster, and use the mean as the current cluster center. Then jump to S2 until the update distance of the current cluster center is less than a preset threshold, and obtain K new clusters after convergence.

[0013] S4. Calculate the second Euclidean distance between each image feature and the target cluster center of its respective cluster. Image features whose second Euclidean distance is greater than a preset distance are identified as abnormal labels.

[0014] Furthermore, it also includes: using image features whose second Euclidean distance is less than or equal to a preset distance as normal labels.

[0015] Furthermore, the annotation information also includes the labels corresponding to each annotation box;

[0016] The method involves using a pre-trained large model to extract feature information from the image to be trained, including: classifying each bounding box according to the labels in the annotation information to obtain bounding boxes with the same label; and using the pre-trained large model to extract image features corresponding to the bounding boxes with the same label in the image to be trained.

[0017] Before performing cluster analysis on the image features corresponding to each extracted bounding box, the process also includes: taking the image features corresponding to bounding boxes with the same label as a single category label data to construct a database corresponding to different category labels;

[0018] When performing cluster analysis on the image features corresponding to each extracted bounding box, the image features are derived from the database.

[0019] Furthermore, after identifying the abnormal label, the process also includes: displaying the abnormal label, modifying the abnormal label, and / or deleting the abnormal label;

[0020] Displaying abnormal labels includes one or more of the following: bolding the abnormal label, displaying a pop-up message, and changing the label color; modifying abnormal labels includes changing the abnormal label to a normal label.

[0021] Furthermore, the trained large model is the DINOv2 model.

[0022] According to a second aspect of the present invention, an electronic device is provided, including a computer-readable storage medium and a processor;

[0023] The computer-readable storage medium is used to store executable instructions;

[0024] The processor is used to read executable instructions stored in the computer-readable storage medium and execute the automatic quality inspection method for the annotation information as described in any of the first aspects.

[0025] According to a third aspect of the present invention, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements an automatic quality inspection method for annotation information as described in any of the first aspects.

[0026] According to a fourth aspect of the present invention, a computer program product is provided, which, when run on a computer, causes the computer to perform an automatic quality inspection method for annotation information as described in any one of the first aspects.

[0027] In summary, the above-described technical solutions conceived in this invention can achieve the following beneficial effects:

[0028] (1) The automatic quality inspection method for annotation information in this invention obtains the annotation information on the image to be trained, inputs the image to be trained and its corresponding annotation information into a pre-trained large model to obtain the image features corresponding to each annotation box, and uses a clustering analysis algorithm to cluster based on the image features corresponding to the annotation boxes to identify abnormal labels, thereby achieving automatic quality inspection of annotation information. The method of this invention does not require manual quality inspection of each label one by one, improving the efficiency of annotation information quality inspection. Moreover, it uses a pre-trained large model to automatically identify the image features within each annotation box without additional model training. At a lower cost, it can achieve the ability to identify fine-grained objects. Using this advanced large model technology, it can more accurately identify annotation errors and inconsistencies, reduce human oversights, and enhance the accuracy of quality inspection. In addition, it can reduce the reliance on manual review, saving a lot of manpower and time costs.

[0029] (2) As a preferred method, when identifying abnormal data points, the K-means clustering algorithm is used to generate clusters and outliers for multiple image features. Each new cluster represents the same category of features and belongs to normal data points. Outliers correspond to abnormal data points and are usually far away from normal data. Based on this, abnormal labels are filtered out. The algorithm is simple to think about and has high accuracy.

[0030] (3) Furthermore, the labels of the bounding boxes can be obtained in advance, and the bounding boxes can be classified according to the labels to obtain bounding boxes with the same label. The image features corresponding to the bounding boxes with the same label are used as data for a category label to construct a database corresponding to different category labels. When performing cluster analysis on the image features corresponding to each bounding box, the image features corresponding to each bounding box can come directly from this database, thereby reducing the amount of computation in the quality inspection process and further improving the efficiency of quality inspection.

[0031] (4) As a preferred option, the trained large model is the DINOv2 model. DINOv2 can learn rich and robust visual features, further improving the accuracy of fine-grained object feature recognition.

[0032] In summary, this invention significantly reduces the time required for manual quality inspection through automated processing and cluster analysis, accelerates the data processing flow, and improves quality inspection efficiency. Furthermore, by utilizing advanced large-scale model technology, it can more accurately identify labeling errors and inconsistencies, reducing human oversights and enhancing quality inspection accuracy. Through a standardized automated quality inspection process, it ensures the consistency of data labeling, improving the learning effect of subsequent algorithms. Moreover, it can reduce reliance on manual review, saving significant human and time costs. Attached Figure Description

[0033] Figure 1 This is a flowchart of the automatic quality inspection method for annotation information in an embodiment of the present invention.

[0034] Figure 2 This is a schematic block diagram of an electronic device in an embodiment of the present invention. Detailed Implementation

[0035] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0036] In this invention, the terms "first," "second," etc., used in the invention and accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0037] Example 1

[0038] like Figure 1 As shown, this embodiment of the invention provides an automatic quality inspection method for labeled information, mainly including:

[0039] Obtain the annotation information on the image to be trained; including the coordinates of the annotation boxes;

[0040] The image to be trained and its corresponding annotation information are input into a pre-trained large model. The pre-trained large model is used to extract the feature information of the image to be trained, thereby obtaining a feature map of the image to be trained. The feature map is then cropped proportionally according to the coordinates of each annotation box to obtain the image features corresponding to each annotation box. The image features corresponding to each annotation box are used to reflect the label of the image. In this embodiment of the invention, the feature information of the image to be trained includes the coordinates of the image to be trained.

[0041] Cluster analysis algorithms are used to perform cluster analysis on the image features corresponding to each extracted annotation box in order to identify abnormal labels and achieve automatic quality inspection of annotation information.

[0042] In this embodiment of the invention, a bounding box refers to a rectangular region used to locate a target in an image. Bounding boxes are commonly used in object detection tasks to indicate the location of a target identified by the model within the image. A bounding box is typically represented by four coordinates: the horizontal and vertical coordinates of the upper left corner, and the horizontal and vertical coordinates of the lower right corner. These four coordinates define the position of the bounding box, enabling the machine learning model to identify and locate specific objects in the image.

[0043] Labels are information attached to data samples, providing descriptive information about the data and telling the model which categories the data samples belong to or what characteristics they have.

[0044] In some embodiments, the large model is a semantic segmentation model. In this embodiment of the invention, the trained large model is the DINOv2 model. DINOv2 (Learning Robust Visual Features without Supervision) can learn rich and robust visual features and has the following characteristics: 1. Unsupervised learning: DINOv2 continues to use a learning method that does not require manual labels, which is particularly important for reducing dependence on large-scale labeled datasets. 2. Improved training strategy: Compared with the original DINO, DINOv2 has optimized the training strategy to improve learning efficiency and the model's generalization ability. 3. More robust visual features: DINOv2 focuses on generating more robust visual features, which enables the model to perform better on a variety of visual tasks. 4. Application of visual Transformer: It continues to use the visual Transformer as the basic architecture, leveraging its powerful representation capabilities to capture complex patterns in images.

[0045] In this embodiment of the invention, the clustering analysis algorithm is the K-means clustering algorithm. The K-means algorithm is simple in concept: for a given sample set, it uses Euclidean distance as an indicator to measure the similarity between data objects. Similarity is inversely proportional to the distance between data objects; the greater the similarity, the smaller the distance. This algorithm is simple and has high accuracy. In other embodiments, hierarchical clustering algorithms, DBSCAN clustering, mean-shift clustering, spectral clustering, etc., can also be used.

[0046] Specifically, in this embodiment of the invention, the K-means clustering algorithm is used to perform clustering analysis on the image features corresponding to each extracted bounding box in order to identify abnormal labels, including:

[0047] S1. Randomly select K image features from the image features corresponding to each labeled box as the initial cluster centers of K clusters;

[0048] S2. Calculate the first Euclidean distance between each image feature and the current cluster center, and assign each image feature to the cluster to which the nearest current cluster center belongs based on the first Euclidean distance, to obtain K new clusters;

[0049] S3. Calculate the mean of all image features in each new cluster and use this mean as the current cluster center. Then jump to S2 until the update distance of the current cluster center is less than the preset threshold, and obtain K new clusters after convergence.

[0050] S4. Calculate the second Euclidean distance between each image feature and the target cluster center of its respective cluster. Image features with a second Euclidean distance greater than a preset distance are identified as anomaly labels. The preset threshold and preset distance are selected based on actual conditions.

[0051] In this embodiment of the invention, when identifying outlier data points, the K-means clustering algorithm is used to generate clusters and outliers for multiple image features. Outliers are points that are far from the target cluster center. Furthermore, each new cluster represents features of the same category and belongs to normal data points; outliers correspond to outlier data points, which are usually far from normal data. Based on this, outlier labels are filtered out.

[0052] In some embodiments, the method further includes: using image features whose second Euclidean distance is less than or equal to a preset distance as normal labels.

[0053] In some embodiments, the labels corresponding to each bounding box can be obtained in advance, and the bounding boxes can be classified according to the labels in the annotation information to obtain bounding boxes with the same label. The image features corresponding to the bounding boxes with the same label are obtained based on a large model, and these image features are used as data for a category label to construct a database corresponding to different category labels. For example, if the labels include "cat," "dog," and "car," then the bounding boxes corresponding to the label "cat" are determined as bounding boxes with the same label, and the image features corresponding to these bounding boxes can be used as data for the category label "cat." When using clustering analysis algorithms to perform clustering analysis on the image features corresponding to each bounding box, the image features corresponding to each bounding box can directly come from this database, reducing the computational load in the quality inspection process and further improving quality inspection efficiency. Furthermore, directly using the image features obtained from the large model for database construction greatly reduces the complexity of database construction.

[0054] In some embodiments, after identifying abnormal labels, the process further includes generating modification suggestions for the abnormal labels, displaying the abnormal labels, and modifying the abnormal labels. Displaying abnormal labels specifically includes one or more of the following: bolding the abnormal labels, displaying pop-up prompts, and changing the label color, to alert the annotator and allow them to visually see which annotation boxes and labels are abnormal.

[0055] Modifying abnormal labels includes changing them to normal labels, thus completing the post-quality inspection modification operation. This modification can be performed electronically or by the labeler.

[0056] In some embodiments, if the labeler finds that the identified abnormal label is a normal label, the labeler can also cancel the identified abnormal label and convert it into a normal label.

[0057] In some embodiments, the method further includes deleting the identified anomalous labels, or deleting the data points corresponding to the identified anomalous labels.

[0058] The automatic quality inspection method for annotation information in this invention obtains the annotation information on the image to be trained, inputs the image and its corresponding annotation information into a pre-trained large model to obtain the image features corresponding to each annotation box, and uses a clustering analysis algorithm to perform clustering based on the image features corresponding to the annotation boxes to identify abnormal labels, thus achieving automatic quality inspection of annotation information. This method eliminates the need for manual quality inspection of each label, improving the efficiency of annotation information quality inspection. Furthermore, it uses a pre-trained large model to automatically identify the image features within each annotation box, eliminating the need for additional model training. At a lower cost, it can achieve the ability to recognize fine-grained objects. Utilizing this advanced large model technology, it can more accurately identify annotation errors and inconsistencies, reduce human error, and enhance the accuracy of quality inspection. Moreover, it reduces reliance on manual review, saving significant labor and time costs.

[0059] Example 2

[0060] like Figure 2 As shown, an electronic device 1 is provided in this embodiment of the invention, including: one or more processors 11; a memory 10 for storing one or more computer programs, wherein the one or more processors 11 are used to execute the one or more computer programs stored in the memory 10 to cause the one or more processors 11 to perform an automatic quality inspection method for labeling information as described in the embodiment.

[0061] Electronic device 1 can be a range of electronic devices such as smartphones, tablets, wearable electronic devices, smart home electronic products, and industrial devices.

[0062] For the specific implementation method, please refer to the detailed description in Example 1, which will not be repeated here.

[0063] The electronic device 1 of this invention significantly reduces the time required for manual quality inspection through automated processing and cluster analysis, accelerates the data processing flow, and improves quality inspection efficiency. Furthermore, by utilizing advanced large model technology, it can more accurately identify labeling errors and inconsistencies, reducing human oversights and enhancing the accuracy of quality inspection. Through a standardized automated quality inspection process, it ensures the consistency of data labeling and improves the learning effect of subsequent algorithms. Moreover, this invention can reduce reliance on manual review, saving significant human and time costs.

[0064] Example 3

[0065] This invention provides a computer-readable storage medium storing computer-executable instructions. When the computer-executable instructions are executed by an electronic device, the electronic device performs an automatic quality inspection method for labeled information as described in the embodiments.

[0066] The relevant technical solutions are described in the corresponding embodiment 1, and will not be repeated here.

[0067] Example 4

[0068] This invention provides a computer program product that, when run on a computer, causes the computer to execute the automatic quality inspection method for annotation information in Embodiment 1.

[0069] The relevant technical solutions are described in the corresponding embodiment 1, and will not be repeated here.

[0070] Those skilled in the art will understand that all or part of the features / steps of the above-described method embodiments can be implemented by methods, data processing systems, or computer programs. These features may be implemented without hardware, entirely in software, or in a combination of hardware and software. The aforementioned computer program may be stored in one or more computer-readable storage media. When the computer program is executed (e.g., by a processor), it performs the steps of the automatic quality inspection method embodiments including the aforementioned annotation information.

[0071] The aforementioned storage media capable of storing program code include: static disks, solid-state drives, random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), optical storage devices, magnetic storage devices, flash memory, magnetic disks or optical disks, and / or combinations of the above devices, that is, they can be implemented by any type of volatile or non-volatile storage devices or combinations thereof.

[0072] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. An automatic quality inspection method for labeled information, characterized in that, include: Obtain annotation information on the image to be trained; wherein, the annotation information includes the coordinates of the bounding box; The image to be trained and its corresponding annotation information are input into a pre-trained large model. The pre-trained large model is used to extract the feature information of the image to be trained to obtain the feature map of the image to be trained. The feature map is then cropped proportionally according to the coordinates of each annotation box to obtain the image features corresponding to each annotation box. The image features corresponding to each annotation box are used to reflect the label of the image within the annotation box. Cluster analysis is performed on the image features corresponding to each extracted annotation box to identify abnormal labels and achieve automatic quality inspection of annotation information; Cluster analysis was performed on the image features corresponding to each extracted bounding box to identify anomalous labels, including: S1. Randomly select K image features from the image features corresponding to each labeled box as the initial cluster centers of K clusters; S2. Calculate the first Euclidean distance between each image feature and the current cluster center, and assign each image feature to the cluster to which the nearest current cluster center belongs based on the first Euclidean distance, to obtain K new clusters; S3. Calculate the mean of all image features in each new cluster, and use the mean as the current cluster center. Then jump to S2 until the update distance of the current cluster center is less than a preset threshold, and obtain K new clusters after convergence. S4. Calculate the second Euclidean distance between each image feature and the target cluster center of its respective cluster. Image features whose second Euclidean distance is greater than a preset distance are considered abnormal labels. The annotation information also includes the labels corresponding to each annotation box; the feature information of the image to be trained is extracted using a pre-trained large model, including: classifying each annotation box according to the labels in the annotation information to obtain annotation boxes with the same label; and extracting the image features corresponding to the annotation boxes with the same label in the image to be trained using a pre-trained large model. Before performing cluster analysis on the image features corresponding to each extracted bounding box, the process also includes: taking the image features corresponding to bounding boxes with the same label as a single category label data to construct a database corresponding to different category labels; When performing cluster analysis on the image features corresponding to each extracted bounding box, the image features are derived from the database.

2. The automatic quality inspection method for labeled information according to claim 1, characterized in that, Also includes: Image features whose second Euclidean distance is less than or equal to a preset distance are used as normal labels.

3. The automatic quality inspection method for labeled information according to claim 1, characterized in that, After identifying the abnormal label, the process also includes: displaying the abnormal label, modifying the abnormal label, and / or deleting the abnormal label; Displaying abnormal labels includes one or more of the following: bolding the abnormal label, displaying a pop-up message, and changing the label color; modifying abnormal labels includes changing the abnormal label to a normal label.

4. The automatic quality inspection method for labeled information according to claim 1, characterized in that, The trained large model is the DINOv2 model.

5. An electronic device, characterized in that, Includes computer-readable storage media and processors; The computer-readable storage medium is used to store executable instructions; The processor is used to read executable instructions stored in the computer-readable storage medium and execute the automatic quality inspection method for the annotation information according to any one of claims 1-4.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the automatic quality inspection method for the labeled information as described in any one of claims 1-4.

7. A computer program product, characterized in that, When the computer program product is run on a computer, the computer performs the automatic quality inspection method for the annotation information as described in any one of claims 1-4.

Images