Automatic quality inspection method for image annotation

Abstract

The invention discloses an image annotation automatic quality inspection method, and relates to the technical field of image annotation quality inspection, and the image annotation automatic quality inspection method mainly comprises the steps: processing original data to obtain training data; constructing a pseudo-twinning neural network, and training the pseudo-twinning neural network by using the training data and the loss function to obtain a quality inspection model; and predicting the image annotation data to be subjected to quality inspection by using the quality inspection model to obtain a quality inspection score, and endowing a quality inspection result to the image annotation data to be subjected to quality inspection. By implementing the image annotation automatic quality inspection method provided by the invention, the accuracy, quantification and efficiency of image annotation automatic quality inspection can be improved.

Description

Technical Field

[0001] This invention relates to the field of image annotation quality inspection technology, and more specifically, to an automated image annotation quality inspection method. Background Technology

[0002] With the rapid development of deep learning and computer vision technologies, automated image annotation has become a common data preprocessing method for training efficient object detection and segmentation models. One existing method is to annotate objects in three-dimensional space and then project them onto the image using camera pose. However, existing automated annotation methods cannot guarantee 100% accuracy of the annotation results, and common problems include mislabeling, omissions, boundary position deviations, and category errors.

[0003] To ensure the quality of training data, manual quality control is typically required for the annotation results of each frame or image—selecting out erroneous annotations and discarding or correcting them. Human intervention brings several drawbacks: high cost, with the cost of manual review by professional annotators increasing linearly or even superlinearly with the scale of annotation; low efficiency, as manual inspection and rejection processes are time-consuming and cannot meet real-time or near-real-time annotation needs; poor consistency, as different quality control personnel have varying subjective judgment standards, leading to inconsistent annotation standards within the same dataset; poor repeatability, in massive datasets or diverse scenarios, manual quality control struggles to ensure identical conditions and standards for each inspection, severely restricting the reproducibility and quantifiable management of the annotation process; and reduced data utilization, as simply discarding inaccurate annotations results in the loss of effective samples, and existing methods lack automatic error correction or feedback iteration mechanisms, failing to fully exploit the value of the data. Therefore, a new quality control method is needed that can reduce manual intervention and increase automation while ensuring annotation accuracy.

[0004] In existing computer vision tasks, two tasks are relatively close to the need for automated quality inspection: semantic segmentation and change detection. Semantic segmentation can segment visible features in an image. By calculating the area of ​​difference (IoU) with the actual annotation results, a threshold can be used to determine whether the quality inspection passes or fails. Change detection can segment areas where the visible features in the image do not match the annotation results. By calculating the area of ​​difference, a threshold can be used to determine whether the quality inspection passes or fails. However, reusing models from these two existing tasks has several drawbacks: 1. Additional cost: The visibility required for training data needs to be additionally labeled; 2. Poor generalization: The segmentation results vary greatly from different perspectives or different scenes, making it difficult to find a universal threshold; 3. Poor accuracy: Area calculation is highly dependent on segmentation accuracy. The actual offset area is very small, and missegmented and missed segments will have a significant impact on the results. 4. Slow speed: Segmentation models are generally slower than classification and detection models; Therefore, a quality inspection model that is fast, accurate, highly generalizable, and does not require additional annotation costs is needed. Summary of the Invention

[0005] The purpose of this invention is to provide an automated quality inspection method for image annotation, which can improve the accuracy, quantification, and efficiency of automated quality inspection for image annotation.

[0006] This invention provides an automated quality inspection method for image annotation, comprising the following steps: S1: Process the original data to obtain training data; S2: Construct a pseudo-twin neural network, which includes an image feature extraction module, a labeled feature extraction module, a feature fusion module, and a classification module; S3: Train the pseudo-twin neural network using the training data and loss function to obtain a quality inspection model; S4: Use the quality inspection model to predict the labeled data of the image to be inspected and obtain the quality inspection score; S5: Assign quality inspection results to the labeled data of the image to be inspected based on the quality inspection score.

[0007] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the above-described automated quality inspection method for image annotation.

[0008] The automated image annotation quality inspection method provided by this invention has the following beneficial effects: This invention obtains training data by processing the original data, and expands a richer training dataset using limited training samples; it includes a pseudo-Twin neural network comprising an image feature extraction module, an annotation feature extraction module, a feature fusion module, and a classification module, and trains the pseudo-Twin neural network using the training data and a loss function to obtain a quality inspection model; the quality inspection model is used to predict the labeled data of the images to be inspected to obtain a quality inspection score, and a quality inspection result is assigned to the labeled data of the images to be inspected accordingly; The quality inspection model provided by this invention is compatible with different sensor models, installation angles, scenarios, and imaging models. It mimics manual quality inspection and can be directly reused in similar quality inspection tasks, exhibiting excellent generalization capabilities. It does not require result segmentation or post-processing, directly outputting the final result of whether the quality inspection passed or failed, thus possessing end-to-end characteristics. The data labels are simple and do not require visibility, with the model focusing on visible elements. It achieves a recall rate of over 50% at 99% accuracy and boasts a faster inference speed, exceeding 10 frames per second. The questionable data from the automatic quality inspection of this invention is confirmed by manual quality inspection and fed back as new training data to the quality inspection model, continuously improving the performance of the quality inspection model and realizing a closed loop of quality inspection model data. This invention can maintain consistent quality inspection standards and has strong consistency; it can provide more detailed scores than manual inspection and has good quantification; this invention can repeatedly perform tasks under the same conditions and has a faster processing speed than manual inspection, thus improving efficiency and reducing the cost of hiring and training quality inspection personnel. Attached Figure Description

[0009] The present invention will be further described below with reference to the accompanying drawings and embodiments. In the accompanying drawings: Figure 1 This is a flowchart of the automated quality inspection method for image annotation provided by the present invention; Figure 2 This is an automated quality inspection flowchart provided by the present invention; Figure 3 This is a flowchart of the quality inspection model training data generation process provided by the present invention; Figure 4 This is a schematic diagram of the quality inspection model structure provided by the present invention. Detailed Implementation

[0010] To provide a clearer understanding of the technical features, objectives, and effects of the present invention, specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0011] Figure 1 A schematic diagram of the automated image annotation quality inspection method of this embodiment is shown. In this embodiment, the automated image annotation quality inspection method includes the following steps: S1: Process the original data to obtain training data; In one exemplary embodiment, the raw data includes raw images, high-precision maps, and camera poses; In one exemplary embodiment, step S1 specifically includes: S11: Based on the camera pose, project the high-precision map onto the original image to obtain the annotation result; S12: Score the labeling results based on the deviation of the elements from the standard to obtain the quality inspection label; In one exemplary embodiment, the quality inspection label indicates whether the inspection passed or failed. S13: Perturb the camera pose to obtain the perturbed camera pose, and obtain data with the quality inspection label as failing based on the perturbed camera pose; S14: Based on the data with the quality inspection label as failing, the annotation results, and the quality inspection label, obtain the training data; S2: Construct a pseudo-twin neural network, which includes an image feature extraction module, a labeled feature extraction module, a feature fusion module, and a classification module; In one exemplary embodiment, the image feature extraction module and the labeled feature extraction module have the same structure, are connected in parallel, and are connected to the feature fusion module, which is connected to the classification module. In one exemplary embodiment, the image feature extraction module and the annotation feature extraction module are structured as ResNet, ConvNeXT, or ViT; In one exemplary embodiment, the feature fusion module includes a self-attention layer, a cross-attention layer, and a feedforward layer, used to fuse input features from different modalities; In one exemplary embodiment, the classification module includes convolutional layers and fully connected layers for obtaining a classification score based on the fused features; S3: Train the pseudo-twin neural network using the training data and loss function to obtain a quality inspection model; In one exemplary embodiment, the loss function is as follows: , , , in The loss function; This represents the predicted probability of the actual category. To balance the weighting coefficients of positive and negative samples; For focusing parameters; These are the probability values ​​output by the model. For sample labels; These are adjustable parameters; In one exemplary embodiment, the loss function is as follows: , in, The loss function; The difference between the predicted value and the actual value; S4: Use the quality inspection model to predict the labeled data of the image to be inspected and obtain the quality inspection score; S5: Assign quality inspection results to the labeled data of the image to be inspected based on the quality inspection score; In one exemplary embodiment, the method further includes: based on the quality inspection results, manually inspecting the questionable data and feeding it back as high-value training data to the quality inspection model for training, thereby gradually improving the performance of the quality inspection model.

[0012] In some embodiments, the above-described automated image annotation quality inspection method can also be implemented in the following ways.

[0013] like Figure 2 The diagram shows the automated quality inspection process; in this embodiment, the automated quality inspection method for image annotation includes: 1. Obtain the image to be inspected and its annotation data. After preprocessing the annotation results, obtain the annotated image and annotation result pair.

[0014] 2. Input the above images and annotation results into the trained quality inspection model to obtain the quality inspection score.

[0015] 3. Based on the quality inspection score, assign quality inspection results to the labeled data of the images to be inspected: data with scores exceeding the preset threshold are directly judged as passing, and data with scores below the preset threshold are directly judged as failing.

[0016] 4. Questionable data undergoes manual quality inspection and is then fed back into the quality inspection model as high-value training data, gradually improving the model's performance.

[0017] like Figure 3 The diagram shown is a flowchart of the process for generating training data for a quality inspection model. The process of acquiring training data and the model training process include: (1) Acquire images, high-precision maps, and camera pose data.

[0018] (2) By using the camera pose, the high-precision map is projected onto the image to obtain the annotation results.

[0019] (3) The labeling results are scored manually according to the deviation of the elements from the standard to obtain the label of whether or not the quality inspection is passed.

[0020] (4) Perturb some of the labeled data that passed the quality inspection, that is, generate training data that failed the quality inspection by perturbing the camera pose.

[0021] (5) Using the above data, i.e., the acquired and generated data, Focal Loss and / or Smooth L1 Loss are used as loss functions to train a pseudo-Siamese neural network with a custom structure to obtain a model for quality inspection. The model inputs the image and the preprocessed annotation results, and outputs the quality inspection accuracy score.

[0022] like Figure 4 The diagram shown is a schematic of the quality inspection model structure. The model structure is as follows: A customized pseudo-twin neural network includes a feature extraction module, a feature fusion module, and a classification module.

[0023] The feature extraction module can be ResNet, ConvNeXT, or ViT. The above images are input into the feature extraction module to obtain features for their respective modalities.

[0024] The feature fusion module consists of a self-attention layer, a cross-attention layer, and a feedforward layer. The different modal features mentioned above are input into the feature fusion module to obtain the fused features.

[0025] The classification module consists of convolutional layers and fully connected layers. The fused features are input into the classification module to obtain a classification score.

[0026] It should be noted that this invention employs a pseudo-twin neural network, using two backbone networks that do not share weights to process the two inputs respectively.

[0027] The loss function designed for the training model in this invention is as follows: , in: , , The probability values ​​output by the model. α and γ are the sample labels, and α and γ are adjustable parameters.

[0028] Another loss function design used in the training model of this invention is as follows: , in This represents the difference between the predicted value and the actual value.

[0029] It should be noted that the above loss functions can be used simultaneously or separately.

[0030] This embodiment provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the above-described automated quality inspection method for image annotation.

[0031] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims. All of these forms are within the protection scope of the present invention.

Claims

1. An automated quality inspection method for image annotation, characterized in that, Includes the following steps: S1: Process the original data to obtain training data; S2: Construct a pseudo-twin neural network, which includes an image feature extraction module, a labeled feature extraction module, a feature fusion module, and a classification module; S3: Train the pseudo-twin neural network using the training data and loss function to obtain a quality inspection model; S4: Use the quality inspection model to predict the labeled data of the image to be inspected and obtain the quality inspection score; S5: Assign quality inspection results to the labeled data of the image to be inspected based on the quality inspection score.

2. The automated quality inspection method for image annotation according to claim 1, characterized in that, Also includes: Based on the quality inspection results, the questionable data will be manually inspected and used as high-value training data to feed back into the quality inspection model for training, thereby gradually improving the performance of the quality inspection model.

3. The automated quality inspection method for image annotation according to claim 1, characterized in that, The raw data includes raw images, high-precision maps, and camera poses.

4. The automated quality inspection method for image annotation according to claim 1, characterized in that, Step S1 specifically includes: S11: Based on the camera pose, project the high-precision map onto the original image to obtain the annotation result; S12: Score the labeling results based on the deviation of the elements from the standard to obtain the quality inspection label; S13: Perturb the camera pose to obtain the perturbed camera pose, and obtain data with the quality inspection label as failing based on the perturbed camera pose; S14: Based on the data that are deemed unqualified by the quality inspection label, the annotation results, and the quality inspection label, training data is obtained.

5. The automated quality inspection method for image annotation according to claim 1, characterized in that, The image feature extraction module and the annotation feature extraction module have the same structure. The image feature extraction module and the annotation feature extraction module are connected in parallel and connected to the feature fusion module. The feature fusion module is connected to the classification module.

6. The automated quality inspection method for image annotation according to claim 1, characterized in that, The image feature extraction module and the annotation feature extraction module are structured as ResNet, ConvNeXT, or ViT.

7. The automated quality inspection method for image annotation according to claim 1, characterized in that, The feature fusion module includes a self-attention layer, a cross-attention layer, and a feedforward layer, which are used to fuse input features from different modalities; the classification module includes a convolutional layer and a fully connected layer, which are used to obtain a classification score based on the fused features.

8. The automated quality inspection method for image annotation according to claim 1, characterized in that, The loss function is as follows: ， ， ， in The loss function; This represents the predicted probability of the actual category. To balance the weighting coefficients of positive and negative samples; For focusing parameters; These are the probability values ​​output by the model. For sample labels; This is an adjustable parameter.

9. The automated quality inspection method for image annotation according to claim 1, characterized in that, The loss function is as follows: ， in, The loss function; This represents the difference between the predicted value and the actual value.

10. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the steps of the automated quality inspection method for image annotation as described in any one of claims 1-9.

Images