Data annotation method and device, storage medium and data annotation equipment
By using a 3D point cloud cross-modal preprocessing and model update mechanism, the problems of low efficiency, high cost and poor consistency in data annotation technology are solved, and high-precision automatic annotation and model optimization in complex scenarios are achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-11-21
- Publication Date
- 2026-03-10
AI Technical Summary
Existing data annotation technologies are inefficient and costly, and struggle to guarantee consistency and quality in complex scenarios. Existing semi-automatic tools have low fidelity and cannot adapt to new scenarios and long-tail problems.
By acquiring the data to be labeled and the trained target model, 3D point cloud cross-modal preprocessing is performed to generate structured data. The target model is used to determine the semantic mask and obtain the uncertainty score. When the uncertainty score is greater than the preset value, the model is updated for labeling. The model is trained and optimized by combining the multimodal training dataset and the discriminator.
It significantly improves the accuracy and completeness of semantic annotation in complex scenarios, dynamically optimizes the model, reduces manual intervention, and improves annotation efficiency and quality.
Smart Images
Figure CN121640216A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of image processing technology, and in particular to a data annotation method and apparatus, storage medium, and data annotation device. Background Technology
[0002] With the deepening development of smart city and digital twin concepts, intelligent transportation systems have become the core support for applications such as traffic flow optimization, automatic accident detection, intelligent traffic light timing, and vehicle-road cooperation. These applications rely on deep, accurate, and real-time semantic understanding of complex traffic scenarios, and massive, high-quality, pixel-level semantically labeled datasets are a prerequisite for training and validating related artificial intelligence algorithms. However, current data labeling technologies face significant bottlenecks: First, traditional purely manual labeling methods are inefficient and costly, and it is difficult to guarantee labeling consistency and quality in large-scale production; second, existing semi-automatic labeling tools have low fidelity in complex scenarios (such as vehicle occlusion and severe weather), still requiring a large amount of manual correction; finally, the labeling models are rigid and cannot learn and iterate from manual intervention, making it difficult to adapt to new scenarios and long-tail problems. Summary of the Invention
[0003] This invention aims to at least partially address one of the technical problems in related technologies. To this end, the first objective of this invention is to propose a data annotation method that significantly improves the accuracy and completeness of semantic annotation in complex scenarios, and can dynamically optimize the model and reduce manual intervention.
[0004] A second objective of this invention is to provide a computer-readable storage medium.
[0005] The third objective of this invention is to provide a data annotation device.
[0006] The fourth objective of this invention is to provide a data annotation device.
[0007] To achieve the above objectives, a first aspect of the present invention proposes a data annotation method, wherein the method includes: acquiring data to be annotated and a trained target model; preprocessing the data to be annotated using 3D point cloud cross-modal and preset models to obtain structured data of the data to be annotated; inputting the structured data into the target model to determine a semantic mask of the data to be annotated; acquiring an uncertainty score of the semantic mask; and updating the target model when the uncertainty score is greater than a preset score, and using the updated target model for data annotation.
[0008] According to the data annotation method of the present invention, the method acquires the data to be annotated and the trained target model, preprocesses the data to be annotated to obtain structured data, inputs the structured data into the target model to determine the semantic mask of the data to be annotated, obtains the uncertainty score of the semantic mask, updates the target model when the uncertainty score is greater than the preset score, and uses the updated target model for data annotation. Thus, the accuracy and completeness of semantic annotation in complex scenarios are significantly improved, and the model can be dynamically optimized and human intervention is reduced. In addition, the data annotation method according to the above embodiments of the present invention may further include the following additional technical features: According to one embodiment of the present invention, the method further includes: determining that the annotation of the data to be annotated is completed when the uncertainty score is less than or equal to the preset score.
[0009] According to an embodiment of the present invention, the method further includes: acquiring a multimodal training dataset and a discriminator for a target model to be trained; labeling the multimodal training dataset to determine a high-fidelity semantic mask for each set of training data in the multimodal training dataset; performing object detection and spatial localization processing on each set of training data to determine the training input data for the target model to be trained; inputting the training input data into the target model to be trained to determine a predicted semantic mask corresponding to the training input data; and training the target model to be trained based on the high-fidelity semantic mask, the predicted semantic mask, and the discriminator to obtain a trained target model.
[0010] According to one embodiment of the present invention, the method further includes cleaning the multimodal training dataset after obtaining it.
[0011] According to one embodiment of the present invention, training the target model to be trained based on the high-fidelity semantic mask, the predicted semantic mask, and the discriminator includes: fixing the weight parameters of the discriminator and calculating the total loss of the target model to be trained, so as to update the weight parameters of the target model to be trained based on the total loss of the target model to be trained; fixing the weight parameters of the target model to be trained, calculating the loss value of the discriminator based on the high-fidelity semantic mask and the predicted semantic mask, so as to update the weight parameters of the discriminator based on the loss value of the discriminator; and repeatedly executing the above training steps until the target model to be trained meets the training completion condition.
[0012] According to one embodiment of the present invention, when the number of iterations in the training step reaches a preset number, or when the loss value of the discriminator no longer decreases, it is determined that the training model meets the training completion condition.
[0013] According to one embodiment of the present invention, updating the target model includes: correcting the semantic mask of the data to be labeled to obtain model evolution training data; and updating the target model according to the model evolution training data when the amount of model evolution training data reaches a preset amount or the time during which the target model has not been updated reaches a preset time.
[0014] To achieve the above objectives, a second aspect of the present invention provides a computer-readable storage medium storing a data annotation program thereon, which, when executed by a processor, implements the data annotation method of the aforementioned embodiments of the present invention.
[0015] According to embodiments of the present invention, a computer-readable storage medium executes a data annotation program via a processor, which significantly improves the accuracy and completeness of semantic annotation in complex scenarios, and can dynamically optimize the model and reduce human intervention.
[0016] To achieve the above objectives, a third aspect of the present invention also proposes a data annotation apparatus, wherein the apparatus comprises: an acquisition module for acquiring data to be annotated and a trained target model; a processing module for preprocessing the data to be annotated using 3D point cloud cross-modal and preset models to obtain structured data of the data to be annotated; inputting the structured data into the target model to determine a semantic mask of the data to be annotated; acquiring an uncertainty score of the semantic mask; and an update module for updating the target model when the uncertainty score is greater than a preset score, and using the updated target model for data annotation.
[0017] According to the data annotation apparatus of the present invention, an acquisition module acquires the data to be annotated and a trained target model, and a processing module preprocesses the data to be annotated to obtain structured data of the data to be annotated. The structured data is then input into the target model to determine the semantic mask of the data to be annotated. By acquiring the uncertainty score of the semantic mask, the update module updates the target model when the uncertainty score is greater than a preset score, and the updated target model is used for data annotation. This significantly improves the accuracy and completeness of semantic annotation in complex scenarios, and can dynamically optimize the model and reduce manual intervention. To achieve the above objectives, a fourth aspect of the present invention provides a data annotation device, including the data annotation apparatus described in the foregoing embodiments of the present invention.
[0018] The data annotation device according to the embodiments of the present invention significantly improves the accuracy and completeness of semantic annotation in complex scenarios by employing the data annotation device of the above embodiments of the present invention, and can dynamically optimize the model and reduce manual intervention.
[0019] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description
[0020] Figure 1 This is a flowchart illustrating a data annotation method according to an embodiment of the present invention; Figure 2 This is a flowchart illustrating a data annotation method according to another embodiment of the present invention; Figure 3 This is a flowchart illustrating a data annotation method according to yet another embodiment of the present invention; Figure 4 This is a flowchart illustrating a data annotation method according to another embodiment of the present invention; Figure 5 This is a flowchart illustrating a data annotation method according to a specific embodiment of the present invention; Figure 6 This is a block diagram of a data annotation device according to an embodiment of the present invention; Figure 7 This is a block diagram of a data annotation device according to an embodiment of the present invention.
[0021] Figure label: Data labeling device 1000, data labeling apparatus 100, acquisition module 10, processing module 20, update module 30. Detailed Implementation
[0022] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.
[0023] The data annotation method, computer-readable storage medium, data annotation apparatus, and data annotation device of the present invention are described below with reference to the accompanying drawings.
[0024] Figure 1 This is a flowchart illustrating a data annotation method according to an embodiment of the present invention.
[0025] Specifically, in some embodiments of the present invention, such as Figure 1 As shown, data annotation methods include: S101, Obtain the data to be labeled and the target model after training.
[0026] Specifically, in this embodiment, the data to be labeled is spatiotemporally aligned multimodal data, including synchronously acquired image data and point cloud data. The image data can be acquired through a camera, and the point cloud data can be acquired through a LiDAR. At the same time, the system loads a trained target model (i.e., generator G). It should be noted that the target model already has the basic ability to generate semantic masks.
[0027] S102 uses 3D point cloud cross-modal and preset models to preprocess the data to be labeled in order to obtain structured data of the data to be labeled.
[0028] Specifically, in this embodiment, the system performs a series of automated preprocessing operations on the acquired data to be labeled, transforming it into a standardized input format required by the target model, i.e., structured data. This process specifically includes: using a preset model (such as the YOLO series models) to quickly infer the image data, obtaining the bounding boxes and initial category labels of objects in the image as preliminary semantic information (i.e., coarse annotation); and using the sensor parameters (rotation matrix R, translation vector t, camera intrinsic parameters K) calibrated during model initialization to accurately project the corresponding 3D LiDAR point cloud data onto the 2D image data, forming a 2D spatial positioning region that can characterize the precise contour and spatial position of the object in the image, serving as strong geometric prior information (i.e., geometric cues). Finally, the preprocessing step outputs a set of structured data containing coarse annotations, geometric cues, and the original image, preparing for subsequent model inference. The original image is the image data within the data to be labeled.
[0029] S103, input structured data into the target model to determine the semantic mask of the data to be labeled.
[0030] Specifically, in this embodiment, the system inputs the structured data produced in step S102 into the current version of the target model (i.e., the generator G). The target model, as a deep neural network, extracts, fuses, and decodes features from the multimodal input through its internal encoder-decoder structure. The model performs one forward propagation inference, ultimately outputting a pixel-level predicted semantic mask with the same size as the original image. The predicted semantic mask assigns a semantic category label (such as road, vehicle, pedestrian, building, etc.) to each pixel, thus completing the deep semantic parsing of the scene.
[0031] S104, obtain the uncertainty score of the semantic mask.
[0032] Specifically, in this embodiment, the system performs a reliability assessment on the predicted semantic mask generated by the target model, quantifying its uncertainty score. This assessment can be achieved using the Monte Carlo dropout method. For example, during the model inference phase, the Dropout layer in the target model (generator G) is kept active, and the system uses the same structured input data <coarse annotation, geometric hints, original image> to perform multiple (e.g., 10) independent forward propagations. Since the Dropout mechanism randomly sets the output of some nodes in the network to zero during each forward propagation, the network substructure activated in each inference is slightly different, resulting in a non-deterministic, slightly different predicted semantic mask each time. Subsequently, the system calculates the variance of these multiple prediction results (e.g., 10 semantic masks) across all pixels. Regions with higher variance indicate that the model's prediction results for that region are more unstable and uncertain. Finally, the system obtains a comprehensive uncertainty score by calculating the average variance of the entire mask or by statistically analyzing the proportion of pixels with high variance to the total number of pixels. This score objectively reflects the model's confidence in the automatic annotation results.
[0033] S105: When the uncertainty score is greater than the preset score, the target model is updated, and the updated target model is used for data labeling.
[0034] Specifically, in this embodiment, the system compares the uncertainty score obtained in step S104 with a preset score, and makes a triage decision based on the comparison result. For example, it corrects the semantic mask of the data to be labeled to obtain model evolution training data. When the amount of model evolution training data reaches a preset amount, or when the target model has not been updated for a preset period of time, the target model is updated based on the model evolution training data. Thus, the updated target model can be used for data labeling.
[0035] Furthermore, in some embodiments of the present invention, the data annotation method further includes: determining that the annotation of the data to be annotated is completed when the uncertainty score is less than or equal to a preset score.
[0036] Specifically, in this embodiment, when the uncertainty score of the semantic mask is less than or equal to a preset threshold, the system determines that the automatic annotation result is reliable and automatically adopts it as the final annotation. The system binds this high-quality result with the original data, directly stores it in the annotation result library, and marks it as completed. This process requires no manual intervention, realizing automated closed-loop processing of high-confidence samples. This mechanism ensures the efficient operation of the annotation process in scenarios where the model is confident, accurately concentrating human resources on difficult cases, thereby significantly improving overall annotation efficiency while ensuring quality.
[0037] Furthermore, in some embodiments of the present invention, such as Figure 2 As shown, data annotation methods also include: S201, Obtain the multimodal training dataset and the discriminator of the target model to be trained.
[0038] Specifically, in this embodiment, firstly, a multimodal training dataset for training is acquired. This dataset is synchronously collected by calibrated sensor arrays (such as LiDAR and cameras) deployed on vehicles or fixed sites, and has undergone rigorous time synchronization and spatial calibration to ensure accurate alignment between images and point cloud frames. The multimodal training dataset covers various typical and edge scenarios (such as different lighting conditions, weather, and traffic densities). Simultaneously, the system initializes a target model to be trained, such as a generator G and a discriminator D. The generator G is typically a convolutional neural network with an encoder-decoder structure, responsible for generating semantic masks from the input. The discriminator D is another convolutional neural network responsible for distinguishing between the generated mask and the real mask; together, they constitute a generative adversarial network (GAN) framework.
[0039] S202, label the multimodal training dataset to determine the high-fidelity semantic mask for each group of training data in the multimodal training dataset.
[0040] Specifically, in this embodiment, the system submits the multimodal training dataset to a manual annotation platform, where professional annotators (annotation experts) perform fine, pixel-level semantic segmentation annotation on each frame of the dataset. During the annotation process, the annotation experts refer to the synchronous point cloud data projected onto the image, using its precise geometric contours to assist in the judgment, especially in areas where object boundaries are blurred or occluded, to ensure the accuracy of the annotation. This step outputs a high-fidelity semantic mask that strictly corresponds to each set of training data (image data, point cloud data). These high-fidelity semantic masks can be used for loss calculation in subsequent supervised training and as "true samples" for the discriminator D.
[0041] S203 performs object detection and spatial localization processing on each set of training data to determine the training input data for the target model to be trained.
[0042] Specifically, in this embodiment, each set of training data includes image data and point cloud data. A pre-trained object detection model (such as YOLO, Faster R-CNN, etc.) can be used to quickly infer the image data, obtaining the bounding boxes and preliminary category labels of each object in the image data as preliminary semantic information (i.e., coarse annotation). Next, using the sensor parameters (rotation matrix R, translation vector t, camera intrinsics K) calibrated during model initialization, the 3D point cloud data collected at the same time is precisely projected onto the 2D image data, forming a 2D spatial localization region that can characterize the precise contour and spatial position of objects in the image, serving as strong geometric prior information (i.e., geometric cues). Finally, the original image, coarse annotation, and geometric cues are combined into a triplet structured data set, which serves as the training input data for the target model to be trained. The original image is the image data in the training data.
[0043] S204. Input the training input data into the target model to be trained in order to determine the prediction semantic mask corresponding to the training input data.
[0044] Specifically, in this embodiment, in each training iteration, the system inputs the training input data (original image, coarse annotation, and geometric cue) prepared in step S203 into the target model (generator G) to be trained. The generator G extracts features from the image, coarse annotation, and geometric cue separately or jointly using its internal network structure, for example, by using an encoder. These features are then fused at the feature layer, and finally upsampled and refined by the decoder, performing a forward propagation computation to output a predicted semantic mask of the same size as the original image. S205, the target model to be trained is trained based on the high-fidelity semantic mask, the predicted semantic mask and the discriminator to obtain the trained target model.
[0045] Specifically, in this embodiment, the weight parameters of the discriminator D are fixed. The predicted semantic mask obtained in S204 and the high-fidelity semantic mask provided in S202 are input into the discriminator D to obtain the probability that D classifies the predicted mask as "true". The total loss of the generator G is calculated. This loss is a weighted sum of pixel-level supervision loss (such as the cross-entropy loss between the predicted mask and the high-fidelity mask) and adversarial loss (the loss that encourages the discriminator D to classify the predicted mask as "true", such as the binary cross-entropy loss). The total loss is used to update the weight parameters of the generator G through the backpropagation algorithm to improve the realism and accuracy of the mask generated by the generator G.
[0046] With the weight parameters of generator G fixed, the high-fidelity semantic mask is used as the "real sample" and the newly generated predicted semantic mask of generator G is used as the "fake sample". These are input into discriminator D respectively, and the loss value of discriminator D is calculated. The goal is to maximize the ability to distinguish real samples from fake samples (e.g., minimize the corresponding binary cross-entropy loss). The weight parameters of discriminator D are updated using the loss value and the backpropagation algorithm to improve the ability of discriminator D to distinguish between real and fake samples.
[0047] The system repeatedly executes the above training steps until the model to be trained meets the training completion conditions. Finally, it outputs a fully trained and stable target model (generator G), which has a strong cross-modal guided generation capability and can serve as the starting point for the self-evolution process.
[0048] Furthermore, in some embodiments of the present invention, the data annotation method further includes cleaning the multimodal training dataset after obtaining it.
[0049] Specifically, in this embodiment, the multimodal training dataset includes image data and point cloud data. For image data, image quality assessment can be performed, blurry frames can be automatically filtered by calculating sharpness indicators, and invalid images that are overexposed or underexposed can be removed based on brightness analysis. At the same time, file integrity can be verified. For point cloud data, noise filtering algorithms (such as statistical outlier removal) can be run to automatically identify and delete isolated noise points caused by environmental interference (such as dust, rain, and fog), retaining pure point clouds that represent the real physical structure. After the above cleaning is completed, the system will also perform data alignment consistency verification to ensure that each pair of images and point clouds strictly corresponds in time and space, and finally output a high-quality, aligned multimodal dataset, providing a reliable guarantee for subsequent manual annotation and model training.
[0050] Furthermore, in some embodiments of the present invention, such as Figure 3 As shown, the training of the target model is performed based on the high-fidelity semantic mask, the predicted semantic mask, and the discriminator, including: S301, fix the weight parameters of the discriminator and calculate the total loss of the target model to be trained, so as to update the weight parameters of the model to be trained according to the total loss of the target model to be trained.
[0051] Specifically, in this embodiment, the system first locks the weight parameters of the discriminator D, making it participate in the calculation as a fixed "referee" at this stage. Next, it calculates the total loss of the generator G (i.e., the target model to be trained), which consists of two parts: 1) pixel-level cross-entropy loss, obtained by comparing the predicted semantic mask with the high-fidelity semantic mask pixel-by-pixel, ensuring that the generated result is structurally aligned with the real annotation; 2) adversarial loss, the value of which depends on the probability that the discriminator D classifies the predicted semantic mask as "true," aiming to improve the realism of the generated result. Finally, the system uses the total loss of the generator G and updates its weight parameters through a backpropagation algorithm to minimize its total loss.
[0052] S302: Fix the weight parameters of the model to be trained, calculate the loss value of the discriminator based on the high-fidelity semantic mask and the predicted semantic mask, and update the weight parameters of the discriminator based on the loss value of the discriminator.
[0053] Specifically, in this embodiment, firstly, the system locks the weight parameters of the generator G. Then, the high-fidelity semantic mask is used as the "real sample," and the most recently generated predicted semantic mask by the generator G is used as the "fake sample," both of which are input into the discriminator D. The loss value of the discriminator D is calculated, typically using binary cross-entropy loss, with the goal of maximizing the discriminator's ability to distinguish between "real" and "fake" masks (i.e., encouraging its output to approach 1 for real samples and 0 for fake samples). Finally, the weight parameters of the discriminator D are updated using the loss value of the discriminator D and through a backpropagation algorithm to improve its "fake detection" ability.
[0054] S303, repeat the above training steps until the model to be trained meets the training completion conditions.
[0055] Specifically, in this embodiment, the system iteratively executes S301 and S302, with each iteration (i.e., updating G and D sequentially) called a training round. This alternating training allows the generator G and discriminator D to co-evolve in competition: the generator G strives to generate realistic masks to deceive the discriminator D, while the discriminator D continuously improves its discrimination ability. This cycle continues until a preset training completion condition is triggered, thereby ensuring that the final initial target model is both accurate and highly realistic.
[0056] Furthermore, in some embodiments of the present invention, when the number of iterations in the training step reaches a preset number, or when the loss value of the discriminator no longer decreases, it is determined that the training model meets the training completion condition.
[0057] Specifically, in this embodiment, when the total number of training rounds reaches a preset number, training is stopped regardless of the model performance. Alternatively, the system monitors the loss value of the discriminator D, and when its decrease within a continuous time window (e.g., in the most recent 5000 training rounds) is less than a preset minimum threshold, the model is considered to have "converged" and its performance no longer shows significant improvement. At this point, training can be stopped, which helps to ensure model performance while avoiding unnecessary consumption of computational resources.
[0058] Furthermore, in some embodiments of the present invention, such as Figure 4 As shown, updating the target model includes: S401, modify the semantic mask of the labeled data to obtain training data for model evolution.
[0059] Specifically, in this embodiment, for problematic samples with excessively high uncertainty scores, the system pushes them to a human-machine collaborative verification platform. Annotation experts review the predicted semantic masks generated by the model and perform efficient, refined modifications and corrections based on them, rather than labeling from scratch. After correction, the system stores this data package, consisting of the original unlabeled data (image data, point cloud data) and the high-quality mask corrected by experts, as training data for model evolution.
[0060] S402, when the amount of training data for model evolution reaches a preset amount, or when the target model has not been updated for a preset period of time, update the target model based on the training data for model evolution.
[0061] Specifically, in this embodiment, when the accumulated training data for model evolution reaches a preset batch size (e.g., 500 data points), the system automatically initiates the retraining process. Alternatively, if the time elapsed since the last model update reaches a preset period (e.g., one week), an update is forcibly initiated even if the data volume is insufficient, ensuring timely model iteration. The system uses all newly accumulated evolution training data to perform offline fine-tuning of the target model currently serving the online service. After training is complete, the old model is seamlessly replaced with a newer, more powerful version, thereby achieving continuous improvement in the intelligence level of the annotation system and forming an efficient self-evolutionary closed loop to complete the update of the target model.
[0062] In summary, in one specific embodiment of the present invention, such as Figure 5As shown, after acquiring the data to be labeled and the trained target model (i.e., generator G), the data to be labeled is preprocessed to obtain coarse annotations and geometric cues. The obtained coarse annotations, geometric cues, and image data from the data to be labeled are input into generator G to obtain the semantic mask of the data to be labeled. An uncertainty score is obtained based on the semantic mask. When the uncertainty score is less than or equal to a preset score, the labeling result is output, and it is determined whether the labeling is complete. If not, the data to be labeled is reacquired; if complete, the labeling of the data is considered complete, and the uncertainty assessment is passed. If the uncertainty score is greater than the preset score, the uncertainty assessment is not passed, and the semantic mask of the data to be labeled is corrected (i.e., manually corrected) to obtain model evolution training data. When the amount of model evolution training data reaches a preset quantity, or the target model has not been updated for a preset duration, the model evolution condition is determined to be met. At this point, the target model is updated based on the model evolution training data.
[0063] In summary, the data annotation method according to embodiments of the present invention acquires the data to be annotated and the trained target model, preprocesses the data to be annotated to obtain structured data, inputs the structured data into the target model to determine the semantic mask of the data to be annotated, obtains the uncertainty score of the semantic mask, updates the target model when the uncertainty score is greater than a preset score, and uses the updated target model for data annotation. Thus, the accuracy and completeness of semantic annotation in complex scenarios are significantly improved, and the model can be dynamically optimized and human intervention reduced.
[0064] Based on the data annotation method proposed in the foregoing embodiments of the present invention, the present invention also proposes a computer-readable storage medium storing a data annotation program thereon, which implements the data annotation method of the above embodiments of the present invention when executed by a processor.
[0065] According to embodiments of the present invention, a computer-readable storage medium executes a data annotation program via a processor, which significantly improves the accuracy and completeness of semantic annotation in complex scenarios, and can dynamically optimize the model and reduce human intervention.
[0066] Figure 6 This is a block diagram of a data annotation device according to an embodiment of the present invention.
[0067] Specifically, such as Figure 6 As shown, the data annotation device 100 includes an acquisition module 10, a processing module 20, and an update module 30.
[0068] The acquisition module 10 is used to acquire the data to be labeled and the trained target model; the processing module 20 is used to preprocess the data to be labeled through 3D point cloud cross-modal and preset model to obtain the structured data of the data to be labeled; input the structured data into the target model to determine the semantic mask of the data to be labeled; obtain the uncertainty score of the semantic mask; and the update module 30 is used to update the target model when the uncertainty score is greater than the preset score, and use the updated target model to perform data labeling.
[0069] In some embodiments of the present invention, the processing module 20 is further configured to determine that the annotation of the data to be annotated is complete when the uncertainty score is less than or equal to a preset score.
[0070] In some embodiments of the present invention, the processing module 20 is further configured to acquire a multimodal training dataset and a discriminator for the target model to be trained; label the multimodal training dataset to determine a high-fidelity semantic mask for each set of training data in the multimodal training dataset; perform object detection and spatial localization processing on each set of training data to determine the training input data for the target model to be trained; input the training input data into the target model to be trained to determine the predicted semantic mask corresponding to the training input data; and train the target model to be trained based on the high-fidelity semantic mask, the predicted semantic mask, and the discriminator to obtain the trained target model.
[0071] In some embodiments of the present invention, the processing module 20 is further configured to clean the multimodal training dataset after obtaining it.
[0072] In some embodiments of the present invention, the processing module 20 is further configured to fix the weight parameters of the discriminator and calculate the total loss of the target model to be trained, so as to update the weight parameters of the model to be trained according to the total loss of the target model to be trained; fix the weight parameters of the model to be trained, calculate the loss value of the discriminator according to the high-fidelity semantic mask and the predicted semantic mask, so as to update the weight parameters of the discriminator according to the loss value of the discriminator; and repeatedly execute the above training steps until the model to be trained meets the training completion conditions.
[0073] In some embodiments of the present invention, when the number of iterations in the training step reaches a preset number or the loss value of the discriminator no longer decreases, it is determined that the training model meets the training completion condition.
[0074] In some embodiments of the present invention, module 30 is specifically used to correct the semantic mask of the data to be labeled in order to obtain model evolution training data; when the amount of model evolution training data reaches a preset amount, or when the target model has not been updated for a preset period of time, the target model is updated according to the model evolution training data.
[0075] It should be noted that other specific implementations of the data annotation device proposed in the embodiments of the present invention can be found in the specific implementations of the data annotation method in the foregoing embodiments of the present invention. To reduce redundancy, they will not be repeated here.
[0076] In summary, the data annotation apparatus according to embodiments of the present invention acquires the data to be annotated and the trained target model through an acquisition module, and preprocesses the data to be annotated to obtain structured data of the data to be annotated through a processing module. The structured data is then input into the target model to determine the semantic mask of the data to be annotated. By acquiring the uncertainty score of the semantic mask, the target model is updated by an update module when the uncertainty score is greater than a preset score. The updated target model is then used for data annotation. This significantly improves the accuracy and completeness of semantic annotation in complex scenarios, and can dynamically optimize the model and reduce manual intervention. Figure 7 This is a block diagram of a data annotation device according to an embodiment of the present invention.
[0077] like Figure 7 As shown, the data annotation device 1000 includes the data annotation apparatus 100 described in the above embodiment of the present invention.
[0078] The data annotation device according to the embodiments of the present invention significantly improves the accuracy and completeness of semantic annotation in complex scenarios by employing the data annotation device of the above embodiments of the present invention, and can dynamically optimize the model and reduce manual intervention.
[0079] It should be noted that the logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be specifically implemented in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.
[0080] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.
[0081] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.
[0082] In the description of this invention, it should be understood that the terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," "clockwise," "counterclockwise," "axial," "radial," and "circumferential" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing this invention and simplifying the description, and are not intended to indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.
[0083] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.
[0084] In this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," "linking," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components, unless otherwise explicitly limited. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.
[0085] In this invention, unless otherwise explicitly specified and limited, "above" or "below" the second feature can mean that the first feature is in direct contact with the second feature, or that the first feature is in indirect contact with the second feature through an intermediate medium. Furthermore, "above," "over," and "on top" of the second feature can mean that the first feature is directly above or diagonally above the second feature, or simply that the first feature is at a higher horizontal level than the second feature. "Below," "below," and "under" the second feature can mean that the first feature is directly below or diagonally below the second feature, or simply that the first feature is at a lower horizontal level than the second feature.
[0086] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.
Claims
1. A data labeling method, characterized by, The method comprises: acquiring to-be-labeled data and a trained target model; preprocessing the to-be-labeled data through 3D point cloud cross-modal and a preset model to obtain structured data of the to-be-labeled data; inputting the structured data into the target model to determine a semantic mask of the to-be-labeled data; acquiring an uncertainty score of the semantic mask; when the uncertainty score is greater than a preset score, updating the target model and using the updated target model to perform data labeling.
2. The data labeling method of claim 1, wherein, The method further comprises: when the uncertainty score is less than or equal to the preset score, determining that the labeling of the to-be-labeled data is completed.
3. The data labeling method of claim 1, wherein, The method further comprises: acquiring a discriminator of a to-be-trained target model and a multi-modal training data set; labeling the multi-modal training data set to determine a high-fidelity semantic mask of each set of training data in the multi-modal training data set; performing target detection and spatial positioning processing on the each set of training data to determine training input data of the to-be-trained target model; inputting the training input data into the to-be-trained target model to determine a predicted semantic mask corresponding to the training input data; training the to-be-trained target model according to the high-fidelity semantic mask, the predicted semantic mask and the discriminator to obtain the trained target model.
4. The data labeling method of claim 3, wherein, The method further comprises: after acquiring the multi-modal training data set, the multi-modal training data set is further cleaned.
5. The data labeling method of claim 3, wherein, The training of the to-be-trained target model according to the high-fidelity semantic mask, the predicted semantic mask and the discriminator comprises: fixing a weight parameter of the discriminator and calculating a total loss of the to-be-trained target model to update the weight parameter of the to-be-trained model according to the total loss of the to-be-trained target model; fixing the weight parameter of the to-be-trained model and calculating a loss value of the discriminator according to the high-fidelity semantic mask and the predicted semantic mask to update the weight parameter of the discriminator according to the loss value of the discriminator; recursively performing the above training steps until the to-be-trained model meets a training completion condition.
6. The data labeling method of claim 5, wherein, When the number of cycles of the training step reaches a preset number or the loss value of the discriminator no longer decreases, it is determined that the to-be-trained model meets the training completion condition.
7. The data labeling method of claim 1, wherein, The updating of the target model comprises: correcting the semantic mask of the to-be-labeled data to obtain model evolution training data; when the number of the model evolution training data reaches a preset number or the time length during which the target model is not updated reaches a preset time length, updating the target model according to the model evolution training data.
8. A computer-readable storage medium, characterized in that, A data labeling program is stored thereon, and the data labeling program is executed by a processor to implement the data labeling method according to any one of claims 1-7.
9. A data labeling apparatus, comprising: The device comprises: an acquisition module configured to acquire to-be-labeled data and a trained target model; The processing module is configured to preprocess the to-be-labeled data through a 3D point cloud cross-modality and a preset model to obtain structured data of the to-be-labeled data; input the structured data into the target model to determine a semantic mask of the to-be-labeled data; and obtain an uncertainty score of the semantic mask. The updating module is configured to update the target model when the uncertainty score is greater than a preset score, and perform data labeling by using the updated target model.
10. A data labeling device, comprising: The data labeling device of claim 9 is included.