Automatic driving scene-oriented data annotation system and method

Abstract

The invention relates to the technical field of data annotation, and discloses a data annotation system and method for an automatic driving scene, and the method comprises the steps: constructing a scene database and a data set for managing to-be-annotated data, working condition data and accepted data; creating an annotation item and obtaining to-be-annotated data; the to-be-labeled data and the corresponding working condition data are associated, scene criticality evaluation is carried out, a scene criticality evaluation driving mechanism is introduced to carry out dynamic strategy generation, and a task strategy and a quality inspection strategy are included; splitting the to-be-labeled data according to a task strategy and performing task allocation; performing intelligent labeling model training based on the data set corresponding to the to-be-labeled data category; pre-labeling by utilizing the trained intelligent labeling model, carrying out manual labeling by utilizing a plurality of built-in labeling tools, and outputting labeled data; performing quality inspection on the labeled data according to a quality inspection strategy so as to output checked and accepted data; and performing intelligent labeling model training and updating based on the checked and accepted data. According to the invention, a high-expansibility system is constructed, and man-machine cooperation high-quality labeling is realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data annotation, in particular to a data annotation system and method for an automatic driving scene. BACKGROUND

[0002] The high-performance perception algorithm of an automatic driving system highly depends on large-scale and high-quality annotated data. In the data-driven automatic driving research and development process, the annotation work provides structured semantic information for model training and evaluation, and directly affects the recognition and decision performance of the system.

[0003] However, the traditional data annotation method mainly relies on manual operation, which is low in efficiency and high in cost. In actual projects, manual annotation often accounts for more than 60% of the cost of the entire AI project, and the annotation cost of a high-precision semantic segmentation image can reach more than ten yuan, and video annotation can consume several times more time. At the same time, the pure manual process is prone to introduce human errors, and it is difficult to guarantee the annotation consistency and accuracy, with an annotation error rate as high as 12%, and multiple rounds of proofreading are needed to reduce the error to below 2%.

[0004] In view of the above problems, although the existing annotation platform increases the intelligent annotation method, it lacks an efficient human-computer collaborative mechanism and model feedback capability, and it is difficult to quickly expand in the face of multi-modal sensor data and complex scenes, and the problems of data modal diversification, low processing efficiency, difficult quality control, and low automation level are particularly prominent. SUMMARY

[0005] The present application aims to provide a data annotation system and method for an automatic driving scene, which realizes a scene-driven, autonomous optimization intelligent collaborative human-computer annotation method through multi-module deep linkage and dynamic strategy generation.

[0006] The basic scheme provided by the present application is: a data annotation system for an automatic driving scene, comprising: an annotation management unit, including a project management module for creating an annotation project and obtaining to-be-annotated data, a task module for splitting the to-be-annotated data according to a task strategy and task allocation, an annotation scene support module with a plurality of built-in annotation tools, a visual annotation module for calling a currently trained intelligent annotation model to pre-annotate before manual annotation, combining the annotation scene support module for manual annotation, and finally outputting annotated data, and a quality control module for quality inspection of the annotated data according to a quality inspection strategy to output accepted data; a working condition association and strategy generation unit for associating the to-be-annotated data with corresponding working condition data, performing scene criticality evaluation, and introducing a scene criticality evaluation driven mechanism to generate a dynamic strategy, the strategy including a task strategy and a quality inspection strategy; The model training management unit is configured to build an intelligent labeling model based on a deep learning framework, and to perform model training based on the accepted data, and the scene criticality evaluation result participates in the model training process. The data management unit is configured to manage the to-be-labeled data, working condition data and accepted data based on the constructed scene database and multidimensional data set.

[0007] The application further provides a data labeling method for an automatic driving scene, which utilizes a data labeling system for the automatic driving scene. S1, constructing a scene database and a multidimensional data set to manage the to-be-labeled data and working condition data; S2, creating a labeling project and obtaining to-be-labeled data, associating the to-be-labeled data with corresponding working condition data, performing scene criticality evaluation, and introducing a scene criticality evaluation driven mechanism to generate a dynamic strategy, the strategy including a task strategy and a quality inspection strategy; S3, splitting the to-be-labeled data according to the task strategy and performing task allocation; S4, training an intelligent labeling model based on a data set corresponding to a general category of the automatic driving scene, performing manual labeling by utilizing a plurality of built-in labeling tools and combining pre-labeling performed by the trained intelligent labeling model, and finally outputting labeled data; S5, performing quality inspection on the labeled data according to the quality inspection strategy to output accepted data meeting a quality standard, and managing the accepted data based on S1; S6, training and updating the intelligent labeling model based on the accepted data, wherein the scene criticality evaluation result participates in the model training process.

[0008] The working principle and advantages of the application are as follows: The application builds a full-link closed-loop system through the deep collaboration of the four units, generates a dynamic strategy driven by scene criticality evaluation, realizes a scene-driven and self-optimized intelligent collaborative human-machine labeling mode, and has multi-dimensional advantages in efficiency improvement, cost control and quality assurance, and the specific advantages are as follows: First, the deep collaboration and strategy closed loop of each module, by breaking through the whole process of working condition association, labeling management, model training and data management, compared with the traditional scattered labeling mode, the real-time flow of each unit information avoids the disconnection of links, at the same time, through the dynamic scheduling of resources by the project management module and the task module, combined with the multi-dimensional data management and control ability of the data management unit, the utilization rate of human and data resources is greatly improved, and the overall labeling process is accelerated. More importantly, the introduction of "working condition association-key importance assessment" double drive upgrades the traditional static labeling process to a scene-aware dynamic strategy generation, so that each module does not operate in isolation, but is driven by scene key importance assessment to automatically generate task strategies and differentiated quality inspection strategies that match the labeling difficulty, task priority and personnel ability, realize resource adaptive allocation, and realize the "working condition analysis-intelligent strategy-accurate labeling-data backflow" pipeline labeling operation; At the same time, the specific algorithm of scene key importance assessment takes into account the importance and rarity of downstream automatic driving algorithm model training, which is beneficial to both upstream and downstream.

[0009] Second, intelligent labeling collaborative man-machine, introduce the enhancement cycle mechanism of "pre-labeling-human verification-model instant retraining", by integrating intelligent labeling and quality detection algorithms, relying on the intelligent labeling model built by the model training management unit to provide high-quality pre-labeling before manual labeling, realize the semi-automatic labeling mode of AI pre-labeling combined with manual verification: for common scenes and targets, the model can automatically generate most of the labeling results, and the human only needs to review and correct; For complex or edge cases, fine labeling is done by human, avoiding the repetitive labor of traditional pure manual labeling. More importantly, the intelligent labeling results are fed back to the model optimization after manual verification and quality acceptance, and the training process integrates scene key importance assessment, so that the model continues to evolve in important scenes, and the model prediction can adapt to more complex scenes and provide more detailed initial labeling, breaking through the scene adaptation limitations of traditional rule labeling. In addition, the visual labeling module links the labeling scene support module to provide targeted tool support for manual labeling, further improving labeling efficiency and operation convenience, and is conducive to system expansion.

[0010] Third, the quality control module builds a multi-level quality inspection system, with the quality inspection strategy driven by scene key importance assessment as the core, strengthens the quality inspection standard for high key importance scenes, accurately identifies labeling deviations through intelligent pre-inspection, manual review and expert arbitration multi-level audit, and ensures labeling quality. At the same time, the accepted data is backflowed to the model training unit through the data management unit, combined with the scene key importance assessment result to optimize the model training, realizing the iterative cycle of labeling quality improvement, model performance optimization and pre-labeling precision upgrade. This dynamic quality control and iterative mode centered on scenes guarantees data quality from the source, and the labeling accuracy is high and continuously improves with the system running.

[0011] Fourth, by constructing a unified scenario database and multidimensional dataset, the system can structurally associate and manage discrete labeled data, working condition information and acceptance results, which not only supports full-process traceability, but also forms a reusable scenario knowledge base, providing in-depth data support for subsequent model training and business decision-making. Attached Figure Description

[0012] Figure 1 This is a schematic diagram of the structure of a data annotation system for autonomous driving scenarios provided in an embodiment of the present invention; Figure 2 This is a flowchart illustrating a data annotation method for autonomous driving scenarios provided in an embodiment of the present invention. Detailed Implementation

[0013] The following detailed explanation illustrates the specific implementation methods: The basic implementation examples are as follows: Figure 1 As shown: A data annotation system for autonomous driving scenarios, including; The annotation management unit includes a project management module for creating annotation projects and obtaining data to be annotated, a task module for splitting data to be annotated and assigning tasks according to task strategies, an annotation scenario support module with built-in annotation tools, a visualization annotation module for calling the currently trained intelligent annotation model to perform pre-annotation before manual annotation and combining it with the annotation scenario support module to perform manual annotation and finally output the annotated data, and a quality control module for performing quality inspection on the annotated data according to the quality inspection strategy to output the accepted data. The working condition association and strategy generation unit is used to associate the data to be labeled with the corresponding working condition data, and to perform scenario criticality assessment. It also introduces a scenario criticality assessment-driven mechanism to generate dynamic strategies, including task strategies and quality inspection strategies. The model training management unit is used to build intelligent labeled models based on deep learning frameworks; it is also used to train models based on accepted data, and the results of scenario criticality assessment are involved in the model training process. The data management unit is used to manage unlabeled data, operational data, and accepted data based on the constructed scenario database and multidimensional dataset.

[0014] Specifically: The annotation management unit (which can be understood as an annotation platform) provides complete data annotation tools and management functions, including project management module, task module, annotation scenario support module, visual annotation module, and quality control module; it also includes process management module and personnel management module.

[0015] The labeling scene support module internally has rich labeling tools and templates, covering different data modalities such as images, videos, audio, text, and 3D point clouds, and supports multiple automatic driving-related data labeling scenes, including but not limited to camera images, laser radar point clouds, high-precision maps, and voice / text data types. It also supports functions such as object detection, polygon labeling, key point labeling, optical flow tracking, lane line extraction, and voice labeling. Specifically: 1) At the image level, it supports labeling of vehicles, pedestrians, non-motor vehicles, traffic signs, traffic lights, lane lines, pedestrian crossings, and speed reduction zones, using two-dimensional bounding boxes, instance segmentation, and semantic segmentation to provide detailed semantic segmentation information for the perception system.

[0016] 2) At the three-dimensional point cloud level, it supports three-dimensional bounding box labeling of targets in point clouds and records the dynamic state (such as stationary, slow-moving, lane-changing, etc.) and time sequence unique ID of the target to construct continuous frame motion trajectories.

[0017] 3) It supports cross-modal joint labeling in multi-sensor fusion scenes, including first labeling in one layer and then mapping the labeling results to another layer for synchronous labeling, such as first labeling in images and then mapping to point clouds for synchronous labeling to ensure the consistency and alignment of multi-modal data.

[0018] 4) To adapt to advanced applications, it can also be extended to natural language level labeling, such as named entity recognition, entity relationship, and driving intent labeling for voice or text, to realize dialogue and intent understanding tasks in human-computer interaction scenarios.

[0019] Through unified labeling support of multi-modal data, this module provides rich and flexible labeling capabilities for the system to meet the needs of multiple data sources in the automatic driving scene.

[0020] The project management module is used to create labeling projects (including project name, task description, requirements, cycle, and budget, etc.) and designate project leaders and participating teams. It is also used to import and obtain data to be labeled.

[0021] The task module is used to split and assign tasks according to task strategies. Specifically, it automatically splits data according to task difficulty, priority, and personnel capabilities and assigns them to different labeling personnel.

[0022] The visual labeling module is used for manual labeling in conjunction with the labeling scene support module, i.e., providing a graphical interface (interactive interface) and tools called from the labeling scene support module for labeling personnel to perform visual manual labeling. At the same time, during manual labeling, the module calls intelligent labeling models to pre-label before manual labeling to assist manual labeling, and finally outputs labeled data.

[0023] A quality control module is configured to set a quality standard and to perform quality inspection on the labeling results according to a quality inspection strategy. The quality inspection strategy includes one or a combination of expert review, multi-person cross-audit, sampling inspection, and AI automatic quality inspection. The quality control module is further configured to initiate manual re-audit based on suspected error items that pass the AI automatic quality inspection. The quality control module is further configured to return data that does not meet the quality standard (fails the acceptance) to a previous step for re-correction by a labeler.

[0024] A process management module is configured to standardize the labeling process and the quality inspection (audit) process.

[0025] A personnel management module is configured to manage the permissions, training, and assessment of the labeling personnel.

[0026] The quality assurance process of "personnel grading + expert arbitration + automatic quality inspection" enables orderly collaboration in each link of task creation, distribution, labeling, review, and acceptance. This process, with the aid of a graphical interface and visual tools, and in combination with AI-assisted labeling and review, can significantly improve labeling efficiency and quality.

[0027] Meanwhile, in project management, task distribution, and quality control, instead of using static or simple priority allocation, dynamic instructions from the working condition association and strategy generation unit are received to achieve task strategy and quality inspection strategy adjustment based on scene criticality.

[0028] The working condition association and strategy generation unit is configured to associate the data to be labeled with corresponding working condition data, perform scene criticality assessment, and introduce a scene criticality assessment-driven mechanism to generate dynamic strategies, including task strategies and quality inspection strategies. A data management module is configured to synchronize and associate data: responsible for timestamp alignment and data binding of the collected sensor data (images, point clouds) and synchronized vehicle CAN bus data (or high-precision IMU data), i.e., working condition data (including vehicle speed v(t), longitudinal acceleration a lon (t), and lateral acceleration a lat (t), and other data representing working conditions).

[0029] A scene criticality assessment module: for each data segment (such as consecutive video frames or point cloud sequences), according to its associated operating conditions, a scenario criticality score (Scenario Criticality Score, ) is calculated. This score is used to quantify the importance and rarity of the data segment for the training of the automatic driving algorithm model (downstream of the labeling result, the labeling data is used for the training of the automatic driving algorithm model). A specific calculation formula is as follows:

[0030] wherein, is the working condition feature vector, containing, for example, a lon (t), a lon (t), etc. is the weight coefficient of the corresponding feature, for example, high acceleration or high angular velocity usually means more complex driving behavior, which should be given higher weight; is a normalization function that maps the numerical values of different physical quantities to a unified scoring interval (such as [0, 1]); is an environmental complexity additive term determined by environmental factors (such as weather, light, road type), input by other sensors or data sources.

[0031] A dynamic strategy generation module is used for scene criticality assessment driven mechanism to generate dynamic strategies, i.e. to generate differentiated task strategies and quality inspection strategies based on the scene criticality assessment results (calculated S c scores). It includes at least one of the following: According to the advantages and disadvantages of the scene criticality assessment results, the priority of task allocation is determined, wherein the data corresponding to high-priority tasks is preferentially split and labeled, for example, the task priority strategy: Data segments with S scores higher than a certain threshold are marked as "high priority" and are preferentially labeled.

[0032] Further, a fusion model is constructed to fuse the priority with the task difficulty and the annotator's ability to determine the final task allocation strategy; specifically: A priority-driven sliding window allocation strategy is adopted, and the first K tasks in the priority queue P are taken out as the current allocation window ; For each window Wt, the matching score is calculated:

[0033] wherein, denotes the tasks in the window; denotes the annotators in the annotator set ; denotes the matching score of the task and the annotator ; denotes the cosine similarity between the ability feature vector of the annotator and the task difficulty feature vector of the task ; Compensation term indicating the task difficulty exceeds the annotator's ability; Adjustment coefficient indicating the sum is 1; Wherein, the personnel ability feature vector characterizes the skill level of the annotator , which can take 5 dimensions: target detection, semantic segmentation, occlusion handling, speed, and attention; for example, =[target detection: 0.8, semantic segmentation: 0.6, occlusion handling: 0.9, speed: 0.7, attention: 0.85].

[0034] Task difficulty feature vector characterizes the difficulty requirements of the task in each dimension, which can take 5 dimensions: target density, segmentation complexity, occlusion degree, time pressure, and precision requirement; for example =[target density: 0.9, segmentation complexity: 0.3, occlusion degree: 0.8, time pressure: 0.6, precision requirement: 0.7].

[0035]

[0036] The value close to 1 indicates that the personnel ability and task difficulty requirements are highly matched (the personnel is good at handling tasks of this difficulty); the value close to 0, the ability and requirement are not related; the value close to -1, the ability and requirement are opposite (the least matching).

[0037]

[0038] wherein, represents the gap between the task difficulty requirements and the actual ability of the personnel, if it is a positive gap: task difficulty > personnel ability (risk point), if it is a negative gap: personnel ability > task difficulty (ability surplus); represents an exponential decay function, when is 0 (perfect match), =1, full compensation, when the gap increases, the value decreases rapidly, reducing the matching degree; represents the tolerance parameter, the smaller the tolerance, the lower the tolerance to the gap, which can be selected as 0.5.

[0039] Based on the binary decision variable optimization allocation, wherein =1 indicates that the task node is assigned to the annotator , and =0 indicates that it is not assigned; the model optimization goal is to find the combination that maximizes the sum of matching scores:

[0040] The simultaneous constraint conditions include that all the tasks undertaken by any annotator j in the set of annotators do not exceed the maximum load of the annotator j :

[0041] Generally 5 or less, which can be dynamically set based on the urgency of the task time and the number of assigned personnel.

[0042] The quality inspection level is determined according to the advantages and disadvantages of the scene criticality evaluation result, and the corresponding quality inspection strategy is determined according to the quality inspection level, wherein the higher the quality inspection level, the stricter the quality inspection strategy adopted, for example, the quality inspection level strategy: according to The score is divided into different levels, for example, The segments with high scores automatically trigger the "expert review" or "multi-person cross-verification" process, while the segments with low scores use the "random inspection" or "AI automatic quality inspection" process. At the same time, three audits (AI automatic quality inspection + multi-person cross-verification + expert review), double audits (AI automatic quality inspection + multi-person cross-verification, or AI automatic quality inspection + expert review), single audit (AI automatic quality inspection), and sampling audit (random inspection) are formed.

[0043] The model training management unit is used for building an intelligent annotation model based on a deep learning framework, and is also used for model training based on accepted data, and the scene criticality evaluation result participates in the model training process. This module is tightly coupled with the annotation management unit (annotation platform) to form a data-algorithm-training closed loop. On the one hand, the system can periodically train or update the deep learning model using the annotated data; on the other hand, the latest model prediction result is used for pre-annotation of the next round of annotation tasks.

[0044] Specifically, the model training management unit supports model selection, training parameter configuration, and computing power management, and provides an end-to-end model training pipeline, so that annotation data can be directly used for model training and iteration. Based on the data set corresponding to the general category of the autonomous driving scene, the intelligent annotation model is trained, and a corresponding intelligent annotation model is trained for each data type, such as an image corresponding to a model and a point cloud corresponding to a model. In the annotation process, the intelligent annotation model (such as a pre-trained target detection model) will automatically generate initial annotation suggestions for manual review. This annotation method through "AI pre-annotation + manual correction" can greatly improve the annotation speed. In addition, this unit optimizes the annotation results in a feedback manner: re-puts the already-inspected data into model training to form a closed-loop mechanism of "annotation-model iteration-reannotation". In this way, annotation and training are iterated cooperatively, the model is gradually refined, and the annotation work is also intelligently assisted by the model, thereby significantly improving the overall production efficiency and data quality.

[0045] ​The scene criticality evaluation result participates in the model training process, that is, the model is trained in a sample weighting manner based on criticality. When the model is trained using the labeled data, the training samples are no longer simply randomly sampled, but are weighted according to the scene criticality evaluation result (score). The higher the scene criticality evaluation result (high score), the higher the weight of the sample in the loss function calculation, or the higher the probability of being selected in batch sampling, thereby guiding the model to focus on learning these critical edge scenes. In this way, a new closed loop of “working condition perception-dynamic labeling-weighted training” is formed, making the model iteration more targeted.

[0046] The weighted loss function can be as follows:

[0047] wherein, is the number of samples in a batch; is the original loss of the i-th sample; is the scene criticality score of the i-th sample; is a hyperparameter for adjusting the influence degree of the criticality score. A data management unit is configured to manage the to-be-labeled data, working condition data, and accepted data based on the constructed scene database and multi-dimensional dataset.

[0048] According to the type of data to be labeled, corresponding datasets such as image, audio, video, and text datasets are created. The data to be labeled includes vehicle exterior perception data (2D / 3D) and vehicle interior perception data (visual interaction, behavior recognition, etc.).

[0049] A scene database is constructed for different driving scenes (such as urban areas, highways, tunnels, parking, and loops), which can store 2D / 3D datasets and the like that have been labeled and accepted for a long time.

[0050] The data management supports centralized storage, classification, and label management of the labeled data, and provides multi-dimensional retrieval functions. Users can quickly retrieve samples according to label categories, scene attributes, timestamps, geographic locations, and the like. The data management supports dataset version management: each labeling output forms an independent version and records the modification history. It supports users to flexibly construct data subsets with different training targets and output datasets in multiple standard formats. Through the perfect data governance and version control of the data management unit, the traceability and reusability of data flow are guaranteed, which provides protection for continuous iteration and data monitoring.

[0051]

[0052] ​​​The embodiment also provides a data labeling method for an automatic driving scene, and utilizes a data labeling system for the automatic driving scene. S1, a scene database and a multi-dimensional data set are constructed, and working condition data and to-be-labeled data are managed; S2, a labeling project is created and to-be-labeled data are obtained; the to-be-labeled data are associated with corresponding working condition data, and scene keyness evaluation is performed, and a scene keyness evaluation driving mechanism is introduced to generate a dynamic strategy, the strategy including a task strategy and a quality inspection strategy; S3, the to-be-labeled data are split according to the task strategy and task allocation is performed; S4, an intelligent labeling model is trained based on a data set corresponding to an automatic driving general category; manual labeling is performed by utilizing a plurality of labeling tools built-in and pre-labeling performed by utilizing the trained intelligent labeling model, and finally labeled data are output; S5, the labeled data are subjected to quality inspection according to the quality inspection strategy to output accepted data meeting a quality standard; the accepted data are managed based on S1; S6, the intelligent labeling model is trained and updated based on the accepted data; wherein a scene keyness evaluation result participates in a model training process.

[0053] As Figure 2 shown is a main labeling process of the application. It can be understood that the execution process and effect of the above method and the above system are completely the same.

[0054] A typical automatic driving data labeling project is taken as an example to illustrate the actual application process of the application, including: S1, project creation and configuration: a labeling project is newly created by a project manager or a data administrator in a labeling platform, labeling specifications (such as target categories, labeling formats, etc.) and personnel permissions are specified, and original data sets (including camera images and laser radar point clouds, etc.) to be labeled are imported. The platform cuts or frames the data according to the settings to generate assignable task sheets.

[0055] S2, task distribution and pre-labeling: the labeling platform distributes the task sheets to suitable labeling personnel or teams according to a task strategy, and simultaneously calls a current pre-training model (an intelligent labeling model) in a model training management unit to automatically generate initial labeling results (such as automatically detecting vehicle bounding boxes, segmenting pedestrians, extracting lane lines, etc.) for each task. These intelligent pre-labeling results serve as an aid, and labeling personnel perform review, correction and refinement on an interactive interface.

[0056] S3, Artificial labeling and collaboration: Labeling personnel use platform tools (polygon labeling, segmentation editing, key point labeling, speech tag labeling, etc.) to manually label data according to task requirements, such as adding a three-dimensional bounding box to laser radar point cloud and setting dynamic attributes, marking event trigger points for video frames, labeling lane center lines on navigation maps, etc. The system also supports multiple people working together, and multiple people can collaborate on the same data to be labeled in complex scenarios and merge the final labeling results.

[0057] S4, Quality control and acceptance: After labeling is completed, the task enters the quality inspection stage. Quality inspectors or automatic quality inspection algorithms will compare multiple labeling results to check labeling integrity, boundary consistency, and label rationality, etc. For example, the system can automatically detect problems such as size abnormalities, category inconsistencies, and ID drift of labeled boxes, and return suspected error items to human re-auditing. After multiple cross-auditing and AI automatic quality inspection, the labeling result meets the quality standard and can pass acceptance. Data that does not pass acceptance will be returned to the previous step for correction by the labeling staff.

[0058] S5, Data storage and version management: The accepted data set is written into the data management unit, and a new data version is generated. The platform automatically records labeling time, participants, and version number, etc. User can query the labeled data through tags, scene categories, collection time or quality score, etc. in the platform retrieval function, and export data subsets as needed.

[0059] S6. Model training and backflow: For accepted data, the model training management module can automatically trigger the training task, and use new labeled data for model training. After training is completed, the new model parameters are deployed back to the labeling platform for automatic pre-labeling of the next round of data, thereby forming a closed-loop feedback of labeling-model iteration. This closed-loop mechanism can continuously improve the model accuracy, while enabling the system to quickly adapt to new scenarios.

[0060] S7. Team organization and management: The platform personnel management function can grade and performance evaluate labeling personnel, and provide training and certification for new personnel; the project management function can monitor project progress in real time and adjust task allocation. For complex labeling tasks, a hybrid mode of "multiple labeling + expert arbitration + automatic quality inspection" is adopted to ensure data quality.

[0061] The actual application process of the present application is illustrated by taking the whole process of enabling an intelligent labeling model from data set creation to intelligent labeling result export as an example, as shown in Figure 2 , including: S1, Data set creation, create corresponding data sets according to the type of data to be labeled, such as image, audio, video, text, etc. S2, project creation, new task project to be labeled, such as image annotation test, and binding with the image dataset created in S1 (one project can only bind one dataset); S3, annotation configuration, annotation type can be selected as general image annotation or image semantic segmentation, including operation configuration of annotation process functionality, single question attribute configuration, annotation tool configuration, and corresponding tool requirement object class attribute configuration; S4, new task, one project corresponds to one configuration, and there can be multiple tasks in one project, such as image annotation or image segmentation; S5, add data, analyze whether the project data (to-be-labeled data) meets the upload format requirements, including compression format and compression package directory structure; S6, automatic annotation configuration: configure intelligent annotation model (including inference algorithm model and intelligent annotation service URL); According to the need for annotation categories, provide the corresponding data set (i.e. the data set corresponding to the general categories of autonomous driving scenes), take car as an example, provide images containing car with quantity ≥1000, and perform stratified sampling on the data set, divide it into training set, validation set and test set according to the ratio of 7:2:1; perform random horizontal flip (probability p=0.5), random cropping (scaling range [0.8, 1.2]) and Mosaic mixing operation (stitching 4 images) on the training set images; perform fixed size scaling (640x640) and normalization processing (pixel value mapping to [-1, 1] interval) on the validation set and test set.

[0062] Model training phase: initialize YOLO network structure and corresponding dataset yaml file, load pre-trained weight; set epoch, batchsize and other training parameters to start training; calculate the average precision mAP@[0.5:0.95] on the validation set, and trigger the early stopping mechanism when there is no improvement for S consecutive epochs (S=15).

[0063] S7, automatic labeling execution: directly call the model for intelligent pre-labeling each time a new labeling task is created, keep the automatic labeling program running in the local area network, use the model trained in S8 as the inference algorithm model and fill in the service URL address 'http: / / 10.130.3131.124.:5000' at the specified position of the platform'service URL', pass in the resource path and prompt information through HTTP request in the labeling process, select the intelligent labeling button on the labeling page to start the inference algorithm model to obtain the intelligent labeling result data [0 0.1865234375 0.3541114058355438 0.080078125 0.18302387267904510 0.3828125 0.38726790450928383 0.08203125 0.17506631299734748 0 0.671875 0.5039787798408488 0.07421875 0.14323607427055704 0 0.8056640625 0.2864721485411141 0.076171875 0.17506631299734748] to realize pre-labeling before manual labeling; S8, labeling result synchronization: manually synchronize the intelligent marking result to the data set; S9, data export: export the data set with intelligent labeling results.

[0064] The practical application process of the application is illustrated and described by taking an automatic emergency braking (AEB) scene data labeling process of a fusion vehicle dynamic working condition as an example, including: S1, data acquisition and synchronization association: a test vehicle performs AEB test in a test field, the vehicle-mounted system records the front camera video, laser radar point cloud, and working condition data such as vehicle speed, longitudinal acceleration and brake pedal opening degree through CAN bus. After the data enters the platform, the 'working condition association and strategy generation module' automatically associates each frame of image and point cloud with the working condition data corresponding to the time stamp.

[0065] S2, scene key evaluation and task distribution: the system analyzes the working condition of the data segment. Before AEB triggering, the vehicle is running at high speed (v is large), and the acceleration is close to 0. When AEB is triggered, the longitudinal acceleration a lon (t) appears a sharp negative peak. The system calculates a very high key score for this data segment according to the preset calculation formula. Then, the labeling management unit marks this task as 'highest priority' and immediately distributes the task to a senior labeler.

[0066] S3, cooperative labeling of working conditions: the labeling interface not only displays the camera image and point cloud, but also displays a synchronized working condition curve (showing the speed and acceleration changes). When labeling the three-dimensional bounding box of the front obstacle, the interface automatically prompts the labeler to add an "emergency braking" event label and requires accurate labeling of the relative distance between the obstacle and the vehicle. lon (t) dramatic changes, automatically prompting the labeler to add an "emergency braking" event label and requiring accurate labeling of the relative distance between the obstacle and the vehicle.

[0067] S4, dynamic quality control: due to the high score of this task, the system automatically triggers the "expert mandatory review" process after the initial labeling is completed. Another experienced expert will review the labeling results to ensure that the accuracy of the bounding box, the continuity of the object ID, and the accuracy of the event label all meet the highest standards.

[0068] S5, data storage and version management: after acceptance, the labeled data (including images, point clouds, labels, and associated working condition scores) are stored in the "emergency obstacle avoidance scene library" of the data management unit and a new data set version is generated.

[0069] S6, weighted model training and closed-loop feedback: in the next round of model training, this AEB data segment is given a higher weight due to its high score in the training process. The model will focus on optimizing its prediction results in this scenario when calculating the loss. The trained new model is deployed back to the labeling management unit (labeling platform), and its pre-labeling ability is significantly enhanced in similar emergency braking scenarios, providing more accurate initial labeling suggestions in subsequent tasks.

[0070] The embodiment provides a data labeling system and method for an automatic driving scene, which realizes a scene-driven, autonomous optimization intelligent cooperative human-machine labeling mode through multi-module deep linkage and dynamic strategy generation.

[0071] The above is only an embodiment of the present application, and the common knowledge of the specific structure and characteristics in the scheme is not described in detail. The ordinary skilled person in the art knows all the ordinary technical knowledge in the field of the application before the application date or the priority date, can know all the prior art in the field, and has the ability to apply conventional experimental means before the date. The ordinary skilled person in the art can improve and implement the present scheme based on their own ability under the guidance of the present application. Some typical known structures or known methods should not be an obstacle for the ordinary skilled person in the art to implement the present application. It should be noted that for those skilled in the art, without departing from the structure of the present application, a number of modifications and improvements can be made, which should be considered as the protection scope of the present application. These will not affect the effect and practicality of the present application.

Claims

1. A data annotation system for autonomous driving scenarios, characterized in that, include: The annotation management unit includes a project management module for creating annotation projects and obtaining data to be annotated, a task module for splitting data to be annotated and assigning tasks according to task strategies, an annotation scenario support module with built-in annotation tools, a visualization annotation module for calling the currently trained intelligent annotation model to perform pre-annotation before manual annotation and combining it with the annotation scenario support module to perform manual annotation and finally output the annotated data, and a quality control module for performing quality inspection on the annotated data according to a multi-level quality inspection strategy to output the accepted data. The working condition association and strategy generation unit is used to associate the data to be labeled with the corresponding working condition data, and to perform scenario criticality assessment. It also introduces a scenario criticality assessment-driven mechanism to generate dynamic strategies, including task strategies and quality inspection strategies. The model training management unit is used to build intelligent labeled models based on deep learning frameworks. It is also used for model training based on accepted data, and the results of scenario criticality assessment are involved in the model training process; The data management unit is used to manage unlabeled data, operational data, and accepted data based on the constructed scenario database and multidimensional dataset.

2. The data annotation system for autonomous driving scenarios according to claim 1, characterized in that, The annotation scenario support module is used for image-level annotation, 3D point cloud-level annotation, cross-modal joint annotation, and natural language-level annotation. Among them, cross-modal joint annotation involves first annotating at one level and then mapping the annotation results to another level for synchronous annotation.

3. The data annotation system for autonomous driving scenarios according to claim 1, characterized in that, The visualization annotation module is also used to support multiple people to collaboratively annotate the same data in complex scenarios and merge the data to form the final annotated data.

4. A data annotation system for autonomous driving scenarios according to claim 1, characterized in that, Scenario criticality assessment involves calculating a scenario criticality score for each data segment based on its associated operating conditions. This quantifies the importance and rarity of the data segment for training autonomous driving algorithm models. Calculate using the following formula: in, This is the feature vector of the working condition; These are the weight coefficients for the corresponding features; This is the normalization function; This is an additional factor for environmental complexity determined by environmental factors.

5. A data annotation system for autonomous driving scenarios according to claim 1, characterized in that, The scenario-criticality assessment-driven mechanism for dynamic strategy generation includes at least one of the following: Based on the assessment results of the criticality of the scenario, the priority of task allocation is determined, and a fusion model is constructed to combine the priority with the difficulty of the task and the ability of the annotators to determine the final task allocation strategy. The quality inspection level is determined based on the assessment results of the criticality of the scenario, and the corresponding quality inspection strategy is determined based on the quality inspection level. The higher the quality inspection level, the more stringent the quality inspection strategy is adopted.

6. A data annotation system for autonomous driving scenarios according to claim 5, characterized in that, The fusion model includes: employing a priority-driven sliding window allocation strategy, where the top K tasks are taken from the priority queue P each time as the current allocation window. ; For each window Calculate the matching score: in, Represents the tasks in the window ; Represents the set of annotators The annotator in ; Represents the tasks in the window and labelers Match score; Indicates the annotator capability feature vector With the task Task difficulty feature vector Cosine similarity; This indicates a compensation item that indicates the task difficulty exceeds the annotator's ability; This represents the adjustment coefficient that sums to 1; Based on binary decision variables Optimize the allocation, where A value of 1 indicates that the task will be completed. Assigned to annotators If the value is 0, no assignment is made; the model optimization objective is to find the maximum sum of matching scores. combination: 。 7. A data annotation system for autonomous driving scenarios according to claim 1, characterized in that, Quality inspection strategies include one or a combination of expert review, cross-audit by multiple personnel, random sampling, and AI-automated quality inspection.

8. A data annotation system for autonomous driving scenarios according to claim 1, characterized in that, In the model training management unit, model training is performed based on a weighted approach for key samples. Samples with better scene keyness evaluation results are given higher weights in the loss function calculation or are selected with higher probability in batch sampling.

9. A data annotation system for autonomous driving scenarios according to claim 1, characterized in that, The data management module manages data in at least one of the following ways: centralized storage, classification and labeling management; creating an independent version for each labeled output and recording the modification history; constructing data subsets for different training objectives and outputting datasets that conform to multiple standard formats.

10. A data annotation method for autonomous driving scenarios, characterized in that, A data annotation system for autonomous driving scenarios according to any one of claims 1-9; the method includes: S1, build a scene database and multidimensional dataset, and manage the data to be labeled and the working condition data; S2, create annotation projects and obtain data to be annotated; associate the data to be annotated with the corresponding working condition data, and perform scenario criticality assessment, and introduce a scenario criticality assessment-driven mechanism to generate dynamic strategies, including task strategies and quality inspection strategies. S3, split the data to be labeled according to the task strategy and assign tasks; S4 trains an intelligent annotation model based on a dataset corresponding to the general categories of autonomous driving scenarios; it then uses several built-in annotation tools and pre-annotations performed by the trained intelligent annotation model to perform manual annotations, and finally outputs the annotated data. S5 performs quality inspection on the labeled data according to the quality inspection strategy to output accepted data that meets the quality standards; and manages the accepted data based on S1. S6 trains and updates an intelligent labeled model based on the accepted data; the scene criticality assessment results are involved in the model training process.

Images