Data set construction method, model training method and device for automatic driving model

Abstract

This invention relates to the field of autonomous driving technology and discloses a dataset construction method, model training method, and device for autonomous driving models. The dataset construction method for autonomous driving models includes: performing object detection on acquired vehicle driving scene data to obtain detection information for the vehicle driving scene data; generating hierarchical semantic annotation information with perception-prediction-decision logic based on the vehicle driving scene data and the detection information; and constructing a dataset for autonomous driving models based on the detection information and the hierarchical semantic annotation information. This invention, through the above scheme, constructs an autonomous driving model dataset with hierarchical semantic annotation information containing perception-prediction-decision logic. When training an autonomous driving model using this dataset, the training effect of the autonomous driving model can be improved, increasing the accuracy and stability of the trained autonomous driving model.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving technology, and in particular to a dataset construction method, model training method, and device for autonomous driving models. Background Technology

[0002] With the widespread application of Vision-Language-Behavior (VLA) large-scale models in autonomous driving technology, large-scale, high-quality datasets have become crucial for training and evaluating these models. Currently, mainstream autonomous driving datasets are Vision-Language Question Answering datasets (such as DriveMLLM and AutoTrust) designed for autonomous driving question answering tasks. However, most current Vision-Language Question Answering datasets employ isolated, single-perspective question-and-answer pairs, lacking a logical progression between them. In contrast, large-scale VLA models encompass perception, prediction, and decision-making stages. Therefore, current Vision-Language Question Answering datasets are insufficient to meet the training requirements of VLA models. Summary of the Invention

[0003] In view of this, the present invention proposes a dataset construction method, model training method and device for autonomous driving models, which solves the technical problem that the question-answer pairs in traditional visual-language question answering datasets are of a single form and lack logical progression, making it difficult to meet the training requirements of VLA models.

[0004] On one hand, embodiments of the present invention provide a method for constructing a dataset for autonomous driving models, characterized by comprising: Target detection is performed on the acquired vehicle driving scene data to obtain the detection information of the vehicle driving scene data; Based on the vehicle driving scenario data and the detection information, hierarchical semantic annotation information with perception-prediction-decision logic is generated; A dataset for autonomous driving models is constructed based on the detection information and the hierarchical semantic annotation information.

[0005] In some implementations, the hierarchical semantic annotation information includes sequentially generated global scene description text, detailed analysis and behavior prediction text of the target to be detected, and vehicle driving decision suggestions.

[0006] In some implementations, generating hierarchical semantic annotation information with perception-prediction-decision logic based on the vehicle driving scenario data and the detection information includes: Based on the vehicle driving scenario data, a global scenario description text for the vehicle driving scenario data is generated; Based on the vehicle driving scene data, the global scene description text, and the detection information of the target to be detected in the vehicle driving scene data, a detailed analysis and behavior prediction text of the target to be detected is generated. Based on the vehicle driving scenario data, the global scenario description text, and the detailed analysis and behavior prediction text of the target to be detected, vehicle driving decision suggestions are generated.

[0007] In some implementations, generating hierarchical semantic annotation information with perception-prediction-decision logic based on the vehicle driving scenario data and the detection information includes: Based on the vehicle driving scenario data and the detection information, guided by a hierarchical question template, a global scenario description text, a detailed analysis and behavior prediction text of the target to be detected, and vehicle driving decision suggestions are generated sequentially.

[0008] In some implementations, after performing target detection on the acquired vehicle driving scene data to obtain detection information for the vehicle driving scene data, the method further includes: Cross-validation is performed based on the detection information to obtain the cross-validation results; In response to the cross-validation result meeting preset conditions, the step of generating hierarchical semantic annotation information with perception-prediction-decision logic based on the vehicle driving scenario data and the detection information is executed.

[0009] In some implementations, cross-validation is performed based on the detection information to obtain cross-validation results, including: Based on the detection information, spatial consistency verification, label semantic consistency verification, and topological rationality verification are performed in parallel to obtain spatial consistency verification results, label semantic consistency verification results, and topological rationality verification results, respectively. Based on the spatial consistency verification result, the label semantic consistency verification result, and the topological rationality verification result, the cross-validation result is determined.

[0010] In some implementations, spatial consistency verification, label semantic consistency verification, and topological rationality verification are performed in parallel based on the detection information, and the spatial consistency verification results, label semantic consistency verification results, and topological rationality verification results are obtained respectively: Spatial consistency verification is performed based on the detection box information of dynamic objects in the detection information to obtain the spatial consistency verification result. Based on the classification information of dynamic objects in the detection information, the semantic consistency of the labels is verified to obtain the semantic consistency verification result of the labels. Topology rationality verification is performed based on the static road detection information and / or the detection frame information of the dynamic object in the detection information to obtain the topology rationality verification result.

[0011] In some implementations, the detection box information includes 2D detection box information and 3D detection box information; spatial consistency verification is performed based on the detection box information of dynamic objects in the detection information to obtain spatial consistency verification results, including: Based on the 3D detection box information of the dynamic object, the 2D projection information of the dynamic object is determined, and spatial consistency verification is performed based on the 2D projection information and the 2D detection box information of the dynamic object to obtain the spatial consistency verification result.

[0012] On the other hand, embodiments of the present invention also provide an autonomous driving model training method, which includes training an autonomous driving model based on a dataset obtained by the method described in any of the above embodiments.

[0013] On the other hand, embodiments of the present invention also provide an electronic device, including: at least one processor; and a memory storing a computer program executable on the processor, wherein the processor executes the steps of the method as described in any of the preceding embodiments when executing the program.

[0014] The present invention has at least the following beneficial effects: This invention provides a dataset construction method, model training method, and device for autonomous driving models. The dataset construction method includes: performing object detection on acquired vehicle driving scene data to obtain detection information; generating hierarchical semantic annotation information with perception-prediction-decision logic based on the vehicle driving scene data and the detection information; and constructing a dataset for autonomous driving models based on the detection information and the hierarchical semantic annotation information. This invention achieves an automatic annotation scheme for autonomous driving datasets. These datasets possess semantic annotation information that is closely related to and logically coherent with perception results (i.e., hierarchical semantic annotation information with perception-prediction-decision logic). When training an autonomous driving model using this dataset, it helps the autonomous driving model better understand vehicle driving scene data, improving the training effect of the autonomous driving model. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other embodiments can be obtained based on these drawings without creative effort.

[0016] Figure 1 A flowchart illustrating a dataset construction method for autonomous driving models provided in this embodiment of the invention; Figure 2 This is a flowchart illustrating the generation of hierarchical semantic annotation information in a dataset construction method for autonomous driving models provided in an embodiment of the present invention. Figure 3 This is a flowchart illustrating another method for generating hierarchical semantic annotation information in the dataset construction method for autonomous driving models provided in this embodiment of the invention. Figure 4 A flowchart illustrating another method for constructing a dataset for an autonomous driving model, as provided in an embodiment of the present invention; Figure 5 A flowchart illustrating another method for constructing a dataset for an autonomous driving model, as provided in an embodiment of the present invention; Figure 6 A flowchart illustrating a cross-validation process in a dataset construction method for autonomous driving models provided in an embodiment of the present invention; Figure 7 A flowchart illustrating another cross-validation method in the dataset construction method for autonomous driving models provided in this embodiment of the invention; Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be further described in detail below with reference to specific examples and the accompanying drawings.

[0018] It should be noted that all uses of "first" and "second" in the embodiments of the present invention are for the purpose of distinguishing two entities or parameters with the same name but different names. It is clear that "first" and "second" are only for the convenience of expression and should not be construed as limiting the embodiments of the present invention. Subsequent embodiments will not explain this in detail.

[0019] With the widespread application of Visual-Language-Behavior (VLA) large-scale models in autonomous driving technology, large-scale, high-quality datasets have become crucial for training and evaluating these models. Currently, mainstream autonomous driving datasets are visual-language question-answering datasets (such as DriveMLLM and AutoTrust) designed for autonomous driving question-answering tasks. However, these datasets mostly employ isolated, single-perspective question-answer pairs, lacking logical progression between questions and answers. This fails to guide VLA models in coherent reasoning, thus limiting their ability to understand and respond to complex scenarios when training VLA models with decision-making capabilities based on existing datasets. This negatively impacts training performance, resulting in low accuracy and poor stability in the trained VLA models.

[0020] The present invention will now be described in detail with reference to the embodiments and accompanying drawings.

[0021] The first aspect of this invention provides a method for constructing a dataset for autonomous driving models, such as... Figure 1 As shown, the method specifically includes steps S10 to S30.

[0022] S10. Perform target detection on the acquired vehicle driving scene data to obtain the detection information of the vehicle driving scene data.

[0023] Vehicle driving scene data may include: vehicle driving scene images (e.g., front-view or multi-view images captured by vehicle cameras) and vehicle driving scene point clouds (e.g., radar point clouds captured by vehicle lidar).

[0024] The detection information can include the detection bounding box information, category labels, confidence scores, and other information for each detection object in the vehicle driving scenario data.

[0025] In this embodiment of the invention, vehicle driving scene data can be obtained from the vehicle, and then target detection can be performed on the obtained vehicle driving scene data to obtain detection information of the vehicle driving scene data.

[0026] S20. Based on vehicle driving scenario data and detection information, generate hierarchical semantic annotation information with perception-prediction-decision logic.

[0027] Hierarchical semantic annotation information can include sequentially generated global scene description text, detailed analysis and behavior prediction text of the target to be detected, and semantic annotation information such as vehicle driving decision suggestions.

[0028] In one specific embodiment, a hierarchical question template with perception-prediction-decision logic can be configured. Under the guidance of the hierarchical question template, based on vehicle driving scenario data and detection information, hierarchical semantic annotation information with perception-prediction-decision logic, such as global scenario description text, detailed analysis and behavior prediction text of the target to be detected, and vehicle driving decision suggestions, can be generated sequentially.

[0029] In one specific embodiment, the hierarchical question template may include a global perception question template, a local perception question template, and a driving suggestion question template, which are used to generate global scene description text, detailed analysis and behavior prediction text of the target to be detected, and vehicle driving decision suggestions, respectively. The global perception question template and global scene description text, the local perception question template and detailed analysis and behavior prediction text of the target to be detected, and the driving suggestion question template and vehicle driving decision suggestions respectively form question-answer pairs for the construction of the dataset.

[0030] In one specific embodiment, the subsequently generated semantic annotation information is generated based on the semantic annotation information generated previously. For example, the detailed analysis and behavior prediction text of the target to be detected is generated based on the global scene description text, and the vehicle driving decision suggestion is generated based on the detailed analysis and behavior prediction text of the target to be detected and the global scene description text.

[0031] In this embodiment of the invention, based on vehicle driving scenario data and its detection information, and guided by a hierarchical question template, a semantic answer that is closely related to the perception result and logically coherent can be generated, thereby obtaining high-quality hierarchical text annotation and completing a complete data loop from perception to decision-making.

[0032] S30. Construct a dataset for autonomous driving models based on detection information and hierarchical semantic annotation information.

[0033] In this embodiment of the invention, a dataset for autonomous driving models (hereinafter referred to as autonomous driving dataset) can be constructed based on vehicle driving scenario data, corresponding detection information, and hierarchical semantic annotation information, thereby improving the training effect of autonomous driving models.

[0034] In this embodiment of the invention, the autonomous driving model can be a VLA model. The VLA model is trained based on a constructed autonomous driving dataset. The hierarchical semantic annotation information in the dataset corresponds to the perception-prediction-decision stage of the VLA model. Specifically, the global scene description text corresponds to the perception stage of the VLA model, the detailed analysis and behavior prediction text of the target to be detected corresponds to the prediction stage of the VLA model, and the vehicle driving decision suggestion corresponds to the decision stage of the VLA model. This hierarchical semantic annotation information can help the VLA model better understand the vehicle driving scene data, thereby improving the training effect of the VLA model and improving the accuracy and stability of the trained VLA model.

[0035] This invention provides an automatic annotation scheme for autonomous driving datasets by performing target detection on acquired vehicle driving scene data to obtain detection information. Based on the vehicle driving scene data and detection information, hierarchical semantic annotation information with perception-prediction-decision logic is generated. Based on the detection information and hierarchical semantic annotation information, a dataset for autonomous driving models is constructed, realizing an automatic annotation scheme for autonomous driving datasets. This autonomous driving dataset has semantic annotation information that is closely related to perception results and is logically coherent. When training an autonomous driving model using this dataset, it can correspond to the perception-prediction-decision stage of the autonomous driving model, thereby helping the autonomous driving model better understand vehicle driving scene data, improving the training effect of the autonomous driving model, and improving the accuracy and stability of the trained autonomous driving model.

[0036] In some embodiments of the present invention, such as Figure 2 As shown, hierarchical semantic annotation information can be generated based on steps S210~S230.

[0037] S210. Based on vehicle driving scenario data, generate a global scenario description text for the vehicle driving scenario data.

[0038] The global scene description text can include an overall picture of the vehicle driving scene data, such as weather, road type, and all visible traffic participants and their approximate locations.

[0039] In one specific embodiment, the vehicle driving scene data can be a vehicle driving scene image, thereby generating a global scene description text for the vehicle driving scene image. More specifically, a global scene description text for the vehicle driving scene data can be generated based on the vehicle driving scene image and a multimodal large language model (e.g., LLM, Qwen-VL, etc.).

[0040] In one specific embodiment, a global scene description text for the vehicle driving scene image can also be generated based on a hierarchical question template with perception-prediction-decision logic and the vehicle driving scene image. More specifically, the hierarchical question template may include an overall perception question template, a local perception question template, and a driving suggestion question template, and a global scene description text for the vehicle driving scene image can be generated based on the overall perception question template and the vehicle driving scene image. For example, the overall perception question template and the vehicle driving scene image can be used as input to a multimodal large language model, and the global scene description text for the vehicle driving scene image can be output based on the multimodal large language model.

[0041] S220. Based on vehicle driving scene data, global scene description text, and detection information of the target to be detected in the vehicle driving scene data, generate detailed analysis and behavior prediction text of the target to be detected.

[0042] The targets to be detected are traffic participants that affect vehicle driving in a vehicle driving scenario.

[0043] The detailed analysis and behavior prediction text of the target to be detected includes the target's attributes, state, and behavior prediction results. For example, in vehicle driving scenario data, the vehicle model, speed, distance to the vehicle in front, and possible behavior in the next few seconds are all data related to a vehicle.

[0044] In one specific embodiment, the vehicle driving scene data can be a vehicle driving scene image. Based on the vehicle driving scene image, its global scene description text, and the detection information of the target object, a detailed analysis and behavior prediction text for the target object can be generated. More specifically, a multimodal large language model can be used to generate the detailed analysis and behavior prediction text for the target object based on the vehicle driving scene image, its global scene description text, and the detection information of the target object.

[0045] In one specific embodiment, detailed analysis and behavior prediction text for the target to be detected can also be generated based on a hierarchical question template with perception-prediction-decision logic, a vehicle driving scene image, a global scene description text for the vehicle driving scene image, and detection information of the target to be detected. More specifically, detailed analysis and behavior prediction text for the target to be detected can be generated based on a local perception question template, a vehicle driving scene image, a global scene description text for the vehicle driving scene image, and detection information of the target to be detected. For example, the local perception question template, the vehicle driving scene image, the global scene description text for the vehicle driving scene image, and detection information of the target to be detected can be used as input to a multimodal large language model, and detailed analysis and behavior prediction text for the target to be detected can be output based on the multimodal large language model.

[0046] S230. Based on vehicle driving scenario data, global scenario description text, and detailed analysis and behavior prediction text of the target to be detected, generate vehicle driving decision suggestions.

[0047] Vehicle driving decision suggestions include: driving instructions for the vehicle at the next moment, such as keeping the lane, slowing down, changing lanes to overtake, or stopping and waiting.

[0048] In this embodiment of the invention, vehicle driving decision suggestions can be generated by using vehicle driving scene data, global scene description text, and detailed analysis and behavior prediction text of the target to be detected.

[0049] In this embodiment of the invention, by using vehicle driving scene data, global scene description text, and detailed analysis and behavior prediction text of the target to be detected, in addition to generating vehicle driving decision suggestions, an explanation of the vehicle driving decision suggestions can also be generated, that is, the reason for generating the vehicle driving decision suggestions.

[0050] In one specific embodiment, the vehicle driving scene data can be a vehicle driving scene image. Based on the vehicle driving scene image, its global scene description text, and detailed analysis and behavior prediction text of the target to be detected, vehicle driving decision suggestions can be generated. More specifically, vehicle driving decision suggestions can be generated based on a multimodal large language model, using the vehicle driving scene image, its global scene description text, and detailed analysis and behavior prediction text of the target to be detected.

[0051] In one specific embodiment, vehicle driving decision suggestions can also be generated based on a hierarchical question template with perception-prediction-decision logic, a vehicle driving scene image, a global scene description text of the vehicle driving scene image, and a detailed analysis and behavior prediction text of the target to be detected. More specifically, vehicle driving decision suggestions can be generated based on a local perception question template, a vehicle driving scene image, a global scene description text of the vehicle driving scene image, and a detailed analysis and behavior prediction text of the target to be detected. For example, the local perception question template, the vehicle driving scene image, the global scene description text of the vehicle driving scene image, and the detailed analysis and behavior prediction text of the target to be detected can be used as input to a multimodal large language model, and vehicle driving decision suggestions can be output based on the multimodal large language model.

[0052] The technical solution of this invention can generate a global scene description text for vehicle driving scene data based on vehicle driving scene data; generate a detailed analysis and behavior prediction text for the target to be detected based on the vehicle driving scene data, the global scene description text, and the detection information of the target to be detected in the vehicle driving scene data; and generate vehicle driving decision suggestions based on the vehicle driving scene data, the global scene description text, and the detailed analysis and behavior prediction text of the target to be detected. Thus, hierarchical semantic annotation information is constructed based on the global scene description text of the vehicle driving scene data, the detailed analysis and behavior prediction text of the target to be detected, and the vehicle driving decision suggestions. This ensures that the hierarchical semantic annotation information corresponds to the perception-prediction-decision stage of the VLA model. This hierarchical semantic annotation information helps the VLA model better understand the vehicle driving scene data, thereby improving the training effect of the VLA model and increasing the accuracy and stability of the trained VLA model.

[0053] The following specific embodiments illustrate the generation process of hierarchical semantic annotation information in the dataset construction method for autonomous driving models provided by the present invention. It should be understood that the following embodiments are only used to explain the present invention and are not intended to limit the present invention.

[0054] like Figure 2 and Figure 3As shown, the generation process of hierarchical semantic annotation information can include three progressive stages: the overall perception stage represented by step S210, the local perception and prediction stage represented by step S220, and the driving decision and suggestion stage represented by step S230.

[0055] In this embodiment, the hierarchical question templates include thought chain question templates for overall perception, local perception and prediction, and driving decision-making and suggestion, respectively. Thus, at each stage of hierarchical semantic annotation information generation, coherent and structured answers can be generated under the guidance of the hierarchical question templates. The specific details of each stage are described below.

[0056] Overall perception stage: Generate global scene description text from the current frame vehicle driving scene image and the overall perception question template.

[0057] The specific content of the overall perception question template can be: describe the entire driving scenario and list all key objects (such as vehicles, pedestrians, and traffic signs) and their global distribution. For example: "Please describe the overall situation of the current driving scenario, including the weather, road type, and all visible key traffic participants and their approximate locations."

[0058] Local perception and prediction stage: Based on the current frame vehicle driving scene image, the global scene description text output by the overall perception stage, the detection information of the target to be detected (e.g., the detection box information of the truck with ID37), and the local perception question template, a detailed analysis and behavior prediction text of the target to be detected is generated.

[0059] The specific content of the local perception question template can be: performing fine-grained attribute analysis, state judgment, and behavior prediction on key objects that affect the vehicle's driving identified in the preceding stage. For example, "For the truck (ID37) in the detection box in the figure, please analyze its vehicle type, speed status, distance from the vehicle in front, and predict its possible behavior in the next 3 seconds."

[0060] Driving decision and suggestion stage: Based on the current frame vehicle driving scene image, the global scene description text output by the overall perception stage, the detailed analysis and behavior prediction text of the target to be detected output by the local perception stage, and the driving suggestion question template, the vehicle's driving decision suggestions are generated and the reasons are explained.

[0061] The specific content of the driving suggestion question template can be as follows: Based on the text output from the first two stages, reason about the vehicle's driving behavior, and provide a final driving behavior decision suggestion with an explanation of the reasoning. For example, "Based on all the above scene information and object analysis, generate safe and reasonable next driving instructions for the ego car (such as keeping the lane, slowing down, changing lanes to overtake, or stopping and waiting), and explain the reasoning."

[0062] In this embodiment, the above process can be implemented using the Qianwen Multimodal Language Model. The current frame vehicle driving scene image, the detection information of the current frame vehicle driving scene image, and the pre-designed overall perception question template, local perception question template, and driving suggestion question template are input into the Qianwen Multimodal Language Model. The multimodal language model is guided to generate coherent and structured answers, and the overall perception question template and its corresponding text, the local perception question template and its corresponding text, and the driving suggestion question template and its corresponding text are output as complete question-answer pairs.

[0063] In this embodiment, the above scheme can simulate the cognitive process of human driving, and guide the large model to conduct coherent and structured reasoning based on the hierarchical thinking chain of "perception-prediction-decision".

[0064] In this embodiment, each question template and its corresponding semantic annotation information constitute a complete, logically coherent language question-and-answer pair that is closely related to the perception results. This can help autonomous driving models (such as VLA models) better understand vehicle driving scenario data, thereby improving the training effect of autonomous driving models and increasing the accuracy and stability of the trained autonomous driving models.

[0065] In some embodiments of the present invention, such as Figure 4 As shown, the dataset construction method for autonomous driving models proposed in this embodiment of the invention may include steps S400 to S450.

[0066] S400: Perform target detection on the acquired vehicle driving scene data to obtain detection information for the vehicle driving scene data.

[0067] Specifically, vehicle driving scene data may include: vehicle driving scene images (e.g., front-view or multi-view images captured by a vehicle camera) and vehicle driving scene point clouds (e.g., radar point clouds captured by a vehicle LiDAR). In this embodiment of the invention, multi-dimensional target detection can be performed on the acquired vehicle driving scene data to obtain multi-dimensional vehicle driving scene data detection information, thereby improving the richness of the final dataset. This helps the autonomous driving model better understand the driving scene data during the training process, thus enabling it to generate better vehicle driving behavior.

[0068] In some specific embodiments, the detection information may include the detection bounding box, category label and confidence level of each detection object in the vehicle driving scene data, as well as information such as lane line topology and drivable area.

[0069] In some specific embodiments, the detection objects may include static objects and dynamic objects. Static objects refer to immovable traffic participants in a driving scenario, such as any one or more of traffic control facilities, road facilities and obstacles, road functional signs, lane lines, and drivable areas. Dynamic objects refer to movable traffic participants in a driving scenario, such as any one or more of pedestrians, motor vehicles, and non-motorized vehicles. For each dynamic object in the vehicle driving scenario data, 2D detection information (e.g., 2D detection boxes and their corresponding category labels and confidence scores) and 3D detection information (e.g., 3D detection boxes and their corresponding type labels and confidence scores) may be included.

[0070] S410. Perform cross-validation based on the detection information to obtain the cross-validation results.

[0071] The objects to be detected can include static objects and dynamic objects. By detecting vehicle driving scene data, the detection information of all objects can be obtained.

[0072] Therefore, in vehicle driving scenarios, dynamic objects are the most critical moving objects and are the objects that autonomous driving models prioritize for accurate identification. Thus, to improve the accuracy of the dataset, cross-validation can be performed based on the detection information of dynamic objects.

[0073] S420. Determine the results of cross-validation.

[0074] If the cross-validation result meets the preset conditions, proceed to step S440; if the cross-validation result does not meet the preset conditions, proceed to step S430.

[0075] S430. Based on the cross-validation results that do not meet the preset conditions, as well as the corresponding vehicle driving scenario data and detection information, a review information is generated for verification.

[0076] Based on the cross-validation results that do not meet the preset conditions, along with the corresponding vehicle driving scenario data and detection information, review information is generated and output to the manual quality inspection platform for verification. Simultaneously, the system records the reasons why the cross-validation results do not meet the preset conditions, which are used for subsequent model performance diagnosis and optimization.

[0077] S440. Based on vehicle driving scenario data and detection information, generate hierarchical semantic annotation information with perception-prediction-decision logic.

[0078] In this embodiment, the specific implementation of step S440 is the same as that of step S20. Therefore, the specific implementation of step S440 will not be described in detail here.

[0079] S450. Construct a dataset for autonomous driving models based on detection information and hierarchical semantic annotation information.

[0080] Specifically, a dataset for autonomous driving models can be constructed based on vehicle driving scenario data and its corresponding multi-dimensional detection information and hierarchical semantic annotation information.

[0081] This invention provides a technical solution for constructing a dataset for autonomous driving models by performing target detection on acquired vehicle driving scene data to obtain detection information; performing cross-validation based on the detection information to obtain cross-validation results; judging the cross-validation results; generating review information for verification based on cross-validation results that do not meet preset conditions, along with the corresponding vehicle driving scene data and detection information; generating hierarchical semantic annotation information based on the vehicle driving scene data and detection information; and constructing a dataset for autonomous driving models based on the detection information and hierarchical semantic annotation information. This solution enables cross-validation of the consistency of annotation results from different modalities and perspectives, and removes annotation results that fail cross-validation, thereby obtaining a high-quality autonomous driving dataset. This improves the accuracy and reliability of the autonomous driving dataset, helping autonomous driving models better understand vehicle driving scene data, thus improving the training effect of autonomous driving models and increasing the accuracy and stability of the trained autonomous driving models.

[0082] The following specific embodiments illustrate the dataset construction method for autonomous driving models provided by the present invention. It should be understood that the following embodiments are only used to explain the present invention and are not intended to limit the present invention.

[0083] like Figure 5 As shown, the dataset construction method for autonomous driving models provided in this embodiment can include five stages: data acquisition, parallel perception, cross-validation, hierarchical semantic annotation, and dataset construction. By acquiring raw vehicle driving scene data from the vehicle and performing parallel perception, cross-validation, and hierarchical semantic annotation, an autonomous driving dataset rich in multi-dimensional annotation information (e.g., 2D / 3D detection, segmentation, topology, and language question-answering pairs) is finally obtained.

[0084] In this embodiment, parallel perception can be performed based on the road static information annotation module and the dynamic key object annotation model, cross-validation can be performed based on the intelligent cross-validation module, and hierarchical semantic annotation can be performed based on hierarchical semantic annotation.

[0085] The following is combined with Figure 5The specific processes of each stage are explained.

[0086] Data Acquisition: Obtain raw vehicle driving scene data from the vehicle, including multi-view driving scene images, driving scene point clouds, etc.

[0087] Parallel perception: Static information annotation and dynamic object annotation based on vehicle driving scene data.

[0088] Static information detection is performed on all detected objects in the vehicle driving scene data to obtain the corresponding static information detection results. Static information detection is used to output the 2D detection information of all detected objects. In this embodiment, static information detection includes three parts: open vocabulary object detection is performed on the input image based on the YOLO-World model to identify and locate four predefined categories of traffic elements (dynamic traffic participants, traffic control facilities, static road facilities and obstacles, and road functional signs); the lane line topology is inferred using the TopoMLP model; and the drivable area is segmented using the HybridNets model.

[0089] Dynamic objects are the core moving objects. Static information annotation can yield 2D detection information for dynamic objects in vehicle driving scene data. Building upon this, to improve the accuracy of annotation results and dataset construction, a 3D object detection network can be used to annotate dynamic objects, obtaining their 3D detection information. This allows for subsequent cross-validation based on the 2D and 3D detection information of the same dynamic object. In this embodiment, the dynamic object detection process is as follows: Based on the Sparse4D-V3 model, multi-view images and temporal information are fused to complete the 3D bounding box detection of dynamic objects and assign cross-frame tracking labels, thereby obtaining the 3D detection information for each dynamic object. The 3D detection information includes 3D bounding box (i.e., 3D detection box) information, confidence score, type label, and tracking label.

[0090] Cross-validation: Cross-validation is performed based on 2D and 3D detection information of dynamic objects, lane topology, and drivable areas. In this embodiment, such as... Figure 5 As shown, cross-validation includes 2D-3D spatial consistency verification, semantic label consistency verification, topological rationality verification, and confidence fusion and conflict labeling identification.

[0091] 2D-3D spatial consistency verification: Project the 3D detection box of the dynamic object onto the 2D image and verify the geometric position consistency between the two.

[0092] Semantic label consistency verification: Compare the classification labels of the same dynamic object in 2D and 3D detection to ensure semantic consistency.

[0093] Topology rationality verification: Using the trajectory and location of dynamic objects, lane topology and drivable area information, verify the rationality of the trajectory, location, drivable area and lane lines of dynamic objects.

[0094] Confidence fusion and conflict labeling identification: Based on the verification results of the first three steps, an overall confidence score is calculated for each dynamic object, and low-confidence labels with conflicts are automatically identified and marked to obtain high-confidence fusion labeling results.

[0095] Hierarchical semantic annotation: Using high-confidence fusion annotation results as structured input, the Qwen2.5-VL multimodal large language model is driven to generate language question-answer pairs that are closely related to the perception results and are logically coherent through the built-in hierarchical "perception-prediction-decision" prompt template. Finally, high-quality hierarchical text annotations are output, completing the complete data loop from perception to decision.

[0096] Dataset Construction: An autonomous driving model dataset is constructed based on the high-confidence fusion annotation results and their corresponding hierarchical semantic annotation information.

[0097] The high-confidence fusion annotation results include validated 2D detection information, 3D detection information, lane topology and drivable area, and confidence score.

[0098] The hierarchical semantic annotation information includes logically rich question-and-answer data corresponding to the high-confidence fusion annotation results.

[0099] The embodiments of the present invention, through the above-described scheme, can cross-validate the consistency of annotation results from different modalities and perspectives, and remove annotation results that fail cross-validation, thereby obtaining a high-quality autonomous driving dataset, improving the accuracy and reliability of the autonomous driving dataset, and thus helping the autonomous driving model to better understand vehicle driving scenario data, thereby improving the training effect of the autonomous driving model and improving the accuracy and stability of the trained autonomous driving model.

[0100] In some embodiments of the present invention, cross-validating the detection information to obtain cross-validation results may include: performing spatial consistency verification, label semantic consistency verification, and topological rationality verification in parallel based on the detection information to obtain spatial consistency verification results, label semantic consistency verification results, and topological rationality verification results, respectively; and determining the cross-validation results based on the spatial consistency verification results, label semantic consistency verification results, and topological rationality verification results.

[0101] Specifically, cross-validation can include spatial consistency verification, semantic label consistency verification, and topological rationality verification. Spatial consistency verification is used to verify the consistency between the 2D and 3D bounding boxes of dynamic objects. Semantic label consistency verification is used to verify the consistency between the 2D and 3D semantic labels of dynamic objects. Topological rationality verification is used to verify the rationality of the position of dynamic objects. In this embodiment of the invention, cross-validation of detection information can be performed based on the detection information of dynamic objects in all detection information.

[0102] In one specific embodiment, during spatial consistency verification, if the overlap between the 2D and 3D bounding boxes of a dynamic object is high, the spatial consistency verification passes, and the corresponding spatial consistency verification result is "verification passed." During semantic label consistency verification, if the 2D and 3D semantic labels of a dynamic object are consistent, the semantic label consistency verification passes, and the corresponding semantic label consistency verification result is "verification passed." During topological rationality verification, the position of the dynamic object on the road can be determined based on its 2D bounding box, and the determined position of the dynamic object is mapped to the drivable area and the road topology. If the mapped position of the dynamic object is within the coverage area of ​​the drivable area and does not conflict with the road topology, the topological rationality verification passes, and the corresponding topological rationality verification result is "verification passed." When spatial consistency verification, semantic label consistency verification, and topological rationality verification all pass, cross-validation passes; otherwise, cross-validation fails.

[0103] In one specific embodiment, the spatial consistency verification result can be represented by a spatial consistency score, the tag language consistency verification result by a tag semantic consistency score, and the topological rationality verification result by a topological rationality score. More specifically, when performing spatial consistency verification, semantic tag consistency verification, and topological rationality verification, spatial consistency score, semantic tag consistency score, and topological rationality score can be calculated, and the calculated spatial consistency score, semantic tag consistency score, and topological rationality score can be weighted and summed to obtain a comprehensive score. Based on this comprehensive score, it is determined whether cross-validation passes. If the comprehensive score is higher than a set threshold, cross-validation passes; otherwise, cross-validation fails.

[0104] In this embodiment of the invention, the vehicle driving scenario data may include one or more dynamic objects. When the vehicle driving scenario data contains one dynamic object, the cross-validation result can be obtained by determining whether the cross-validation passes based on the comprehensive score of the dynamic object. When the vehicle driving scenario data contains multiple dynamic objects, the comprehensive score of all dynamic objects can be calculated, and then the minimum comprehensive score can be obtained. The cross-validation result can be obtained by determining whether the cross-validation passes based on the minimum comprehensive score.

[0105] In this embodiment of the invention, spatial consistency verification, label semantic consistency verification, and topological rationality verification can be performed using detection information. Based on the results of these verifications, cross-validation results can be determined. Unreliable detection information can be eliminated based on the cross-validation results. Thus, based on reliable detection information and its corresponding hierarchical semantic annotation of vehicle driving scenario data, the accuracy and reliability of hierarchical semantic annotation information are improved, thereby enhancing the accuracy and reliability of the final constructed autonomous driving dataset. This results in a high-quality dataset, which helps the autonomous driving model better understand vehicle driving scenario data, thereby improving the training effect of the autonomous driving model and increasing the accuracy and stability of the trained autonomous driving model.

[0106] In some embodiments of the present invention, spatial consistency verification, label semantic consistency verification, and topological rationality verification can be performed in parallel based on the detection information of dynamic objects to obtain spatial consistency verification results, label semantic consistency verification results, and topological rationality verification results.

[0107] Specifically, such as Figure 6 As shown, spatial consistency verification, label semantic consistency verification, and topological rationality verification can be performed in parallel based on steps S610~S630 to obtain spatial consistency verification results, label semantic consistency verification results, and topological rationality verification results.

[0108] S610. Perform spatial consistency verification based on the detection box information of dynamic objects in the detection information to obtain the spatial consistency verification result.

[0109] In this embodiment of the invention, the detection box information of the dynamic object includes 2D detection box information and 3D detection box information. When determining the spatial consistency verification result, the 2D projection information of the dynamic object can be determined based on the 3D detection box information of the dynamic object, and spatial consistency verification can be performed based on the 2D projection information and the 2D detection box information of the dynamic object to obtain the spatial consistency verification result.

[0110] In one specific embodiment, the spatial consistency verification result can be represented by a spatial consistency score. When performing spatial consistency verification based on the 2D projection information and 2D detection box information of a dynamic object, the overlap between the 2D projection information and the 2D detection box information can be calculated, and the spatial consistency score can be determined based on the calculated overlap. A higher overlap indicates a higher degree of physical alignment between the 2D and 3D detection box information, resulting in a higher assigned spatial consistency score.

[0111] S620. Perform label semantic consistency verification based on the classification information of dynamic objects in the detection information to obtain the label semantic consistency verification result.

[0112] In this embodiment of the invention, the tag semantic consistency score can be used to represent the tag semantic consistency verification result. When performing tag semantic consistency verification, the semantic classification tags in the 2D detection information and the 3D detection information of the dynamic object can be compared, and a tag semantic consistency score is generated based on the comparison result. In one specific embodiment, if the comparison result shows that the semantic tags are consistent, a higher tag semantic consistency score is generated; if the comparison result shows that the semantic tags are inconsistent, a lower tag semantic consistency score is generated. Here, "higher" and "lower" are relative; specifically, it can be understood that the tag semantic consistency score generated when the semantic tags are consistent is greater than the tag semantic consistency score generated when the semantic tags are inconsistent. In one specific embodiment, if the comparison result shows that the semantic tags are consistent, a positive tag semantic consistency score is generated; if the comparison result shows that the semantic tags are inconsistent, a negative tag semantic consistency score is generated.

[0113] S630. Perform topology rationality verification based on static road detection information and / or dynamic object detection box information in the detection information to obtain the topology rationality verification result.

[0114] Specifically, static road detection information refers to the detection information about the road itself within a static object. Static road detection information can include lane line topology and / or drivable areas.

[0115] In this embodiment of the invention, the topology rationality verification result can be represented by a topology rationality score.

[0116] In some specific embodiments, when performing topology rationality verification, the position of a dynamic object on the road can be determined based on the 2D detection box of the dynamic object, and the determined position of the dynamic object can be mapped to the drivable area and the road topology. If the mapped position of the dynamic object is within the coverage of the drivable area and does not conflict with the road topology, a higher topology rationality score is assigned to the topology rationality verification; otherwise, a lower topology rationality score is assigned to the topology rationality verification.

[0117] In some specific embodiments, during topology rationality verification, a trajectory sequence of the dynamic object can be formed based on its historical and current positions. A conflict determination is then made between this trajectory sequence and the road topology. If no conflict occurs, a higher topology rationality score is assigned; otherwise, a lower score is assigned. This topology rationality score based on the vehicle trajectory sequence can be weighted and summed with the topology rationality score based on the vehicle position to obtain the final topology rationality score.

[0118] In some specific embodiments, conflict determination can also be performed based on the drivable area and lane line topology during topology verification. Specifically, the drivable area is mapped onto the lane line topology to determine whether the position of the drivable area in the lane line topology complies with traffic rules. If it does, no conflict is determined; otherwise, a conflict is determined, and a lower topology rationality score is assigned to the topology rationality verification. In some embodiments, the topology rationality score based on the drivable area and lane line topology can be weighted and summed with the topology rationality score based on vehicle position to obtain the final topology rationality score. In some embodiments, the topology rationality score based on the drivable area and lane line topology can be weighted and summed with the topology rationality score based on vehicle position and the topology rationality score based on vehicle trajectory sequence to obtain the final topology rationality score.

[0119] Through the above technical solutions, this invention can accurately determine spatial consistency scores, label semantic consistency verification scores, and topological rationality scores, thereby determining the accuracy of cross-validation results. Based on these cross-validation results, low-reliability detection information and their corresponding semantic annotation information and vehicle driving scenario data can be excluded from the final constructed autonomous driving dataset. This results in a high-quality dataset, improving the accuracy and reliability of the final constructed autonomous driving dataset. Consequently, when training an autonomous driving model based on this dataset, it helps the autonomous driving model better understand vehicle driving scenario data, thereby improving the training effect of the autonomous driving model and enhancing the accuracy and stability of the trained autonomous driving model.

[0120] The cross-validation process in the dataset construction method for autonomous driving models provided by the present invention will be described below through specific embodiments. It should be understood that the following embodiments are only used to explain the present invention and are not intended to limit the present invention.

[0121] like Figure 7As shown, in this embodiment, cross-validation can include three parts: data association and matching, parallel validation, and confidence fusion and decision-making. The specific workflow of each part of cross-validation is described below.

[0122] Data association and matching: Processing asynchronous data streams from upstream.

[0123] Asynchronous data streams include: 2D labeled data streams (detection boxes and category labels from YOLO-World, etc.), 3D labeled data streams (3D bounding boxes, tracking markers and category labels from Sparse4D-V3, etc.), and topology and map data streams (lane line topology from TopoMLP and drivable areas from HybridNets).

[0124] The process involves mapping annotations from different sources onto the same real-world object. Through timestamp synchronization, 3D-to-2D projection matching, and tracking identifier association, a unique internal association ID is assigned to the same dynamic object across annotation results in different modalities.

[0125] Parallel verification process: After the asynchronous data stream is successfully associated, spatial consistency verification, semantic label consistency verification, and topological rationality verification are performed in parallel based on the various data of the dynamic object.

[0126] Spatial consistency verification: Calculate the IoU (Intersection over Union) between the region after the 3D bounding box is projected onto the image plane and the 2D detection box. The higher the IoU, the better the physical location matches between the 3D and 2D perceptions, and the higher the spatial consistency score.

[0127] Semantic label consistency verification: Compare the classification labels (e.g., car, truck, pedestrian) of the same dynamic object in 2D labeled data streams and 3D labeled data streams. Based on a pre-defined label mapping table, determine whether the two are semantically consistent (e.g., car with car is consistent, car with truck is conflicting), and output a semantic label consistency score.

[0128] Topology rationality verification: Based on static topological information such as lane lines, curbs, and drivable areas, determine whether the position and trajectory of dynamic objects are reasonable. For example, verify whether vehicles are driving within lanes, whether pedestrians are on reasonable road surfaces, whether their trajectories conflict with the road topology, and output a topology rationality score.

[0129] Confidence fusion and decision: Based on spatial consistency score, semantic label consistency score, topological rationality score and pre-learned weights (e.g., spatial consistency weight may be the highest), the three scores are weighted and fused to calculate the comprehensive confidence score of the dynamic object.

[0130] The overall confidence score is compared with a set threshold. If the overall confidence score is higher than the set threshold, it is determined to be a high-confidence result, marked as "verified," and output to the hierarchical semantic annotation module to drive question-answering generation. If the overall confidence score is lower than the threshold or a certain verification (such as label conflict) triggers a serious alarm, it is determined to be a low-confidence / conflicting label, packaged into a "label list requiring review," and output to the manual quality inspection platform to prompt reviewers to conduct key checks. Simultaneously, the conflict type is recorded for subsequent model performance diagnosis and optimization.

[0131] Through the above technical solutions, this invention can accurately determine spatial consistency scores, label semantic consistency verification scores, and topological rationality scores, thereby determining the accuracy of cross-validation results. Based on these cross-validation results, low-reliability detection information and their corresponding semantic annotation information and vehicle driving scenario data can be excluded from the final constructed autonomous driving dataset. This results in a high-quality dataset, improving the accuracy and reliability of the final constructed autonomous driving dataset. Consequently, when training an autonomous driving model based on this dataset, it helps the autonomous driving model better understand vehicle driving scenario data, thereby improving the training effect of the autonomous driving model and enhancing the accuracy and stability of the trained autonomous driving model.

[0132] Based on the same inventive concept, according to another aspect of the present invention, embodiments of the present invention also provide an autonomous driving model training method, which includes: training an autonomous driving model based on a dataset obtained by the method described in any of the above embodiments.

[0133] In this embodiment of the invention, the autonomous driving model can be a VLA model. The VLA model is trained based on a constructed autonomous driving dataset. The hierarchical semantic annotation information in the dataset corresponds to the perception-prediction-decision stage of the VLA model. Specifically, the global scene description text corresponds to the perception stage of the VLA model, the detailed analysis and behavior prediction text of the target to be detected corresponds to the prediction stage of the VLA model, and the vehicle driving decision suggestion corresponds to the decision stage of the VLA model. This hierarchical semantic annotation information can help the VLA model better understand the vehicle driving scene data, thereby improving the training effect of the VLA model and improving the accuracy and stability of the trained VLA model.

[0134] This invention provides a method for constructing an automatically labeled dataset by performing target detection on acquired vehicle driving scene data to obtain detection information. Based on the vehicle driving scene data and detection information, hierarchical semantic annotation information with perception-prediction-decision logic is generated. A dataset for autonomous driving models is constructed based on the detection information and hierarchical semantic annotation information. This autonomous driving dataset has language question-answer pairs that are closely related to the perception results and are logically coherent. When the VLA model is trained using this autonomous driving dataset, it corresponds to the perception-prediction-decision stage of the VLA model, thereby helping the VLA model to better understand the vehicle driving scene data, improving the training effect of the VLA model, and improving the accuracy and stability of the trained VLA model.

[0135] Based on the same inventive concept, according to another aspect of the present invention, such as Figure 8 As shown, an embodiment of the present invention also provides an electronic device 100, which includes a processor 110 and a memory 120. The memory 120 stores a computer program 121 that can run on the processor. When the processor 110 executes the program, it performs the steps of the method described above.

[0136] The memory, as a non-volatile storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules, such as the program instructions / modules corresponding to the methods described in the embodiments of this application. The processor executes various functional applications and data processing of the device by running the non-volatile software programs, instructions, and modules stored in the memory, thereby implementing the methods described in the above embodiments.

[0137] The memory may include a program storage area and a data storage area, wherein the program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the device, etc. Furthermore, the memory may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory may optionally include memory remotely located relative to the processor, which can be connected to the local module via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0138] Finally, it should be noted that those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium for the program can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc. The above computer program embodiments can achieve the same or similar effects as any of the corresponding foregoing method embodiments.

[0139] Those skilled in the art will also understand that the various exemplary logic blocks, modules, circuits, and algorithm steps described in conjunction with the disclosure herein can be implemented as electronic hardware, computer software, or a combination of both. To clearly illustrate this interchangeability between hardware and software, the functionality of various illustrative components, blocks, modules, circuits, and steps has been generally described. Whether this functionality is implemented as software or as hardware depends on the specific application and the design constraints imposed on the system as a whole. Those skilled in the art can implement the functionality in various ways for each specific application, but such implementation decisions should not be construed as departing from the scope of the embodiments disclosed herein.

[0140] The above are exemplary embodiments disclosed in this invention. However, it should be noted that various changes and modifications can be made without departing from the scope of the embodiments of this invention as defined by the claims. The functions, steps, and / or actions of the methods according to the disclosed embodiments described herein do not need to be performed in any particular order. The sequence numbers of the disclosed embodiments of this invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments. Furthermore, although the elements disclosed in the embodiments of this invention may be described or claimed individually, they may be understood as multiple unless explicitly limited to a singular number.

[0141] It should be understood that, as used herein, the singular form “a” is intended to include the plural form as well, unless the context clearly supports an exception. It should also be understood that, as used herein, “and / or” refers to any and all possible combinations of one or more of the associated listed items.

[0142] Those skilled in the art should understand that the discussion of any of the above embodiments is merely exemplary and is not intended to imply that the scope of the invention (including the claims) is limited to these examples. Within the framework of the invention, technical features of the above embodiments or different embodiments can be combined, and many other variations of different aspects of the invention exist, which are not provided in the details for the sake of brevity. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the invention should be included within the protection scope of the invention.

Claims

1. A method for constructing a dataset for autonomous driving models, characterized in that, include: Target detection is performed on the acquired vehicle driving scene data to obtain the detection information of the vehicle driving scene data; Based on the vehicle driving scenario data and the detection information, hierarchical semantic annotation information with perception-prediction-decision logic is generated; A dataset for autonomous driving models is constructed based on the detection information and the hierarchical semantic annotation information.

2. The method according to claim 1, characterized in that, The hierarchical semantic annotation information includes sequentially generated global scene description text, detailed analysis and behavior prediction text of the target to be detected, and vehicle driving decision suggestions.

3. The method according to claim 2, characterized in that, Based on the vehicle driving scenario data and the detection information, hierarchical semantic annotation information with perception-prediction-decision logic is generated, including: Based on the vehicle driving scenario data, a global scenario description text for the vehicle driving scenario data is generated; Based on the vehicle driving scene data, the global scene description text, and the detection information of the target to be detected in the vehicle driving scene data, a detailed analysis and behavior prediction text of the target to be detected is generated. Based on the vehicle driving scenario data, the global scenario description text, and the detailed analysis and behavior prediction text of the target to be detected, vehicle driving decision suggestions are generated.

4. The method according to claim 2, characterized in that, Based on the vehicle driving scenario data and the detection information, hierarchical semantic annotation information with perception-prediction-decision logic is generated, including: Based on the vehicle driving scenario data and the detection information, guided by a hierarchical question template, a global scenario description text, a detailed analysis and behavior prediction text of the target to be detected, and vehicle driving decision suggestions are generated sequentially.

5. The method according to claim 1, characterized in that, After performing target detection on the acquired vehicle driving scene data to obtain the detection information of the vehicle driving scene data, the method further includes: Cross-validation is performed based on the detection information to obtain the cross-validation results; In response to the cross-validation result meeting preset conditions, the step of generating hierarchical semantic annotation information with perception-prediction-decision logic based on the vehicle driving scenario data and the detection information is executed.

6. The method according to claim 5, characterized in that, Based on the detection information, cross-validation is performed to obtain the following cross-validation results: Based on the detection information, spatial consistency verification, label semantic consistency verification, and topological rationality verification are performed in parallel to obtain spatial consistency verification results, label semantic consistency verification results, and topological rationality verification results, respectively. Based on the spatial consistency verification result, the label semantic consistency verification result, and the topological rationality verification result, the cross-validation result is determined.

7. The method according to claim 6, characterized in that, Based on the detection information, spatial consistency verification, label semantic consistency verification, and topological rationality verification are performed in parallel, yielding spatial consistency verification results, label semantic consistency verification results, and topological rationality verification results, respectively: Spatial consistency verification is performed based on the detection box information of dynamic objects in the detection information to obtain the spatial consistency verification result. Based on the classification information of dynamic objects in the detection information, the semantic consistency of the labels is verified to obtain the semantic consistency verification result of the labels. Topology rationality verification is performed based on the static road detection information and / or the detection frame information of the dynamic object in the detection information to obtain the topology rationality verification result.

8. The method according to claim 7, characterized in that, The detection box information includes 2D detection box information and 3D detection box information; spatial consistency verification is performed based on the detection box information of dynamic objects in the detection information, and the spatial consistency verification results include: Based on the 3D detection box information of the dynamic object, the 2D projection information of the dynamic object is determined, and spatial consistency verification is performed based on the 2D projection information and the 2D detection box information of the dynamic object to obtain the spatial consistency verification result.

9. A method for training an autonomous driving model, characterized in that, include: The autonomous driving model is trained based on the dataset obtained by the method described in any one of claims 1-8.

10. An electronic device, comprising: At least one processor; as well as A memory storing a computer program executable on the processor, characterized in that the processor executes the program and performs the steps of the method as described in any one of claims 1 to 9.

Images