Tag-Driven Closed-Loop Simulation Method and System for Autonomous Driving Logs

Abstract

This application provides a label-driven closed-loop simulation method and system for autonomous driving logs, relating to the field of autonomous driving testing technology, aiming to solve the problems of efficiency, realism, and traceability in algorithm testing. The method includes: acquiring and cleaning vehicle log data of a fixed duration; performing consistency verification and adaptive semantic slicing based on event-type and fragment-type tags associated with the log data to generate a scene index; performing closed-loop backfeeding based on this index, updating sensor data affected by the output of the autonomous vehicle planning and control algorithm through a minimum necessary regeneration strategy; performing spatiotemporal alignment of the outputs of different algorithms and calculating a set of difference indicators; generating a difference report bound to the scene signature; automatically determining based on a preset threshold expression and triggering task orchestration based on a directed acyclic graph to drive data re-mining and secondary backfeeding, forming a data closed loop. This application realizes the intelligent and systematic iteration of autonomous driving algorithms.

Description

Technical Field

[0001] This invention relates to the field of closed-loop simulation of autonomous driving, and in particular to a label-driven closed-loop simulation method and system for autonomous driving logs. Background Technology

[0002] With the rapid development of autonomous driving and advanced driver assistance systems (ADAS) technologies, efficient and safe testing and verification of algorithms using simulation technology has become a crucial step in the R&D process. Traditional real-vehicle road testing is costly, time-consuming, and difficult to cover extreme scenarios. Therefore, playing back and testing driving logs collected from real roads in a simulation environment has become an important means of evaluating algorithm performance and accelerating iteration.

[0003] Currently, there are various existing technical solutions in this field. For example, a typical solution is log-based open-loop playback, which simply re-inputs the recorded data into the algorithm, failing to reflect the cascading effects of the algorithm's decisions on the environment, and thus limiting the depth of testing. Another solution involves semi-closed-loop object-level environment reconstruction, reproducing the vehicle environment and traffic flow in a simulator, but this typically simplifies the physical details of the sensors, affecting the realism of the simulation. Furthermore, there are some more automated evaluation schemes that acquire real vehicle and simulation data, batch-inject them to generate log datasets, parse and time-align them, evaluate the results, and output test reports. However, these methods primarily focus on comparing perception results.

[0004] Overall, existing technologies generally use fixed time windows or simple event triggers to slice scenarios, which makes it difficult to ensure the semantic coherence of complex driving scenarios; the degree of automation and standardization in the evaluation process is insufficient, and the generated reports lack strong correlation with the original scenarios and test parameters, resulting in poor auditability and reproducibility of the results. Summary of the Invention

[0005] This invention aims to at least solve the aforementioned technical problems existing in the prior art. To this end, the first aspect of this invention proposes a method for closed-loop simulation of autonomous driving logs based on label-driven methods, the method comprising:

[0006] Acquire and clean log data collected from vehicles, the log data including data segments of fixed duration; wherein, the log data collected from vehicles is the output of the first algorithm;

[0007] Obtain event-type tags and fragment-type tags associated with the log data, and perform consistency verification and semantic slicing on the log data based on the tags to generate a non-physical scene index;

[0008] Based on the scenario index, while keeping the interface contract of the existing network system unchanged, the second algorithm is used to perform closed-loop backfeedback, and the output results of the second algorithm and the system behavior induced by the output results are obtained.

[0009] In the same closed-loop scenario, the output results of the first algorithm and the output results of the second algorithm are aligned by clock and event, and a set of difference indicators is calculated;

[0010] A difference report is generated based on the set of difference indicators. The difference report includes a single-scenario report and a batch statistical report. The difference report is then scenario-signed, and an interface contract list is generated.

[0011] The set of difference indicators is judged according to the preset threshold strategy, and data re-mining and secondary injection are performed based on the judgment results.

[0012] This invention employs an end-to-end tag-driven closed-loop simulation process. First, tags are used to achieve high-quality semantic slicing, ensuring the causal logic integrity and semantic coherence of the test scenario. Second, by performing closed-loop backfeeding and calculating the difference index between the first and second algorithms, the dynamic impact of algorithm changes on system behavior can be realistically reflected, resulting in more accurate evaluation. Finally, based on the judgment results, re-mining and backfeeding are automatically triggered, forming a data-driven closed loop of "test-evaluation-optimization." This achieves automated and intelligent iteration of autonomous driving algorithm testing and verification, significantly improving development efficiency and system reliability.

[0013] Optionally, consistency verification and semantic slicing of log data based on the tags include:

[0014] For the event-type tags, the look-ahead and look-back durations of the slices are adaptively determined based on the vehicle's average speed before the event, road curvature, and road grade.

[0015] For the fragment type label, a continuity marker is automatically written at the boundary of the physical data fragment;

[0016] The sliced ​​scenes are subjected to consistency quality inspection, and the quality inspection scores include temporal continuity, heading consistency, causal coverage, and label source consistency.

[0017] This method effectively overcomes the semantic fragmentation problem caused by fixed-duration slicing by performing adaptive window slicing based on vehicle speed and road conditions for event-type labels and applying boundary continuity markings to fragment-type labels, thus ensuring the integrity of complex driving scenarios. Combined with multi-dimensional consistency quality inspection, the quality of scene slicing is ensured from the source, laying the foundation for subsequent high-fidelity closed-loop simulation and reliable evaluation results.

[0018] Optionally, the use of the second algorithm for closed-loop recirculation includes:

[0019] Based on the output of the second algorithm, the vehicle trajectory is recalculated using discrete integration.

[0020] Regeneration is performed only on sensor inputs affected by changes in the vehicle's behavior. The regeneration includes updating the obstruction chain, rewriting the radar Doppler frequency and radial velocity, and performing time resampling and interpolation.

[0021] During the recharge process, system status checkpoints are generated at fixed time intervals. These checkpoints include the vehicle status, target status, random seed, and interface mapping version.

[0022] This method preserves the true details of the original data to the greatest extent possible, significantly reducing computational complexity and simulation resource consumption while ensuring the rationality and physical accuracy of changes in the simulation environment. By generating system state checkpoints, the simulation process becomes rollback-able and reproducible, greatly enhancing the robustness of the testing process and the ease of debugging.

[0023] Optionally, clock and event alignment is performed between the output of the first algorithm and the output of the second algorithm, including:

[0024] Time alignment of the two results is performed based on a unified clock;

[0025] Using the event marked by the event-type label as the anchor point, calculate the time difference between the output of the first algorithm and the output of the second algorithm at the anchor point.

[0026] This method employs a dual alignment strategy—clock unification and event anchor point alignment—to ensure the comparability of different algorithm outputs across both the time and key event dimensions. It eliminates comparison biases caused by system latency or varying triggering times, enabling subsequent calculations to accurately and truthfully reflect the fundamental changes in algorithm performance.

[0027] Optionally, the set of difference indicators includes the following indicators:

[0028] Safety metrics include collision count, minimum time to collision, minimum lateral clearance, and minimum longitudinal clearance.

[0029] Compliance metrics include the number of times the line was crossed, the duration of driving against traffic, and the percentage of speeding.

[0030] Comfort-related indicators include peak longitudinal acceleration, peak lateral acceleration, and jerk distribution;

[0031] Consistency metrics include trajectory dynamic time warping distance, trajectory Friesian distance, event recurrence rate, and critical moment deviation;

[0032] Extended indicators for commercial vehicles include brake settling delay error, load sensitivity, and steering rate limit hit rate.

[0033] The differentiated index set constructed by this method comprehensively covers safety, compliance, comfort, consistency, and commercial vehicle-specific concerns, enabling a multi-dimensional and refined evaluation of algorithm performance. In particular, the introduction of extended indicators for commercial vehicles allows the evaluation to accurately reflect the special dynamic characteristics and operational requirements of heavy-duty vehicles, making the test results more practically instructive.

[0034] Optionally, the step of performing scenario signing on the difference report and generating an interface contract list includes:

[0035] An audit signature is generated based on the hash value of the scenario index, the version of the interface contract list, the strategy parameters, the random seed, and the evaluation script version.

[0036] Write the audit signature into the metadata of the difference report.

[0037] This method strongly binds the test report with the audit signature generated from scenario indexes, parameter configurations, etc., establishing a complete traceability chain from the final evaluation results to the original test inputs and environment; it ensures the reproducibility and auditability of the test results, meets the requirements of functional safety and compliance processes for strict recording of test activities, and provides tamper-proof evidence support for problem localization and responsibility definition.

[0038] Optionally, the set of difference indicators is judged according to a preset threshold strategy, and data re-mining and secondary backfeeding are performed based on the judgment results, including:

[0039] The set of difference indicators is substituted into a preset threshold expression for determination;

[0040] If the judgment result is out of bounds, then based on the out-of-bounds label type and scene meta-features, the search conditions are automatically constructed to search and expand similar scenes in the data lake;

[0041] The task chain, which includes nodes for data retrieval, downloading, semantic slicing, closed-loop backfeeding, automated evaluation, and report generation, is organized in the form of a directed acyclic graph.

[0042] This method automates the judgment process through a preset threshold expression and intelligently triggers further analysis and task orchestration based on the judgment results. This mechanism can automatically mine peripheral test cases similar to failure scenarios from massive amounts of data and organize a complete verification loop. It achieves problem-driven automated test expansion and regression verification, significantly improving the efficiency and systematic nature of discovering and resolving long-tail problems, and enabling the entire testing system to have the ability to self-improve and continuously evolve.

[0043] A second aspect of this invention proposes a label-driven closed-loop simulation system for autonomous driving logs, the system comprising:

[0044] The data ingestion and cleaning module is used to acquire and clean log data collected from vehicles, wherein the log data includes data segments of fixed duration; and the log data collected from vehicles is the output result of the first algorithm.

[0045] The tag slicing and indexing module is used to obtain event-type tags and fragment-type tags associated with the log data, and to perform consistency verification and semantic slicing on the log data based on the tags to generate a non-physical scene index.

[0046] The closed-loop backfeeding execution module is used to perform closed-loop backfeeding based on the scenario index, keeping the interface contract of the existing network system unchanged, and using the second algorithm to obtain the output result of the second algorithm and the system behavior induced by the output result.

[0047] The evaluation metrics module is used to perform clock and event alignment on the output results of the first algorithm and the output results of the second algorithm in the same closed-loop scenario, and to calculate a set of difference metrics.

[0048] The report generation and signing module is used to generate a difference report based on the difference indicator set. The difference report includes a single-scenario report and a batch statistical report. The module also performs scenario signing on the difference report and generates an interface contract list.

[0049] The threshold determination and arrangement module is used to determine the set of difference indicators according to the preset threshold strategy, and to perform data re-mining and secondary injection based on the determination results.

[0050] Optionally, the tag slicing and indexing module includes:

[0051] The adaptive window determination submodule is used to dynamically determine the slice duration for event-type labels;

[0052] The boundary marker submodule is used to write continuity markers at the physical boundaries of fragment tags;

[0053] The quality inspection scoring submodule is used to score the consistency of sliced ​​scenes.

[0054] Optionally, the closed-loop backfeed execution module includes:

[0055] The trajectory recalculation submodule is used to integrate the output of the second algorithm to obtain the vehicle trajectory.

[0056] The minimum necessary regeneration submodule is used to selectively update the occlusion chain, radar point cloud attributes, and image time series. Attached Figure Description

[0057] Figure 1 This is a flowchart illustrating a tag-driven closed-loop simulation method for autonomous driving logs, provided in an embodiment of the present invention. Detailed Implementation

[0058] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0059] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of embodiments of this disclosure, unless otherwise stated, "a plurality of" means two or more. Furthermore, the use of "based on" or "according to" implies openness and inclusiveness, because processes, steps, calculations, or other actions "based on" or "according to" one or more of the stated conditions or values ​​may in practice be based on additional conditions or beyond the stated values.

[0060] This invention provides a label-driven closed-loop simulation method and system for autonomous driving logs, such as... Figure 1 As shown, the method may include the following steps:

[0061] Step 101: Acquire and clean the log data collected from the vehicle. The log data includes data segments of fixed duration. The log data collected from the vehicle is the output of the first algorithm.

[0062] Specifically, the first algorithm acquires raw log data from the data acquisition system of the autonomous vehicle. The raw log data is the complete data recorded during the acquisition process of the first algorithm, typically including multi-sensor messages, vehicle bus signals (CAN / FD messages, including vehicle speed, heading angle, control commands, etc.), and time synchronization information. The multi-sensor messages include both raw perception data such as camera images, LiDAR point clouds, and millimeter-wave radar data, as well as structured data synchronously recorded during the acquisition process and output by the sensors. Examples include target structured information output by the camera (target size, position, speed, target type), road environment structured information (lane lines, speed limit signs, traffic light status), and target structured information output by the radar (target position, speed), all stored in message form.

[0063] The raw log data is cleaned, including removing invalid frames caused by transmission packet loss or sensor failure, smoothing abnormal jump signals, etc., and then the continuous log stream is divided into physical data clips of fixed duration (e.g., 5 minutes) for storage and management, so as to facilitate subsequent retrieval and processing.

[0064] Step 102: Obtain event-type tags and fragment-type tags associated with log data, and perform consistency verification and semantic slicing on the log data based on the tags to generate a non-physical scene index.

[0065] In one possible implementation, the tags undergo consistency verification and semantic slicing, including:

[0066] Based on event-type labels, the look-ahead and look-back durations of slices are adaptively determined based on the vehicle's average speed before the event, road curvature, and road grade.

[0067] Based on the fragment type label, a continuity marker is automatically written at the boundary of the physical data fragment;

[0068] The sliced ​​scenes are subjected to consistency quality inspection. The quality inspection scores include temporal continuity, heading consistency, causal coverage, and label source consistency.

[0069] Specifically, the input includes event-based labels (such as "LDW triggered" and "preceding vehicle cuts in," marking a specific moment) and segment-based labels (such as "continuous following" and "driving on a curve," marking a time period). The sources of event-based and segment-based labels include the output of automatic detection algorithms that post-process log data, as well as manual annotation results. Each label contains its corresponding time information, used to establish a correlation with the log data, and includes metadata such as source, confidence level, and time quality.

[0070] First, perform adaptive semantic slicing:

[0071] For event-based tags, instead of using fixed-duration slicing, the look-ahead duration L of the slice is adaptively determined based on the vehicle state and road environment before the event occurs. post With backtracking duration L pre Specifically, it is calculated using the following formula:

[0072] Rewind duration: ;

[0073] Preview duration: ;

[0074] Where v is the average speed of the vehicle in the second before the event. For road curvature estimation, road_grade represents the road slope or grade. , As a configurable coefficient, the clip function is used to constrain the result within a preset minimum and maximum value range, L + L is the minimum allowed length for an event-type tab window. - The maximum allowed duration for an event-type tab window. Backtrack duration L pre L - The range of values ​​for L is used to ensure that enough causes can be seen before an event occurs, but not too long; the look-ahead duration is L. post L + The value range is used to cover key outcome segments following the impact of the event. This method can dynamically adjust the slice range based on factors such as vehicle speed and the sharpness of curves to ensure that the complete causal context is included.

[0075] For fragment tags, when their time span covers multiple physical data clips, continuity markers (such as the end marker of the previous clip and the start marker of the next clip) are automatically written at the boundaries of the physical clips, thereby logically ensuring semantic continuity across physical clips.

[0076] Secondly, a consistency quality check is performed: the candidate scenes generated by the slices are scored in multiple dimensions, including:

[0077] Time continuity score Calculated based on the time interval deviation between adjacent frames. ;in, Let be the time tolerance constant, and let the time difference |Δt| be... At the same level, the score drops significantly; The smaller the value, the stricter the scoring criteria.

[0078] Heading Consistency Score : Evaluate the smoothness of vehicle heading changes within the scenario. ;in, Let be the heading tolerance constant, and be the heading difference |Δ | with At the same level, the score drops significantly; The smaller the value, the stricter the scoring criteria.

[0079] Causal coverage Assess whether key causal events are fully included;

[0080] Tag source consistency score : Evaluate the consistency between automatically labeled and manually verified labels.

[0081] Any scenario with a score below the preset threshold will be marked and entered into a manual review or rollback process to ensure the quality of the scenario input closed-loop feedback.

[0082] Finally, generate the non-physical index:

[0083] The results of slicing and quality inspection are aggregated to generate a non-physical scene index file (such as SceneIndex). This index does not copy the original massive log data, but instead defines the time range (t_start, t_end) of the scene by recording the physical Clip identifier (clip_id) and frame offset array (offsets[]). The index file also contains a data source digest (source_digest), the aforementioned quality inspection scores (qc_scores), and the hash value of the entire index content. This design makes scene creation, management, and sharing extremely efficient, and ensures data integrity and traceability through the hash value.

[0084] Step 103: Based on the scenario index, keep the interface contract of the existing network system unchanged, use the second algorithm to perform closed-loop backfeeding, and obtain the output result of the second algorithm and the system behavior induced by the output result.

[0085] Specifically, the goal of this step is to re-drive the original scene in a simulation environment using the second algorithm and generate changes in system behavior that conform to physical reality. The key is to achieve minimal necessary regeneration to balance realism and efficiency. Both the first and second algorithms are planning and control algorithms in autonomous driving systems.

[0086] In one possible implementation, closed-loop recharge is performed, including:

[0087] Based on the output of the second algorithm, the vehicle trajectory is recalculated using discrete integration.

[0088] Regeneration is performed only on sensor inputs affected by changes in the vehicle's behavior. Regeneration includes updating the occlusion chain, rewriting the radar Doppler frequency and radial velocity, and performing time resampling and interpolation.

[0089] During the reflow process, system status checkpoints are generated at fixed time intervals. The checkpoints include the status of the vehicle itself, the status of the target body, the random seed, and the interface mapping version.

[0090] Specifically, the steps are as follows:

[0091] Step 301: Maintain the interface contract.

[0092] When loading the raw data pointed to by the scene index, an interface contract manifest is used to ensure that the output topic name, field format, timestamp and frame_id are strictly consistent with the interface of the vehicle network system, so as to ensure that the second algorithm can be seamlessly integrated.

[0093] Step 302: Perform closed-loop recharge and minimum necessary regeneration.

[0094] A. Vehicle Trajectory Recalculation: The planning and control commands output by the second algorithm (the algorithm to be evaluated) are used to recalculate the vehicle's pose, velocity, acceleration, and trajectory curvature at future moments through discrete integration. In specific embodiments such as commercial vehicles, parameters such as braking setup delay, steering rate limit, and load adaptation are injected into this model to more realistically reflect vehicle dynamics.

[0095] B. Impact Domain Analysis and Data Regeneration: Analyze the new vehicle trajectory to determine which external traffic participants and environmental elements will be affected by the changes in vehicle behavior (e.g., changes in relative position leading to changes in occlusion relationships, changes in radar beam illumination angle). Only perform sensor data regeneration on these affected objects. The regeneration method is as follows:

[0096] Update occlusion chain: Recalculate the occlusion and visibility of targets in LiDAR point cloud or camera image based on the new geometric relationship;

[0097] Rewrite radar properties: For millimeter-wave radar point clouds, recalculate and rewrite the Doppler frequency and radial velocity of the affected targets;

[0098] Time resampling and interpolation: For sensor data such as video streams, resampling and interpolation are performed based on the new time series to maintain time synchronization.

[0099] For static backgrounds and distant, unrelated traffic participants that do not interact with the vehicle, the original log data is directly inherited without any changes to avoid introducing simulation artifacts.

[0100] Step 303: Establish checkpoints and rollback mechanisms.

[0101] During the reflow process, a system status checkpoint is generated at fixed intervals (e.g., 0.5 seconds). Each checkpoint stores the current vehicle state, key target state, the random seed used, the interface mapping version, and all injected parameters. If a subsequent regeneration process triggers an anomaly (such as non-monotonic timestamps, coordinate transformation breaks, or risk indicators exceeding limits), it automatically reverts to the most recent valid checkpoint, narrows the parameter change range, and retryes. The entire process ensures idempotency, guaranteeing the robustness and reproducibility of the simulation.

[0102] Step 104: In the same closed-loop scenario, align the clock and event results of the first algorithm and the second algorithm, and calculate the set of difference indicators.

[0103] In one possible implementation, clock and event alignment is performed between the outputs of the first algorithm and the second algorithm, including:

[0104] Time alignment of the two results is performed based on a unified clock;

[0105] Using the events marked by event-type tags as anchor points, calculate the time difference between the output of the first algorithm and the output of the second algorithm at the anchor point.

[0106] In one possible implementation, the set of difference indicators includes the following indicators:

[0107] Safety metrics include collision count, minimum time to collision, minimum lateral clearance, and minimum longitudinal clearance.

[0108] Compliance metrics include the number of times the line was crossed, the duration of driving against traffic, and the percentage of speeding.

[0109] Comfort-related indicators include peak longitudinal acceleration, peak lateral acceleration, and jerk distribution;

[0110] Consistency metrics include trajectory dynamic time warping distance, trajectory Friesian distance, event recurrence rate, and critical moment deviation;

[0111] Extended indicators for commercial vehicles include brake settling delay error, load sensitivity, and steering rate limit hit rate.

[0112] Specifically, to accurately assess the performance changes brought about by algorithm iterations, a rigorous comparison needs to be made between the baseline (first algorithm) results and the output of this feedback (second algorithm). This step includes the following procedures:

[0113] Step 401: Align the results.

[0114] Time alignment: Unify the two results to the same high-precision clock source, and perform bidirectional resampling to a common frame rate when necessary to ensure that the data at each moment is comparable;

[0115] Event anchor alignment: Using the time marked by the event label (such as "AEB triggered") as the anchor point, calculate the deviation between the first algorithm and the second algorithm at the event trigger time. ), used to correct and evaluate time-series response differences.

[0116] Step 402: Calculate a comprehensive set of difference indicators.

[0117] Based on the aligned data, calculate multi-dimensional metrics, including but not limited to:

[0118] Safety metrics: Collision count, Time to collision (TTC), Time to frontage (THW), Minimum lateral and longitudinal clearance;

[0119] Compliance indicators: number of times lanes were crossed, duration of driving against traffic, and percentage of speed limits exceeded;

[0120] Comfort-related indicators: peak longitudinal and lateral acceleration, statistical distribution of jerk (Jerk acceleration);

[0121] Consistency metrics: Dynamic Time Warping (DTW) distance of the trajectory, Fréchet distance, recurrence rate of critical events (such as lane change completion), critical moments ( Deviation;

[0122] Extended metrics for commercial vehicles include: brake settling-off delay error, sensitivity of braking distance to load changes, and steering rate limit hit rate. These metrics are specifically designed to evaluate performance characteristics unique to commercial vehicles.

[0123] Step 403: Conduct a significance assessment.

[0124] When conducting batch scenario testing, statistical testing methods (such as Kolmogorov-Smirnov test or Anderson-Darling test) are used to analyze the batch results of the above indicators, combined with Bootstrap confidence interval calculation, in order to identify statistically significant performance changes between the first and second algorithms, rather than just random fluctuations.

[0125] Step 105: Generate a single-scenario difference report and a batch statistical report based on the difference indicator set, and bind the reports with the scenario signature and interface contract list.

[0126] Specifically, this step of the report generation includes:

[0127] Single-Scene Difference Report: Presented in a visually appealing format. For example, an image may overlay the original trajectory, the trajectory of the first algorithm, and the trajectory of the second algorithm; it may provide comparison screenshots of keyframes (such as the moment of highest risk); it may list all core metrics in a table and highlight any metrics that exceed the limits.

[0128] Batch statistical reports: Summarize and analyze all test scenarios. Statistics are categorized by event type, weather conditions, road type, traffic density, etc., and histograms and box plots of indicator distribution are generated to visually display the distribution and drift of algorithm performance, and to list detected abnormal scenario clusters.

[0129] In one possible implementation, the report is bound to the scenario signature and interface contract list, including:

[0130] An audit signature is generated based on the hash value of the scenario index, the version of the interface contract list, the strategy parameters, the random seed, and the evaluation script version.

[0131] Write the audit signature into the metadata of the difference report.

[0132] Specifically, to ensure absolute traceability and reproducibility of the evaluation results, an audit signature is generated. This signature is produced by calculating an irreversible cryptographic digest (such as SHA-256) of the following elements: the hash value of the scene index (SceneIndex.hash), the interface contract manifest version (Manifest.version), all policy parameters used in this backfeedback, the random seed, the time source and frame rate settings, and the version number of the evaluation script. The generated audit signature is written into the metadata of the final difference report and is bound to the report file itself. Any tampering with the original data, configuration, or process will cause the signature verification to fail, thus ensuring the integrity and credibility of the evaluation process.

[0133] Step 106: Determine the set of differential indicators according to the preset threshold strategy, and automatically arrange data re-mining and secondary injection tasks based on the determination results.

[0134] In one possible implementation, the set of difference indicators is judged according to a preset threshold strategy, and data re-mining and secondary injection tasks are automatically arranged based on the judgment results, including:

[0135] Substitute the set of difference indicators into the preset threshold expression for judgment;

[0136] If the judgment result is out of bounds, then based on the out-of-bounds label type and scene meta-features, the search conditions are automatically constructed to search and expand similar scenes in the data lake;

[0137] The task chain, which includes nodes for data retrieval, downloading, semantic slicing, closed-loop backfeeding, automated evaluation, and report generation, is organized in the form of a directed acyclic graph.

[0138] Specifically, the determination method is as follows: one or more threshold expressions are preset to automatically determine whether the algorithm version passes the test. The expressions consist of difference indicators and logical operators, such as: This expression indicates that if "minimum TTC is less than 1.5 seconds", "at least one collision occurs", or "the trajectory DTW distance exceeds a threshold θ", then the condition is met. dtw If any condition is met, it is judged as "out of bounds".

[0139] Specifically, the triggering actions and automatic orchestration include:

[0140] Version control (Gating): If an out-of-bounds error is detected, the current algorithm version is marked as "failed" or "canary release only" in the release pipeline.

[0141] Automated re-mining: Based on the tag type that leads to out-of-bounds access (such as "aggressive entry") and scene meta-features (such as "urban roads, rainy days, high density"), search conditions are automatically constructed to retrieve and expand scene data with similar features from the vast data lake to enrich the test set.

[0142] Task orchestration: A complete task chain named "Retrieve → Download → Slice → Recycle → Evaluation → Report Generation" is automatically orchestrated in the form of a Directed Acyclic Graph (DAG). Each task node is designed to be idempotent and retryable. If a node fails, it will automatically retry N times; if it still fails, it will fall back to the last successful node and automatically generate a defect ticket to notify the developers. This makes the entire data loop operation fully automated, scalable, and exceptionally robust.

[0143] In summary, this invention achieves several significant technological advancements and beneficial effects through the end-to-end closed-loop simulation and evaluation method. First, by using tag-driven adaptive semantic slicing and multi-dimensional consistency verification, the semantic integrity and high quality of the test scenario are fundamentally guaranteed, overcoming the problem of missing causal information caused by fixed-duration pruning. Second, the "minimum necessary regeneration" strategy employed in the closed-loop backfeeding process accurately simulates dynamic environmental changes caused by algorithm decisions (such as sensor occlusion and changes in radar echo characteristics) while significantly reducing the computational overhead of full-scene reconstruction, achieving a balance between simulation efficiency and fidelity. Furthermore, through automated and standardized calculation of difference indicators, report generation, and strong signature binding with scene data and configuration parameters, a complete, reliable, and reproducible evaluation audit trail chain is established, greatly improving the engineering rigor and reliability of the test process. Ultimately, relying on intelligent threshold determination and automated re-mining task orchestration, this method can automatically transform evaluation results into optimization momentum, forming a data-driven closed loop of "problem discovery - scenario expansion - regression verification," thereby continuously and efficiently mining long-tail scenarios, driving the intelligent and systematic iteration of autonomous driving algorithms, and significantly improving R&D efficiency and system reliability.

[0144] This invention provides a label-driven closed-loop simulation system for autonomous driving logs, comprising:

[0145] The data ingestion and cleaning module is used to acquire and clean vehicle log data;

[0146] The tag slicing and indexing module is used to perform consistency checks and semantic slicing based on event-type tags and fragment-type tags, generating non-physical scene indexes;

[0147] The closed-loop backfeed execution module is used to perform closed-loop backfeed based on the scenario index.

[0148] The evaluation metrics module is used to align the results of the first algorithm with the output of the second algorithm and calculate a set of difference metrics.

[0149] The report generation and signing module is used to generate difference reports and bind them to scenario signatures;

[0150] The threshold determination and orchestration module is used to orchestrate further mining tasks based on the results of the difference index determination.

[0151] In one possible implementation, the tag slicing and indexing module includes:

[0152] The adaptive window determination submodule is used to dynamically determine the slice duration for event-type labels;

[0153] The boundary marker submodule is used to write continuity markers at the physical boundaries of fragment tags;

[0154] The quality inspection scoring submodule is used to score the consistency of sliced ​​scenes.

[0155] In one possible implementation, the closed-loop backfeed execution module includes:

[0156] The trajectory recalculation submodule is used to integrate the output of the second algorithm to obtain the vehicle trajectory.

[0157] The minimum necessary regeneration submodule is used to selectively update the occlusion chain, radar point cloud attributes, and image time series.

[0158] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the device described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0159] In another embodiment of the present invention, an electronic device is also provided, which includes a processor and a memory. The memory stores at least one instruction or at least one program. The at least one instruction or at least one program is loaded and executed by the processor to implement the label-driven closed-loop simulation method for autonomous driving logs proposed in the embodiments of the present invention.

[0160] In another embodiment of the present invention, a computer-readable storage medium is also provided, which stores at least one instruction or at least one program, wherein the at least one instruction or at least one program is loaded and executed by a processor to implement the label-driven closed-loop simulation method for autonomous driving logs proposed in the embodiments of the present invention.

[0161] The foregoing primarily describes the solutions provided by the embodiments of the present invention from the perspective of the device. It is understood that, in order to achieve the above functions, the device includes corresponding hardware structures and / or software modules for executing each function. Those skilled in the art should readily recognize that, in conjunction with the algorithmic steps of the examples described in the embodiments disclosed herein, the present invention can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present invention.

[0162] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions within the technical scope disclosed in the present invention should be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A label-driven closed-loop simulation method for autonomous driving logs, characterized in that, include: Acquire and clean log data collected from vehicles, the log data including data segments of fixed duration; wherein, the log data collected from vehicles is the output of the first algorithm; Obtain event-type tags and fragment-type tags associated with the log data, and perform consistency verification and semantic slicing on the log data based on the tags to generate a non-physical scene index; Based on the scenario index, while keeping the interface contract of the existing network system unchanged, the second algorithm is used to perform closed-loop backfeedback, and the output results of the second algorithm and the system behavior induced by the output results are obtained. In the same closed-loop scenario, the output results of the first algorithm and the output results of the second algorithm are aligned by clock and event, and a set of difference indicators is calculated; A difference report is generated based on the set of difference indicators. The difference report includes a single-scenario report and a batch statistical report. The difference report is then scenario-signed, and an interface contract list is generated. The set of difference indicators is judged according to the preset threshold strategy, and data re-mining and secondary injection are performed based on the judgment results.

2. The method according to claim 1, characterized in that, The consistency check and semantic slicing of log data based on the tags include: For the event-type tags, the look-ahead and look-back durations of the slices are adaptively determined based on the vehicle's average speed before the event, road curvature, and road grade. For the fragment type label, a continuity marker is automatically written at the boundary of the physical data fragment; The sliced ​​scenes are subjected to consistency quality inspection, and the quality inspection scores include temporal continuity, heading consistency, causal coverage, and label source consistency.

3. The method according to claim 1, characterized in that, The method of using the second algorithm for closed-loop recirculation includes: Based on the output of the second algorithm, the vehicle trajectory is recalculated using discrete integration. Regeneration is performed only on sensor inputs affected by changes in the vehicle's behavior. The regeneration includes updating the obstruction chain, rewriting the radar Doppler frequency and radial velocity, and performing time resampling and interpolation. During the recharge process, system status checkpoints are generated at fixed time intervals. These checkpoints include the vehicle status, target status, random seed, and interface mapping version.

4. The method according to claim 1, characterized in that, The step of aligning the output of the first algorithm with the output of the second algorithm using clock and event synchronization includes: Time alignment of the two results is performed based on a unified clock; Using the event marked by the event-type label as the anchor point, calculate the time difference between the output of the first algorithm and the output of the second algorithm at the anchor point.

5. The method according to claim 1, characterized in that, The set of difference indicators includes the following indicators: Safety metrics include collision count, minimum time to collision, minimum lateral clearance, and minimum longitudinal clearance. Compliance metrics include the number of times the line was crossed, the duration of driving against traffic, and the percentage of speeding. Comfort-related indicators include peak longitudinal acceleration, peak lateral acceleration, and jerk distribution; Consistency metrics include trajectory dynamic time warping distance, trajectory Friesian distance, event recurrence rate, and critical moment deviation; Extended indicators for commercial vehicles include brake settling delay error, load sensitivity, and steering rate limit hit rate.

6. The method according to claim 1, characterized in that, The step of performing scenario signing on the difference report and generating an interface contract list includes: An audit signature is generated based on the hash value of the scenario index, the version of the interface contract list, the strategy parameters, the random seed, and the evaluation script version. Write the audit signature into the metadata of the difference report.

7. The method according to claim 1, characterized in that, The step of determining the set of difference indicators according to a preset threshold strategy, and performing data re-mining and secondary backfeeding based on the determination results, includes: The set of difference indicators is substituted into a preset threshold expression for determination; If the judgment result is out of bounds, then based on the out-of-bounds label type and scene meta-features, the search conditions are automatically constructed to search and expand similar scenes in the data lake; The task chain, which includes nodes for data retrieval, downloading, semantic slicing, closed-loop backfeeding, automated evaluation, and report generation, is organized in the form of a directed acyclic graph.

8. A label-driven closed-loop simulation system for autonomous driving logs, characterized in that, include: The data ingestion and cleaning module is used to acquire and clean log data collected from vehicles, wherein the log data includes data segments of fixed duration; and the log data collected from vehicles is the output result of the first algorithm. The tag slicing and indexing module is used to obtain event-type tags and fragment-type tags associated with the log data, and to perform consistency verification and semantic slicing on the log data based on the tags to generate a non-physical scene index. The closed-loop backfeeding execution module is used to perform closed-loop backfeeding based on the scenario index, keeping the interface contract of the existing network system unchanged, and using the second algorithm to obtain the output result of the second algorithm and the system behavior induced by the output result. The evaluation metrics module is used to perform clock and event alignment on the output results of the first algorithm and the output results of the second algorithm in the same closed-loop scenario, and to calculate a set of difference metrics. The report generation and signing module is used to generate a difference report based on the difference indicator set. The difference report includes a single-scenario report and a batch statistical report. The module also performs scenario signing on the difference report and generates an interface contract list. The threshold determination and arrangement module is used to determine the set of difference indicators according to the preset threshold strategy, and to perform data re-mining and secondary injection based on the determination results.

9. The system according to claim 8, characterized in that, The tag slicing and indexing module includes: The adaptive window determination submodule is used to dynamically determine the slice duration for event-type labels; The boundary marker submodule is used to write continuity markers at the physical boundaries of fragment tags; The quality inspection scoring submodule is used to score the consistency of sliced ​​scenes.

10. The system according to claim 8, characterized in that, The closed-loop backfeed execution module includes: The trajectory recalculation submodule is used to integrate the output of the second algorithm to obtain the vehicle trajectory. The minimum necessary regeneration submodule is used to selectively update the occlusion chain, radar point cloud attributes, and image time series.

Images