Large model driven automatic driving failure mode cluster mining and classification method

Abstract

The invention belongs to the technical field of intelligent driving, and particularly relates to a large-model-driven automatic driving failure mode cluster mining and classifying method, which comprises the following steps: firstly, acquiring automatic driving test data, detecting a failure event according to preset conditions such as AEB triggering and manual takeover, and generating failure fragments through time window slicing; according to a structured template containing dimensions such as an Ego vehicle track, the fragments are converted into data in a unified format; inputting a large model to generate semantic information such as a failure abstract, a preliminary attribution and a risk causal chain, splicing the semantic information, encoding the semantic information into a fixed-length vector, and clustering the vector into a failure cluster in combination with cosine similarity and an HDBSCAN algorithm; and finally, extracting a cluster-level failure mode, a typical trigger condition and the like by means of a large model, and outputting a standardized failure cluster dictionary and a differential analysis report. According to the method, the problem that automatic driving complete failure cluster mining and classification are lacked in an existing automatic driving failure analysis mode can be solved.

Description

Technical Field

[0001] This invention belongs to the field of intelligent driving technology, and in particular relates to a method for mining and classifying failure mode clusters of autonomous driving driven by a large model. Background Technology

[0002] As autonomous driving technology rapidly evolves towards higher levels (L3 and above), automakers and testing institutions generate massive amounts of test data daily during the R&D and verification phases, covering simulation test data, closed-course test data, and real-world road test data. Among this data, only a few segments are "failure segments" that are of critical value for system safety analysis and iterative optimization. These are specific time segments during the operation of the intelligent driving system where performance significantly degrades, risks significantly increase, or manual intervention or safety mechanisms are triggered.

[0003] Currently, the safety performance of autonomous driving systems has become a core bottleneck for the technology's implementation. Regulatory agencies are increasingly stringent in their safety verification requirements for high-level autonomous driving. The industry urgently needs efficient and accurate failure mode analysis technology to uncover common weaknesses in systems from massive amounts of test data, supporting system-level safety assessments and pre-production reviews.

[0004] Currently, the commonly used failure analysis methods for autonomous driving in the industry mainly fall into three categories, but all of them have obvious technical defects and cannot meet the needs of large-scale, high-precision, and systematic failure analysis: Manual review and analysis: Engineers manually review videos, tracks, and CAN signals, analyzing the causes of failures one by one. This method relies heavily on the individual experience of engineers, resulting in extremely low review efficiency. It cannot support the massive amounts of test data generated daily, and different engineers lack a unified format for describing the "cause of failure," making it impossible to compare and analyze failure data across vehicle models and projects.

[0005] Screening analysis based on simple rules or thresholds: Suspected risk segments are screened by setting preset numerical thresholds, and then further judged by humans to determine whether they are failure segments. Although this method achieves preliminary screening of "suspected segments", it can only identify the existence of "abnormal operating conditions" and cannot explain the core reasons for the abnormality. Moreover, the threshold setting depends on experience, which is prone to missed screening or false screening.

[0006] Rough classification based on scenario conditions: Failure fragments are classified according to the basic attributes of the test scenario. This method can only achieve "surface scenario classification" and cannot delve into the essential mechanism of failure, nor can it automatically summarize "failure mode clusters" from a large amount of failure data, making it difficult for engineers to identify the systemic weaknesses of the system in specific scenarios.

[0007] Among the closest existing technologies, the main ones include: 1. Threshold-based anomaly detection: It automatically filters out "abnormal operating conditions" by setting minimum distance, TTC threshold, acceleration threshold, etc. Its shortcoming is that it can only identify anomalies, but cannot explain the cause of the anomalies.

[0008] 2. Trajectory clustering based on numerical features: This method performs cluster analysis on failed trajectories, such as using Dynamic Time Warping (DTW) to measure the differences between trajectories and divide them into several clusters. The drawback is that this method can only reflect the similarity of trajectory shapes and cannot reveal the mechanisms at the levels of perception, prediction, and decision-making.

[0009] 3. Text-based accident case aggregation: It performs natural language analysis on public accident reports or internal accident texts to extract accident types and scenario elements; however, it is not bound to specific system operation data and lacks fine-grained failure analysis at the "behavioral process level".

[0010] Therefore, existing methods for analyzing failures in autonomous driving technology lack a comprehensive set of methods that cannot be developed based on "test data". Failure fragment Mechanism Explanation The complete method chain of "failure clusters". Summary of the Invention

[0011] The technical problem solved by this invention is to provide a large model-driven method for mining and classifying failure mode clusters in autonomous driving, so as to solve the problem that existing methods for analyzing failures in autonomous driving lack the ability to mine and classify complete failure clusters in autonomous driving.

[0012] The basic solution provided by this invention is a large model-driven method for mining and classifying failure mode clusters in autonomous driving, comprising: S1: Acquire test data in autonomous driving mode, detect failure events according to preset trigger conditions, and call the time window to slice each failure event to generate multiple failure segments; S2: Describe each failed segment according to a unified structured description template to generate structured data in a unified format; S3: Input structured data into the large model, and the large model will output failure summary, preliminary failure attribution, and risk causal chain, generating semantic information of the corresponding failure fragments; S4: Concatenate the semantic information of each failure segment into text, convert it into a fixed-length vector, and use the fixed-length vector as a feature to calculate the semantic distance between any two failure segments. Then, use a clustering algorithm to divide all failure segments into several failure clusters. S5: Based on the large model, the failure modes of each failure cluster are described, typical triggering conditions are extracted, representative scenarios are selected, and systemic weaknesses are highlighted to obtain the analysis and summary results of the failure cluster. S6: Output a failure cluster dictionary and failure analysis report based on the analysis and summary results.

[0013] Furthermore, S1 includes: S1-1: Acquire continuous test data in autonomous driving mode, and automatically detect failure events in the test data according to preset trigger conditions; the preset trigger conditions include but are not limited to: automatic emergency braking (AEB) trigger flag is 1, manual takeover flag is 1, minimum vehicle distance drops below a set threshold within a preset time, TTC decreases within a preset time, longitudinal or lateral acceleration exceeds a preset comfort threshold, and the system enters degraded mode or protection mode. S1-2: Extract the triggering event from the acquired failure event, and select a preset duration before and after the triggering event as the failure segment time window. The expression is:

[0014] in, To trigger the event at the time of triggering. The preset time before the triggering time in the failure event. The preset duration after the trigger event is triggered in the failure event; S1-3: Obtain multiple failure segments from the failure events based on the failure segment time windows divided in S1-2.

[0015] Furthermore, S2 includes: S2-1: Construct a structured description template based on the dimensions of Ego vehicle trajectory features, target object trajectory features, system control and decision data, scene semantic information, and key event node information; S2-2: Organize the failed fragments into structured data in a uniform format according to the structured description template.

[0016] Furthermore, S3 includes: S3-1: After preprocessing the structured data, input it into the large model; S3-2: The large model extracts the core features of the scene, the behavioral deviations of the system modules, and the failure results from the preprocessed structured data, and generates a short text failure summary according to the logic of scene, deviation, and result. S3-3: The large model extracts failure evidence of the perception, prediction, decision-making and control modules of the autonomous driving system from the preprocessed structured data, and combines it with the logic of the autonomous driving system to determine the correlation between module failure and results, and generate preliminary failure attribution. S3-4: The large model connects the scenario causes, system module behaviors, physical quantity changes and failure results in the time sequence of the preprocessed structured data, and binds the corresponding indicators in the structured data to each link to generate a risk causal chain; S3-5: Integrate the failure summary, preliminary failure attribution, and risk causal chain to generate semantic information for the corresponding failure fragment.

[0017] Furthermore, S4 includes: S4-1: Obtain the semantic information of the i-th failure segment, and concatenate the failure summary, preliminary failure attribution, and risk causal chain in the semantic information, using a delimiter in between. Distinguish and generate complete semantic text The expression is:

[0018] in, This refers to the failure summary content in the semantic information of the i-th failure segment; This refers to the initial attribution of failure in the semantic information of the i-th failed segment; The risk causal chain content in the semantic information of the i-th failed segment; S4-2: Convert the complete semantic text into a fixed-length vector using a large model, as shown in the expression:

[0019] in, The output is a fixed-length vector. For encoding functions in large models; and fixed-length vectors Standardization is performed to obtain a standardized vector. ; S4-3: Calculate the semantic distance between the standardized vectors of any two failed segments based on cosine similarity, and generate semantic vector similarity; S4-4: Based on semantic vector similarity, the HDBSCAN clustering algorithm is used to divide all failure segments into several failure clusters, as expressed in the following expression:

[0020] Each failed cluster This represents a semantically similar failure mode. The number of clusters.

[0021] Furthermore, S5 includes: S5-1: Based on the semantic information within each failure cluster, a cluster-level failure mode description is generated through semantic high-frequency word extraction and large model aggregation; S5-2: Based on the structured data within each failure cluster, the scenario triggering conditions, target triggering conditions, and system state triggering conditions are extracted through statistical analysis and threshold fitting methods, and typical triggering conditions are generated by integrating them. S5-3: Calculate the distance to the cluster center based on the semantic vector within each failure cluster, and select several representative failure segments as representative scenarios for that cluster; S5-4: Combining the cluster-level failure mode description, typical triggering conditions, and autonomous driving system module logic of each failure cluster, systemic weakness prompts are generated through large model and domain knowledge matching; S5-5: Integrate cluster-level failure mode descriptions, typical triggering conditions, representative scenarios, and systemic weakness alerts, and store the analysis and summary results of the generated failure clusters in a unified format.

[0022] Furthermore, S6 includes: S6-1: Fill in the analysis and summary results of the failure clusters output in step S5 according to the fields in the preset failure cluster dictionary; S6-2: Design a failure analysis report based on user-differentiated needs, integrate the analysis summary results of failure clusters according to the designed failure analysis report, and output the failure analysis report corresponding to the failure cluster.

[0023] An electronic device includes a processor and a memory, wherein the memory stores programs or instructions, and the processor executes the above-described large model-driven autonomous driving failure mode cluster mining and classification method by calling the programs or instructions stored in the memory.

[0024] A computer-readable storage medium storing a program or instructions that causes a computer to execute a large-model-driven method for mining and classifying failure mode clusters of autonomous driving, as described above.

[0025] The principle and advantages of this invention are as follows: The technical solution of this application is driven by the core of "large-scale model semantic understanding and reasoning ability", and constructs a full-link technical system of "data screening - structured description - semantic parsing - vector clustering - cluster-level induction - result output": First, from the massive autonomous driving test data, failure events are detected and sliced ​​to generate failure fragments through preset trigger conditions; then, based on five dimensions of Ego vehicle trajectory, target object features, system control, scene semantics, and key events, a structured template is constructed to convert the failure fragments into data in a unified format; then, the large-scale model is used to parse the structured data to generate semantic information containing failure summary, preliminary attribution, and risk causal chain; then, the semantic information is spliced ​​and encoded into a fixed-length vector through the large-scale model, and combined with cosine similarity calculation and HDBSCAN clustering algorithm, semantically similar failure fragments are divided into failure clusters; finally, based on the large-scale model, the commonalities of each failure cluster are extracted to generate failure mode description, typical trigger conditions, representative scenarios and systemic weakness prompts, and finally output a standardized failure cluster dictionary and differentiated analysis report, realizing the transformation from "fragmented failure data" to "systemic failure knowledge".

[0026] The advantages are: the technical solution of this application accurately solves the four core problems of existing autonomous driving failure analysis technologies: I. To address the issues of low efficiency and reliance on experience in manual review, the efficiency of failure analysis is greatly improved through structured templates and automated parsing of large models, and the unified description format eliminates the barriers to cross-project data comparison. Second, to address the issue that threshold-based screening can only identify anomalies but cannot explain the causes, we use large-scale model reasoning to output a complete risk causal chain and module attribution, thus achieving the integration of "anomaly identification - mechanism analysis". Third, to address the problem that scenario-based classification cannot uncover the essential failure mechanism, we use semantic vectorization clustering to form failure clusters, accurately extract the common patterns of similar failures, and help identify systemic weaknesses. Fourth, addressing the lack of a complete methodological chain in existing technologies, we construct a system starting from "test data". Failure fragment Mechanism Explanation Failure Cluster The closed loop of "application of results" supports multiple scenarios such as R&D optimization, test case design, and regulatory compliance verification, providing key technical support for the safety assessment and mass production review of high-level autonomous driving systems. Attached Figure Description

[0027] Figure 1 This is a flowchart of an embodiment of the present invention; Figure 2 This is a schematic diagram of the electronic device structure according to an embodiment of the present invention. Detailed Implementation

[0028] The following detailed description illustrates the specific implementation method: The reference numerals in the accompanying drawings include: electronic device 400, processor 401, memory 402, input device 403, and output device 404.

[0029] The basic implementation examples are as follows: Figure 1 As shown: A large model-driven method for mining and classifying failure mode clusters in autonomous driving, including: S1: Acquire test data in autonomous driving mode, detect failure events according to preset trigger conditions, and slice each failure event using a time window to generate multiple failure segments; wherein, S1 includes: S1-1: Acquire continuous test data in autonomous driving mode and automatically detect failure events in the test data according to preset trigger conditions; the preset trigger conditions include, but are not limited to: automatic emergency braking (AEB) trigger flag being 1, manual intervention flag being 1, minimum vehicle distance decreasing to below a set threshold within a preset time, TTC decreasing within a preset time, longitudinal or lateral acceleration exceeding a preset comfort threshold, and the system entering a degraded mode or protection mode; In this embodiment, a stable data transmission link is established with various sensors and controllers of the autonomous vehicle through relevant professional data acquisition equipment and corresponding software systems, thereby acquiring continuous test data in autonomous driving mode. This data covers vehicle operating status information, sensor perception data, and system internal status parameters; For preset trigger conditions, corresponding algorithmic logic is written to monitor in real time during data acquisition or subsequent data processing. For example, for the condition that the Automatic Emergency Braking (AEB) trigger flag is 1, a program is set in the data acquisition software to monitor the output signal of the AEB control module. Once the signal value becomes 1, the trigger condition is immediately determined to be met. For the manual takeover flag being 1, the system monitors the switching signals of vehicle steering, braking, and other control permissions. When a switch from the autonomous driving system to manual control is detected and the flag is 1, the condition is considered met. For the minimum distance between vehicles falling below a set threshold within a preset time, millimeter-wave radar or lidar is used to acquire the distance data of the vehicle in front, and the relative speed is calculated in combination with vehicle speed information. A time window is set (e.g., 5 seconds), and the distance between vehicles is calculated in real time within this time window. If the distance between vehicles falls below a set threshold (e.g., 2 meters), a failure event detection is triggered. Similarly, for the time to collision (TTC) decreasing within a preset time, the TTC is calculated based on sensor data. If the TTC is within a preset time (e.g., 3 seconds), the failure event detection is triggered. If the value continues to decrease, the trigger condition is met; if the longitudinal or lateral acceleration exceeds a preset comfort threshold, acceleration data is acquired through the vehicle's acceleration sensor and compared with the preset comfort threshold (e.g., longitudinal acceleration). lateral acceleration If the value exceeds the specified range, the system will be triggered. The system will enter a degraded mode or a protection mode, and monitor the status flag of the autonomous driving system. When the flag indicates that the corresponding mode has been entered, the failure event detection will be triggered.

[0030] S1-2: Extract the triggering event from the acquired failure event, and select a preset duration before and after the triggering event as the failure segment time window. The expression is:

[0031] in, To trigger the event at the time of triggering. The preset time before the triggering time in the failure event. The preset duration after the trigger event is triggered in the failure event; in this embodiment, and The value of is determined based on the specific test scenario and analysis requirements. For example, in urban road test scenarios, considering the vehicle's response characteristics in complex traffic environments, can be... Set to 3 seconds. Set to 5 seconds; in highway test scenarios, due to higher vehicle speeds and faster vehicle dynamics, adjustments can be made as needed. It lasts for 2 seconds. It lasts for 4 seconds.

[0032] S1-3: Multiple failure segments are obtained from the failure event based on the failure segment time window defined in S1-2. In this embodiment, multiple failure segments are accurately extracted from the continuous test data corresponding to the failure event based on the failure segment time window defined in S1-2. Specifically, the indexing function of the data storage system is used to locate and extract the corresponding data segments from the large amount of stored data according to the start and end times of the time window. For example, in a database indexed by timestamps, by inputting the start and end timestamps of the failure segment time window, the system can automatically filter out all relevant data within that time period, including sensor data, vehicle status data, system control data, etc., forming independent failure segments, providing basic data support for subsequent failure analysis.

[0033] S2: Describe each failed segment according to a unified structured description template to generate structured data in a unified format; wherein, S2 includes: S2-1: A structured description template is constructed based on the dimensions of Ego vehicle trajectory features, target object trajectory features, system control and decision data, scene semantic information, and key event node information. In this embodiment, the Ego vehicle trajectory features include longitudinal speed, longitudinal acceleration, lateral position, heading angle, and lane change status. The longitudinal speed is extracted from the CAN bus signal; the longitudinal acceleration is calculated based on the speed signal differential or directly read from the CAN bus signal; the lateral position is calculated based on the lane centerline of the high-precision map, and the lateral distance between the Ego vehicle's center point and the centerline is calculated; the heading angle is read from the CAN bus signal; and the lane change status is determined by combining the steering angle signal and the decision status signal. The target trajectory features include target ID and type, target absolute velocity, target relative position, and target relative velocity. The target ID and type are extracted from the signals output by the perception module in the autonomous driving system. The target absolute velocity is obtained by reading the signals from the perception module or by calculating the target absolute position differentially. The target relative position is calculated based on the coordinates of the Ego vehicle center point and the target center point. The target relative velocity is calculated from the Ego longitudinal velocity and the target longitudinal velocity.

[0034] The system control and decision data include braking / drive torque commands, steering angle commands, and high-level decision status. The braking / drive torque commands and steering angle commands are obtained by reading CAN bus signals, while the high-level decision status is obtained by reading domain controller signals and combined with lane change requests and obstacle avoidance trigger flags for auxiliary judgment.

[0035] Scene semantic information includes road type, lane information, traffic control facilities, and environmental information. Among them, the road type is based on the road type field of the high-precision map or obtained by matching the road database through GPS positioning; the lane information is obtained by extracting the number of lanes and the current lane ID from the high-precision map; in the traffic control facilities, the speed limit value is obtained from the speed mark of the high-precision map, and the traffic light status is obtained from the on-board camera perception module; in the environmental information, the weather is determined from the rain sensor and the reflectivity of the lidar point cloud, and the illumination is determined from the illumination sensor or the camera exposure parameters.

[0036] Key event node information includes the moment of sudden deceleration of the vehicle in front, the moment of the cut-in vehicle crossing the lane, the moment of obvious braking of the ego, the moment of minimum clearance, and the moment of AEB / intervention triggering. Among them, the moment of sudden deceleration of the vehicle in front is based on the longitudinal acceleration of the monitored target object. When the longitudinal acceleration of the target object is less than the sudden deceleration threshold and remains so for a preset time, for example, less than If the deceleration lasts for 0.5 seconds, then that moment is the moment when the vehicle in front decelerates suddenly; the moment a vehicle crosses the lane is when the lateral position of an object in the adjacent lane is monitored, and the start time of the lane crossing is recorded when the lateral position moves from the adjacent lane into the current lane; the moment of significant Ego braking is when the Ego braking torque is monitored, and when the braking torque is less than the significant braking threshold, for example, less than The start time is recorded; the minimum distance occurs when the relative longitudinal distance between Ego and the target is traversed to find the time and value corresponding to the minimum value; the AEB / takeover trigger time is obtained by reading the trigger timestamp of the AEB or takeover command in the system log.

[0037] Therefore, based on the information from various dimensions, including Ego vehicle trajectory features, target object trajectory features, system control and decision-making data, scene semantic information, and key event node information, a structured description template is constructed.

[0038] S2-2: Organize the failed fragments into structured data in a unified format according to the structured description template; in this embodiment, JSON format is used for storage.

[0039] S3: Input structured data into the large model, and the large model outputs failure summaries, preliminary failure attributions, and risk causal chains, generating semantic information for the corresponding failure segments; S3 includes: S3-1: After preprocessing the structured data, it is input into the large model. In this embodiment, the preprocessing includes data filtering and data format standardization. Data filtering involves selecting structured fields that are key to the failure explanation, including the time sequence of key event nodes, abnormal feature data, and scenario constraints, and removing fields that are irrelevant to the failure mechanism to avoid interfering with the large model's attention. Data format standardization involves using a unified format of field labels + numerical descriptions + units to transform the structured data into natural language feature descriptions, as shown in the following example: Ego longitudinal acceleration, its standardized input description is as follows: [Ego Vehicle Characteristics] The maximum longitudinal acceleration within the failure segment is (Exceeding the comfort threshold) ), appearing at the time -0.8s; For example, the standardized input description of key event nodes is as follows:

Key Event Sequence

[0040] After the data format is standardized, a clear task instruction, namely the prompt instruction definition, needs to be added before the standardized data to guide the large model to output the specified content. An example instruction template is as follows: "Based on the structured feature data of the following autonomous driving failure segments, complete three tasks: 1. Generate a failure summary of no more than 100 characters, which must include the scenario, system behavior deviation, and failure result; 2. From the four system modules of perception, prediction, decision-making, and control, provide preliminary attributions and weights for the failures (the sum of weights does not have to be 1, it only represents the relative probability); 3. Organize the 'scenario triggers' in chronological order." System behavior The causal link of the risk of failure must be supported by key data. Meanwhile, to avoid the problem of insufficient explanation accuracy of general large models in the field of autonomous driving, the specific steps for the selection, adaptation and optimization of large models are as follows: 1. Select a large model with a foundation of "pre-training in the autonomous driving field", or a general large model that supports domain fine-tuning, such as GPT-4; 2. Construct a fine-tuning dataset: Collect 500+ manually labeled paired samples of "structured data - failure explanation - attribution results", covering typical failure modes such as perception lag, decision delay, and control instability; 3. Fine-tuning objectives: Optimize the model's understanding of autonomous driving terminology (such as "TTC", "AEB", "lateral control offset") and the accuracy of semantic conversion of numerical features (such as accurately identifying "lateral offset 0.5m" as "normal lane departure" instead of "lane deviation"). 4. Prompt Optimization: Through a small number of samples, determine the optimal instruction expression, such as supplementing domain constraints ("Attribution must comply with ISO 21448 expected functional safety standards") and clarifying the output format ("Attribution weights are retained to one decimal place, causal links are expressed using single quotes"). 'connect").

[0041] The preprocessed, standardized input prompt is then fed into the adapted large model to trigger model inference. The inference process is shown in S3-2 to S3-5 below: S3-2: The large model extracts the core features of the scene, the behavioral deviations of the system modules, and the failure results from the preprocessed structured data, and generates a short text-based failure summary according to the logic of scene, deviation, and result; an example output is shown below: "In the scenario of an urban expressway, the vehicle in front..." -3s rapid deceleration ( The system failed to adjust its following strategy in a timely manner until... The braking started only after -1.2 seconds, causing the distance between vehicles to decrease from 50 meters to 2.5 meters. AEB is triggered at any time. S3-3: The large model extracts failure evidence from the perception, prediction, decision-making, and control modules of the autonomous driving system from the preprocessed structured data, and combines it with the logic of the autonomous driving system to determine the correlation between module failures and results, generating preliminary failure attributions; an example of the output is as follows: "Perception lag: 0.7 (Evidence: the speed of the vehicle in front from..." Down to The data update latency was as follows: (perception module updates data 300ms later); decision delay: 0.5 (evidence: braking command output 150ms after perception update); underprediction: 0.2 (evidence: failure to predict the deceleration trend of the vehicle ahead); control execution lag: 0.1 (evidence: response begins 50ms after braking command output, within the normal range). S3-4: The large model connects the scenario triggers, system module behaviors, physical quantity changes, and failure results in chronological order based on the preprocessed structured data, and binds corresponding indicators from the structured data to each link to generate a risk causal chain; an example of the output is as follows: 1. Scene triggers: -3s The car in front begins to decelerate rapidly (deceleration) ) 2. Perception module deviation: The perception module failed to update the speed of the vehicle in front in a timely manner (delay 300ms). 3. Prediction module deviation: Collision risk was not predicted based on the forward vehicle's motion trend (no warning was issued when TTC dropped from 5s to 1.2s). 4. Decision module bias: Braking command is output 150ms after perception update. 5. Changes in physical quantities: The relative speed between Ego and the vehicle in front changes from... Become The distance between vehicles has been reduced from 50m to 2.5m. 6. Failure Result: "It constantly triggers AEB to avoid collisions, but this exceeds the comfort threshold." S3-5: Integrate the failure summary, preliminary failure attribution, and risk causal chain to generate semantic information for the corresponding failure fragment.

[0042] S4: Concatenate the semantic information of each failed segment into text, convert it into a fixed-length vector, and use the fixed-length vector as a feature to calculate the semantic distance between any two failed segments. Then, use a clustering algorithm to divide all failed segments into several failed clusters. S4 includes: S4-1: Obtain the semantic information of the i-th failure segment, and concatenate the failure summary, preliminary failure attribution, and risk causal chain in the semantic information, using a delimiter in between. Distinguish and generate complete semantic text The expression is:

[0043] in, The failure summary content in the semantic information of the i-th failure segment is, in this embodiment, the failure core description within 100 characters output by step S3. The initial attribution content of the failure in the semantic information of the i-th failure segment is, in this embodiment, the attribution result containing modules and weights output by step S3. The risk causal chain content in the semantic information of the i-th failure segment is, in this embodiment, the failure causal description output by step S3 in chronological order. S4-2: Convert the complete semantic text into a fixed-length vector using a large model, as shown in the expression:

[0044] in, The output is a fixed-length vector. For encoding functions in large models; and fixed-length vectors Standardization is performed to obtain a standardized vector. In this embodiment, the selected large model supports long text semantic understanding and is adapted to the autonomous driving field. The standardization process adopts L2 standardization, which ensures that the standardized semantic vectors are on the same numerical scale for all failed segments.

[0045] S4-3: Calculate the semantic distance between the standardized vectors of any two failed segments based on cosine similarity to generate semantic vector similarity. In this embodiment, to reduce the computational load of subsequent clustering, a similarity threshold is first set. Failed segments with semantic vector similarity greater than or equal to the similarity threshold are marked as "suspected to be of the same type" and given priority consideration in subsequent clustering. The recommended value logic for the similarity threshold is as follows: Initial value: Calculate the minimum similarity of similar failure segments based on historical failure case annotation data; Dynamic adjustment: If the proportion of marked "suspected similar" segments... If the threshold is too low, the initial value of the similarity threshold will be increased by 0.05 each time; if the proportion is less than If the threshold is too high, the similarity threshold will be lowered by -0.05 each time.

[0046] S4-4: Based on semantic vector similarity, the HDBSCAN clustering algorithm is used to divide all failure segments into several failure clusters, as expressed in the following expression:

[0047] Each failed cluster This represents a semantically similar failure mode. The number of clusters is denoted as 'number'. In this embodiment, the HDBSCAN clustering algorithm is a mature existing technology, and will not be described in detail here.

[0048] Finally, the generated failed clusters are verified and stored in a unified JSON format to provide input for the cluster-level analysis in step S5.

[0049] S5: Based on the large model, the failure modes of each failure cluster are described, typical triggering conditions are extracted, representative scenarios are selected, and systemic weaknesses are highlighted, resulting in an analysis summary of the failure clusters; S5 includes: S5-1: Based on the semantic information within each failure cluster, a cluster-level failure mode description is generated through semantic high-frequency word extraction and large model aggregation. In this embodiment, jieba word segmentation combined with the TF-IDF algorithm is used to extract keywords from all failure summaries and causal chains within the cluster. The extracted dimensions include high-frequency failure scenarios, high-frequency system biases, and high-frequency failure results. For example, if the failure cluster is the "perception hysteresis cluster for sudden deceleration of the vehicle in front," then high-frequency failure scenarios include sudden deceleration of the vehicle in front, urban expressways, etc.; high-frequency system biases include perception hysteresis and underprediction; and high-frequency failure results include triggering AEB. An example of integrating these into the prompt for input to the large model is as follows: "Based on the following failure cluster characteristics, generate a cluster-level failure mode description of no more than 150 characters (which must include the scenario, system bias, failure result, and core data support): 1. High-frequency scenario: Sudden deceleration of the vehicle in front ( ), urban expressways ( ); 2. High-frequency system bias: sensing hysteresis ( (Average delay 0.8s), under-prediction ( ); 3. High-frequency results: Triggering AEB ( ), average minimum spacing 2.3m; 4. Intra-cluster sample size: 85 valid fragments. The final description output by the large model includes scenario constraints, core biases, causal logic, and results. An example of the final cluster-level failure mode description is as follows: "The failure mode of this cluster is: in an urban expressway scenario, the vehicle in front decelerates rapidly (deceleration)." At that time, the system exhibited an average perception lag of 0.8 seconds and failed to predict the collision risk in advance, causing the decision module to delay outputting braking commands, ultimately... The fragment triggered AEB, with an average minimum spacing of only 2.3m, exceeding the safety redundancy threshold. S5-2: Based on the structured data within each failure cluster, scene triggering conditions, target triggering conditions, and system state triggering conditions are extracted using statistical analysis and threshold fitting methods, and integrated to generate typical triggering conditions. In this embodiment, the scene triggering conditions are extracted by considering road type, weather conditions, and speed limit zones. The road type is determined by statistically analyzing the frequency of occurrence of each road type. The types of weather conditions are determined by the frequency of occurrence of each weather type. The type; the speed limit interval is obtained by calculating the mean and standard deviation of the speed limit values ​​within the cluster.

[0050] The target triggering conditions include the preceding vehicle deceleration triggering threshold and the preceding vehicle relative distance triggering range; the system state triggering conditions include the Ego initial vehicle speed, system decision mode, and braking command delay.

[0051] S5-3: Calculate the distance to the cluster center based on the semantic vector within each failure cluster, and select several representative failure segments as representative scenarios of the cluster. In this embodiment, the segment whose semantic vector is closest to the cluster center is selected. First, the similarity between each segment and the cluster center is calculated. Then, the two segments with the highest similarity are selected as representative failure segments to represent the representative scenarios of the cluster, which is convenient for engineers to conduct subsequent analysis and research.

[0052] S5-4: Combining the cluster-level failure mode description, typical triggering conditions, and autonomous driving system module logic of each failure cluster, systemic weakness hints are generated through large-scale model and domain knowledge matching; in this embodiment, the input data construction example for large-scale model inference is as follows: "Based on the following cluster-level information, output 2-3 systemic weakness alerts (which need to be linked to the perception / prediction / decision / control modules, and clearly define the optimization direction): 1. Failure mode: When the vehicle in front decelerates suddenly on an urban expressway, perception lag (average 0.8s) + insufficient prediction leads to braking delay triggering AEB; 2. Triggering condition: deceleration of the vehicle in front." Ego speed 1. Following mode; 3. Data characteristics: The perception module updates the speed of the vehicle in front with a delay of 0.8s, and the prediction module fails to warn of the risk that the TTC (Traffic Time Control) will decrease from 5s to 1.2s; 4. Logic of the autonomous driving system module: perception predict decision making The total delay of the control response chain needs to be (Safety requirements). The large model outputs a link based on failure mechanisms, module responsibilities, and optimization directions. An example of the output is shown below: Weakness of the perception module: It is unable to detect sudden deceleration of the vehicle in front. The average speed update delay of the target is 0.8s, which exceeds the system safety response chain threshold (0.5s). It is speculated that there is room for optimization in the inter-frame correlation logic of the perception algorithm for high-speed moving targets. It is recommended to optimize the update frequency of the optical flow method or the deep learning target tracking model. Weakness of the prediction module: Failure to provide early warning of the collision risk (TTC) dropping from 5s to 1.2s (within the cluster) (The sample has no warning), suggesting that the prediction module's time-series prediction window for the "trend of sudden deceleration of the vehicle in front" is too short (currently possibly only 1 second). It is recommended to extend the prediction window to... And add "deceleration threshold trigger warning" logic; S5-5: Integrate cluster-level failure mode descriptions, typical triggering conditions, representative scenarios, and systemic weakness alerts, and store the analysis and summary results of the generated failure clusters in a unified format.

[0053] S6: Output a failure cluster dictionary and a failure analysis report based on the analysis and summary results; S6 includes: S6-1: The analysis and summary results of the failure clusters output in step S5 are filled in according to the fields in the preset failure cluster dictionary. In this embodiment, the preset fields include failure cluster ID, failure mode name, detailed description of failure mode, typical triggering conditions, representative scenario information, system weakness list, cluster basic statistical data, risk level, applicable test scenario, data source batch, etc. The failure cluster dictionary is stored in JSON format to ensure that it can be parsed by machines and read by humans.

[0054] S6-2: Design a failure analysis report based on user-differentiated needs. Integrate the analysis and summary results of failure clusters according to the designed failure analysis report, and output the failure analysis report corresponding to the failure cluster. In this embodiment, different report modules are designed for different users. For example, for R&D engineers, the report optional modules include failure cluster distribution statistics and system weakness priority ranking. For the testing team, the report optional modules include risk level distribution and representative scenario list.

[0055] Therefore, the protection points of the above-mentioned technical solution of this application include: 1. A "failure fragment structured description template" for autonomous driving testing is proposed, which can uniformly extract ego trajectory, target object trajectory, control command, scene semantics and key event information from multi-source data, providing standard input for subsequent semantic analysis.

[0056] 2. A failure semantic interpretation mechanism based on a large model is proposed, which automatically outputs failure summaries, preliminary attributions, and risk causal chains, replacing the traditional review method that relies on human experience.

[0057] 3. A method for constructing "failure clusters / families" based on semantic vectorization and clustering is proposed, which focuses on failure mechanism as the core of clustering, rather than solely on numerical trajectory similarity.

[0058] 4. For each failed cluster, an automatic summarization method for cluster-level triggering conditions and representative scenarios is proposed to support the identification of systemic weaknesses and the construction of regression test scenarios.

[0059] 5. This method is based entirely on external behavioral data of the system, without requiring access to the internal structure of the model. It has good black-box compatibility and is easy to promote and apply across different vehicle models and different supplier systems.

[0060] like Figure 2 As shown, in another embodiment of this embodiment, an electronic device is also included, the electronic device 400 including one or more processors 401 and memory 402.

[0061] The processor 401 may be a central processing unit (CPU) or other form of processing unit with data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 400 to perform desired functions.

[0062] The memory 402 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 401 may execute the program instructions to implement the large model-driven autonomous driving failure mode cluster mining and classification method described above in any embodiment of the present invention, and / or other desired functions. Various contents such as initial extrinsic parameters and thresholds may also be stored in the computer-readable storage medium.

[0063] In one example, the electronic device 400 may further include an input device 403 and an output device 404, these components being interconnected via a bus system and / or other forms of connection mechanisms (not shown). The input device 403 may include, for example, a keyboard, a mouse, etc. The output device 404 may output various information to the outside, including warning messages, braking force, etc. The output device 404 may include, for example, a display, a speaker, a printer, and a communication network and its connected remote output devices, etc.

[0064] Of course, for the sake of simplicity, Figure 2Only some of the components of the electronic device 400 relevant to the present invention are shown, omitting components such as buses, input / output interfaces, etc. In addition, the electronic device 400 may include any other suitable components depending on the specific application.

[0065] In addition to the methods and devices described above, embodiments of the present invention may also be computer program products, which include computer program instructions that, when executed by a processor, cause the processor to perform the steps of a large model-driven autonomous driving failure mode cluster mining and classification method provided in any embodiment of the present invention.

[0066] The computer program product can be written in any combination of one or more programming languages ​​to perform the operations of the embodiments of the present invention. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0067] Furthermore, embodiments of the present invention may also be computer-readable storage media storing computer program instructions, which, when executed by a processor, cause the processor to perform the steps of a large model-driven autonomous driving failure mode cluster mining and classification method provided in any embodiment of the present invention.

[0068] The above are merely embodiments of the present invention. Commonly known structures and characteristics are not described in detail here. Those skilled in the art are aware of all common technical knowledge in the field prior to the application date or priority date, are aware of all existing technologies in that field, and have the ability to apply conventional experimental methods prior to that date. Those skilled in the art can, under the guidance of this application, improve and implement this solution in combination with their own capabilities. Some typical known structures or methods should not be obstacles for those skilled in the art to implement this application. It should be noted that those skilled in the art can make several modifications and improvements without departing from the structure of the present invention. These should also be considered within the scope of protection of the present invention, and will not affect the effectiveness of the implementation of the present invention or the practicality of the patent. The scope of protection claimed in this application should be determined by the content of its claims, and the specific embodiments described in the specification can be used to interpret the content of the claims.

Claims

1. A large model driven automatic driving failure mode cluster mining and classification method, characterized in that: include: S1: Acquire test data in autonomous driving mode, detect failure events according to preset trigger conditions, and call the time window to slice each failure event to generate multiple failure segments; S2: Describe each failed segment according to a unified structured description template to generate structured data in a unified format; S3: Input structured data into the large model, and the large model will output failure summary, preliminary failure attribution, and risk causal chain, generating semantic information of the corresponding failure fragments; S4: Concatenate the semantic information of each failure segment into text, convert it into a fixed-length vector, and use the fixed-length vector as a feature to calculate the semantic distance between any two failure segments. Then, use a clustering algorithm to divide all failure segments into several failure clusters. S5: Based on the large model, the failure modes of each failure cluster are described, typical triggering conditions are extracted, representative scenarios are selected, and systemic weaknesses are highlighted to obtain the analysis and summary results of the failure cluster. S6: Output a failure cluster dictionary and failure analysis report based on the analysis and summary results.

2. The large model-driven automatic driving failure mode cluster mining and classification method according to claim 1, characterized in that: S1 includes: S1-1: Acquire continuous test data in autonomous driving mode, and automatically detect failure events in the test data according to preset trigger conditions; the preset trigger conditions include but are not limited to: automatic emergency braking (AEB) trigger flag is 1, manual takeover flag is 1, minimum vehicle distance drops below a set threshold within a preset time, TTC decreases within a preset time, longitudinal or lateral acceleration exceeds a preset comfort threshold, and the system enters degraded mode or protection mode. S1-2: Extract the triggering event from the acquired failure event, and select a preset duration before and after the triggering event as the failure segment time window. The expression is: wherein, is a triggering time of the triggering event, is a preset time length before the triggering time of the triggering event in the invalidation event, is a preset time length after the triggering time of the triggering event in the invalidation event; S1-3: Obtain multiple failure segments from the failure events based on the failure segment time windows divided in S1-2.

3. The large model-driven automatic driving failure mode cluster mining and classification method according to claim 1, characterized in that: S2 includes: S2-1: Construct a structured description template based on the dimensions of Ego vehicle trajectory features, target object trajectory features, system control and decision data, scene semantic information, and key event node information; S2-2: Organize the failed fragments into structured data in a uniform format according to the structured description template.

4. The large model driven automatic driving failure mode cluster mining and classification method according to claim 1, characterized in that: S3 includes: S3-1: After preprocessing the structured data, input it into the large model; S3-2: The large model extracts the core features of the scene, the behavioral deviations of the system modules, and the failure results from the preprocessed structured data, and generates a short text failure summary according to the logic of scene, deviation, and result. S3-3: The large model extracts failure evidence of the perception, prediction, decision-making and control modules of the autonomous driving system from the preprocessed structured data, and combines it with the logic of the autonomous driving system to determine the correlation between module failure and results, and generate preliminary failure attribution. S3-4: The large model connects the scenario causes, system module behaviors, physical quantity changes and failure results in the time sequence of the preprocessed structured data, and binds the corresponding indicators in the structured data to each link to generate a risk causal chain; S3-5: Integrate the failure summary, preliminary failure attribution, and risk causal chain to generate semantic information for the corresponding failure fragment.

5. The large model-driven automatic driving failure mode cluster mining and classification method according to claim 4, characterized in that: S4 includes: S4-1: Obtain the semantic information of the i-th failed fragment, splice the failure summary, failure preliminary attribution and risk causal chain in the semantic information, and pass through a separator in the middle distinguish, generate complete semantic text , the expression is: wherein, is a failure summary content in the semantic information of the i-th failure segment; is a failure preliminary cause content in the semantic information of the i-th failure segment; is a risk causal chain content in the semantic information of the i-th failure segment; S4-2: Convert the complete semantic text into a fixed-length vector using a large model, as shown in the expression: wherein, is an output fixed-length vector, is an encoding function in a large model; and the fixed-length vector is normalized to obtain a normalized vector ; S4-3: Calculate the semantic distance between the standardized vectors of any two failed segments based on cosine similarity, and generate semantic vector similarity; S4-4: Based on semantic vector similarity, the HDBSCAN clustering algorithm is used to divide all failure segments into several failure clusters, as expressed in the following expression: wherein each failure cluster represents a semantically similar failure mode, is the number of clusters.

6. The large model-driven automatic driving failure mode cluster mining and classification method according to claim 5, characterized in that: S5 includes: S5-1: Based on the semantic information within each failure cluster, a cluster-level failure mode description is generated through semantic high-frequency word extraction and large model aggregation; S5-2: Based on the structured data within each failure cluster, the scenario triggering conditions, target triggering conditions, and system state triggering conditions are extracted through statistical analysis and threshold fitting methods, and typical triggering conditions are generated by integrating them. S5-3: Calculate the distance to the cluster center based on the semantic vector within each failure cluster, and select several representative failure segments as representative scenarios for that cluster; S5-4: Combining the cluster-level failure mode description, typical triggering conditions, and autonomous driving system module logic of each failure cluster, systemic weakness prompts are generated through large model and domain knowledge matching; S5-5: Integrate cluster-level failure mode descriptions, typical triggering conditions, representative scenarios, and systemic weakness alerts, and store the analysis and summary results of the generated failure clusters in a unified format.

7. The method for mining and classifying failure mode clusters of autonomous driving driven by a large model according to claim 6, characterized in that: S6 includes: S6-1: Fill in the analysis and summary results of the failure clusters output in step S5 according to the fields in the preset failure cluster dictionary; S6-2: Design a failure analysis report based on user-differentiated needs, integrate the analysis summary results of failure clusters according to the designed failure analysis report, and output the failure analysis report corresponding to the failure cluster.

8. An electronic device, characterized in that: It includes a processor and a memory, wherein the memory stores programs or instructions, and the processor executes a large model-driven autonomous driving failure mode cluster mining and classification method as described in any one of claims 1-7 by calling the programs or instructions stored in the memory.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a program or instructions that cause a computer to execute a large model-driven method for mining and classifying failure mode clusters for autonomous driving as described in any one of claims 1-7.

Images