An intelligent driving simulation test credibility evaluation method and device and a storage medium

Abstract

This invention relates to the technical field of intelligent driving simulation testing, and particularly to a method, device, and storage medium for evaluating the credibility of intelligent driving simulation testing. The core solution of this invention is as follows: extracting simulation scene elements and constructing a causal graph centered on driving decisions, transforming it into a Bayesian network; learning conditional probability distributions based on simulation and real-vehicle test data, calculating the local credibility of nodes using likelihood ratios; calculating the causal effect strength of each node on driving decisions as a keyness weight; weighted aggregation to obtain an overall credibility score and mapping level; and simultaneously tracing and locating weak links in credibility. This invention achieves causal interpretability and scene adaptability in overall credibility evaluation, locating weak links at the element level, and is suitable for pure software virtual simulation environments, accurately guiding the optimization and correction of simulation models and scenarios.

Description

Technical Field

[0001] This invention relates to the technical field of intelligent driving simulation testing, and in particular to a method, apparatus and storage medium for evaluating the credibility of intelligent driving simulation testing. Background Technology

[0002] With the rapid development of various intelligent driving technologies such as advanced driver assistance systems (ADAS), combined driver assistance systems (CAMS), and high-level autonomous driving, virtual simulation-based testing and verification has become a key core means for the development and safety assessment of intelligent driving systems. Compared with closed-course testing and open road testing, virtual simulation testing has significant advantages such as high scenario coverage and testing efficiency, low testing cost, and strong repeatability. It can support intelligent driving researchers and experimenters to complete testing and verification of a large number of typical and dangerous scenarios within limited time and space. In the New Assessment / Test Method (NATM) proposed by the Informal Working Group on Validation Methods for Automated Driving (VMAD) of the United Nations World Forum for Harmonization of Vehicle Regulations (UN / WP.29), simulation testing has been listed as an important component of its proposed "multi-pillar approach". In November 2023, the Ministry of Industry and Information Technology, the Ministry of Public Security, the Ministry of Housing and Urban-Rural Development, and the Ministry of Transport of my country jointly issued the "Implementation Guidelines for Pilot Projects on Access and Road Passage of Intelligent Connected Vehicles (Trial)," which also clearly requires that the confidence level of the simulation test toolchain used, as well as the credibility of vehicle dynamics, sensor and other models, should be proved, and the credibility of the simulation test should be verified by means of comparison with closed site and actual road test results.

[0003] However, current credibility assessment of intelligent driving simulation tests still faces the following technical challenges:

[0004] (1) Lack of ability to assess the overall credibility of simulation results. Existing methods mostly focus on the fidelity assessment of a single sub-model (such as vehicle sensor model, vehicle dynamics model) or a single level. Each sub-model or level is assessed independently and then summarized. The comprehensive impact of the causal driving relationship and causal transmission effect between various elements of the simulation scene on the credibility of the overall simulation results is not considered at the modeling level. For example, the patent "A Simulation Confidence Testing Method, Apparatus, and Testing Equipment for a Virtual Camera" (Publication No. CN117540564A) only tests the simulation confidence of a single perception component of a virtual camera, evaluating it using three dimensions: static image simulation accuracy, static recognition accuracy, and dynamic recognition accuracy. It fails to answer whether the overall simulation results are reliable and where the deviations come from. The paper "Perception Sensor Model Fidelity Evaluation for Automated Driving System Scenario-Based Simulation Testing" (International Journal of Automotive Technology, 2025) proposes a three-layer fidelity evaluation method for perception sensor models: the base layer, the sensor layer, and the target layer. However, the evaluation object is limited to a single sub-model of a LiDAR sensor, with each layer's fidelity calculated independently, without modeling the causal transmission mechanism between layers. These methods cannot answer the crucial question of "whether the overall results of intelligent driving simulation tests are reliable."

[0005] (2) Traditional statistical comparison and evaluation methods lack causal interpretability. Existing methods mostly use purely data-driven means such as statistical index comparison (e.g., mean square error, correlation coefficient, Hausdorff distance, etc.) or fuzzy comprehensive evaluation for credibility assessment. They do not establish a causal relationship model between elements of the simulation scenario, resulting in evaluation results that can only reflect "how much the simulation differs from reality", but cannot explain "why the deviation exists" and "along what causal chain the deviation is transmitted to the driving decision output". Although the patent "A Credibility Evaluation Method for a Vehicle-in-the-Loop Simulation Test System" (Publication No. CN121807693A) proposes multi-dimensional index quantification and subjective and objective weighting, its evaluation object is the vehicle-in-the-loop simulation system (including hardware equipment), and the weight determination depends on the subjective weighting method, lacking modeling and analysis of the internal causal mechanism of the simulation scenario.

[0006] (3) The hierarchical evaluation method only establishes correlations between layers, failing to distinguish the causal direction of deviation propagation. The patent "An Evaluation Method and System for the Confidence of a High-Level Autonomous Driving System Simulation" (Publication No. CN121388467A) proposes a hierarchical evaluation model that decomposes simulation confidence into four levels: scene layer, sensor layer, actuator / dynamics layer, and algorithm decision layer, based on system function. It then uses a "cross-layer interactive verification mechanism" to locate the root cause level based on the correlation between sub-confidences at each level. While this method achieves hierarchical confidence decomposition and cross-layer correlation analysis, the inter-layer relationships it analyzes are essentially correlations rather than causal relationships. Correlation only indicates that the sub-confidences of two levels change together, failing to distinguish "which level's deviation caused the deviation of another level," and also failing to reflect the specific path and intensity of deviation propagation along the causal chain. Therefore, this method can only locate the "root cause level" (such as the sensor layer), but cannot further pinpoint the specific root cause elements within that level (such as deviations in the preceding vehicle distance model), thus lacking sufficient guidance for correcting intelligent driving simulation models.

[0007] (4) The sub-confidence scores of each layer are calculated independently, without considering the cumulative effect of causal transmission between layers during the calculation process. Existing hierarchical evaluation methods calculate the sub-confidence scores of each layer independently and then synthesize them through cross-layer correlation analysis. However, simulation biases can accumulate or be amplified in the causal transmission between layers. For example, the bias in the ahead distance model of the sensor layer can be causally transmitted to the following decision bias of the algorithm decision layer, and then to the longitudinal acceleration bias of the actuator layer. The independent calculation of each layer cannot reflect this causal transmission effect during the scoring process, and can only be indirectly inferred afterward through correlation analysis, which limits the accuracy and interpretability of the confidence assessment results.

[0008] (5) The determination of weights is highly subjective or the fixed structure cannot adapt to changes in the scenario. The patent "A Credibility Evaluation Method for a Vehicle-in-the-Loop Simulation Testing System" (Publication No. CN121807693A) uses a weighting method that combines subjective and objective factors to calculate the fusion weights of credibility metrics. The paper "Perception Sensor Model Fidelity Evaluation for Automated Driving System Scenario-Based Simulation Testing" (International Journal of Automotive Technology, 2025) uses the analytic hierarchy process (AHP) to determine the weights of different fidelity evaluation metrics. These methods rely on expert experience or static data distribution and cannot adapt to the dynamic changes in the degree of influence of each element on driving decisions in different simulation scenarios. For example, in a highway scenario, the motion state of the vehicle in front has a much greater impact on driving decisions than the road curvature; while in a complex intersection scenario in an urban area, the criticality of traffic light status and pedestrian motion status increases significantly. The fixed structure of the aforementioned patents and literature cannot dynamically adjust the criticality weights of elements within each layer according to the scenario type.

[0009] (6) The granularity of weak link location is coarse, lacking the ability to trace the cause and effect at the element level. When the simulation credibility does not meet the requirements, the patent "A method and system for evaluating the confidence of simulation of a high-level autonomous driving system" (publication number CN121388467A) can locate the "root level" (such as the sensor layer), but a level contains a large number of different types of elements, and locating the level is still too coarse. The patent "A method for determining the credibility of a simulation scene" (publication number CN117787005A) determines the credibility of the scene by comparing the probability distribution of preset attention indicators in the simulation scene and the real scene, but it only evaluates the credibility of the scene itself, does not involve the overall credibility evaluation of the simulation results, and does not have the ability to locate weak links. The paper "A Simulation Credibility Evaluation Method Based on Improved Fuzzy Comprehensive Evaluation" (Journal of System Simulation, 2020) uses an improved fuzzy comprehensive evaluation to evaluate the credibility of general simulation systems. Although the weight determination method is improved, the evaluation object is a general simulation system, which is not applicable to intelligent driving simulation. Furthermore, it cannot locate the weak links in credibility and their causal roots, which is not conducive to targeted improvements in the adjustment of intelligent driving simulation model parameters and the correction of simulation scenarios.

[0010] Therefore, there is an urgent need for a credibility assessment method for intelligent driving simulation testing that can comprehensively consider the causal coupling relationship between scene elements, adapt to changes in scene criticality, and possess causal interpretability and weak link location capabilities in a software virtual simulation environment. Summary of the Invention

[0011] The purpose of this invention is to provide a method, apparatus, and storage medium for evaluating the credibility of intelligent driving simulation tests, aiming to solve the technical problems that are still faced in the current evaluation of the credibility of intelligent driving simulation tests as mentioned in the background art.

[0012] To achieve the above objectives, the present invention provides the following technical solution: a method for evaluating the credibility of intelligent driving simulation testing, comprising the following steps:

[0013] S1. Simulation scene element extraction and causal graph construction

[0014] Scene elements are extracted from the virtual simulation environment of the intelligent driving simulation test software. These scene elements include vehicle motion elements, traffic participant elements, road environment elements, and driving decision elements. With the driving decision elements as the central node, directed edges of causal relationships between scene elements are established to construct a simulation scene causal graph. The directed edges of causal relationships represent the causal driving relationship between upstream scene elements and downstream scene elements.

[0015] Specifically, step S1 includes:

[0016] S11. Based on the semantic description information of the intelligent driving simulation test scenario, the scenario elements are hierarchically classified to obtain vehicle motion layer elements, traffic participant layer elements, road environment layer elements, and driving decision layer elements; among which:

[0017] The vehicle motion layer elements include one or more of the following: longitudinal velocity, lateral acceleration, heading angle, longitudinal acceleration, and lateral velocity.

[0018] The traffic participant layer elements include one or more of the following: relative position and speed of surrounding vehicles, pedestrian position and motion state, and non-motorized vehicle motion state.

[0019] The road environment layer elements include one or more of the following: road geometric parameters, pavement adhesion coefficient, traffic signal status, weather and lighting conditions.

[0020] The driving decision-making elements include one or more of the following: target lane selection, desired longitudinal acceleration, desired steering angle, and following distance setting.

[0021] S12. Based on prior knowledge in the driving domain and physical constraint rules of the simulation scenario, establish causal driving relationships among the hierarchically categorized scenario elements: traffic participant layer elements and road environment layer elements drive driving decision layer elements respectively; driving decision layer elements drive vehicle motion layer elements; at the same time, road environment layer elements drive traffic participant layer elements.

[0022] S13. Set the driving decision-making layer elements as the central node, and represent the causal driving relationship between the elements of each layer in the form of a directed acyclic graph to form the causal graph of the simulation scene.

[0023] S2. Construction of Bayesian Networks Based on Causal Graphs

[0024] The causal graph of the simulated scene is transformed into a Bayesian network structure, where the scene element nodes in the causal graph are used as random variable nodes of the Bayesian network, and the directed edges of causal relationships are used as conditional dependency edges of the Bayesian network. Based on the simulation test data and the actual vehicle test data in the closed test field, the conditional probability distribution parameters of each node in the Bayesian network are learned respectively to obtain the simulated conditional probability distribution and the real conditional probability distribution.

[0025] Specifically, step S2 includes:

[0026] S21. Keep the direction of the directed edges of the causal relationships in the causal graph of the simulation scene unchanged, and use them as directed edges of the Bayesian network.

[0027] S22. For each node in the Bayesian network, using the simulation test dataset and the closed test field real vehicle test dataset respectively, the maximum likelihood estimation or Bayesian parameter estimation method is used to learn the conditional probability distribution parameters of the node under the condition of its parent node, and the simulation conditional probability table and the real conditional probability table are obtained respectively.

[0028] S23. Perform a conditional independence test on the Bayesian network structure, remove directed edges that do not satisfy the conditional independence constraint, and obtain the simplified Bayesian network.

[0029] Furthermore, prior to step S2, an alignment step is included between the simulation test data and the actual vehicle test data from the closed proving ground:

[0030] The simulation test data and the real vehicle test data at the test site are timestamped to ensure that the simulation data and the real data correspond at the same time step in the same driving scenario.

[0031] For continuous scene element variables, the equal-frequency binning method is used to discretize them into a finite number of states to adapt to the conditional probability distribution calculation of Bayesian networks.

[0032] S3. Probabilistic Reasoning and Node Credibility Calculation Based on Likelihood Ratio

[0033] For each node in the Bayesian network, a local credibility index is calculated based on the likelihood ratio between the simulated conditional probability distribution and the real conditional probability distribution of that node; the likelihood ratio is the ratio of the marginal likelihood of that node under real data to the marginal likelihood under simulated data.

[0034] Specifically, step S3 includes:

[0035] S31. For any node in a Bayesian network Let its parent node set be Based on real-vehicle test data from a closed test track, the log-marginal likelihood of this node under the true conditional probability distribution is calculated. :

[0036]

[0037] in, This represents the number of test data samples.

[0038] S32. Based on simulation test data, calculate the log marginal likelihood of this node under the simulated conditional probability distribution. :

[0039]

[0040] S33, Computation Node Local credibility index :

[0041]

[0042] in, , The closer the value is to 1, the higher the credibility of the scene elements corresponding to that node.

[0043] S4, Scene Keyness Calculation

[0044] Based on the causal graph of the simulation scene, the causal effect intensity of each scene element node relative to the driving decision center node is calculated, and the normalized causal effect intensity is used as the scene criticality weight of the node.

[0045] Specifically, step S4 includes:

[0046] S41. In the cause-effect graph of the simulation scenario, taking the central node of the driving decision-making layer as the endpoint, for each scenario element node... The sum of the products of mediation effects along all causal paths from the node to the central node is used as the strength of the causal effect of that node on the central node. :

[0047]

[0048] in, As the driving decision center node, From arrive The set of all causal paths, For directed edges on the path The causal strength coefficient is calculated based on the conditional mutual information between corresponding variables in the simulation data;

[0049] S42. Normalize the causal effect intensity of all scene element nodes to obtain the scene criticality weight. :

[0050]

[0051] in, This represents the total number of scene element nodes.

[0052] S5, Weighted Aggregation and Overall Credibility Assessment

[0053] The local credibility index of each node is weighted and aggregated according to the scene criticality weight to obtain the overall credibility score of the simulation test; the overall credibility score is mapped to a preset credibility level range to output the credibility level result of the simulation test.

[0054] Specifically, step S5 includes:

[0055] S51. Calculate the overall credibility score based on the scene criticality weight and local credibility index of each node. :

[0056]

[0057] in, This represents the total number of scene element nodes. For nodes The criticality weight of the scenario For nodes Local credibility index;

[0058] S52. Preset credibility level mapping rules: Map the overall credibility score Mapped to a credibility level range, the credibility level range includes at least an "untrustworthy" level, a "low credibility" level, a "medium credibility" level, and a "high credibility" level, and the scoring threshold corresponding to each level is set based on the statistical quantile of historical simulation test data or industry standards.

[0059] S53, Output overall credibility score The credibility level and its corresponding credibility level are used as the credibility assessment conclusion of this intelligent driving simulation test.

[0060] S6. Location and Feedback of Weak Links in Simulation Credibility

[0061] S61. Identifying Local Credibility Indicators Scene element nodes that fall below a preset weak threshold are considered as nodes with weak credibility.

[0062] S62. Following the causal relationship direction of the causal graph of the simulation scene, trace the upstream causal path of the weak credibility node to determine the root cause scene element that leads to the decrease in credibility.

[0063] S63. Output the weak credibility nodes and root cause scenario elements as a simulation credibility weak link diagnosis report, which is used to guide the adjustment of simulation model parameters or the correction of simulation scenarios.

[0064] The present invention also provides a reliability evaluation device for intelligent driving simulation testing, comprising:

[0065] The causal graph construction module is used to extract scene elements from the software virtual simulation environment of intelligent driving simulation testing and construct a causal graph of the simulation scene.

[0066] The Bayesian network construction module is used to transform the causal graph of the simulation scenario into a Bayesian network structure, and to learn the conditional probability distribution parameters of each node based on the simulation test data and the actual vehicle test data in the closed test field.

[0067] The node credibility calculation module is used to calculate the local credibility index of each node based on the likelihood ratio between the simulated and real conditional probability distributions of each node in the Bayesian network.

[0068] The scene criticality calculation module is used to calculate the causal effect strength of each scene element node relative to the driving decision center node based on the causal graph of the simulation scene, and normalize it into scene criticality weight.

[0069] The overall credibility assessment module is used to weight and aggregate the local credibility indicators of each node according to the scene criticality weight, obtain the overall credibility score, and map it to the credibility level range for output.

[0070] Furthermore, the device also includes:

[0071] The weak link localization module is used to identify weak nodes in credibility where the local credibility index is lower than the preset weak threshold, trace the upstream causal path along the causal graph of the simulation scene, determine the root cause scene elements, and output a simulation credibility weak link diagnosis report.

[0072] The present invention also provides an electronic device, comprising:

[0073] At least one processor; and a memory communicatively connected to said at least one processor; wherein,

[0074] The memory stores instructions that can be executed by the at least one processor, which enables the at least one processor to perform the above-described intelligent driving simulation test credibility evaluation method.

[0075] The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the above-described intelligent driving simulation test credibility evaluation method.

[0076] Compared with the prior art, the beneficial effects of the present invention are:

[0077] 1. This invention is the first to construct a causal graph for intelligent driving simulation scenarios, which can explicitly model the causal driving relationships between simulation scenario elements and achieve a reliable assessment of causal interpretability. The invention explicitly models the causal driving relationships between elements in the vehicle motion layer, traffic participant layer, road environment layer, and driving decision layer as a directed acyclic graph (DAG) causal graph for simulation scenarios, overcoming the limitation of existing methods that only establish inter-layer relationships but lack causal directionality.

[0078] 2. This invention employs Bayesian network-based likelihood ratio inference, embedding causal transmission effects within the credibility calculation process of intelligent driving simulation, eliminating the need for additional cross-layer verification steps. The invention transforms the causal graph of the intelligent driving simulation scenario into a Bayesian network. The local credibility index of each node is calculated based on the likelihood ratio of its conditional probability distribution. This likelihood ratio is itself calculated under the conditional constraints of the parent node; therefore, the causal transmission effect is embedded within the conditional probability structure, and the deviation of upstream nodes is naturally reflected in the likelihood ratio of downstream nodes.

[0079] 3. By utilizing scene-adaptive weighting based on the strength of causal effects, the limitations of static weights and fixed hierarchical structures can be overcome. This invention automatically calculates the scene criticality weights based on the sum of the products of the mediation effects of causal paths from each node to the driving decision center node in the causal graph of the intelligent driving simulation scene, so that the weights dynamically change with the scene type and the importance of the elements.

[0080] 4. The granularity of weak link location reaches the element level, and the causal tracing is accurate to a single scene element variable. This invention identifies nodes with local credibility indices below the weak threshold and traces the root cause scene element along the upstream causal path of the simulation scene causal graph. This allows it to locate specific scene element variables (such as "the deviation of the relative distance model between the vehicles in front"), rather than locating only the layer level (such as "the sensor layer") of existing methods.

[0081] 5. Natural quantification of uncertainty is achieved based on Bayesian network likelihood ratio inference. This invention transforms the causal graph of the simulated scene into a Bayesian network, performs likelihood ratio inference using the conditional probability distribution of simulated and real dual-source data, and utilizes local credibility indices. The exponential decay form of this invention naturally quantifies uncertainty, avoiding the inaccurate evaluation results caused by subjective selection of membership functions and unreasonable selection of fuzzy synthesis operations in existing methods such as fuzzy comprehensive evaluation. Specifically, the local credibility index of this invention... It has a clear statistical meaning, reflecting the statistical distance between two conditional probability distributions, rather than being a heuristic indicator.

[0082] 6. Applicable to pure software virtual simulation environments, filling the gap in overall credibility assessment methods for causal-driven approaches in this field. This invention is specifically designed for software virtual simulation environments, evaluating causal-driven approaches by comparing the probability distributions of simulation and real-vehicle test data. It does not rely on physical measurement signals from hardware-in-the-loop systems, thus broadening its applicability. Furthermore, this invention achieves overall credibility assessment of causal-driven approaches through causal graphs and Bayesian networks, filling the gap in such methods within pure software virtual simulation environments. Attached Figure Description

[0083] Figure 1 This is a flowchart illustrating the reliability assessment method for intelligent driving simulation testing of the present invention.

[0084] Figure 2 This is a schematic diagram of the causal graph structure of the simulation scene of the present invention;

[0085] Figure 3 This is a schematic diagram of the structure of the present invention, which transforms a causal graph into a Bayesian network;

[0086] Figure 4 This is a structural block diagram of the intelligent driving simulation test credibility evaluation device of the present invention. Detailed Implementation

[0087] The technical solutions in the embodiments of the present invention will be clearly and completely described below. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0088] Example 1: Credibility Assessment of Intelligent Driving Simulation Test in Highway Lane Changing Scenario

[0089] Step S1: Simulation scene element extraction and causal graph construction

[0090] refer to Figure 1 and Figure 2 This embodiment evaluates the credibility of intelligent driving simulation tests for highway lane-changing scenarios.

[0091] S11. Scene element hierarchical classification:

[0092] Extract scene elements from the software virtual simulation environment and categorize them hierarchically:

[0093] Vehicle motion layer elements: longitudinal speed lateral acceleration Heading angle Longitudinal acceleration ;

[0094] Traffic participant level elements: relative distance to the vehicle in front Relative speed of the vehicle in front Relative distance of vehicles to the side and rear ;

[0095] Road environmental layer elements: road curvature Road surface adhesion coefficient ;

[0096] Driving decision-making elements: target lane selection Desired longitudinal acceleration Desired steering angle .

[0097] S12. Establish causal driving relationships:

[0098] Based on prior knowledge in the field of road driving, the following causal driving relationship is established:

[0099] Road curvature →Relative distance to the vehicle in front (Curves affect following distance);

[0100] Road curvature →Desired longitudinal acceleration (curving reduces expected acceleration);

[0101] Road surface adhesion coefficient →Desired longitudinal acceleration (Low adhesion reduces acceleration requirements);

[0102] Relative distance of the vehicle in front Relative speed of the vehicle in front →Desired longitudinal acceleration (Decision-making based on the vehicle's condition);

[0103] Relative distance of vehicles to the side and rear →Target Lane Selection (Feasibility assessment of lane changing);

[0104] Target lane selection →Desired steering angle (Operation by changing lanes);

[0105] Relative distance of the vehicle in front →Target Lane Selection (Too close to trigger lane-changing intention);

[0106] Desired longitudinal acceleration →Longitudinal acceleration (Longitudinal motion response);

[0107] longitudinal acceleration → Longitudinal velocity (Longitudinal motion response);

[0108] Desired steering angle → Lateral acceleration (Lateral motion response);

[0109] lateral acceleration → heading angle (Lateral motion response).

[0110] S13. Construct a cause-effect graph for the simulation scenario:

[0111] Based on driving decision-making elements ( , , Using as the central node, the above causal driving relationships are represented in the form of a directed acyclic graph, forming a causal graph of the simulation scene containing 12 nodes and 12 directed edges, as shown below. Figure 2 As shown.

[0112] Step S2: Construction of Bayesian Network Based on Causal Graph

[0113] S21. Transforming causal graphs into Bayesian network structures:

[0114] refer to Figure 3 The direction of the directed edges of causal relationships in the causal graph of the simulation scene remains unchanged. The 12 scene element nodes are used as random variable nodes of the Bayesian network, and the 12 directed edges of causal relationships are used as conditional dependency edges to form the initial structure of the Bayesian network.

[0115] Data alignment and discretization:

[0116] Obtain 5000 time steps each from the simulation test dataset and the real test dataset for the same highway lane-changing scenario, and align the timestamps. Discretize continuous variables using an equal-frequency binning method.

[0117] Longitudinal velocity It is divided into 5 states (0-20, 20-40, 40-60, 60-80, 80-100 km / h);

[0118] Relative distance of the vehicle in front It is divided into 5 states (0-10, 10-20, 20-40, 40-60, 60+m);

[0119] Desired longitudinal acceleration It is divided into 4 states (strong deceleration, weak deceleration, weak acceleration, and strong acceleration).

[0120] Target lane selection Two states: (Stay in the current lane, Change to the target lane);

[0121] The same applies to the other variables.

[0122] S22, Learning conditional probability distribution parameters:

[0123] For each node in the Bayesian network, the conditional probability distribution parameters of that node under the conditions of its parent node are learned using both simulation test datasets and closed test track real vehicle test datasets. For example, for the node "expected longitudinal acceleration", the conditional probability distribution parameters are learned using the maximum likelihood estimation method. Its parent node is "relative distance to the preceding vehicle". "Relative speed of the vehicle in front" "Road adhesion coefficient" "Road curvature" "The data from the simulation test dataset and the closed test track real vehicle test dataset are analyzed separately." The conditional probability table.

[0124] S23. Test of conditional independence:

[0125] The conditional independence of the Bayesian network structure was tested using the chi-square test with a significance level of 0.05, and directed edges that did not satisfy the conditional independence constraint were removed. In this embodiment, the road curvature was found to be... For desired longitudinal acceleration The direct causal edge does not satisfy the conditional independence constraint (the influence is through the relative distance of the preceding vehicle). (Indirect transmission), therefore, removing the direct directed edge, the simplified Bayesian network contains 12 nodes and 11 directed edges.

[0126] Step S3: Probabilistic inference and node credibility calculation based on likelihood ratio

[0127] Calculate the local credibility index for each node in the Bayesian network. The index is based on the node's "relative distance to the vehicle in front". For example:

[0128] S31. Based on real-vehicle test data from a closed test track, calculate the log-marginal likelihood of this node under the true conditional probability distribution:

[0129]

[0130] Calculations show that .

[0131] S32. Based on simulation test data, calculate the log-marginal likelihood of this node under the simulation conditional probability distribution:

[0132]

[0133] Calculations show that .

[0134] S33, Computation Node Local credibility index:

[0135]

[0136] Using this method, the local credibility index of all 12 nodes was calculated. The results of this embodiment are as follows:

[0137]

[0138] Step S4: Calculation of Scene Criticality

[0139] The scene criticality weight of each node is calculated based on the causal graph of the simulated scene.

[0140] S41. Calculate the strength of causal effect:

[0141] With the driving decision center node ( , , aggregation node Using ) as the endpoint, calculate the distance from each node to The sum of the products of the causal path mediation effects.

[0142] Using the node "relative distance to the vehicle in front" For example, its journey to the driving decision center node... The causal pathways are:

[0143] Path 1: → Directed edge causality strength coefficient (Calculated based on conditional mutual information);

[0144] Path 2: → Directed edge causality strength coefficient .

[0145] Then the strength of the causal effect:

[0146]

[0147] Similarly, calculate the causal effect strength for all nodes:

[0148]

[0149] S42. Normalization yields scene criticality weights:

[0150] Normalize the strength of causal effects only for Normalize the nodes ( The downstream response nodes do not participate in the weighting, but their local confidence level in step S3 is still retained as reference information.

[0151]

[0152] The criticality weights of each node in the scenario are as follows:

[0153]

[0154] Step S5: Weighted Aggregation and Overall Credibility Assessment

[0155] S51. Calculate the overall credibility score:

[0156]

[0157] S52, Credibility Level Mapping:

[0158] Preset credibility level mapping rules:

[0159]

[0160] S53. Output evaluation results:

[0161] This is mapped to a "high credibility" level. The credibility assessment conclusion of this highway lane-changing scenario simulation test is: overall credibility score 0.9685, level "high credibility".

[0162] Step S6: Locating and Feedback on Weak Links in Simulation Credibility

[0163] S61. Identify vulnerable nodes:

[0164] Set the weak threshold to The local credibility index is lower than... The nodes are:

[0165] Target lane selection ( (If the node is below the threshold, it is marked as a weak node).

[0166] Desired steering angle ( If the value is below the threshold, it is marked as a weak node.

[0167] S62. Causal Tracing:

[0168] Tracing the source along the causal graph of the simulation scenario:

[0169] Upstream causal path: → , → The relative distance to the vehicle in front and the relative distance to vehicles to the side and rear are the key factors affecting the credibility of lane-changing decisions.

[0170] Upstream causal path: → The lack of credibility in the lane-changing decision led to the execution of the turnaround.

[0171] S63. Output diagnostic report:

[0172] Two weak points were identified:

[0173] Weak point 1: Target lane selection ( ), root causes: bias in the relative distance model of the vehicle in front and bias in the motion prediction model of the vehicle to the side and rear;

[0174] Weak point 2: Desired turning angle ( The root cause is the insufficient credibility of the lane-changing decision, leading to causal transmission.

[0175] Improvement suggestions: Prioritize correcting the parameters of the side and rear vehicle motion prediction model and the simulation accuracy of the relative distance to the vehicle in front.

[0176] Example 2: Credibility Assessment of Intelligent Driving Simulation Test in Complex Urban Intersection Scenarios

[0177] Step S1: Simulation scene element extraction and causal graph construction

[0178] This embodiment has the same method steps as Embodiment 1, the difference lies in the scene type and scene elements.

[0179] Scene elements:

[0180] Vehicle motion layer: longitudinal speed lateral acceleration Heading angle ;

[0181] Traffic participant level: relative positions of oncoming vehicles Pedestrian crossing distance Non-motorized vehicle movement status ;

[0182] Road environment layer: Traffic signal light status Intersection geometric parameters Weather conditions ;

[0183] Driving decision-making elements: Traffic decision (Traffic / Parking Waiting), Expected Longitudinal Acceleration Desired steering angle .

[0184] Causal driving relationship:

[0185] Traffic light status →Passage Decision (Stop at red lights, go at green lights);

[0186] Pedestrian crossing distance →Passage Decision (Pedestrians must yield to parked vehicles when crossing the road);

[0187] Relative position of oncoming vehicles →Passage Decision (When turning left, you must yield to oncoming traffic going straight.)

[0188] Non-motorized vehicle movement status →Passage Decision (Yield to non-motorized vehicles);

[0189] Weather conditions W → Desired longitudinal acceleration (Reduce vehicle speed in rainy weather);

[0190] Weather conditions W → Pedestrian crossing distance (Changes in pedestrian behavior on rainy days);

[0191] Traffic Decision →Desired longitudinal acceleration Desired steering angle (Motion response);

[0192] Desired longitudinal acceleration → Longitudinal velocity (Longitudinal motion response);

[0193] Desired steering angle → Lateral acceleration Heading angle (Lateral motion response).

[0194] The complex intersection scenario in urban areas includes elements not found in highway scenarios, such as traffic light status, pedestrian crossing distance, and non-motorized vehicle movement. In this scenario, the sum of the products of the causal path mediation effects of these new elements on driving decisions is relatively large, thus receiving higher weights in the scenario's criticality weight calculation. For example, the causal effect strength of traffic light status... The causal effect strength of pedestrian crossing distance These elements automatically receive high weight in urban scenes (e.g.) and This reflects the scene-adaptive characteristics of scene criticality weights.

[0195] Based on calculations, the overall credibility score of this embodiment is: This is mapped to a "medium confidence" level. The weak node is the "pedestrian crossing distance". ( The causal analysis revealed that the root cause was a significant discrepancy between the pedestrian behavior model's movement patterns in rainy weather and the actual data. It is recommended to correct the rainy weather parameters of the pedestrian behavior model.

[0196] Example 3: Intelligent Driving Simulation Test Credibility Evaluation Device

[0197] refer to Figure 4 This embodiment provides a reliability evaluation device for intelligent driving simulation testing, including:

[0198] The causal graph construction module 401 is used to extract vehicle motion layer elements, traffic participant layer elements, road environment layer elements and driving decision layer elements from the software virtual simulation environment of intelligent driving simulation test, establish causal relationship directed edges based on prior knowledge in the driving domain and physical constraint rules, and construct a causal graph of the simulation scene.

[0199] The Bayesian network construction module 402 is used to transform the causal graph of the simulation scene into a Bayesian network structure, perform time stamp alignment and equal frequency bin discretization on the simulation test data and the real vehicle test data in the closed test field, learn the simulation conditional probability table and the real conditional probability table of each node respectively, and perform conditional independence test to simplify the network structure.

[0200] The node credibility calculation module 403 is used to calculate the local credibility index based on the log marginal likelihood difference of each node under the simulated and real conditional probability distributions using the exponential decay formula.

[0201] The scene criticality calculation module 404 is used to calculate the causal effect intensity based on the sum of the products of the mediation effects of the causal paths from each node to the driving decision center node in the causal graph of the simulation scene, and obtain the scene criticality weight after normalization.

[0202] The overall credibility assessment module 405 is used to weight and aggregate the local credibility indicators of each node according to the scene criticality weight, obtain the overall credibility score, and map it to one of the four levels of "untrustworthy", "low credibility", "medium credibility" and "high credibility" output.

[0203] The weak link localization module 406 is used to identify nodes whose local credibility index is lower than the weak threshold, trace the root cause scene elements along the upstream path of the causal graph of the simulation scene, and output a credibility weak link diagnosis report.

[0204] Example 4: Electronic Equipment

[0205] In one implementation of the method of the present invention, the electronic device includes at least one processor and a memory communicatively connected to the at least one processor. The memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enable the at least one processor to perform the intelligent driving simulation test credibility evaluation method described in Embodiment 1 or Embodiment 2.

[0206] The processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices. The memory can include volatile memory (such as RAM) or non-volatile memory (such as ROM, flash memory, hard disk, or optical disk).

[0207] Example 5: Computer-readable storage medium

[0208] This embodiment provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the intelligent driving simulation test credibility evaluation method described in Embodiment 1 or Embodiment 2.

[0209] The computer-readable storage medium may include various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A method for evaluating the credibility of intelligent driving simulation testing, characterized in that, Includes the following steps: S1. Simulation Scene Element Extraction and Causal Graph Construction: Scene elements are extracted from the virtual simulation environment of the intelligent driving simulation test software. The scene elements include vehicle motion elements, traffic participant elements, road environment elements, and driving decision elements. With the driving decision elements as the central node, directed edges of causal relationships between scene elements are established to construct a simulation scene causal graph. The directed edges of causal relationships represent the causal driving relationship between upstream scene elements and downstream scene elements. S2. Construction of Bayesian Network Based on Causal Graph: The causal graph of the simulation scene is transformed into a Bayesian network structure, wherein the scene element nodes in the causal graph are used as random variable nodes of the Bayesian network, and the directed edges of causal relationships are used as conditional dependency edges of the Bayesian network. Based on the simulation test data and the actual vehicle test data in the closed test field, the conditional probability distribution parameters of each node in the Bayesian network are learned respectively to obtain the simulation conditional probability distribution and the real conditional probability distribution. S3. Probabilistic reasoning and node credibility calculation based on likelihood ratio: For each node in a Bayesian network, the local credibility index of the node is calculated based on the likelihood ratio between the simulated conditional probability distribution and the real conditional probability distribution. The likelihood ratio is the ratio of the marginal likelihood of the node under real data to the marginal likelihood under simulated data. S4. Calculation of scene criticality: Based on the causal graph of the simulation scene, calculate the causal effect intensity of each scene element node relative to the driving decision center node, and normalize the causal effect intensity as the scene criticality weight of the node. S5. Weighted Aggregation and Overall Credibility Assessment: The local credibility index of each node is weighted and aggregated according to the scene criticality weight to obtain the overall credibility score of the simulation test. The overall credibility score is mapped to a preset credibility level range, and the credibility level result of the simulation test is output.

2. The method for evaluating the credibility of intelligent driving simulation testing according to claim 1, characterized in that, The method for constructing the causal graph of the simulation scene in step S1 includes: S11. Based on the semantic description information of the intelligent driving simulation test scenario, the scenario elements are hierarchically classified to obtain the elements of the vehicle motion layer, the elements of the traffic participants layer, the elements of the road environment layer, and the elements of the driving decision layer. S12. Based on prior knowledge in the driving domain and physical constraint rules of the simulation scenario, establish causal driving relationships among the hierarchically categorized scenario elements: traffic participant layer elements and road environment layer elements drive driving decision layer elements respectively; driving decision layer elements drive vehicle motion layer elements, and road environment layer elements drive traffic participant layer elements. S13. Set the driving decision-making layer elements as the central node, and represent the causal driving relationship between the elements of each layer in the form of a directed acyclic graph to form the causal graph of the simulation scene.

3. The method for evaluating the credibility of intelligent driving simulation testing according to claim 1, characterized in that, The method for constructing the Bayesian network in step S2 includes: First, the simulation test data and the real vehicle test data at the test site are aligned: the simulation test data and the real vehicle test data at the test site are time-stamp aligned so that the simulation data and the real data correspond at the same time step in the same driving scenario. For continuous scenario element variables, the equal frequency binning method is used to discretize them into a finite number of states to adapt to the conditional probability distribution calculation of Bayesian network. Then perform the following steps: S21. Keep the direction of the directed edges of the causal relationships in the causal graph of the simulation scene unchanged, and use them as directed edges of the Bayesian network. S22. For each node in the Bayesian network, using the simulation test dataset and the closed test field real vehicle test dataset respectively, the maximum likelihood estimation or Bayesian parameter estimation method is used to learn the conditional probability distribution parameters of the node under the condition of its parent node, and the simulation conditional probability table and the real conditional probability table are obtained respectively. S23. Perform a conditional independence test on the Bayesian network structure, remove directed edges that do not satisfy the conditional independence constraint, and obtain the simplified Bayesian network.

4. The method for evaluating the credibility of intelligent driving simulation testing according to claim 1, characterized in that, The method for calculating the local credibility index of the node in step S3 includes: S31. For any node in a Bayesian network Let its parent node set be Based on real-vehicle test data from a closed test track, the log-marginal likelihood of this node under the true conditional probability distribution is calculated. : ； in, The number of test data samples; S32. Based on simulation test data, calculate the log marginal likelihood of this node under the simulated conditional probability distribution. : ； S33, Computation Node Local credibility index : ； in, , The closer the value is to 1, the higher the credibility of the scene elements corresponding to that node.

5. The method for evaluating the credibility of intelligent driving simulation testing according to claim 1, characterized in that, The method for calculating the scene criticality weight in step S4 includes: S41. In the cause-effect graph of the simulation scenario, taking the central node of the driving decision-making layer as the endpoint, for each scenario element node... The sum of the products of mediation effects along all causal paths from the node to the central node is used as the strength of the causal effect of that node on the central node. : ； in, As the driving decision center node, From arrive The set of all causal paths, For directed edges on the path The causal strength coefficient is calculated based on the conditional mutual information between corresponding variables in the simulation data; S42. Normalize the causal effect intensity of all scene element nodes to obtain the scene criticality weight. : ； in, This represents the total number of scene element nodes.

6. The method for evaluating the credibility of intelligent driving simulation testing according to claim 1, characterized in that, The weighted aggregation and credibility level mapping method described in step S5 includes: S51. Calculate the overall credibility score based on the scene criticality weight and local credibility index of each node. : ； in, This represents the total number of scene element nodes. For nodes The criticality weight of the scenario For nodes Local credibility index; S52. Preset credibility level mapping rules: Map the overall credibility score Mapped to a credibility level range, the credibility level range includes at least an "untrustworthy" level, a "low credibility" level, a "medium credibility" level, and a "high credibility" level. The scoring threshold corresponding to each level is set based on the statistical quantile of historical simulation test data or industry standards. S53, Output overall credibility score The credibility level and its corresponding credibility level are used as the credibility assessment conclusion of this intelligent driving simulation test.

7. The method for evaluating the credibility of intelligent driving simulation testing according to claim 1, characterized in that, It also includes step S6: locating and providing feedback on weak points in simulation credibility, specifically including: S61. Identify scene element nodes whose local credibility index is lower than the preset weak threshold and designate them as weak credibility nodes. S62. Following the causal relationship direction of the causal graph of the simulation scene, trace the upstream causal path of the weak credibility node to determine the root cause scene element that leads to the decrease in credibility. S63. Output the weak credibility nodes and root cause scenario elements as a simulation credibility weakness diagnosis report, which is used to guide the adjustment of simulation model parameters or the correction of simulation scenarios.

8. A reliability evaluation device for intelligent driving simulation testing, characterized in that, include: The causal graph construction module is used to extract scene elements from the software virtual simulation environment of intelligent driving simulation testing and construct a causal graph of the simulation scene. The Bayesian network construction module is used to transform the causal graph of the simulation scene into a Bayesian network structure, and learn the conditional probability distribution parameters of each node based on the simulation test data and the real test data. The node credibility calculation module is used to calculate the local credibility index of each node based on the likelihood ratio between the simulated and real conditional probability distributions of each node in the Bayesian network. The scenario criticality calculation module is used to calculate the causal effect strength of each scenario element node relative to the driving decision center node based on the causal graph of the simulation scenario, and normalize it into scenario criticality weights. The overall credibility assessment module is used to weight and aggregate the local credibility indicators of each node according to the scene criticality weight, obtain the overall credibility score, and map it to the credibility level range for output. The weak link localization module is used to identify weak nodes in credibility where the local credibility index is lower than the preset weak threshold, trace the upstream causal path along the causal graph of the simulation scene, determine the root cause scene elements, and output a simulation credibility weak link diagnosis report.

9. An electronic device, characterized in that, include: At least one processor; The system includes a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enable the at least one processor to perform the intelligent driving simulation test credibility assessment method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the intelligent driving simulation test credibility evaluation method as described in any one of claims 1 to 7.

Images