Inplausible automatic driving decision-making method based on causal knowledge

Abstract

The invention belongs to the technical field of automatic driving testing, and particularly relates to an interpretable automatic driving decision-making method based on causal knowledge. Comprising the following steps: step 1, discovering causals oriented to automatic decision knowledge; 2, constructing an interpretable automatic driving decision model; 3, evaluating the interpretability of the automatic driving decision-making system; and 4, evaluating the performance of the automatic driving decision-making system. The method comprises four steps of causal discovery for automatic driving decision-making knowledge, construction of an interpretable automatic driving decision-making model, evaluation of the interpretability of the automatic driving decision-making system, and evaluation of the performance of the automatic driving decision-making system. Based on a causal discovery model of decision observation data, key variables of a causal relationship influencing self-vehicle driving behaviors are found from the data, an interpretability standard and a decision performance evaluation index are defined, the interpretability and decision performance of an automatic driving decision system are evaluated, and the automatic driving decision system is continuously optimized and iterated.

Description

Technical Field

[0001] The present invention belongs to the technical field of autonomous driving testing, and specifically is an explainable autonomous driving decision-making method based on causal knowledge. Background Art

[0002] With the continuous development of deep learning theory, the continuous reduction of sensor and computing hardware resource costs, and the emergence of policies and regulations supporting the development of autonomous driving technology, the commercial application process of autonomous driving technology has been promoted. Rule-based autonomous driving decision-making methods cannot adapt to complex driving environments due to their excessive reliance on manually written rules. With the continuous development of massive driving data and large models, end-to-end autonomous driving technology is expected to become the mainstream technology for the application of intelligent vehicles. Intelligent vehicles need to consider the impact of the surrounding environment to make safe and reliable decision-making behaviors. Traditional end-to-end autonomous driving technology, which is mainly data-driven, takes the surrounding environment information as input and outputs the self-driving behavior action sequence. However, a lot of environmental information is irrelevant to the self-vehicle decision-making behavior, and such a large amount of irrelevant data affects the performance of the model. In addition, since the essence of end-to-end large models is to learn the correlation between data rather than the causal relationship between data, the interpretability of the autonomous driving decision-making system is still an important barrier to the development of high-level autonomous driving systems.

[0003] In order to accelerate the development of autonomous driving technology, the development of interpretable decision-making methods for autonomous driving and the comprehensive and effective evaluation of the interpretability and decision-making performance of the system are key technologies that researchers are currently focusing on. The interpretable decision-making method for autonomous driving based on causal knowledge aims to provide a clear and transparent decision-making process. Through massive driving data, the causal knowledge that affects the behavior decisions of autonomous driving vehicles is learned, and the causal knowledge is embedded in the spatiotemporal graph neural network model to intuitively reflect the impact of each node or each feature on the decision, thereby improving the interpretability of the model. By constructing a causal discovery model of autonomous driving decision-making knowledge, the causal relationship between the cause and result of the decision-making of the vehicle behavior is revealed. By constructing an interpretable decision-making model for autonomous driving and understanding the decision-making process of the model, safe and reliable decisions of the autonomous driving system can be achieved. Formulate interpretability evaluation standards and system evaluation indicators to evaluate the interpretability and decision-making performance of the autonomous driving decision-making system.

[0004] There are few patent forms around the explainable autonomous driving decision-making method based on causal knowledge in the field of intelligent vehicle decision-making. Chinese patent CN202211445291.6 discloses an explainable autonomous driving decision-making system and method thereof, which extracts key feature vectors from the traffic environment, inputs the deep Q network model, and outputs the corresponding decision instructions. Chinese patent CN202210201601.3 discloses an autonomous driving decision-making method and system based on knowledge graph, which realizes the explainability of autonomous driving decision information through the knowledge graph of the knowledge graph and increases the credibility of the autonomous driving decision-making system. Chinese patent CN202410487199.9 discloses a risk estimation-based reinforcement learning autonomous driving safety explainable decision-making method, which makes understandable optimal decisions under the premise of ensuring safety constraints. The above three patents can realize explainable autonomous driving decisions. Data-driven decision-making systems can only learn the correlation between variables, not the causal relationship. Therefore, it is still very critical to embed decision-making causal knowledge into deep learning models to realize explainable autonomous driving decisions. Summary of the invention

[0005] The present invention provides an explainable autonomous driving decision-making method based on causal knowledge, which can derive a safe and explainable autonomous driving mode, solving the problem that traditional data-driven autonomous driving decision-making systems cannot utilize prior knowledge and the resulting weak user acceptability.

[0006] The technical solution of the present invention is described as follows in conjunction with the accompanying drawings: A method for interpretable autonomous driving decision-making based on causal knowledge includes the following steps: Step 1: Discover the causal relationship for automatic decision-making knowledge; Step 2: Build an explainable autonomous driving decision model; Step 3: Evaluate the interpretability of the autonomous driving decision system; Step 4: Evaluate the performance of the autonomous driving decision system.

[0007] Furthermore, the specific method of step one is as follows: S11, collect multi-source heterogeneous driving data; S12. Establish a model based on causal discovery of decision-making observation data; S13. Integrate autonomous driving decision-making knowledge.

[0008] Furthermore, the specific method of S11 is as follows: S111, collecting virtual simulation data; By building a driving simulator and simulation platform, we can design various driving scenarios, including urban roads, highways, and rural roads, covering different weather conditions, traffic flows, and road conditions; add dynamic elements, including vehicles, pedestrians, and bicycles; simulate the real traffic environment, including roads, buildings, traffic signs, and signal lights; build a complete virtual driving environment, and the feature information of all scene elements constitutes a multi-source heterogeneous driving data set in the virtual simulation environment. v , as shown in formula (1): (1) In the formula, h e , h n , h b , h p , h l , h s are the feature sets of the scene elements of the vehicle, adjacent vehicles, adjacent bicycles, pedestrians, traffic lights and signboards; among them, h e ={ x e , y e , vx e, vy e, s e , t e , b e}, h n ={ s n , x n , y n , vx n, vy n}, h b ={ x b , y b , vx b, vy b}, h p ={ x p , y p , vx p, vy p}, h l ={ x l , y l , sl}, h s ={ x s , y s , s s}; x e 、y e 、vx e、vy e、s e 、t e and b e They are the lateral position, longitudinal position, lateral speed, longitudinal speed, steering wheel angle, accelerator pedal opening and brake pedal opening of the vehicle; s n 、x n 、y n 、vx n、vy n They are the type, lateral position, longitudinal position, lateral speed, and longitudinal speed of the adjacent vehicles respectively; x b 、y b , vx b, vy b are the Lateral position, longitudinal position, lateral speed, longitudinal speed; x p 、y p 、vx p、vy p They are the pedestrian’s lateral position, longitudinal position, lateral speed, and longitudinal speed respectively; x l 、y l 、s l They are the horizontal position, vertical position and status of the traffic light respectively; x s 、y s 、s s They are the horizontal position, vertical position and status of the traffic sign respectively; S112, collecting real vehicle data; The raw data obtained by the inertial navigation positioning device, laser radar, millimeter wave radar, and camera are processed to obtain target-level data, and a multi-source heterogeneous driving data set Ω is obtained in the real vehicle environment. r ; S113, classifying the scenes; The simulation data Ω v and multi-source heterogeneous driving data set, namely real vehicle driving data Ωr Slice the scene according to the road topology structure and determine the starting and ending points of each scene slice. The road topology structure is divided into roads with different curvatures, intersections with or without traffic lights, T-intersections, and roundabouts. Ensure that the elements such as roads, traffic signs, and signal lights in the slices are complete and consistent. Further classify the scene segments according to the number of vehicles in the environment around the vehicle, and the same number of vehicles in the environment are classified into the same type of scene. S114, integrating multi-source heterogeneous driving data; The generated scene slices are stored as independent simulation scenes, and a slice index is established to save the type, location, and environment information of each slice. The slice library is expanded as needed, and new slice types and scenes are added to obtain a classification set Ω of surrounding vehicles of different road sections and different numbers. v,n and Ω r,n ; The specific method of S12 is as follows: S121, preprocessing the data; The obtained Ω v,n and Ω r,n Data cleaning and data standardization are performed; first, missing values ​​are filled in the data using the Lagrange interpolation method, and data standardization is used to convert data of different scales into a unified scale. The calculation formula is (2): (2) In the formula, X max and X min are the maximum and minimum values ​​of the data respectively; S122, build a causal relationship model; Kernel regression is used as a nonlinear model, and environmental vehicles, traffic lights and traffic signs are regarded as variables. Based on the driving behavior of the ego vehicle in different road sections, with different numbers of environmental vehicles, and with or without traffic lights and traffic signs, the nonlinear causal relationship between the variables that affect the ego vehicle's driving behavior is analyzed. S123, fitting and evaluating the model; The Granger causality test is used to fit the model parameters, and the causal skeleton structure is determined through significance test; a causal graph is generated to represent the causal relationship between the variables that affect the driving behavior of the self-driving car, and to quantify the degree of influence of different variables on the driving behavior of the self-driving car, and finally a directed acyclic graph representing the causal relationship is output; The specific method of S13 is as follows: S131, defining road section driving knowledge; Integrate the knowledge that affects the behavior of autonomous vehicles on the road; consider the interaction between the self-driving vehicle and the surrounding vehicles and traffic signs, use causal diagrams to represent the causal relationship between the variables that affect the self-driving behavior, and classify the causal diagrams according to the presence or absence of traffic signs and the number of vehicles in the surrounding environment of the self-driving vehicle; S132, define intersection decision knowledge; Define the decision-making knowledge that affects the driving behavior of the vehicle in the intersection scenario; consider the impact of traffic lights, pedestrians and environmental vehicles on the vehicle; and classify and save the causal graph that can represent the causal relationship between the variables that affect the driving behavior of the vehicle according to the presence or absence of traffic lights, the number of pedestrians and environmental vehicles.

[0009] Furthermore, the specific method of step 2 is as follows: S21. Constructing driving scenarios for autonomous vehicles; S22. Build an autonomous driving decision model; S23. Evaluate the model offline test results.

[0010] Furthermore, the specific method of S21 is as follows: S211, extracting scene elements; Extract scene elements in the driving scene of the autonomous driving vehicle. Dynamic scene elements include the vehicle, adjacent vehicles, pedestrians, bicycles and traffic lights, and static scene elements include traffic signs. S212, extracting causal relationships; The similarity of the integrated autonomous driving decision causal graph and the spatiotemporal graph structure is calculated, and the causal graph with high similarity is selected. The autonomous driving decision causal knowledge is initialized to the spatiotemporal graph. The similarity calculation is shown in formula (11): (11) In the formula, G c and G d They are cause-effect diagram and driving scenario diagram respectively; mcs ( G c , G d ) is the number of nodes of the largest common subgraph of the two graphs; max (| G c |,| G d |) is the ratio of the product of the number of nodes in the two graphs; S213, representing an autonomous driving driving scenario based on a spatiotemporal graph structure; The driving scene of the autonomous driving vehicle is represented by a spatiotemporal graph. For a certain moment, the scene elements are represented by the nodes of the graph, and the relationship between the scene elements is represented by the directed edges between the nodes. The dynamic scene modeling is as follows: on the basis of the graph structure, the time dimension is added to form a graph model in the form of a sequence. The graph structure is represented as shown in formula (12): (12) In the formula, V ={ v e , v n , v b , v p , v l , v s};node v e , v n , v b , v p , v l , v s They are the own car, neighboring car, bicycle, pedestrian, traffic light and sign; E ={ ee n , ee b , ee p , ee l , ee s}, ee n , ee b , ee p , ee l , ee s are directed edges between the vehicle and neighboring vehicles, bicycles, pedestrians, traffic lights, and traffic signs. The connection relationship of the directed edges is determined by the distance between the nodes. v e and v i Distance d ( v e , v i ) is calculated by formulas (13) and (14): (13) (14) when d ( v e , vi )<100m, v e and v i There are directed edges between them; the weights of the directed edges are initialized by the extracted causal graph to ensure that the sum of the weights of the directed edges of all nodes is 1; The specific method of S22 is as follows: S221. Build a dataset for autonomous driving decision-making; The collected autonomous driving vehicle driving data is divided into data sets; a fixed-size sliding window is used to create training samples; the historical 3s autonomous driving vehicle driving data is used to predict the driving behavior action sequence of the next 2s; 70%, 20% and 10% of the sample quantity are divided into training set, test set and validation set respectively; the number of samples N tot The calculation formula is shown in (15): (15) In the formula, N is the total length of the time series; the window size W The number of time steps included for each sample; step size S is the number of time steps for each window movement; S222, embedding causal knowledge in autonomous driving decision making; The causal graph with high similarity is calculated, the causal relationship of the causal graph is extracted, and the causal relationship is embedded into the spatiotemporal graph model; including the relationship between nodes and edges that affect the driving behavior of the vehicle; S223, learning the spatiotemporal correlation of “ego vehicle-scene elements”; The spatiotemporal interaction relationship between the ego vehicle and scene elements is acquired through the spatiotemporal graph neural network. First, the feature representation of the ego vehicle node is calculated through the temporal attention mechanism. Then, a graph convolutional neural network is designed to mine the graph structure at each moment, obtain the feature representation of the ego vehicle node that aggregates the surrounding nodes, and learn the importance of the ego vehicle feature at each moment through a temporal attention mechanism. Finally, the spatiotemporal correlation between the ego vehicle and scene elements is fused by the channel attention mechanism. S224, mapping the driving behavior of the vehicle; The learned encoding vector is mapped into a three-dimensional vector through dimension transformation, deformation and linear transformation, which respectively represent the steering wheel angle, accelerator pedal opening and brake pedal opening of the vehicle; The specific method of S23 is as follows: S231. Define evaluation indicators; The mean absolute error (MAE), root mean square error (RMSE), and mean absolute percentage error (MAPE) are used as indicators to evaluate the difference between the model prediction value and the true value. The calculation formulas (16), (17), and (18) of MAE, RMSE, and MAPE are as follows: (16) (17) (18) In the formula, and They are the actual driving action sequence and the predicted driving behavior action sequence of the autonomous driving vehicle respectively; N Indicates the number of samples; S232, performing offline testing on the model; By analyzing MAE, RMSE and MAPE, the degree of deviation between the predicted values ​​and the actual values ​​of the steering wheel angle, accelerator pedal opening and brake pedal opening of the autonomous driving vehicle is measured; the data set of autonomous driving decision-making is divided into several parts, and the model is trained and verified on different data subsets; the data set is evenly divided into k mutually exclusive subsets; k-1 subsets are selected each time for training, and the remaining subset is used for verification; the model is trained and verified k times, and the verification set is different each time; finally, the average of the k verification results is taken as the evaluation indicator of the model.

[0011] Furthermore, the specific method of step three is as follows: S31. Define interpretability criteria; S32, provide prior explanation of the autonomous driving decision model; S33, conduct post hoc explanation of the autonomous driving decision model; S34. Evaluate the autonomous driving decision system.

[0012] Furthermore, the specific method of S31 is as follows: S311, analytical model transparency; Analyze the decision-making mechanism of the model itself, and obtain explanations by studying the internal structure and parameters of the model; use the prior explanation results of the model as the model transparency analysis results; evaluate and score the model's input layer, graph attention layer, temporal convolution layer, spatiotemporal fusion layer, fully connected layer, normalization layer, activation layer, residual connection layer, and output layer. The specific formula and operation of graph convolution explain how node features are aggregated through neighboring nodes; similar input data has stable output results at each layer; output model transparency analysis results U ; S312, analyzing the sensitivity of the model; By introducing the importance of perturbation analysis features, it is used as an indicator to evaluate the ex post interpretability of the model; using the counterfactual interpretation method, the input features of the model are changed before prediction to analyze the changes in model prediction; the counterfactual interpretation results are used as the model sensitivity analysis results, and the model sensitivity analysis results are output V ; S313, conduct objective evaluation; Objectively evaluate the autonomous driving system; score the scene elements that affect the self-driving behavior according to the preset situation. Each scene element has a unique corresponding score. According to the score, the degree of influence on the self-driving behavior is classified into strong, general and weak. Finally, the objective evaluation results of the scene elements that affect the self-driving behavior in different scenarios are obtained. W ; S314. Define the interpretability comprehensive evaluation model; Define an interpretability comprehensive evaluation model, and output the comprehensive evaluation results of transparency, sensitivity and objective evaluation; analyze the key features obtained U , V and W The importance is sorted, and the sorting rules and calculation formula (33) are as follows: (33) In the formula, S The intersection set of key features obtained from S311, S312 and S313, the remaining features are based on the principle of two-by-two intersection, and the priority is defined as > > > > U > W > V , get the feature importance ranking result; The specific method of S32 is as follows: S321, analyzing the model structure; The defined graph structure is analyzed. The edges of the graph structure are the key factors for message transmission. The connection edges between nodes are determined by the interactive relationship between scene elements with different semantics and features. In order to distinguish the importance of different edges in message transmission, the fixed weights of different edges are first initialized according to the causal relationship model. During the model training process, the weights are parameterized, and the edge weights are updated through backpropagation and gradient descent algorithms, and the learned edge weights are visualized. S322, analyzing and visualizing the importance of nodes; Select and extract features that have a significant impact on model predictions, and visualize the results; Based on the adjacency matrix between the ego vehicle node and other scene element nodes, take the ego vehicle node as the central node, aggregate the features of the surrounding nodes, and visualize the importance of the nodes through a heat map; S323, obtaining prior explanation results; The nodes with the greatest impact on the driving behavior of the vehicle in the same scenario are counted and sorted according to the calculation results of the node importance. The frequency distribution of the characteristic variables with different degrees of influence on the vehicle behavior decision is displayed through histograms and box plots; the ranking results of the influence of different node characteristics on the model prediction results are determined U , generate a model explanation report, describing in detail the distribution of each feature to the prediction results; The specific method of S33 is as follows: S331, input key variables; Input key variables; specifically: trained autonomous driving decision model f , key variables defined X i ; S332, construct counterfactual scenarios; By constructing counterfactual scenarios, using the do operator to generate counterfactuals, intervening in multiple features X 1, X 2, …, X k , generate new input features, and the other features remain unchanged. The calculation formula (34) is: (34) In the formula, v i is the value after intervention, and the generated counterfactual state is a Similar inputs but with some features interfered with; then use the intervened input state to change the vehicle’s driving environment and calculate the counterfactual decision result , the calculation formula (35) is: (35) In order to find the best counterfactual state, an optimization algorithm is used to minimize the comprehensive loss function. The loss function calculation formula (36) is: (36) Where the first term is the decision difference and the second term is the similarity of the input. λ is a hyperparameter that balances these two terms; use the gradient descent algorithm to solve and find ; is the input feature; is the state after intervention; Using weighted Euclidean distance evaluation X and The similarity of is calculated by formula (37): (37) S333, Evaluation of key factors; Evaluate the key variables of ego-vehicle driving behavior; first, evaluate counterfactual decision outcomes The impact of the system, including safety, efficiency and compliance, defines an impact function The difference value is quantified, and the difference value quantification calculation formula (38) is: (38) In the formula, s , e and l They are the safety, efficiency and compliance evaluation score functions respectively; w s , w e and w l are the weight coefficients of safety, efficiency and compliance respectively; find the counterfactual decision according to the difference value safety, efficiency and compliance reduction key variables; S334, obtaining post-explanation results; Analyze the key variables, count the key variables that affect safety, efficiency and compliance through histograms and box plots, and record the counterfactual instance scenarios and key variables; determine the ranking results of the key variables that affect the safety, efficiency and compliance of the vehicle under the same scenario V ; The specific method of S34 is as follows: S341. Define a questionnaire; obtain an evaluation score based on the questionnaire for objective evaluation of the autonomous driving decision system; the questionnaire is designed as follows: the lane changing scenario determines the scene elements and features that affect the lane changing of the ego vehicle; the emergency braking scenario determines the scene elements and features that affect the braking of the ego vehicle; the intersection scenario with traffic lights determines the scene elements and features that affect the driving behavior of the ego vehicle.

[0013] S342. Statistically analyze the evaluation results of the questionnaire in S41, obtain the ranking results W of the scene elements and key feature variables according to the evaluation scores, and intuitively display the influence of different scene elements on the vehicle driving behavior through a graphical visualization method.

[0014] Furthermore, the specific method of step 4 is as follows: S41, automatic driving decision system is tested; S42. Setting evaluation indicators for the autonomous driving decision system; S43. Analyze the performance of the autonomous driving decision system.

[0015] Furthermore, the specific method of S41 is as follows: S411, perform model-in-the-loop testing; The self-driving vehicle, surrounding vehicles, pedestrians and traffic light signals are input into the trained model through the interface in the simulation software. The model calculates and outputs the steering wheel angle, accelerator pedal opening and brake pedal opening signal control instructions, and sends the control signal to the autonomous driving vehicle through the virtual interface to realize the state update of the autonomous driving vehicle; the starting point and the end point are set in the simulation software, and traffic participants with different densities and different flow rates are randomly generated. The sections of the road with different curvatures, intersections with traffic lights, intersections without traffic lights, and roundabouts are passed in the middle; the autonomous driving vehicle drives from the starting point to the end point, and the simulation ends when a collision occurs. If there is no collision and the vehicle drives in accordance with traffic rules, it is recorded as a successful test, and the number of collisions is recorded; S412, conduct real vehicle online testing; Carry out real-vehicle online testing of the algorithm on the road sections where data is collected, test the autonomous driving decision system's long-distance driving, lane keeping, lane changing, and overtaking functions in highway environments, as well as the traffic capacity of congested sections, multi-lane intersections, and intersections with or without traffic lights in urban road environments; conduct real-vehicle tests under different traffic densities and flows, and set starting and end points; when the autonomous driving decision system cannot cope with the current driving scenario, the driver can take over control of the vehicle at any time; when the driver takes over, it means that the autonomous driving decision system has failed to make a decision in the current scenario, and the number of times the driver takes over is recorded; The specific method of S42 is as follows: S421, conduct driving safety evaluation; In the model-in-the-loop test, the collision rate is used as an indicator to evaluate the driving safety of the autonomous driving vehicle. In the real vehicle online test, the takeover rate is used as an indicator to evaluate the driving safety. The calculation formulas (39) and (40) of the collision rate and takeover rate are as follows: (39) (40) In the formula, T tot is the total number of trials; T c is the number of collisions that occurred in the virtual simulation test; T t The number of times the driver took over during the actual vehicle test; S422, evaluating driving efficiency; The performance of the decision system is evaluated by the driving efficiency of the autonomous driving vehicle. The driving efficiency calculation formula (41) is as follows: (41) In the formula, T e The number of tests required to meet the minimum travel time; T suc is the number of successes; S423, conduct driving compliance evaluation; The performance of the decision system is evaluated through the driving compliance of the autonomous driving vehicle. The driving compliance calculation formula (42) is as follows: (42) In the formula, T l The number of tests to meet traffic regulations; The specific method of S43 is as follows: S431, obtaining model-in-the-loop test results; Evaluate the decision-making performance of the autonomous driving vehicle by analyzing the evaluation results of driving safety, driving efficiency and driving compliance, calculate the overall collision rate, evaluate the overall collision situation of the system in all test scenarios, draw a histogram or pie chart to show the distribution of the overall collision rate; calculate the collision rate in each specific scenario, evaluate the collision situation of the system in different scenarios, draw a bar chart or line chart to show the difference in collision rate in each scenario, and record the information of the self-vehicle, adjacent vehicles, pedestrians and environment involved in the collision accident; S432, obtaining the actual vehicle online test result; By drawing histograms or pie charts, the driving safety, driving efficiency and driving compliance evaluation results of autonomous vehicles in various scenarios are statistically analyzed; during the driving process of the autonomous vehicle, the information of the vehicle, adjacent vehicles, pedestrians and the environment is recorded when the driver takes over; S433, optimizing result feedback; Analyze and record the model-in-the-loop test results and the real vehicle online test results; study the decision-making mechanism of the model in failed use cases, locate the cause of decision failure, and improve the performance of the algorithm; analyze the cause of each collision, draw a grouped bar chart, and show the number of collisions caused by different reasons.

[0016] The beneficial effects of the present invention are: 1. The explainable autonomous driving decision-making method based on causal knowledge described in the present invention includes four steps: causal discovery for autonomous driving decision knowledge, building an explainable autonomous driving decision model, evaluating the explainability of the autonomous driving decision system, and evaluating the performance of the autonomous driving decision system. Based on the causal discovery model of decision observation data, the key variables of the causal relationship that affect the driving behavior of the vehicle are found from the data. Define the explainability standards and decision performance evaluation indicators, evaluate the explainability and decision performance of the autonomous driving decision system, and continuously optimize and iterate the autonomous driving decision system; 2. The explainable autonomous driving decision-making method based on causal knowledge described in the present invention constructs a causal discovery model for autonomous driving decision-making knowledge, and provides key variables that affect the driving behavior of the vehicle for the decision-making model based on spatiotemporal graph neural network; 3. The explainable autonomous driving decision-making method based on causal knowledge described in the present invention constructs an explainable autonomous driving decision-making model, realizes the embedded representation of decision-making causal knowledge and the spatiotemporal correlation learning of "self-vehicle-scene elements", and outputs a safe and accurate driving behavior action sequence based on the spatiotemporal graph neural network; 4. The causal knowledge-based explainable autonomous driving decision-making method described in the present invention, by defining the explainability standard, realizes the ex ante and ex post explanation of the autonomous driving decision-making model, as well as the objective evaluation of the autonomous driving decision-making system, determines the key features that affect the behavior decision of the autonomous driving vehicle in different scenarios, and performs statistical and visual analysis; 5. The explainable autonomous driving decision-making method based on causal knowledge described in the present invention analyzes the decision-making performance of the autonomous driving decision-making system through safety, comfort and compliance evaluation indicators through the testing methods of model-in-the-loop testing and real vehicle online testing. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0018] Figure 1 It is a schematic diagram of the process of the present invention; Figure 2 It is a schematic diagram of step 1 of the present invention; Figure 3 It is a schematic diagram of step 2 of the present invention; Figure 4 It is a schematic diagram of step three of the present invention; Figure 5 It is a schematic diagram of step 4 of the present invention; Figure 6 This is an example diagram of the causal discovery model algorithm based on decision observation data; Figure 7 This is an example picture of a driving scene; Figure 8 This is an example diagram of an explainable autonomous driving decision algorithm. DETAILED DESCRIPTION

[0019] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only parts related to the present invention, rather than all structures, are shown in the accompanying drawings.

[0020] See also Figure 1 The present invention provides an explainable autonomous driving decision-making method based on causal knowledge, comprising the following steps: See also Figure 2 and Figure 6 ,Step 1: Discover the causal relationship of automatic decision-making knowledge, as follows: S11. Collect multi-source heterogeneous driving data, as follows: S111, collecting virtual simulation data; By building a driving simulator and simulation platform, we can design various driving scenarios, including urban roads, highways, and rural roads, covering different weather conditions, traffic flows, and road conditions; add dynamic elements, including vehicles, pedestrians, and bicycles; simulate the real traffic environment, including roads, buildings, traffic signs, and signal lights; build a complete virtual driving environment, and the feature information of all scene elements constitutes a multi-source heterogeneous driving data set in the virtual simulation environment. v , as shown in formula (1): (1) In the formula, h e , h n , h b , h p , h l , h s are the feature sets of the scene elements of the vehicle, adjacent vehicles, adjacent bicycles, pedestrians, traffic lights and signboards; among them, h e ={ x e , y e , vx e, vy e, se , t e , b e}, h n ={ s n , x n , y n , vx n, vy n}, h b ={ x b , y b , vx b, vy b}, h p ={ x p , y p , vx p, vy p}, h l ={ x l , y l , s l}, h s ={ x s , y s , s s}; x e 、y e 、vx e、vy e、s e 、t e and b e They are the lateral position, longitudinal position, lateral speed, longitudinal speed, steering wheel angle, accelerator pedal opening and brake pedal opening of the vehicle; s n 、x n 、y n 、vx n、vy n They are the type, lateral position, longitudinal position, lateral speed, and longitudinal speed of the adjacent vehicles respectively; x b 、y b , vx b, vy b are theLateral position, longitudinal position, lateral speed, longitudinal speed; x p 、y p 、vx p、vy p They are the pedestrian’s lateral position, longitudinal position, lateral speed, and longitudinal speed respectively; x l 、y l 、s l They are the horizontal position, vertical position and status of the traffic light respectively; x s 、y s 、s s They are the horizontal position, vertical position and status of the traffic sign respectively; S112, collecting real vehicle data; The raw data obtained by the inertial navigation positioning device, laser radar, millimeter wave radar, and camera are processed to obtain target-level data. The data type is the same as step 111), and a multi-source heterogeneous driving data set Ω is obtained in a real vehicle environment. r ; Among them, the processing can be carried out through a sensor fusion algorithm based on deep learning or other algorithms.

[0021] S113, classifying the scenes; The simulation data collected by S111 and S112 are v and multi-source heterogeneous driving data set, namely real vehicle driving data Ω r Slice the scene according to the road topology structure and determine the starting and ending points of each scene slice. The road topology structure is divided into roads with different curvatures, intersections with or without traffic lights, T-intersections, and roundabouts. Ensure that the elements such as roads, traffic signs, and signal lights in the slices are complete and consistent. Further classify the scene segments according to the number of vehicles in the environment around the vehicle, and the same number of vehicles in the environment are classified into the same type of scene. S114, integrating multi-source heterogeneous driving data; The scene slices generated in S113 are stored as independent simulation scenes, and a slice index is established to save the type, location, and environment information of each slice for easy query and management; the slice library is expanded as needed, new slice types and scenes are added, the richness and coverage of the data are improved, and the classification set Ω of the surrounding environment vehicles of different road sections and different numbers is obtained. v,n and Ω r,n ; Among them, the extended slice library refers to slicing types and scenarios according to scenarios. For example, a library has three categories of books: literature, engineering, and linguistics. When a new history book is stored in the library, a new location (ie, ID) is allocated.

[0022] S12. Establish a model based on causal discovery of decision-making observation data, as follows: S121, preprocessing the data; The obtained Ω v,n and Ω r,n Data cleaning and data standardization are performed; first, missing values ​​are filled in the data using the Lagrange interpolation method, and data standardization is used to convert data of different scales into a unified scale. The calculation formula is (2): (2) In the formula, X max and X min are the maximum and minimum values ​​of the data respectively; S122, build a causal relationship model; Kernel regression is used as a nonlinear model, and environmental vehicles, traffic lights and traffic signs are regarded as variables. Based on the driving behavior of the ego vehicle in different road sections, with different numbers of environmental vehicles, and with or without traffic lights and traffic signs, the nonlinear causal relationship between the variables that affect the ego vehicle's driving behavior is analyzed. An exemplary embodiment of this step is as follows Figure 6 shown.

[0023] The Hilbert-Schmidt Independence Criterion (HSIC) is used to determine the input feature variables. X ={ h e , h n , h b , h p , h l , h s , h r}, and the target feature variable Y ={ s e , t e , b e}, h e, h n , h b , h p , h l , h s , h r , s e , t e and b e From the first step, step by step, step by step, definition. First, for each feature X p and Y q , define the kernel matrix K p and L q Formulas (3) and (4) are: (3) (4) In the formula, k p Features X p The kernel function of l q The target variable Y q The kernel function of n is the dimension of the input feature variable; The Laplace kernel function is used to perform nonlinear mapping on the input feature variables and the target feature variables. The calculation formulas (5) and (6) of the Laplace kernel function are as follows: (5) (6) Where ||.|| is the Euclidean distance between two input vectors; x and y They are nuclear center; and The bandwidth parameter is used to control the smoothness of the kernel function; In order to eliminate the bias, the kernel matrix is ​​centered and the calculation formulas are (7) and (8): (7) (8) In the formula, I is the identity matrix; Ais a vector of all 1s, .

[0024] For each feature and each target variable, the HSIC between the two is calculated using formula (9): (9) In the formula, the larger the HSIC value, the better the feature X p With the target variable Y q The stronger the dependence between them; tr(.) is the trace of the matrix, that is, the sum of the diagonal elements of the matrix.

[0025] Using the Hilbert-Schmidt independence criterion and the elastic network regression model, a nonlinear feature selection model is constructed. The optimization objective function formula (10) is: (10) In the formula, β q For the q The regression coefficient of the target variable; is the regularization parameter; is the trade-off parameter between L1 and L2 regularization.

[0026] Through this optimization objective function, select the target variable Y The most dependent nonlinear feature. The coordinate descent method is used to solve the objective function. First, the regression coefficients are initialized β is a zero matrix, For each target variable Y q , calculate the loss term and regularization term in the objective function, when the regression coefficient matrix β When the update amplitude is small enough, the iteration is stopped. The final regression coefficient matrix β Represents each feature X p With each target variable Y q The larger the dependency between β pq Representation characteristics X p right Y q Have a strong impact.

[0027] S123, fitting and evaluating the model; The Granger causality test is used to fit the model parameters, and the causal skeleton structure is determined through significance test; a causal graph is generated to represent the causal relationship between the variables that affect the driving behavior of the vehicle, and to quantify the degree of influence of different variables on the driving behavior of the vehicle, and finally a directed acyclic graph (DAG) representing the causal relationship is output. G c ; S13. Integrate autonomous driving decision-making knowledge, as follows: S131, defining road section driving knowledge; Integrate the knowledge that affects the behavior of autonomous vehicles on the road; consider the interaction between the self-driving vehicle and the surrounding vehicles and traffic signs, use causal diagrams to represent the causal relationship between the variables that affect the self-driving behavior, and classify the causal diagrams according to the presence or absence of traffic signs and the number of vehicles in the surrounding environment of the self-driving vehicle; S132, define intersection decision knowledge; Define the decision-making knowledge that affects the driving behavior of the vehicle in the intersection scenario; consider the impact of traffic lights, pedestrians and environmental vehicles on the vehicle; and classify and save the causal graph that can represent the causal relationship between the variables that affect the driving behavior of the vehicle according to the presence or absence of traffic lights, the number of pedestrians and environmental vehicles.

[0028] Step 2: See Figure 3 and Figure 7 , build an explainable autonomous driving decision model. The specific method is as follows: S21. Construct a driving scenario for an autonomous driving vehicle, as follows: S211, extracting scene elements; Extract scene elements from the driving scene of the autonomous driving vehicle. Dynamic scene elements include the vehicle, adjacent vehicles, pedestrians, bicycles, and traffic lights. Static scene elements include traffic signs. The features of the scene elements are the same as those defined in S111. S212, extracting causal relationships; The similarity between the autonomous driving decision causal graph and the spatiotemporal graph structure integrated by S13 is calculated, and the causal graph with high similarity (customizable, greater than 0.8 is considered high similarity) is selected. The autonomous driving decision causal knowledge is initialized to the spatiotemporal graph. The similarity calculation is shown in formula (11): (11) In the formula, G c and G d They are cause-effect diagram and driving scenario diagram respectively; mcs ( G c , G d) is the number of nodes of the largest common subgraph of the two graphs; max (| G c |,| G d |) is the ratio of the product of the number of nodes in the two graphs; S213, representing an autonomous driving driving scenario based on a spatiotemporal graph structure; The driving scene of the autonomous driving vehicle is represented by a spatiotemporal graph. For a certain moment, the scene elements are represented by the nodes of the graph, and the relationship between the scene elements is represented by the directed edges between the nodes. The dynamic scene modeling is as follows: on the basis of the graph structure, the time dimension is added to form a sequence graph model, such as Figure 7 As shown, the graph structure is expressed as shown in formula (12): (12) In the formula, V ={ v e , v n , v b , v p , v l , v s};node v e , v n , v b , v p , v l , v s They are the own car, neighboring car, bicycle, pedestrian, traffic light and sign; E ={ ee n , ee b , ee p , ee l , ee s}, ee n , ee b , ee p , ee l , ee s are directed edges between the vehicle and neighboring vehicles, bicycles, pedestrians, traffic lights, and traffic signs. The connection relationship of the directed edges is determined by the distance between the nodes. v e and v i Distance d ( v e ,v i ) is calculated by formulas (13) and (14): (13) (14) when d ( v e , v i )<100m, v e and v i There are directed edges between them; the weights of the directed edges are initialized by the extracted causal graph to ensure that the sum of the weights of the directed edges of all nodes is 1; S22. Build an autonomous driving decision model, as follows: S221. Build a dataset for autonomous driving decision-making; The collected driving data of autonomous vehicles are divided into data sets; a sliding window of fixed size is used to create training samples, where the window can slide at a frequency of 60 Hz, that is, every 60 consecutive discrete points; the historical 3s driving data of autonomous vehicles is used to predict the driving behavior action sequence of the next 2s; 70%, 20% and 10% of the sample quantity are divided into training set, test set and validation set respectively; the number of samples N tot The calculation formula is shown in (15): (15) In the formula, N is the total length of the time series; the window size W The number of time steps included for each sample; step size S is the number of time steps for each window movement; S222, embedding causal knowledge in autonomous driving decision making; S212 is used to calculate a causal graph with high similarity, and the causal relationship of the causal graph is extracted, and the causal relationship is embedded into the spatiotemporal graph model; including the relationship between nodes and edges that affect the driving behavior of the vehicle; S223, learning the spatiotemporal correlation of “ego vehicle-scene elements”; The spatiotemporal interaction relationship between the ego vehicle and scene elements is acquired through the spatiotemporal graph neural network. First, the feature representation of the ego vehicle node is calculated through the temporal attention mechanism. Then, a graph convolutional neural network is designed to mine the graph structure at each moment, obtain the feature representation of the ego vehicle node that aggregates the surrounding nodes, and learn the importance of the ego vehicle feature at each moment through a temporal attention mechanism. Finally, the spatiotemporal correlation between the ego vehicle and scene elements is fused by the channel attention mechanism. S224, mapping the driving behavior of the vehicle; The encoding vector learned in S223 is mapped into a three-dimensional vector through dimension transformation, deformation and linear transformation, which respectively represent the steering wheel angle, accelerator pedal opening and brake pedal opening of the vehicle; S23, evaluation model offline test results, as follows: S231. Define evaluation indicators; Define the model evaluation indicators, using mean absolute error (MAE), root mean square error (RMSE) and mean absolute percentage error (MAPE) as indicators to evaluate the difference between the model prediction value and the true value. The calculation formulas (16), (17) and (18) of MAE, RMSE and MAPE are as follows: (16) (17) (18) In the formula, and They are the actual driving action sequence and the predicted driving behavior action sequence of the autonomous driving vehicle respectively; N Indicates the number of samples; S232, performing offline testing on the model; By analyzing MAE, RMSE and MAPE, the deviation between the predicted value and the actual value of the steering wheel angle, accelerator pedal opening and brake pedal opening of the autonomous driving vehicle is measured; the data set of the autonomous driving decision is divided into several parts, and the model is trained and verified on different data subsets; the data set is evenly divided into k mutually exclusive subsets; k-1 subsets are selected for training each time, and the remaining subset is used for verification; the model is trained and verified k times, and the verification set is different each time; finally, the average value of k verification results is taken as the evaluation indicator of the model. The value of k is 5 or 10.

[0029] Exemplary embodiments of S22 and S23 are as follows Figure 8 shown.

[0030] S22 builds an autonomous driving decision model. According to the self-driving data of S221, after the data set is divided, the scene fragments at each moment are represented by a graph structure. The representation of the graph is represented by S213. The continuous scenes composed of each sample are represented in the form of a list of graphs. When constructing the graph structure and feature data, all tensor parameters are copied to the GPU device. When training the model, first load the data set constructed by the self-driving car, and use the causal relationship to guide the decision of the model according to the causal relationship graph extracted by S212. Then perform self-driving time correlation learning. The input is based on the driving data of the self-driving car. The dimension of the data is [ B × T × F ], B Indicates the size of Batchsize, T represents the length of the input sequence, F represents the dimension of the sequence, that is, the number of features of the ego vehicle. First, the matrix W is used to map the ego vehicle feature dimension to D dimension, D is a hyperparameter. Then, the temporal attention mechanism is used to learn the impact of the vehicle’s historical data on future data. The temporal attention weight calculation formula (19) is expressed as: (19) In the formula, W Q , W K are the weight matrices to be trained respectively; α t,i For at the moment t and i relationship; The feature representation of the ego-vehicle node is calculated through the temporal attention mechanism, and the calculation formula (20) is: (20) Then, the spatial correlation learning of “ego vehicle-scene element” is performed. First, all the feature dimensions of heterogeneous nodes are mapped to the same dimension. Then, the features are aggregated according to the different connection relationships between nodes. Assume that each type of edge For a certain connection relationship, define a message constructor MsgE to generate v i To the vehicle node v e The message constructor formula (21) can be expressed as: (twenty one) In the formula, E i,v The feature of the edge.

[0031] For self-driving nodes v e , all types are E i Aggregate the messages of the edges to get the aggregated messages. The aggregation formula (22) is: (twenty two) In the formula, Agg Ei is an aggregation function, such as mean aggregation or max pooling.

[0032] The node features are updated using the aggregated messages of all types of edges, and the calculation formula (23) is: (twenty three) In the formula, For the l Layer Node v e The hidden state of ; Update is an update function.

[0033] Finally, propagation is performed through the graph convolutional network, and the calculation formula (24) is: (twenty four) In the formula, and is a learnable weight matrix representing the activation function.

[0034] Finally, the spatiotemporal correlation of the vehicle and the scene elements is fused through the channel attention mechanism to obtain the fused representation result. and Perform global average pooling to obtain two vectors Z 1 and Z 2, the calculation formulas (25) and (26) are: (25) (26) In the formula, Z 1 and Z The shape of 2 is 1× f .

[0035] Then, a fully connected layer and a nonlinear activation function are used to learn the channel weights, and the calculation formulas (27) and (28) are: (27) (28) Formula (27) and Formula (28) use ReLU activation function and Sigmoid activation function respectively. W r and Ws is the weight matrix of the fully connected layer.

[0036] Use the learned channel weights S 1 and S 2 pairs of input tensors and Perform weighted fusion to obtain the final representation result. The calculation formula (29) is: (29) In the formula, is element-wise multiplication, .

[0037] Then, two LSTM networks are used to map the driving behavior features and output the driving behavior action information, including the steering wheel angle, accelerator pedal opening, and brake pedal opening. The calculation formulas (30) and (31) are: (30) (31) In the formula, For the model prediction results, , ; is the length of the output sequence.

[0038] Loss function during training L The calculation formula (32) is: (32) In the formula, and are the true value and predicted value of the model output respectively.

[0039] Use MAE, RMSE, and MAPE in S23 to evaluate the training effect of the model. If the training loss of the model changes less than 0.0001 within 5 consecutive epochs, or if the set maximum number of epochs is reached, the training is stopped. Usually after each epoch, check the performance of the model on the validation set. If the performance is better than the previous best performance, save the current model weights. Determine a set of hyperparameters for the model, train the model with the training set, find the optimal function that minimizes the loss function, measure the performance of the optimal function on the validation set, and determine the optimal specified hyperparameter combination. Save the model parameters, load them, and verify them on the test set and validation. Select the model with the smallest error on the validation set, and merge the training set and validation set as the overall training model to find the optimal function; measure the generalization performance of the optimal function on the test set.

[0040] Step 3: Refer to Figure 4, evaluate the interpretability of the autonomous driving decision system, the specific method is as follows: S31. Define the interpretability criteria as follows: S311, analytical model transparency; Analyze the decision-making mechanism of the model itself, and obtain explanations by studying the internal structure and parameters of the model; use the prior explanation results of the model as the model transparency analysis results; evaluate and score the model's input layer, graph attention layer, temporal convolution layer, spatiotemporal fusion layer, fully connected layer, normalization layer, activation layer, residual connection layer, and output layer. The specific formula and operation of graph convolution explain how node features are aggregated through neighboring nodes; similar input data has stable output results at each layer; output model transparency analysis results U ; S312, analyzing the sensitivity of the model; By introducing the importance of perturbation analysis features, it is used as an indicator to evaluate the ex post interpretability of the model; using the counterfactual interpretation method, the input features of the model are changed before prediction to analyze the changes in model prediction; the counterfactual interpretation results are used as the model sensitivity analysis results, and the model sensitivity analysis results are output V ; S313, conduct objective evaluation; Objectively evaluate the autonomous driving system; score the scene elements that affect the self-driving behavior according to the preset situation. Each scene element has a unique corresponding score. According to the score, the degree of influence on the self-driving behavior is classified into strong, general and weak. Finally, the objective evaluation results of the scene elements that affect the self-driving behavior in different scenarios are obtained. W ; S314. Define the interpretability comprehensive evaluation model; Define an interpretability comprehensive evaluation model, and output the comprehensive evaluation results of transparency, sensitivity and objective evaluation; analyze the key features obtained U , V and W The importance is sorted, and the sorting rules and calculation formula (33) are as follows: (33) In the formula, S The intersection set of key features obtained from S311, S312 and S313, the remaining features are based on the principle of two-by-two intersection, and the priority is defined as > > > > U > W > V , get the feature importance ranking result; S32. Provide a prior explanation of the autonomous driving decision model, as follows: S321, analyzing the model structure; The graph structure defined in S211 is analyzed. The edge of the graph structure is the key factor for message transmission. The connection edge between nodes is determined by the interaction relationship between scene elements with different semantics and features. In order to distinguish the importance of different edges in message transmission, the fixed weights of different edges are first initialized according to the causal relationship model constructed in step S122. During the model training process, the weights are parameterized, the edge weights are updated through back propagation and gradient descent algorithms, and the learned edge weights are visualized. S322, analyzing and visualizing the importance of nodes; Select and extract features that have a significant impact on model predictions, and visualize the results; Based on the adjacency matrix between the ego vehicle node and other scene element nodes, take the ego vehicle node as the central node, aggregate the features of the surrounding nodes, and visualize the importance of the nodes through a heat map; S323, obtaining prior explanation results; The nodes with the greatest impact on the driving behavior of the vehicle in the same scenario are counted and sorted according to the calculation results of the node importance. The frequency distribution of the characteristic variables with different degrees of influence on the vehicle behavior decision is displayed through histograms and box plots; the ranking results of the influence of different node characteristics on the model prediction results are determined U , generate a model explanation report, describing in detail the distribution of each feature to the prediction results; S33. Perform a post-facto explanation of the autonomous driving decision model, as follows: S331, input key variables; The following steps use the autonomous driving decision model trained by S22 f , key variables defined in S111 X i ; S332, construct counterfactual scenarios; By constructing counterfactual scenarios, using the do operator to generate counterfactuals, intervening in multiple features X 1, X 2, …, X k , generate new input features, and the other features remain unchanged. The calculation formula (34) is: (34) In the formula, v i is the value after intervention, and the generated counterfactual state is a Similar inputs but with some features interfered with; then use the intervened input state to change the vehicle’s driving environment and calculate the counterfactual decision result , the calculation formula (35) is: (35) In order to find the best counterfactual state, an optimization algorithm is used to minimize the comprehensive loss function. The loss function calculation formula (36) is: (36) Where the first term is the decision difference and the second term is the similarity of the input. λ is a hyperparameter that balances these two terms; use the gradient descent algorithm to solve and find ; is the input feature; is the state after intervention; Using weighted Euclidean distance evaluation X and The similarity of is calculated by formula (37): (37) S333, Evaluation of key factors; Evaluate the key variables of ego-vehicle driving behavior; first, evaluate counterfactual decision outcomes The impact of the system, including safety, efficiency and compliance, defines an impact function The difference value is quantified, and the difference value quantification calculation formula (38) is: (38) In the formula, s , e and l They are the safety, efficiency and compliance evaluation score functions respectively; w s , w e and w l are the weight coefficients of safety, efficiency and compliance respectively; find the counterfactual decision according to the difference value safety, efficiency and compliance reduction key variables; S334, obtaining post-explanation results; Analyze the key variables, count the key variables that affect safety, efficiency and compliance through histograms and box plots, and record the counterfactual instance scenarios and key variables; determine the ranking results of the key variables that affect the safety, efficiency and compliance of the vehicle under the same scenario V .

[0041] S34. Evaluate the autonomous driving decision system as follows: S341, define questionnaire; The evaluation score results are obtained based on the questionnaire for objective evaluation of the autonomous driving decision-making system; the questionnaire is designed as follows: the lane changing scenario determines the scene elements and characteristics that affect the lane changing of the ego vehicle; the emergency braking scenario determines the scene elements and characteristics that affect the braking of the ego vehicle; the intersection scenario with traffic lights determines the scene elements and characteristics that affect the driving behavior of the ego vehicle; S342. Statistically analyze the evaluation results of the questionnaire in S41, obtain the ranking results W of the scene elements and key feature variables according to the evaluation scores, and intuitively display the influence of different scene elements on the vehicle driving behavior through a graphical visualization method.

[0042] Step 4: Read Figure 5 , evaluate the performance of the autonomous driving decision system. The specific method is as follows: S41. The autonomous driving decision system is tested as follows: S411, perform model-in-the-loop testing; By using the interface in the simulation software to input the self-driving car, environmental vehicles, pedestrians, and traffic light signals into the trained model, the model calculates and outputs the steering wheel angle, accelerator pedal opening, and brake pedal opening signal control instructions, and sends the control signal to the autonomous driving vehicle through the virtual interface to achieve the state update of the autonomous driving vehicle. Set the starting point and end point in the simulation software, randomly generate traffic participants with different densities and different flow rates, and pass through sections of different curvatures, intersections with traffic lights, intersections without traffic lights, roundabouts, and other sections. The number of experimental tests is set to 20,000. The autonomous driving vehicle drives from the starting point to the end point, and the simulation ends when a collision occurs. If there is no collision and the vehicle drives in accordance with traffic rules, it is recorded as a successful test, and the number of collisions is recorded; S412, conduct real vehicle online testing; This step conducts real-vehicle online testing of the algorithm on the road section where S11 collects data, and tests the functions of the autonomous driving decision system in long-distance driving, lane keeping, lane changing, overtaking, etc. in a highway environment, as well as the traffic capacity of congested sections, multi-lane intersections, and intersections with or without traffic lights in an urban road environment. Real-vehicle tests are conducted under different traffic densities and flow rates, with a starting point and an end point set, and the number of tests is also set to 20,000 times. In order to ensure driving safety, when the autonomous driving decision system cannot cope with the current driving scenario, the driver can take over control of the vehicle at any time; when the driver takes over, it means that the autonomous driving decision system has failed to make a decision in the current scenario, and the number of times the driver takes over is recorded; S42. Set the evaluation indicators of the autonomous driving decision system, as follows: S421, conduct driving safety evaluation; In the model-in-the-loop test, the collision rate is used as an indicator to evaluate the driving safety of the autonomous driving vehicle. In the real vehicle online test, the takeover rate is used as an indicator to evaluate the driving safety. The calculation formulas (39) and (40) of the collision rate and takeover rate are as follows: (39) (40) In the formula, T tot is the total number of trials; T c is the number of collisions that occurred in the virtual simulation test; T t The number of times the driver took over during the actual vehicle test; S422, evaluating driving efficiency; The performance of the decision system is evaluated by the driving efficiency of the autonomous driving vehicle. The driving efficiency calculation formula (41) is as follows: (41) In the formula, T e The number of tests required to meet the minimum travel time; T suc is the number of successes; S423, conduct driving compliance evaluation; The performance of the decision system is evaluated through the driving compliance of the autonomous driving vehicle. The driving compliance calculation formula (42) is as follows: (42) In the formula, T l The number of tests to meet traffic regulations; S43. Analyze the performance of the autonomous driving decision system, as follows: S431, obtaining model-in-the-loop test results; Evaluate the decision-making performance of the autonomous driving vehicle by analyzing the evaluation results of driving safety, driving efficiency and driving compliance, calculate the overall collision rate, evaluate the overall collision situation of the system in all test scenarios, draw a histogram or pie chart to show the distribution of the overall collision rate; calculate the collision rate in each specific scenario separately, evaluate the collision situation of the system in different scenarios, draw a bar chart or line chart to show the difference in collision rate in each scenario, and record environmental information such as the vehicle, adjacent vehicles, pedestrians and traffic lights where the collision accident occurred; S432, obtaining the actual vehicle online test result; By drawing histograms or pie charts, the driving safety, driving efficiency and driving compliance evaluation results of autonomous vehicles in various scenarios are statistically analyzed; during the driving process of the autonomous vehicle, environmental information such as the vehicle itself, adjacent vehicles, pedestrians and traffic lights is recorded when the driver takes over; S433, optimizing result feedback; This phase analyzes and records the model's in-loop test results and real-vehicle online test results. Further study the model's decision-making mechanism in failed cases, locate the cause of decision failure, and continuously improve the algorithm's performance. Analyze the cause of each collision, such as decision error, sensor failure, environmental interference, etc., draw a grouped bar chart, and show the number of collisions caused by different reasons.

[0043] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. An explainable autonomous driving decision-making method based on causal knowledge, characterized in that: The following steps are involved: Step 1: Discover the causal relationship for automatic decision-making knowledge; Step 2: Build an explainable autonomous driving decision model; Step 3: Evaluate the interpretability of the autonomous driving decision system; Step 4: Evaluate the performance of the autonomous driving decision system.

2. The explainable autonomous driving decision-making method based on causal knowledge according to claim 1, characterized in that: The specific method of step one is as follows: S11, collect multi-source heterogeneous driving data; S12. Establish a model based on causal discovery of decision-making observation data; S13. Integrate autonomous driving decision-making knowledge.

3. The explainable autonomous driving decision-making method based on causal knowledge according to claim 2, characterized in that: The specific method of S11 is as follows: S111, collecting virtual simulation data; By building a driving simulator and simulation platform, we can design various driving scenarios, including urban roads, highways, and rural roads, covering different weather conditions, traffic flows, and road conditions; add dynamic elements, including vehicles, pedestrians, and bicycles; simulate the real traffic environment, including roads, buildings, traffic signs, and signal lights; build a complete virtual driving environment, and the feature information of all scene elements constitutes a multi-source heterogeneous driving data set in the virtual simulation environment. v , as shown in formula (1): （1） In the formula, h e , h n , h b , h p , h l , h s are the feature sets of the scene elements of the vehicle, adjacent vehicles, adjacent bicycles, pedestrians, traffic lights and signboards; among them, h e ={ x e , y e , vx e, vy e, s e , t e , b e }, h n ={ s n , x n , y n , vx n, vy n }, h b ={ x b , y b , vx b, vy b }, h p ={ x p , y p , vx p, vy p }, h l ={ x l , y l , s l }, h s ={ x s , y s , s s }; x e 、y e 、vx e、vy e、s e 、t e and b e They are the lateral position, longitudinal position, lateral speed, longitudinal speed, steering wheel angle, accelerator pedal opening and brake pedal opening of the vehicle; s n 、x n 、y n 、vx n、vy n They are the type, lateral position, longitudinal position, lateral speed, and longitudinal speed of the adjacent vehicles respectively; x b 、y b , vx b, vy b are the Lateral position, longitudinal position, lateral speed, longitudinal speed; x p 、y p 、vx p、vy p They are the pedestrian’s lateral position, longitudinal position, lateral speed, and longitudinal speed respectively; x l 、y l 、s l They are the horizontal position, vertical position and status of the traffic light respectively; x s 、y s 、s s They are the horizontal position, vertical position and status of the traffic sign respectively; S112, collecting real vehicle data; The raw data obtained by the inertial navigation positioning device, laser radar, millimeter wave radar, and camera are processed to obtain target-level data, and a multi-source heterogeneous driving data set Ω is obtained in the real vehicle environment. r ; S113, classifying the scenes; The simulation data Ω v and multi-source heterogeneous driving data set, namely real vehicle driving data Ω r Slice the scene according to the road topology structure and determine the starting and ending points of each scene slice. The road topology structure is divided into roads with different curvatures, intersections with or without traffic lights, T-intersections, and roundabouts. Ensure that the elements such as roads, traffic signs, and signal lights in the slices are complete and consistent. Further classify the scene segments according to the number of vehicles in the environment around the vehicle, and the same number of vehicles in the environment are classified into the same type of scene. S114, integrating multi-source heterogeneous driving data; The generated scene slices are stored as independent simulation scenes, and a slice index is established to save the type, location, and environment information of each slice. The slice library is expanded as needed, and new slice types and scenes are added to obtain a classification set Ω of surrounding vehicles of different road sections and different numbers. v,n and Ω r,n ; The specific method of S12 is as follows: S121, preprocessing the data; The obtained Ω v,n and Ω r,n Data cleaning and data standardization are performed; first, missing values ​​are filled in the data using the Lagrange interpolation method, and data standardization is used to convert data of different scales into a unified scale. The calculation formula is (2): （2） In the formula, X max and X min are the maximum and minimum values ​​of the data respectively; S122, build a causal relationship model; Kernel regression is used as a nonlinear model, and environmental vehicles, traffic lights and traffic signs are regarded as variables. Based on the driving behavior of the ego vehicle in different road sections, with different numbers of environmental vehicles, and with or without traffic lights and traffic signs, the nonlinear causal relationship between the variables that affect the ego vehicle's driving behavior is analyzed. S123, fitting and evaluating the model; The Granger causality test is used to fit the model parameters, and the causal skeleton structure is determined through significance test; a causal graph is generated to represent the causal relationship between the variables that affect the driving behavior of the self-driving car, and to quantify the degree of influence of different variables on the driving behavior of the self-driving car, and finally a directed acyclic graph representing the causal relationship is output; The specific method of S13 is as follows: S131, defining road section driving knowledge; Integrate the knowledge that affects the behavior of autonomous vehicles on the road; consider the interaction between the self-driving vehicle and the surrounding vehicles and traffic signs, use causal diagrams to represent the causal relationship between the variables that affect the self-driving behavior, and classify the causal diagrams according to the presence or absence of traffic signs and the number of vehicles in the surrounding environment of the self-driving vehicle; S132, define intersection decision knowledge; Define the decision-making knowledge that affects the driving behavior of the vehicle in the intersection scenario; consider the impact of traffic lights, pedestrians and environmental vehicles on the vehicle; and classify and save the causal graph that can represent the causal relationship between the variables that affect the driving behavior of the vehicle according to the presence or absence of traffic lights, the number of pedestrians and environmental vehicles.

4. The explainable autonomous driving decision-making method based on causal knowledge according to claim 3, characterized in that: The specific method of step 2 is as follows: S21. Constructing driving scenarios for autonomous vehicles; S22. Build an autonomous driving decision model; S23. Evaluate the model offline test results.

5. The explainable autonomous driving decision-making method based on causal knowledge according to claim 4, characterized in that: The specific method of S21 is as follows: S211, extracting scene elements; Extract scene elements in the driving scene of the autonomous driving vehicle. Dynamic scene elements include the vehicle, adjacent vehicles, pedestrians, bicycles and traffic lights, and static scene elements include traffic signs. S212, extracting causal relationships; The similarity of the integrated autonomous driving decision causal graph and the spatiotemporal graph structure is calculated, and the causal graph with high similarity is selected. The autonomous driving decision causal knowledge is initialized to the spatiotemporal graph. The similarity calculation is shown in formula (11): （11） In the formula, G c and G d They are cause-effect diagram and driving scenario diagram respectively; mcs ( G c , G d ) is the number of nodes of the largest common subgraph of the two graphs; max (| G c |,| G d |) is the ratio of the product of the number of nodes in the two graphs; S213, representing an autonomous driving driving scenario based on a spatiotemporal graph structure; The driving scene of the autonomous driving vehicle is represented by a spatiotemporal graph. For a certain moment, the scene elements are represented by the nodes of the graph, and the relationship between the scene elements is represented by the directed edges between the nodes. The dynamic scene modeling is as follows: on the basis of the graph structure, the time dimension is added to form a graph model in the form of a sequence. The graph structure is represented as shown in formula (12): （12） In the formula, V ={ v e , v n , v b , v p , v l , v s };node v e , v n , v b , v p , v l , v s They are the own car, neighboring car, bicycle, pedestrian, traffic light and sign; E ={ ee n , ee b , ee p , ee l , ee s }, ee n , ee b , ee p , ee l , ee s are directed edges between the vehicle and neighboring vehicles, bicycles, pedestrians, traffic lights, and traffic signs. The connection relationship of the directed edges is determined by the distance between the nodes. v e and v i Distance d ( v e , v i ) is calculated by formulas (13) and (14): （13） （14） when d ( v e , v i )<100m, v e and v i There are directed edges between them; the weights of the directed edges are initialized by the extracted causal graph to ensure that the sum of the weights of the directed edges of all nodes is 1; The specific method of S22 is as follows: S221. Build a dataset for autonomous driving decision-making; The collected autonomous driving vehicle driving data is divided into data sets; a fixed-size sliding window is used to create training samples; the historical 3s autonomous driving vehicle driving data is used to predict the driving behavior action sequence of the next 2s; 70%, 20% and 10% of the sample quantity are divided into training set, test set and validation set respectively; the number of samples N tot The calculation formula is shown in (15): （15） In the formula, N is the total length of the time series; the window size W The number of time steps included for each sample; step size S is the number of time steps for each window movement; S222, embedding causal knowledge in autonomous driving decision making; The causal graph with high similarity is calculated, the causal relationship of the causal graph is extracted, and the causal relationship is embedded into the spatiotemporal graph model; Including the relationships between nodes and edges that affect the driving behavior of the vehicle; S223, learning the spatiotemporal correlation of "ego vehicle-scene elements"; The spatiotemporal interaction relationship between the ego vehicle and scene elements is acquired through learning the spatiotemporal graph neural network. First, the feature representation of the ego vehicle node is calculated through the temporal attention mechanism. Then, a graph convolutional neural network is designed to mine the graph structure at each moment, obtain the feature representation of the ego vehicle node that aggregates the surrounding nodes, and learn the importance of the ego vehicle features at each moment through a temporal attention mechanism; finally, a channel attention mechanism is used to fuse the spatiotemporal correlation between the ego vehicle and scene elements; S224, mapping the driving behavior of the vehicle; The learned encoding vector is mapped into a three-dimensional vector through dimension transformation, deformation and linear transformation, which respectively represent the steering wheel angle, accelerator pedal opening and brake pedal opening of the vehicle; The specific method of S23 is as follows: S231 defines evaluation indicators; The mean absolute error (MAE), root mean square error (RMSE), and mean absolute percentage error (MAPE) are used as indicators to evaluate the difference between the model prediction value and the true value. The calculation formulas (16), (17), and (18) of MAE, RMSE, and MAPE are as follows: （16） （17） （18） In the formula, and They are the actual driving action sequence and the predicted driving behavior action sequence of the autonomous driving vehicle respectively; N Indicates the number of samples; S232, performing offline testing on the model; By analyzing MAE, RMSE and MAPE, the degree of deviation between the predicted values ​​and the actual values ​​of the steering wheel angle, accelerator pedal opening and brake pedal opening of the autonomous driving vehicle is measured; the data set of autonomous driving decision-making is divided into several parts, and the model is trained and verified on different data subsets; the data set is evenly divided into k mutually exclusive subsets; k-1 subsets are selected each time for training, and the remaining subset is used for verification; the model is trained and verified k times, and the verification set is different each time; finally, the average of the k verification results is taken as the evaluation indicator of the model.

6. The explainable autonomous driving decision-making method based on causal knowledge according to claim 1, characterized in that: The specific method of step three is as follows: S31. Define interpretability criteria; S32, provide prior explanation of the autonomous driving decision model; S33, conduct post hoc explanation of the autonomous driving decision model; S34. Evaluate the autonomous driving decision system.

7. The explainable autonomous driving decision-making method based on causal knowledge according to claim 6, characterized in that: The specific method of S31 is as follows: S311, analytical model transparency; Analyze the decision-making mechanism of the model itself, and obtain explanations by studying the internal structure and parameters of the model; use the prior explanation results of the model as the model transparency analysis results; evaluate and score the model's input layer, graph attention layer, temporal convolution layer, spatiotemporal fusion layer, fully connected layer, normalization layer, activation layer, residual connection layer, and output layer. The specific formula and operation of graph convolution explain how node features are aggregated through neighboring nodes; similar input data has stable output results at each layer; output model transparency analysis results U ; S312, analyzing the sensitivity of the model; By introducing perturbations to analyze feature importance, it is used as a metric to evaluate the model’s ex post interpretability; Use the counterfactual explanation method to change the input features of the model before prediction and analyze the changes in model prediction; use the counterfactual explanation results as the model sensitivity analysis results and output the model sensitivity analysis results V ; S313, conduct objective evaluation; Objectively evaluate the autonomous driving system; score the scene elements that affect the self-driving behavior according to the preset situation. Each scene element has a unique corresponding score. According to the score, the degree of influence on the self-driving behavior is classified into strong, general and weak. Finally, the objective evaluation results of the scene elements that affect the self-driving behavior in different scenarios are obtained. W ; S314. Define the interpretability comprehensive evaluation model; Define an interpretability comprehensive evaluation model, and output the comprehensive evaluation results of transparency, sensitivity and objective evaluation; analyze the key features obtained U , V and W The importance is sorted, and the sorting rules and calculation formula (33) are as follows: （33） In the formula, S The intersection set of key features obtained from S311, S312 and S313, the remaining features are based on the principle of two-by-two intersection, and the priority is defined as > > > > U > W > V , get the feature importance ranking result; The specific method of S32 is as follows: S321, analyzing the model structure; The defined graph structure is analyzed. The edges of the graph structure are the key factors for message transmission. The connection edges between nodes are determined by the interactive relationship between scene elements with different semantics and features. In order to distinguish the importance of different edges in message transmission, the fixed weights of different edges are first initialized according to the causal relationship model. During the model training process, the weights are parameterized, and the edge weights are updated through backpropagation and gradient descent algorithms, and the learned edge weights are visualized. S322, analyzing and visualizing the importance of nodes; Select and extract features that have a significant impact on model predictions, and visualize the results; Based on the adjacency matrix between the ego vehicle node and other scene element nodes, take the ego vehicle node as the central node, aggregate the features of the surrounding nodes, and visualize the importance of the nodes through a heat map; S323, obtaining prior explanation results; The nodes with the greatest impact on the driving behavior of the vehicle in the same scenario are counted and sorted according to the calculation results of the node importance. The frequency distribution of the characteristic variables with different degrees of influence on the vehicle behavior decision is displayed through histograms and box plots; the ranking results of the influence of different node characteristics on the model prediction results are determined U , generate a model explanation report, describing in detail the distribution of each feature to the prediction results; The specific method of S33 is as follows: S331, input key variables; Input key variables; specifically: trained autonomous driving decision model f , key variables defined X i ; S332, construct counterfactual scenarios; By constructing counterfactual scenarios, using the do operator to generate counterfactuals, intervening in multiple features X 1, X 2, …, X k , generate new input features, and the other features remain unchanged. The calculation formula (34) is: （34） In the formula, v i is the value after intervention, and the generated counterfactual state is a Similar inputs but with some features interfered with; then use the intervened input state to change the vehicle’s driving environment and calculate the counterfactual decision result , the calculation formula (35) is: （35） In order to find the best counterfactual state, an optimization algorithm is used to minimize the comprehensive loss function. The loss function calculation formula (36) is: （36） Where the first term is the decision difference and the second term is the similarity of the input. λ is a hyperparameter that balances these two terms; use the gradient descent algorithm to solve and find ; is the input feature; is the state after intervention; Using weighted Euclidean distance evaluation X and The similarity of is calculated by formula (37): （37） S333, Evaluation of key factors; Evaluate the key variables of ego-vehicle driving behavior; first, evaluate counterfactual decision outcomes The impact of the system, including safety, efficiency and compliance, defines an impact function The difference value is quantified, and the difference value quantification calculation formula (38) is: （38） In the formula, s , e and l They are the safety, efficiency and compliance evaluation score functions respectively; w s , w e and w l are the weight coefficients of safety, efficiency and compliance respectively; Find the counterfactual decision based on the difference value safety, efficiency and compliance reduction key variables; S334, obtaining post-explanation results; Analyze key variables, use histograms and box plots to count key variables that affect safety, efficiency, and compliance, and record counterfactual instance scenarios and key variables; Determine the ranking results of key variables that affect the safety, efficiency and compliance of the self-driving car in the same scenario V ; S341, define questionnaire; The evaluation score results are obtained based on the questionnaire for objective evaluation of the autonomous driving decision-making system; the questionnaire is designed as follows: the lane changing scenario determines the scene elements and characteristics that affect the lane changing of the ego vehicle; the emergency braking scenario determines the scene elements and characteristics that affect the braking of the ego vehicle; the intersection scenario with traffic lights determines the scene elements and characteristics that affect the driving behavior of the ego vehicle; S342. Statistically analyze the evaluation results of the questionnaire in S41, obtain the ranking results W of the scene elements and key feature variables according to the evaluation scores, and intuitively display the influence of different scene elements on the vehicle driving behavior through a graphical visualization method.

8. The explainable autonomous driving decision-making method based on causal knowledge according to claim 1, characterized in that: The specific method of step 4 is as follows: S41, automatic driving decision system is tested; S42. Setting evaluation indicators for the autonomous driving decision system; S43. Analyze the performance of the autonomous driving decision system.

9. The explainable autonomous driving decision-making method based on causal knowledge according to claim 8, characterized in that: The specific method of S41 is as follows: S411, perform model-in-the-loop testing; By inputting the self-vehicle, surrounding vehicles, pedestrians and traffic light signals into the trained model through the interface in the simulation software, the model calculates and outputs the steering wheel angle, accelerator pedal opening and brake pedal opening signal control instructions, and sends the control signal to the autonomous driving vehicle through the virtual interface to achieve the state update of the autonomous driving vehicle; set the starting point and end point in the simulation software, randomly generate traffic participants with different densities and different flow rates, and pass through sections of different curvatures, intersections with traffic lights, intersections without traffic lights, and roundabout sections; The autonomous driving vehicle drives from the starting point to the end point, and the collision simulation ends. If there is no collision and the vehicle drives according to traffic rules, it is recorded as a successful test, and the number of collisions is recorded; S412, conduct real vehicle online testing; Carry out real-vehicle online testing of the algorithm on the road sections where data is collected, test the autonomous driving decision system's long-distance driving, lane keeping, lane changing, and overtaking functions in highway environments, as well as the traffic capacity of congested sections, multi-lane intersections, and intersections with or without traffic lights in urban road environments; conduct real-vehicle tests under different traffic densities and flows, and set starting and end points; when the autonomous driving decision system cannot cope with the current driving scenario, the driver can take over control of the vehicle at any time; when the driver takes over, it means that the autonomous driving decision system has failed to make a decision in the current scenario, and the number of times the driver takes over is recorded; The specific method of S42 is as follows: S421, conduct driving safety evaluation; In the model-in-the-loop test, the collision rate is used as an indicator to evaluate the driving safety of the autonomous driving vehicle. In the real vehicle online test, the takeover rate is used as an indicator to evaluate the driving safety. The calculation formulas (39) and (40) of the collision rate and takeover rate are as follows: （39） （40） In the formula, T tot is the total number of trials; T c is the number of collisions that occurred in the virtual simulation test; T t The number of times the driver took over during the actual vehicle test; S422, evaluating driving efficiency; The performance of the decision system is evaluated by the driving efficiency of the autonomous driving vehicle. The driving efficiency calculation formula (41) is as follows: （41） In the formula, T e The number of tests required to meet the minimum travel time; T suc is the number of successes; S423, conduct driving compliance evaluation; The performance of the decision system is evaluated through the driving compliance of the autonomous driving vehicle. The driving compliance calculation formula (42) is as follows: （42） In the formula, T l The number of tests to meet traffic regulations; The specific method of S43 is as follows: S431, obtaining model-in-the-loop test results; Evaluate the decision-making performance of the autonomous driving vehicle by analyzing the evaluation results of driving safety, driving efficiency and driving compliance, calculate the overall collision rate, evaluate the overall collision situation of the system in all test scenarios, draw a histogram or pie chart to show the distribution of the overall collision rate; calculate the collision rate in each specific scenario, evaluate the collision situation of the system in different scenarios, draw a bar chart or line chart to show the difference in collision rate in each scenario, and record the information of the self-vehicle, adjacent vehicles, pedestrians and environment involved in the collision accident; S432, obtaining the actual vehicle online test result; By drawing histograms or pie charts, the driving safety, driving efficiency and driving compliance evaluation results of autonomous vehicles in various scenarios are statistically analyzed; during the driving process of the autonomous vehicle, the information of the vehicle, adjacent vehicles, pedestrians and the environment is recorded when the driver takes over; S433 optimizes the result feedback; Analyze and record the model-in-the-loop test results and the real vehicle online test results; study the decision-making mechanism of the model in failed use cases, locate the cause of decision failure, and improve the performance of the algorithm; analyze the cause of each collision, draw a grouped bar chart, and show the number of collisions caused by different reasons.

Images