An interpretable autonomous driving decision system and method thereof
By combining a deep Q-network model and the DeepSHAP module, the Shapley value is calculated to explain autonomous driving decisions, solving the problem of users' difficulty in understanding the black box of deep reinforcement learning and improving user trust and transparency.
Patent Information
- Application Number
- CN202211445291.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-11-18
- Publication Date
- 2025-12-23
- Estimated Expiration
- 2042-11-18
AI Technical Summary
Existing autonomous driving decision-making systems, based on deep reinforcement learning neural networks, are black-box in nature, making it difficult for users to understand and trust autonomous driving decisions, thus affecting user acceptance.
Design an interpretable autonomous driving decision-making system that combines a deep Q-network model and a DeepSHAP module. The system calculates Shapley value to explain the input-output mapping relationship of autonomous driving decisions. It uses traffic environment feature vectors collected by the onboard perception module to output decision commands and display them to the user.
It increases user transparency and trust in autonomous driving decisions, helps users understand the decision-making process, and enhances their trust in autonomous driving systems.
Smart Images
Figure CN115719477B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of automatic driving, in particular to an interpretable automatic driving decision system and method thereof. BACKGROUND
[0002] Highly automated driving is considered to be the trend of future intelligent travel, because it can reduce manpower and cost, improve safety and reliability, reduce emissions and energy consumption. At present, most of the decision systems in autonomous vehicles use pre-designed, artificially intensive, highly modular rule / model-based algorithms.
[0003] However, in real-world environments, it is difficult for rule / model-based frameworks to generalize human heuristic strategies to an endless number of new scenarios and tasks, resulting in a great limitation of this decision-making approach; on the other hand, deep reinforcement learning, which combines reinforcement learning and deep learning, has become a hot topic in the field of artificial intelligence in recent years and has shown amazing potential in autonomous vehicles, such as deep reinforcement learning has successfully trained intelligent race cars to drive better than human top race car drivers, or trained intelligent environments to speed up the testing and evaluation process. Although deep reinforcement learning technology has made breakthrough progress in the application of autonomous driving, whether deep reinforcement learning-based autonomous driving decision technology can be successfully deployed in the real world largely depends on user acceptance and trust. Because the neural networks used in deep reinforcement learning have black box properties, the public still have doubts about them. Therefore, it is necessary to design an interpretable framework that can explain deep reinforcement learning-based autonomous driving decisions to users, thereby enhancing the trust of passengers in the autonomous driving system. SUMMARY
[0004] The purpose of the present application is to overcome the defects of the prior art and provide an interpretable automatic driving decision system and method thereof, which can intuitively explain the input and output mapping relationship of automatic driving decisions to users, and improve the understanding and trust of users in automatic driving decisions.
[0005] The purpose of the present application can be achieved by the following technical solution: an interpretable automatic driving decision system, comprising an automatic driving decision module and a DeepSHAP module, the automatic driving decision module is connected with a vehicle control module, a vehicle-mounted perception module and the DeepSHAP module, the DeepSHAP module is connected with a vehicle-mounted human-computer interaction unit, the automatic driving decision module is based on a deep Q network model, combines the traffic environment feature vector collected by the vehicle-mounted perception module, outputs the corresponding decision instruction, and transmits it to the vehicle control module for execution.
[0006] The DeepSHAP module is used to calculate the Shapley value corresponding to the data generated by the interaction between the deep Q-network model and the traffic environment, and transmit it to the in-vehicle human-machine interaction unit for display.
[0007] Furthermore, the vehicle control module includes a lateral control module and a longitudinal control module.
[0008] An interpretable autonomous driving decision-making method includes the following steps:
[0009] S1. Based on the driving scenario, determine the decision vector and state vector of the autonomous driving decision module, and use the deep Q network algorithm to train the neural network and build a deep Q network model.
[0010] S2. Extract key feature vectors from the traffic environment, input them into the deep Q-network model, output the corresponding decision instructions, and transmit them to the vehicle control module for execution.
[0011] S3. While the vehicle control module executes the decision command, the on-board perception module outputs the corresponding background dataset.
[0012] S4. The background dataset and the deep Q-network model are used as inputs to the DeepSHAP module. The DeepSHAP module calculates the Shapley value of all features in the background dataset and transmits it to the in-vehicle human-machine interaction unit for display.
[0013] Furthermore, in step S1, the driving scenario is a typical two-lane driving scenario, and the decision set of the autonomous driving decision module is determined as follows:
[0014] A={a left ,a idle ,a right}
[0015] Where A is the decision set, a left Indicates a left lane change, a idle To indicate maintaining the current action, a right Indicates a right lane change;
[0016] The state vector of the autonomous driving decision module is as follows:
[0017] S = [n ego ,n lat ,d]
[0018]
[0019]
[0020] Where S is the state vector, d represents the relative distance between the vehicle and the surrounding vehicles, and nego represents the lane where the ego vehicle is located, n lat represents the lane where the surrounding vehicle is located.
[0021] Further, the specific process of constructing the deep Q network model in step S1 is as follows:
[0022] The neural network is trained by using the deep Q network algorithm to approximate the action value function Q(s t , a t ), wherein the neural network is a 4-layer fully connected network, and in the deep Q network algorithm, Q(s t , a t ) is defined as the cumulative return obtained by taking the decision a t under the policy π with the state s t starting from time t:
[0023]
[0024] r = w1r1 + w2r2
[0025]
[0026]
[0027] wherein γ ∈ (0, 1) represents a discount factor, r is a reward function, r1 represents a collision penalty, w1 is a weight corresponding to r1, r2 represents a lane changing penalty, and w2 is a weight corresponding to r2.
[0028] Further, the key feature vector in step S2 includes the lane where the ego vehicle is located, the lane where the surrounding vehicle is located, and the distance between the ego vehicle and the surrounding vehicle.
[0029] Further, the specific process of step S2 is as follows: according to the current state data collected by the vehicle-mounted perception module, the key feature vector is extracted from the environment, the decision that maximizes the Q function is selected by using the greedy strategy, and the current decision of the autonomous vehicle is taken as the current decision of the autonomous vehicle, and then the decision quantity is transmitted to the vehicle control module for execution.
[0030] Further, the working process of the DeepSHAP module in step S4 is as follows:
[0031] Given a reference feature vector r, an interpreted feature vector x, and a corresponding model v(·), we have:
[0032] Δy = v(x) - v(r)
[0033] Δx i = x i -r i
[0034] wherein xi represents the i-th feature, r i represents the i-th reference feature;
[0035] A L-layer feedforward neural network v NN is adopted to represent the model v(x):
[0036]
[0037]
[0038] where g l (·) represents the i-th neural network layer with n l neurons;
[0039] Further, the Shapley value is calculated by the chain rule and linear approximation.
[0040] Further, the Shapley value is specifically:
[0041]
[0042]
[0043]
[0044] where, is the Shapley value of the i-th feature, represents the Shapley value of the j l -th neuron in the l-th layer with respect to the i-th feature x i represents the multiplier corresponding to the j l -th neuron in the l-th layer, represents the cumulative multiplier of the i-th feature x i .
[0045] Further, if the neurons of the feedforward neural network are linear functions , the Shapley value is calculated by the following formula:
[0046] Linear:
[0047] If the neurons of the feedforward neural network are ReLU activation functions , the Shapley value is calculated by the following formula:
[0048] ReLU:
[0049] Compared with the prior art, the automatic driving decision module and the DeepSHAP module are arranged, the automatic driving decision module based on a deep Q network model is used, the traffic environment feature vector collected by the vehicle-mounted perception module is combined to output corresponding decision instructions and transmit the decision instructions to the vehicle control module for execution, the DeepSHAP module is used to calculate Shapley values corresponding to data generated by interaction between the deep Q network model and the traffic environment, and the Shapley values are transmitted to the vehicle-mounted man-machine interaction unit for display. Therefore, a user can know the mapping mechanism relationship between automatic driving decision input and output through the Shapley values, so that the transparency of the automatic driving decision is effectively improved, and the trust degree of a driver or passenger to the automatic driving system is improved.
[0050] The application designs an explainable framework including a deep Q network automatic driving decision model and a DeepSHAP explanation process, first extracts key feature vectors from an environment, including a lane where a vehicle is located, lanes where surrounding vehicles are located and distances between the vehicle and the surrounding vehicles, then realizes automatic driving decision by using a deep Q network algorithm, and executes decision instructions by using a bottom horizontal control module and a vertical control module, finally obtains a background data set by interaction between the deep Q network model and the environment, and inputs the background data set and the Q network model to a DeepSHAP module to calculate Shapley values of all features in the data set, so that the automatic driving decision is intuitively explained from three aspects of local explanation, global explanation and feature dependence analysis, the influence of input features of the deep reinforcement learning model on the decision and the correlation are shown to a user, and the user is helped to understand the automatic driving decision in time. BRIEF DESCRIPTION OF DRAWINGS
[0051] Figure 1 It is a method flowchart of the application;
[0052] Figure 2 It is a framework application diagram in the embodiment;
[0053] Figure 3 It is an automatic driving environment diagram in the embodiment;
[0054] Figure 4 It is a decision process diagram of the deep Q network model
[0055] Figure 5 It is a decision local explanation diagram based on Shapley values in the embodiment;
[0056] Figure 6 It is a decision global explanation diagram based on Shapley values in the embodiment;
[0057] Figure 7 An example of a decision feature dependency analysis based on Shapley value. DETAILED DESCRIPTION
[0058] The application will be described in detail below with reference to the accompanying drawings and specific embodiments.
[0059] Embodiments
[0060] An interpretable autonomous driving decision system includes an autonomous driving decision module and a DeepSHAP module. The autonomous driving decision module is connected to a vehicle control module, a vehicle-mounted perception module, and the DeepSHAP module. The DeepSHAP module is connected to a vehicle-mounted human-computer interaction unit. The autonomous driving decision module is based on a deep Q network model and combines traffic environment feature vectors collected by the vehicle-mounted perception module to output corresponding decision instructions and transmit them to the vehicle control module (including a lateral control module and a longitudinal control module) for execution.
[0061] The DeepSHAP module is used to calculate Shapley values corresponding to data generated by the interaction of the deep Q network model and the traffic environment and transmit them to the vehicle-mounted human-computer interaction unit for display.
[0062] Based on the above system, an interpretable autonomous driving decision method is implemented, as shown in Figure 1 , which includes the following steps:
[0063] S1. According to the driving scene, determine the decision vector and state vector of the autonomous driving decision module, and train the neural network using the deep Q network algorithm to construct a deep Q network model.
[0064] S2. Extract the key feature vector from the traffic environment, input it into the deep Q network model, output the corresponding decision instruction, and transmit it to the vehicle control module for execution.
[0065] S3. While the vehicle control module executes the decision instruction, the vehicle-mounted perception module outputs the corresponding background data set.
[0066] S4. The background data set and the deep Q network model are jointly input into the DeepSHAP module. The DeepSHAP module calculates the Shapley values of all features in the background data set and transmits them to the vehicle-mounted human-computer interaction unit for display.
[0067] This embodiment applies the above technical solution to construct an interpretable architecture of a deep Q network autonomous driving decision model based on DeepSHAP, as shown in Figure 2 , which specifically includes two parts: deep Q network autonomous driving decision model design and DeepSHAP interpretation method.
[0068] In the design of the deep Q-network autonomous driving decision model, this embodiment uses, as follows: Figure 3 Taking the typical two-lane driving scenario shown as an example, the decision set A is defined as follows:
[0069] A={a left ,a idle ,a right} (1)
[0070] Where a left Indicates a left lane change, a idle To indicate maintaining the current action, a right This indicates a right lane change.
[0071] The state vector S is defined as follows:
[0072] S = [n ego ,n lat ,d] (2)
[0073] In the formula, d represents the relative distance between the vehicle and the surrounding vehicles, and n ego Indicates the lane where the vehicle is located, n lat Indicates the lane where Zhou's car is located.
[0074]
[0075] Then, the classic deep Q-network algorithm is used to train the neural network to approximate the action value function Q(s). t ,a t The network is a 4-layer fully connected network. In the deep Q-network algorithm, Q(s) t ,a t Defined as starting from time t, with state s t Given the input, make a decision a under policy π. t Cumulative returns obtained:
[0076]
[0077] In the formula, γ∈(0,1) represents the discount factor, and the reward function r is designed as follows:
[0078] r = w1r1 + w2r2 (5)
[0079] In the formula, r1 represents the collision penalty:
[0080]
[0081] The corresponding weight w1 = 1.
[0082] r2 indicates lane change penalty:
[0083]
[0084] The corresponding weight is w2 = 0.2.
[0085] Thus, corresponding to Figure 2 The deep Q-network model for the autonomous driving decision-making module has been constructed. Then, the deployed deep Q-network model is placed in the simulation environment. Based on the current state, a greedy strategy is used to select the decision that maximizes the Q-function as the current decision for the autonomous vehicle (the process is as follows). Figure 4 (As shown). The decision quantity is then passed to the underlying horizontal control model and vertical control model (such as...). Figure 2 (As shown in the control module), thereby generating a dataset for interpretable analysis in the DeepSHAP section, such as... Figure 2 As shown in the DeepSHAP section.
[0086] In the design of the interpretable method DeepSHAP: the feature importance value, or Shapley value, is quickly approximated through recursive backpropagation. Given a reference feature vector r, an interpretable feature vector x, and the corresponding model v(·), we obtain:
[0087] Δy=v(x)-v(r) (8)
[0088] and
[0089] Δx i =x i -r i (9)
[0090] In the formula, x i Let r represent the i-th feature. i This represents the i-th reference feature. This embodiment considers using an L-layer feedforward neural network v NN (·) is used to represent the complex model v(x):
[0091]
[0092] In the formula
[0093]
[0094] Indicates having n l The l-th layer of the neural network with n neurons. Then, through the chain rule and linear approximation, DeepSHAP can calculate the feature importance value, i.e., the Shapley value. as follows:
[0095]
[0096]
[0097] where represents the jth neuron in the lth layer l Shapley value of the ith feature x i represents the jth neuron in the lth layer l corresponding to the multiplicative factor of the ith feature x represents the cumulative multiplicative factor of the ith feature x i . Note that if the neuron is a linear or ReLU activation function, its Shapley value can be solved analytically. For example, for a linear function its Shapley value is:
[0098] Linear:
[0099] For a ReLU activation function its Shapley value is
[0100] ReLU:
[0101] So far, the Deep SHAP theoretical method corresponding to Figure 2 is constructed.
[0102] Finally, based on DeepSHAP, the Shapley value corresponding to the data generated by the interaction of the deep Q network model and the environment is calculated, so that the intuitive explanation of the deep Q network automatic driving decision model can be obtained depending on the Shapley value of the feature.
[0103] As shown in Figure 5 is the local explanation of the decision. For the left lane change decision, when n ego = 1, it plays a small positive contribution to the decision (corresponding to ), when n lat = 1 it plays a large positive contribution to the decision (corresponding to ), the relative distance d < d_mean (the mean of the relative distance) plays a large positive contribution to the left lane change (corresponding to ), and the sum of the contributions of all features and the base value (corresponding to ) is equal to the action value corresponding to the left lane change. Based on the above decision explanation, it is helpful for the operator to understand the decision-making process of the model, thereby performing defect analysis, fault diagnosis, and improving the performance of the model.
[0104] As shown in Figure 6 is the global explanation of the decision. For the left lane change decision, the observation lane feature of the ego vehicle (n ego ) in the left lane, and the solid box indicates in the right lane. It can be seen that when the host vehicle is in the right lane, the corresponding Shapley value is large, which increases the possibility of left turn, because the host vehicle is originally in the right lane, and the turning can only be left, which is close to human intuition. When the front vehicle (n lat ) is in the right lane, the corresponding Shapley value is large, which increases the possibility of left turn, because only the front vehicle is in the right lane, which blocks the driving of the host vehicle, so it will increase the possibility of left turn of the host vehicle. In the relative distance feature (d), from the overall distribution, the smaller the value, the larger the corresponding Shapley value. Therefore, compared to the case of large distance, small distance will increase the possibility of left turn.
[0105] As Figure 7 shown is the feature dependence analysis. For left lane change decision, when the host vehicle is in the right lane, the front vehicle is also in the right lane (in the box), which contributes a lot to the left lane change decision.
[0106] In summary, the technical solution first extracts the key feature vector from the environment, including the lane of the host vehicle, the lane of the surrounding vehicle, and the distance between the host vehicle and the surrounding vehicle. Then, the deep Q network algorithm is used to realize the automatic driving decision, and the decision instruction is executed through the underlying lateral control module and longitudinal control module. Finally, the background data set is obtained by interacting the deep Q network model with the simulation environment, and the Q network model is used as input to the DeepSHAP module. The Shapley values of all features in the data set are calculated to intuitively explain the automatic driving decision from three aspects of local explanation, global explanation and feature dependence analysis. Through human-computer interaction, the influence of the input features of the deep reinforcement learning model on the decision and the correlation can be displayed from three aspects of local explanation, global explanation and feature dependence, which greatly enhances the transparency of the deep reinforcement learning model, thereby helping human drivers to understand the automatic driving decision in time and enhancing the trust of the driver in the decision system.
Claims
1. An interpretable autonomous driving decision-making method, characterized in that, The method comprises the following steps: S1, determining a decision vector and a state vector of an automatic driving decision module according to a driving scene, training a neural network by using a deep Q network algorithm, and constructing a deep Q network model; S2, extracting a key feature vector from a traffic environment, inputting the deep Q network model, outputting a corresponding decision instruction, and transmitting the decision instruction to a vehicle control module for execution; S3, the vehicle control module executes the decision instruction, and a vehicle-mounted perception module outputs a corresponding background data set; S4, the background data set and the deep Q network model are jointly input into a DeepSHAP module, the DeepSHAP module calculates Shapley values of all features in the background data set, and the Shapley values are transmitted to a vehicle-mounted human-computer interaction unit for display; The specific process of constructing the deep Q network model in step S1 is as follows: training a neural network using a deep Q-network algorithm to approximate an action value function wherein the neural network is a 4-layer fully connected network, in the deep Q-network algorithm, defined as the cumulative reward obtained from time t starting with state as input, taking decisions under a policy wherein, denotes a discount factor, is a reward function, denotes a collision penalty, is a corresponding weight, denotes a lane change penalty, is a corresponding weight; The key feature vector in step S2 comprises a lane where a vehicle is located, a lane where a surrounding vehicle is located, and a distance between the vehicle and the surrounding vehicle; The working process of the DeepSHAP module in step S4 is as follows: Given a reference feature vector , an interpreted feature vector , and a corresponding model , one obtains: wherein, represents the th feature, represents the th reference feature; Using a feedforward neural network to represent the model : wherein, represents a first neural network layer having neurons; and neurons. The feature importance value, that is, the Shapley value, is calculated through the chain rule and linear approximation; The Shapley value is specifically as follows: in, For the first i Shapley value of each feature Indicates the first The first layer Neuron about the first Features Shapley value Indicates the first The first layer The multipliers corresponding to each neuron express For the first Features The cumulative multiplier.
2. The interpretable autonomous driving decision-making method of claim 1, wherein, The driving scene in step S1 is a typical two-lane driving scene, and the decision of the automatic driving decision module is specifically as follows: wherein, is a set of decisions, represents a left lane change, represents maintaining the current action, represents a right lane change; The state vector of the automatic driving decision module is specifically as follows: wherein, is a state vector, denotes the relative distance of the ego vehicle to the surrounding vehicle, denotes the lane in which the ego vehicle is located, denotes the lane in which the surrounding vehicle is located.
3. The interpretable autonomous driving decision-making method of claim 1, wherein, The specific process of step S2 is as follows: according to current state data collected by the vehicle-mounted perception module, a key feature vector is extracted from the environment, a decision that maximizes a Q function is selected by using a greedy strategy, and the decision is used as a current decision of the automatic driving vehicle, and then the decision is transmitted to the vehicle control module for execution.
4. The interpretable autonomous driving decision-making method of claim 1, wherein, If the neurons of the feedforward neural network are linear functions The Shapley value is then computed by the following equation: If the neurons of the feedforward neural network are ReLU activation functions Shapley values are computed by the following equation: 。 5. An interpretable autonomous driving decision system, implemented based on the interpretable autonomous driving decision method of claim 1, characterized in that, The automatic driving decision module and the DeepSHAP module are connected with the vehicle control module, the vehicle-mounted perception module, and the DeepSHAP module, the DeepSHAP module is connected with the vehicle-mounted human-computer interaction unit, the automatic driving decision module is based on the deep Q network model, combines the traffic environment feature vector collected by the vehicle-mounted perception module, outputs a corresponding decision instruction, and transmits the decision instruction to the vehicle control module for execution; The DeepSHAP module is used for calculating Shapley values corresponding to data generated by interaction between the deep Q network model and the traffic environment, and transmitting the Shapley values to the vehicle-mounted human-computer interaction unit for display.
6. The interpretable autonomous driving decision system of claim 5, wherein, The vehicle control module comprises a lateral control module and a longitudinal control module.