Double-layer adversarial attack method for automatic driving decision model

Abstract

The invention discloses a double-layer adversarial attack method for an automatic driving decision model. The method comprises the following steps: establishing a mixed traffic automatic driving scene; designing a man-driven vehicle model; designing an automatic driving vehicle intelligent body; training an automatic driving automobile decision model through reinforcement learning; designing an upper-layer agent of the attack model; designing a lower-layer gradient calculation loss function of the attack model; and training the attack model. According to the method, an improved Dueling DQN algorithm is adopted on the upper layer to obtain the optimal motion disturbance direction of the automatic driving automobile, specific state disturbance is generated on the lower layer through momentum projection gradient descent, and an attack model is obtained through interactive training with the trained automatic driving automobile. Compared with the prior art, the method has the advantages that the attack efficiency is greatly improved while the optimal attack effect is ensured, and a brand-new efficient method is provided for the safety test of the automatic driving system.

Description

Technical Field

[0001] The present invention belongs to the field of autonomous driving safety, and specifically relates to a double-layer adversarial attack method for an autonomous driving decision-making model, which is used to attack autonomous driving vehicles using reinforcement learning as a decision-making algorithm. Background Art

[0002] Autonomous driving vehicles, as the cornerstone of future intelligent transportation systems, are expected to revolutionize road safety, reduce congestion, and improve traffic efficiency. Reinforcement learning is committed to dealing with complex and dynamic environments and learning optimal control strategies. More and more researchers are dedicated to the application of reinforcement learning in the field of autonomous driving. However, reinforcement learning is overly dependent on the accuracy of sensors. Unfortunately, sensors are also the most vulnerable components in autonomous driving vehicles. By interfering with sensor data, the output of the reinforcement learning agent can be changed, leading to danger. Currently, the methods for generating interfering data can be divided into two types: (1) generating strong state interferences through a heuristic approach. (2) generating strong state interferences through reinforcement learning by continuously interacting with autonomous driving vehicles.

[0003] However, most of the current methods for generating interfering data have more or less deficiencies. For example, generating interferences through a heuristic approach is very short-sighted, only focusing on immediate destruction and not considering the long-term impact of attacks. However, autonomous driving is a long-term activity, which is not reasonable. Directly using reinforcement learning to directly generate state perturbations will lead to a very large action space, sometimes even as large as the state space, resulting in low training and exploration efficiency. Summary of the Invention

[0004] Object of the Invention: The object of the present invention is to solve the problem that the existing solutions for adversarial attacking autonomous driving vehicles either adopt a heuristic method that is very short-sighted or adopt a reinforcement learning method with low efficiency.

[0005] Technical Solution: To solve the above problems, the present invention provides the following technical solution: A double-layer adversarial attack method for an autonomous driving decision-making model, including the following steps:

[0006] Step 1: Establish an autonomous driving scenario, which provides a basic environment for the entire attack method. The vehicle model and the attacking agent in the subsequent steps will be trained in this scenario;

[0007] Step 2: Design the human-driven vehicle model involved in the autonomous driving scenario in Step 1, where longitudinal decision-making is controlled by the IDM model and lateral decision-making is controlled by the MOBIL model;

[0008] Step 3: Design the autonomous driving vehicle agent involved in the autonomous driving scenario in Step 1, and design its state space, action space, and reward function;

[0009] Step 4: Select a reinforcement learning algorithm to train the autonomous driving vehicle agent in step 3 to obtain the autonomous driving vehicle decision model;

[0010] Step 5: Design the upper-layer agent of the attack model, and design its state space, action space, and reward function;

[0011] Step 6: Design the loss function for calculating the gradient of the lower layer of the attack model;

[0012] Step 7: By continuously interacting with the autonomous vehicle agent obtained in step 4, use the improved Dueling DQN algorithm and momentum projected gradient descent to jointly train the attack model defined in steps 5 and 6.

[0013] As a further preferred solution, in step 1, the autonomous driving scenario established is a gate merging scenario under a highway.

[0014] As a further preferred solution, the longitudinal decision of the human-driven vehicle model in step 2 is controlled by the IDM model, and the control formula is as follows:

[0015]

[0016] in is the calculated vehicle acceleration, a is the maximum acceleration allowed, v is the current speed, v0 is the desired speed, s * is the expected minimum following distance, s is the following distance between the current vehicle and the vehicle in front, b is a comfortable deceleration of the vehicle, Δv is the speed difference between the current vehicle and the vehicle in front, and T is the safe time head distance of the vehicle.

[0017] As a further preferred solution, the lateral decision of the human-driven vehicle model in step 2 is controlled by the MOBIL model, and the control formula is as follows:

[0018]

[0019] in is the acceleration of the front vehicle in the new lane after the lane change, a n is the vehicle's acceleration before changing lanes, is the predicted acceleration of the current vehicle after changing lanes, a c is the current vehicle acceleration before changing lanes, is the acceleration of the front vehicle in the current lane after changing lanes, a o is the vehicle acceleration before changing lanes, Δa th is the lane change threshold, b safe The safe deceleration "threshold".

[0020] As a further preferred solution, in step 3, the state space of the autonomous vehicle agent is {s1, s2, s3, …, s N}, where s i = {present, loc x , loc y , v x , v y}, representing whether the i-th vehicle can be observed, and the x and y coordinates of the position and speed of the i-th vehicle respectively; the action space is A = {a1, a2, a3, a4, a5}, representing 5 discrete actions of turning left, turning right, staying unchanged, accelerating, and decelerating respectively; the reward function is R t = R re + R s + R m - C t , where R re is the reward for reaching the end point, R s is the reward for maintaining a target speed, R m is the reward for successful merging, and C t is the cost of the autonomous vehicle at time t.

[0021] As a further preferred solution, in step 4, the specific training process of the autonomous vehicle decision-making model is as follows:

[0022] S41. Initialize the value network, policy network, and environment parameters of the reinforcement learning model;

[0023] S42. At each time step t, perform state observation, action selection, obtain action feedback, and store data;

[0024] S43. Determine whether the network update standard is reached. If so, execute S44; otherwise, execute S42;

[0025] S44. Calculate the value estimate of each state through the value network, and use this to update the value network and the policy network;

[0026] S45. Determine whether the reinforcement learning model has converged. If so, end the training process; otherwise, execute S42;

[0027] As a further preferred solution, since the goal of the attack model is to interfere with the observation state of the autonomous vehicle, in step 5, the state space of the upper-layer agent of the attack model is the same as the state space of the autonomous vehicle, which is also {s1, s2, s3, …, s N}, where s i = {present, loc x , loc y , v x , v y}; The action space is also the same as that of the autonomous vehicle, with 5 discrete actions: turning left, turning right, staying the same, accelerating, and decelerating. However, the meanings are different. The purpose of the autonomous vehicle is to find the safest action, while the purpose of the upper-layer agent of the attack model is to find the most dangerous and costly action for the autonomous vehicle; the reward function is R attacker = λ1·(-r t ) + λ2·c t + λ3·f ch (ch) + λ4·f s (v) + λ3·P(δ), where r t is the reward of the autonomous vehicle under this attack, c t is the cost of the autonomous vehicle under this attack, f ch (ch) is the reward for causing a collision of the autonomous vehicle, f s (v) is the reward for maintaining the autonomous vehicle at a low speed, and P(δ) is the penalty for the interference range limit. The specific formula is as follows:

[0028]

[0029] where ch is a boolean variable used to indicate whether a collision occurs in the vehicle, R ch is the positive reward when a collision occurs, γ safe is the penalty term when no collision occurs, v represents the current vehicle speed, v target represents the target speed that the autonomous vehicle is expected to reach through the attack, σ is a parameter controlling the influence of speed deviation, and ε is a parameter controlling the influence of speed deviation.

[0030] As a further preferred solution, the loss function for calculating the gradient of the lower layer of the attack model in step 6 is:

[0031]

[0032] where π1 is the policy network of the autonomous vehicle in step 3 obtained through training in step 4, π2 is the online network of the improved Dueling DQN, δ is the output interference, D KL (P||Q) is the calculation formula of the KL divergence, a attack is the output of the Dueling DQN online network, that is, the most dangerous and costly action for the autonomous vehicle.

[0033] As a further preferred solution, the formula of the improved Dueling DQN in step 7 is as follows:

[0034]

[0035] θ target←τ·θ attacker +(1 - τ)·θ target

[0036] Where V(s) represents the value of a given state, A(s,a) represents the advantage of taking a specific action in that state, y i represents the TD error calculation for network update, r i is the reward at the current time step, s i+1 is the state at the next time step, θ attacker are the parameters of the Dueling DQN online network, L(θ attacker ) is its loss function. To improve the training performance and efficiency of Dueling DQN, a target network is introduced during training. θ target are the parameters of the target network, and a smooth update method is introduced to update the target network. τ is the smooth update coefficient, which controls the degree of gradual update of the parameters of the target network from the online network.

[0037] As a further preferred solution, the momentum projection gradient descent formula in step 7 is as follows:

[0038]

[0039] δ k+1 =v k -v t

[0040] Where represents normalizing its loss function, is the policy network of the autonomous vehicle, δ k represents the state perturbation generated by momentum projection gradient descent at the k-th iteration step, a t is the attack policy generated by Dueling DQN, v t is the momentum considering multi-step update gradient, γ is the corresponding momentum coefficient, δ k+1 is the updated state perturbation.

[0041] As a further preferred solution, the training process of the attack model in step 7 is as follows:

[0042] S71: Prepare the trained autonomous vehicle agent, randomly initialize the parameters of the Dueling DQN online network, and the autonomous driving scenario;

[0043] S72: Perform state observation at each time step t to obtain the current state;

[0044] S73: The upper layer of the attack model outputs a state perturbation orientation;

[0045] S74: The lower layer of the attack model continuously iterates to output a specific state perturbation;

[0046] S75: Apply the state perturbation to the decision-making model of the autonomous vehicle;

[0047] S76: The decision-making model of the autonomous vehicle performs state observation, action selection, and obtains action feedback;

[0048] S77: The upper layer of the attack model calculates the reward at the current time step through the action feedback and stores the data;

[0049] S78: Determine whether the online network update criterion is met. If so, execute S79; otherwise, execute S72.

[0050] S79: Calculate the loss function and use it to update the Dueling DQN online network;

[0051] S710: Determine whether the target network update criterion is met. If so, execute S711; otherwise, execute S712.

[0052] S711: Update the Dueling DQN target network through smooth update;

[0053] S712: Determine whether the reinforcement learning model has converged. If so, end the training process; otherwise, execute S72.

[0054] Beneficial effects

[0055] Compared with the prior art, the present invention:

[0056] 1. The proposed adversarial attack method has a better attack effect than traditional heuristic schemes.

[0057] 2. The proposed adversarial attack method has a better training efficiency than traditional reinforcement learning schemes.

[0058] 3. Innovatively combines the reinforcement learning attack method with the heuristic attack method, taking into account the advantages of both and avoiding their disadvantages.

[0059] 4. Applying this attack method to the adversarial training of autonomous vehicles can enhance the robustness of the system in extreme environments and improve the safety and reliability of autonomous vehicles. Description of the drawings

[0060] Figure 1 It is a schematic structural diagram of the double-layer adversarial attack method for the autonomous driving decision-making model of the present invention;

[0061] Figure 2 It is a training result diagram of using multiple reinforcement learning algorithms to train autonomous vehicles;

[0062] Figure 3 It is the training result graph of the attack model. Specific implementation manners

[0063] The present invention will be further described below with reference to the accompanying drawings:

[0064] Figure 1 It is a schematic structural diagram of a double-layer adversarial attack method for an autonomous driving decision-making model. The present invention will be described in detail according to this structural diagram.

[0065] The first step is to establish an autonomous driving scenario. In this invention, the established scenario is the ramp merging scenario under a highway.

[0066] The second step is that the longitudinal decision-making of the human-driven vehicle is controlled by the IDM model, and the control formula is as follows:

[0067]

[0068] Where is the calculated vehicle acceleration, a is the maximum allowable acceleration, v is the current speed, v0 is the desired speed, s * is the desired minimum following distance, s is the following distance between the current vehicle and the vehicle in front, b is a comfortable deceleration of the vehicle, Δv is the speed difference between the current vehicle and the vehicle in front, and T is the safe time headway of the vehicle.

[0069] The third step is that the lateral decision-making of the human-driven vehicle is controlled by the MOBIL model, and the control formula is as follows:

[0070]

[0071] Where is the acceleration of the vehicle in front in the new lane after lane change, a n is the acceleration of this vehicle before lane change, is the predicted acceleration of the current vehicle after lane change, a c is the acceleration of the current vehicle before lane change, is the acceleration of the vehicle in front in the current lane after lane change, a o is the acceleration of this vehicle before lane change, Δa th is the lane change threshold, b safe is the "threshold" for safe deceleration.

[0072] The fourth step is to design an autonomous driving vehicle agent, and design its state space, action space, and reward function. The state space is {s1, s2, s3,..., s N}, where s i = {present, loc x , loc y , v x , vy}, respectively representing whether the i-th vehicle can be observed, and the x and y coordinates of the position and speed of the i-th vehicle. The action space is A = {a1, a2, a3, a4, a5}, representing five discrete actions of turning left, turning right, staying the same, accelerating, and decelerating respectively. The reward function is R t = R re + R s + R m - C t , where R re is the reward for reaching the end, R s is the reward for maintaining a target speed, R m is the reward for successful merging, and C t is the cost of the autonomous vehicle at time t.

[0073] Step 5, design the specific training process of the autonomous vehicle decision-making model, which is as follows:

[0074] S51. Initialize the value network, policy network, and environment parameters of the reinforcement learning model;

[0075] S52. Perform state observation, action selection, obtain action feedback, and store data at each time step t;

[0076] S53. Determine whether the network update standard is reached. If so, execute S64; otherwise, execute S62;

[0077] S54. Calculate the value estimate of each state through the value network and use it to update the value network and policy network;

[0078] S55. Determine whether the reinforcement learning model has converged. If so, end the training process; otherwise, execute S62;

[0079] Step 6, design the upper-layer intelligent agent of the attack algorithm, and design its state space, action space, and reward function. Since the goal of the attack model is to interfere with the observation state of the autonomous vehicle, the state space of the upper-layer intelligent agent of the attack model is the same as that of the autonomous vehicle, which is also {s1, s2, s3,..., s N}, where s i = {present, loc x , loc y , v x , v y}; the action space is also the same as that of the autonomous vehicle, with five discrete actions of turning left, turning right, staying the same, accelerating, and decelerating. However, the meanings are different. The purpose of the autonomous vehicle is to find the safest action, while the purpose of the upper-layer intelligent agent of the attack model is to find the most dangerous and costly action for the autonomous vehicle. The reward function is R attacker = λ1·(-rt ) + λ2·c t + λ3·f ch (ch) + λ4·f s (v) + λ3·P(δ), where r t is the reward of the autonomous vehicle under this attack, c t is the cost of the autonomous vehicle under this attack, f ch (ch) is the reward for causing a collision of the autonomous vehicle, f s (v) is the reward for maintaining the autonomous vehicle at a low speed, P(δ) is the interference range limit penalty, and the specific formula is as follows:

[0080]

[0081] where ch is a Boolean variable used to indicate whether a vehicle collision occurs, R ch is the positive reward when a collision occurs, γ safe is the penalty term when no collision occurs, v represents the current vehicle speed, v target represents the target speed that the autonomous vehicle is expected to reach through the attack, σ is a parameter controlling the influence of speed deviation, and ε is a parameter controlling the influence of speed deviation.

[0082] Step 7, design the loss function for calculating the lower layer gradient of the attack model, which is:

[0083]

[0084]

[0085] where π1 is the policy network of the autonomous vehicle, π2 is the online network of the improved Dueling DQN, δ is the output interference, D KL (P||Q) is the calculation formula of KL divergence, a attack is the output of the Dueling DQN online network, that is, the most dangerous and costly action for the autonomous vehicle.

[0086] Step 8, design the improved Dueling DQN network structure, and the formula is as follows:

[0087]

[0088] θ target ← τ·θ attacker +(1 - τ)·θ target

[0089] where V(s) represents the value of the given state, A(s,a) represents the advantage of taking a specific action in this state, y iRepresents the TD error calculation for network update, r i is the reward at the current time step, s i+1 is the state at the next time step, θ attacker are the parameters of the Dueling DQN online network, L(θ attacker ) is its loss function. During the training process, we introduce a target network, θ target are the parameters of the target network, and we introduce a smooth update method to update the target network. τ is the smooth update coefficient, which controls the degree of gradual update of the target network's parameters from the online network.

[0090] Step 9: Design the momentum projection gradient descent formula as follows:

[0091]

[0092] δ k+1 =δ k -v t

[0093] where represents normalizing its loss function, is the policy network of the autonomous driving vehicle, δ k represents the state perturbation generated by momentum projection gradient descent at the k-th iteration step, a t is the attack policy generated by Dueling DQN, v t is the momentum considering multi-step update gradient, γ is the corresponding momentum coefficient, δ k+1 is the updated state perturbation.

[0094] Step 10: Design the training process of the attack model as follows:

[0095] S101: Prepare the trained autonomous driving vehicle model, randomly initialize the parameters of the Dueling DQN online network, and the autonomous driving scenario;

[0096] S102: Perform state observation at each time step t to obtain the current state;

[0097] S103: The upper layer of the attack model outputs a state perturbation orientation;

[0098] S104: The lower layer of the attack model continuously iterates to output a specific state perturbation;

[0099] S105: Apply the state perturbation to the autonomous driving vehicle;

[0100] S106: The autonomous driving vehicle performs state observation, action selection, and obtains action feedback;

[0101] S107: The attack model calculates the reward at the current time step through action feedback and stores the data.

[0102] S108: Determine whether the online network update standard is reached. If so, execute S109; otherwise, execute S102.

[0103] S109: Calculate the loss function and use it to update the Dueling DQN online network.

[0104] S1010: Determine whether the target network update standard is reached. If so, execute S1011; otherwise, execute S1012.

[0105] S1011: Update the Dueling DQN target network through smooth update.

[0106] S1012: Determine whether the reinforcement learning model has converged. If so, end the training process; otherwise, execute S112.

[0107] Figure 2 This is the training curve of the autonomous driving vehicle. We used four common reinforcement learning algorithms to train the autonomous driving vehicle. As shown in the figure, SAC-Lag has the best performance. Therefore, the training of the attack model is also completed in the process of continuously interacting with SAC-Lag.

[0108] Figure 3 This is the training curve of the attack model. A total of 12,000 rounds of iteration were performed. It can be seen that the algorithm began to converge when it reached 8,000 rounds, indicating that the attack model had achieved a good attack effect at this time.

[0109] The present invention is also applicable to any autonomous driving vehicle using reinforcement learning as the decision-making algorithm, which will not be elaborated here.

Claims

1. A two-layer adversarial attack method for an autonomous driving decision model, characterized by: The following steps are involved: Step 1: Establish an autonomous driving scenario to provide a basic environment for the entire attack method. The vehicle model and attack agent in subsequent steps will be trained in this scenario. Step 2: Design the human-driven vehicle model involved in the autonomous driving scenario in step 1, where the longitudinal decision is controlled by the IDM model and the lateral decision is controlled by the MOBIL model; Step 3: Design the autonomous driving vehicle agent involved in the autonomous driving scenario in step 1, and design its state space, action space, and reward function; Step 4: Select a reinforcement learning algorithm to train the autonomous driving vehicle agent in step 3 to obtain the autonomous driving vehicle decision model; Step 5: Design the upper-layer agent of the attack model, and design its state space, action space, and reward function; Step 6: Design the loss function for calculating the gradient of the lower layer of the attack model; Step 7: By continuously interacting with the autonomous vehicle agent obtained in step 4, use the improved Dueling DQN algorithm and momentum projected gradient descent to jointly train the attack model defined in steps 5 and 6.

2. The two-layer adversarial attack method for the autonomous driving decision model according to claim 1, characterized in that: In step 1, the established autonomous driving scenario is a gate merging scenario on a highway; the longitudinal decision of the human-driven vehicle model in step 2 is controlled by the IDM model, and the control formula is as follows: in is the calculated vehicle acceleration, a is the maximum acceleration allowed, v is the current speed, v0 is the desired speed, s * is the expected minimum following distance, s is the following distance between the current vehicle and the vehicle in front, b is a comfortable deceleration of the vehicle, Δv is the speed difference between the current vehicle and the vehicle in front, and T is the safe time head distance of the vehicle.

3. The two-layer adversarial attack method for the autonomous driving decision model according to claim 1, characterized in that: The lateral decision of the human-driven vehicle model in step 2 is controlled by the MOBIL model, and the control formula is as follows: in is the acceleration of the front vehicle in the new lane after the lane change, a n is the vehicle’s acceleration before changing lanes, is the predicted acceleration of the current vehicle after changing lanes, a c is the current vehicle acceleration before changing lanes, is the acceleration of the front vehicle in the current lane after changing lanes, a o is the vehicle acceleration before changing lanes, Δa th is the lane change threshold, b safe The "threshold" for safe deceleration.

4. The two-layer adversarial attack method for the autonomous driving decision model according to claim 1, characterized in that: The state space of the autonomous vehicle agent in step 3 is {s1,s2,s3,…,s N }, where s i = {present,loc x ,loc y ,v x ,v y }, respectively representing whether the i-th vehicle can be observed, the x and y coordinates of the position and speed of the i-th vehicle; the action space is A = {a1, a2, a3, a4, a5}, representing five discrete actions: turn left, turn right, remain unchanged, accelerate, and decelerate; the reward function is R t =R re +R s +R m -C t , where R re is the reward for reaching the end point, R s The reward for maintaining a target speed, R m As a reward for successful merging, C t is the cost of the autonomous vehicle at time t.

5. The two-layer adversarial attack method for the autonomous driving decision model according to claim 1, characterized in that: The specific training process of the autonomous driving car decision model in step 4 is: S41. Initialize the value network, policy network and environment parameters of the reinforcement learning model; S42, performing state observation, action selection, obtaining action feedback and storing data in each time step t; S43, determine whether the network update standard is met, if so, execute S44, otherwise, execute S42; S44, calculating the value estimate of each state through the value network, and using this to update the value network and the strategy network; S45, determine whether the reinforcement learning model has converged. If so, end the training process; otherwise, execute S42.

6. The two-layer adversarial attack method for the autonomous driving decision model according to claim 1, characterized in that: Since the goal of the attack model is to interfere with the observed state of the autonomous vehicle, the state space of the upper agent of the attack model in step 5 is consistent with the state space of the autonomous vehicle, which is also {s1,s2,s3,…,s N }, where s i = {present,loc x ,loc y ,v x ,v y The action space is also consistent with the action space of the self-driving car, with five discrete actions: turn left, turn right, stay the same, accelerate, and decelerate. The goal of the self-driving car is to find the safest action, and the goal of the upper-level agent of the attack model is to find the most dangerous and costly action for the self-driving car. The reward function is R attacker =λ1·(-r t )+λ2·c t +λ3·f ch (ch)+λ4·f s (v)+λ3·P(δ), where r t is the reward of the self-driving car under this attack, c t is the cost of the self-driving car under this attack, f ch (ch) a reward for causing a collision with an autonomous vehicle, f s (v) The reward for maintaining the autonomous vehicle at a lower speed, P(δ) is the range limitation penalty for interference, and the specific formula is as follows: Where ch is a Boolean variable used to indicate whether the vehicle has collided. ch is the positive reward when a collision occurs, γ safe is the penalty term when no collision occurs, v represents the current vehicle speed, and v target It represents the target speed that the autonomous vehicle hopes to reach through the attack, σ is the parameter that controls the influence of speed deviation, and ε is the parameter that controls the influence of speed deviation.

7. The two-layer adversarial attack method for the autonomous driving decision model according to claim 1, characterized in that: The loss function for calculating the gradient of the lower layer of the attack model in step 6 is: Where π1 is the policy network of the autonomous vehicle in step 3 trained by step 4, π2 is the online network of the improved Dueling DQN, δ is the output interference, and D KL (P||Q) is the calculation formula of KL divergence, a attack is the output of the DuelingDQN online network, which is the most dangerous and costly action for the self-driving car.

8. The two-layer adversarial attack method for the autonomous driving decision model according to claim 1, characterized in that: The improved Dueling DQN formula in step 7 is as follows: i target ←t·i attacker +(1-τ)·θ target where V(s) represents the value of a given state, A(s,a) represents the advantage of taking a specific action in that state, and y i represents the TD error calculation used for network update, r i is the reward at the current time step, s i+1 is the state at the next time step, θ attacker is the parameter of Dueling DQN online network, L(θ attacker ) is its loss function. In order to improve the performance and efficiency of Dueling DQN training, the target network θ is introduced during the training process. target is the parameter of the target network, and a smooth update method is introduced to update the target network. τ is the smooth update coefficient, which controls the degree of gradual update of the target network parameters from the online network.

9. The two-layer adversarial attack method for the autonomous driving decision model according to claim 1, characterized in that: The momentum projected gradient descent formula in step 7 is as follows: d k+1 =d k -v t in Indicates the normalization of its loss function. is the policy network of the autonomous vehicle, δ k represents the state disturbance generated by iterative momentum projected gradient descent at step k, a t Generated attack strategy for Dueling DQN, v t To consider the momentum of multi-step gradient updates, γ is the corresponding momentum coefficient, δ k+1 is the updated state interference.

10. The two-layer adversarial attack method for the autonomous driving decision model according to claim 1, characterized in that: The attack model training process in step 7 is: S71: Prepare the trained autonomous driving vehicle agent, randomly initialize the parameters of the Dueling DQN online network, and the autonomous driving scenario; S72: perform state observation in each time step t to obtain the current state; S73: The upper layer of the attack model outputs a state interference guide; S74: The lower layer of the attack model continuously iterates and outputs a specific state interference; S75: Applying state disturbance to the autonomous vehicle decision model; S76: Decision model for autonomous vehicles performs state observation, action selection, and action feedback; S77: The upper layer of the attack model calculates the reward of the current time step through action feedback and stores the data; S78: Determine whether the online network update standard is met. If so, execute S79; otherwise, execute S72 S79: Calculate the loss function and use it to update the Dueling DQN online network; S710: Determine whether the target network update standard is met, if so, execute S711, otherwise, execute S712 S711: Update the Dueling DQN target network by smoothing update; S712: Determine whether the reinforcement learning model has converged. If so, end the training process. Otherwise, execute S72.

Images