An automatic driving vehicle adaptive decision control method considering disturbance factors

Abstract

This invention belongs to the technical field of autonomous driving and intelligent transportation systems, specifically relating to an adaptive decision-making control method for autonomous vehicles that considers disturbance factors. The method includes: Step S1: Outputting a scenario demand weight vector based on the PSO algorithm according to the current traffic flow situation around the vehicle; Step S2: Collecting the user preference vector at the current moment; Step S3: Constructing a hybrid preference vector at the current moment, based on a weighted sum of the user preference vector and the scenario demand weight vector; Step S4: Outputting adaptive robust driving actions for the autonomous vehicle based on the CMOR-AC reinforcement learning algorithm. The advantages of this invention are: improving the decision-making robustness of autonomous vehicles in complex traffic scenarios, achieving adaptive and personalized human-like decision-making control, while ensuring efficient matching of strategies and preferences through nonlinear constraints, enabling the search for Pareto optimal solutions on non-convex surfaces, and adapting to complex and ever-changing traffic scenarios.

Description

Technical Field

[0001] This invention belongs to the field of autonomous driving and intelligent transportation systems, and specifically relates to an adaptive decision control method for autonomous vehicles that takes into account disturbance factors. Background Technology

[0002] Autonomous driving technology deeply integrates multiple disciplines such as artificial intelligence, electronic information, vehicle engineering, and automation. Its potential social value is reflected in many aspects, including improving traffic efficiency, reducing traffic accidents, alleviating traffic congestion, and reducing exhaust emissions. It has become a key breakthrough direction and important research area in the new round of technological revolution and industrial transformation. Autonomous driving decision-making and control technology is a crucial link in achieving safe and efficient vehicle operation. It needs to cope with various unpredictable disturbances during driving and balance multiple conflicting goals such as safety, efficiency, and comfort, while adapting to the personalized preferences of different drivers and passengers and dynamically changing traffic scenarios.

[0003] Existing autonomous driving decision-making and control technologies have several limitations: First, real-world traffic scenarios involve multiple uncertainties, such as sensor noise, measurement errors, changes in road surface adhesion coefficient, and vehicle load variations. Existing technologies do not consider multiple types of disturbances simultaneously, or only design coping strategies for specific disturbances, leading to undesirable or even dangerous driving behaviors by autonomous vehicles in uncertain environments, resulting in insufficient decision robustness. Second, existing multi-objective decision-making methods typically employ statically weighted reward functions, where objective weights cannot be dynamically adjusted according to the time-varying characteristics of traffic conditions, and it is difficult to effectively integrate personalized user preferences, easily causing imbalances in driving behavior and failing to achieve adaptive and personalized decision control. Third, traditional multi-objective reinforcement learning methods have failed to cover Pareto optimal policies across all preference spaces in a discrete-continuous mixed action space using a single model, exhibiting low policy generalization and computational efficiency, making it difficult to find Pareto optimal solutions on non-convex surfaces, and difficult to extend to high-dimensional preference spaces.

[0004] Therefore, there is an urgent need for an adaptive decision-making and control method for autonomous vehicles that can comprehensively consider multiple types of disturbance factors, dynamically adjust the weights of multiple objectives, and integrate user preferences. This method can improve the decision-making robustness, scenario adaptability, and human-like driving level of autonomous vehicles in uncertain environments, and meet the needs of the practical application and large-scale deployment of autonomous driving technology. Summary of the Invention

[0005] In view of the above problems, the purpose of this invention is to provide an adaptive decision control method for autonomous vehicles that takes into account disturbance factors. This method combines adversarial training with dynamic adjustment of multi-objective preferences to achieve robust, human-like, and adaptive decision control of autonomous vehicles in uncertain environments, thereby overcoming the shortcomings of the prior art.

[0006] This invention provides an adaptive decision control method for autonomous vehicles that considers disturbance factors, comprising the following steps: Step S1: Based on the current traffic flow situation around the vehicle, output the scene demand weight vector using the PSO algorithm. : Formula 1: ; Formula 2: ; In the formula, , , These are weights for scenario requirements, efficiency, collaboration, and security, with their values ​​all constrained within a certain range. ; Step S2: Collect the user preference vector at the current moment. : Formula 17: ; Formula 18: ; In the formula, , , These are user-defined efficiency weights, collaboration weights, and security weights, with their value ranges constrained within a certain range. If the user does not specify a preference, a default value will be set. That is, safety should be the top priority, followed by efficiency and collaboration; Step S3: Construct the mixture preference vector at the current time step ,in Based on user preference vectors With the scene requirement weight vector Weighted sum design: Formula 19: ; In the formula, These are preference weight coefficients, used to balance user preferences with the weights of scenario requirements, forming a hybrid preference vector. This will serve as the basis for decision-making in subsequent CMOR-AC reinforcement learning algorithms that drive autonomous driving agents. Step S4: Based on the CMOR-AC reinforcement learning algorithm, output the adaptive and robust driving actions of the autonomous vehicle.

[0007] As a preferred embodiment of the present invention, step S1 further includes the following step: Step S1-1: In At any given time, initialize an empty state and weight cache pool. Build capacity is Status and weighted cache pool This is used to store matching pairs of historical states and their corresponding optimal weights. Formula 3: ; In the formula, Representing the A set of historical traffic situation feature vectors, This represents the optimal weight combination of efficiency, cooperation, and security objectives obtained through the PSO algorithm under the corresponding situation; when the situation and weight cache pool Reaching maximum capacity At that time, the first-in-first-out (FIFO) strategy is used to eliminate the earliest-entered position and weight pair; Step S1-2: Construct the traffic situation feature vector at the current moment ; A traffic situation feature vector for the current moment is constructed by extracting four core features from the vehicle's observation space data. All core feature values ​​are constrained within a certain range. Ensure that all core features are scaled uniformly; ① Traffic pressure characteristics The distance is represented by the mean of the Euclidean distance between the vehicle and other vehicles within the observation range. Formula 4: ; In the formula, This represents the number of other vehicles within the observation range. Representing the The European-style distance between the car and the driver. Represents the vehicle's safety threshold. The larger the size, the more space the vehicle has to move freely, and the less traffic pressure there is. The smaller the size, the less space the vehicle has to move freely, and the greater the traffic pressure. When there are no other vehicles within the observation range, it leads to... =1; ② Traffic flow characteristics The speed is represented by the average relative speed between the vehicle and other vehicles within the observation range. Formula 5: ; Formula 6: ; In the formula, This represents the average relative speed between the observed vehicle and other vehicles within the observation range. Represents the first within the observation range The speed of the car For the vehicle's speed, when When the value is positive, vehicles should increase their speed to meet the overall traffic flow efficiency requirements. When the value is negative, the weight of the vehicle in the efficiency index should be reduced. When there are no other vehicles within the observation range, let... =1; ③ Conflict risk characteristics The distance between the vehicle and other vehicles within the observation range is expressed as the minimum Euclidean distance. Formula 7: ; The larger the [velocity], the higher the risk of conflict during lane changing / lane following, especially when there are no other vehicles within the observation range. =0; ④ Characteristics of behavioral influence The impact of a driver's behavior on traffic flow is represented by the speed changes of vehicles following behind the vehicle. The expression is as follows: Formula 8: ; In the formula, This represents the speed difference between two adjacent time steps of a vehicle following behind the first vehicle. The larger the value, the greater the impact of the driver's behavior on traffic flow. More attention should be paid to cooperation with other vehicles. When there are no following vehicles within the observation range, [the driver should]... It is 0.

[0008] As a preferred embodiment of the present invention, step S1 further includes the following step: Steps S1-3: Determine the similarity between the current situation and the historical situation, retrieve the historical situation and the weight cache pool, and if the current traffic situation feature vector... Historical traffic situation feature vector The L1 distance of the core feature value is less than the threshold, i.e., the Manhattan distance. If the value is less than the threshold, it is determined to be a similar situation, and the historical best weight is directly reused to reduce computational overhead, proceeding to step S1-10. If no similar situation exists, proceed to step S1-4, where: Formula 9: ; In the formula, The first feature vector of the traffic situation at the current moment. One portion, The first of the historical traffic situation feature vectors One component; Step S1-4: Initialize the particle swarm based on the characteristic distribution of the current situation and set constraints; Based on traffic pressure characteristics The traffic scenarios are divided into three categories, and a differentiated weight initialization range is set for each category. During the initialization process, unnormalized weight values ​​are first randomly generated within the corresponding range, and then the weights are summed to 1 through particle constraints to ensure that the distribution of the initial particle swarm matches the current traffic situation. The particle's dimension corresponds to the three target weights to be optimized; that is, a single particle is encoded as... To ensure the physical meaning of the weights, the following constraints must be satisfied: Formula 10:

[0009] The constraint processing method is as follows: the result after updating the particle position is first taken as non-negative, and then normalized. Formula 11: ; In the formula, To determine the final weights that satisfy the constraints after correction, if the sum of the weights before normalization is less than 10... -6 Then assign default weights. In other words, autonomous vehicles prioritize safety first, followed by efficiency and collaboration; Step S1-5: Determine if the maximum number of iterations has been reached. If it has, proceed to step S1-9; otherwise, proceed to step S1-6.

[0010] As a preferred embodiment of the present invention, step S1 further includes the following step: Steps S1-6: Calculate the fitness of all particles; Design a fitness function that balances situational fit with short-term gains. The calculation formula is: Formula 12: ; In the formula: ① Situational matching degree The weights represent the degree of matching between the current traffic situation and the current traffic conditions. This is calculated by combining the situation characteristics with the weights. Formula 13: ; In the formula, Prioritizing efficiency Prioritize collaboration. Prioritize safety; ② Short-term gains The weighted average of the short-term rewards obtained by the autonomous driving agent through reinforcement learning after interacting with the environment represents the effectiveness of the weighted average in a real traffic environment. Formula 14: ; In the formula, For short-term reward sample size, For the first The multi-objective reward vector of each sample. Transpose of the scenario requirement weight vector for each time step; ③ Weights for situational matching degree; Steps S1-7: Update the individual optimal solution and the global optimal solution for each particle; The iterative update of particles includes two stages: velocity update and position update. The inertial weight is dynamically adjusted to balance global exploration and local utilization capabilities, employing a linear decreasing approach. In the early stages of iteration, a large weight is used to enhance global search, while in the later stages, the inertial weight is gradually reduced to enhance local utilization. The calculation formula is as follows: Formula 15: ; In the formula, For the first Inertia weights in the next iteration This represents the current iteration number. The maximum number of iterations, and These represent the minimum and maximum values ​​of the inertia weight, respectively. Step S1-8: Update particle velocity and position and return to step S1-5.

[0011] As a preferred embodiment of the present invention, step S1 further includes the following step: Steps S1-9: Smooth transition of scenario requirement weights; A weight smoothing strategy is designed based on L1 distance, that is, using Manhattan distance. Design a weight smoothing strategy: Formula 16: ; In the formula, The weighting of the scenario requirements for the final adoption of the vehicle The globally optimal weights obtained by the PSO algorithm. The weights output at the previous time step; Step S1-10: Output the scene requirement weight vector at the current moment. And save the current situation vector and optimal weights to the situation and weight cache pool. .

[0012] As a preferred embodiment of the present invention, step S4 further includes the following step: Introducing adversarial agents to model disturbance factors under different traffic scenarios, and outputting the optimal observed disturbance. and optimal motion perturbation The goal of an autonomous driving agent is to maximize expected utility in an adversarial environment, while the goal of an adversarial agent is to minimize expected utility. Therefore, a two-person zero-sum Markov game is used to model the interaction between the two agents. A Nash equilibrium solution exists in the two-person zero-sum Markov game. Formula 20: ; In the formula, It is the value function of the optimal equilibrium state in the game. and These represent the optimal strategies of the autonomous driving agent and the adversarial agent when satisfying Nash equilibrium, respectively. To maximize the operator, the autonomous driving agent selects the optimal strategy with the goal of maximizing its long-term cumulative utility. To minimize the operator, representing the adversarial agent choosing the optimal adversarial strategy with the objective of minimizing the utility of the autonomous driving agent, For the maximum time step, For strategic utility, for Real-time traffic conditions , Represent At any time, autonomous driving agents and adversarial agents follow the optimal strategy. and The actions taken for At time 1, the mixed preference vectors satisfy Nash equilibrium, and neither party can increase their respective utility by unilaterally changing their strategies.

[0013] As a preferred embodiment of the present invention, step S4 further includes the following step: Construct the CMOR-AC reinforcement learning algorithm; Step S4-1: Setting the optimization objectives and constraints for the CMOR-AC reinforcement learning algorithm; (1) Optimization objective: The CMOR-AC reinforcement learning algorithm solves constrained optimization problems. Equation 21: ; ; ; In the formula, , These represent the preference constraint threshold and the policy deviation constraint threshold, respectively. In its original state, Given a mixed preference vector, the policy utility function is defined as: Equation 22: ; Equation 23: ; In the formula, For a multi-objective action value function vector, To account for the mixed actions of adversarial agents and autonomous driving agents, The original actions output by the autonomous driving agent policy network. For the action perturbation weights, It is the transpose symbol; (2) Constraints: ① Prefer nonlinear constraints Through the mixed preference vector With multi-objective action value function vector The modulo representation of the cross product ensures the diversity of learned decisions and achieves efficient matching between Pareto robust optimal policies and mixed preference spaces. Its expression is: Formula 24: ; In the formula, for and The angle between them The larger the value, the more the strategy deviates from the mixed preference direction, and the worse the strategy is. ②Strategy bias nonlinear constraint The JS divergence is used to represent the difference between two probability distributions. Here, it represents the policy bias of an autonomous driving agent after being disturbed by observations. The larger the JS divergence, the worse the policy robustness, and vice versa. Its expression is: Formula 25: ; Equation 26: ; Equation 27: ; In the formula, Let JS divergence be the metric. Let KL divergence be the KL divergence. Distribution of actions in the real state Distribution of actions in disturbance state The average mixed distribution, To observe the perturbation weights, In its original state, This is a disturbance state. For autonomous driving agents based on disturbance states The perturbation action performed.

[0014] As a preferred embodiment of the present invention, step S4 further includes the following step: Step S4-2: Design the state space; Select the European distance between the vehicle and the destination. ,speed acceleration Yaw angle Current lane index And, taking the vehicle as the center, the speeds and Euclidean distances of the six closest HDVs within a 200-meter radius of the vehicle itself. The state space of autonomous driving decision-making agents (both horizontal and vertical) and adversarial agents; Step S4-3: Design the motion space; The autonomous driving lateral decision-making agent outputs discrete lane-changing actions. This includes left lane changing, lane keeping, and right lane changing; the autonomous driving longitudinal decision-making agent outputs the vehicle's continuous longitudinal acceleration. ; Adversarial agent outputs optimal observation perturbation and optimal motion perturbation ; Step S4-4: Design the reward function; To address the three objectives of efficiency, collaboration, and safety, competitive or conflicting reward function vectors are designed, while simultaneously considering comfort constraints, trip completion rate, and collision penalties. When the autonomous driving agent engages in rapid acceleration, rapid deceleration, or a collision occurs, a penalty term is added to each sub-objective in its reward function vector; the specific settings are as follows: ①Efficiency sub-goal rewards : Formula 28: ; In the formula, For the vehicle's speed, and These represent the minimum and maximum speed limits for the road segment, respectively. ② Collaborative Sub-goal Rewards : The reward function for the collaborative sub-objective is designed as follows: Equation 29: ; In the formula, This represents the speed of vehicles following behind the vehicle before the vehicle performs a lane-changing or overtaking maneuver. This represents the speed of vehicles following behind a vehicle after the first vehicle performs a lane change or overtaking maneuver. This represents the expected speed of the following vehicle; when the speed difference between the following vehicles is positive, the autonomous vehicle is rewarded, guiding the autonomous driving agent to learn a collaborative driving strategy. ③ Safety sub-target reward : Formula 30: ; In the formula, This is the distance between the front of your vehicle and the vehicle in front of you in the current lane. This is a preset headway threshold; therefore, when the headway is less than... At that time, the vehicle will be penalized; only if the front distance of the vehicle is greater than [a certain value]. At that time, the vehicle will receive a reward; ④ Comfort constraints When the acceleration exceeds 4 m / s² 2 The jerk exceeds 1.5 m / s². 3 At times, this will affect driving comfort; Equation 31: ; Equation 32: ; Equation 33: ; In the formula, For acceleration comfort penalty, As a penalty for aggression, For the vehicle's acceleration, This is the maximum acceleration of the vehicle. This represents the change in acceleration between two adjacent time steps; ⑤ Trip completion rate The design endpoint-driven rewards are as follows: Equation 34: ; In the formula, The distance between the vehicle and the destination is expressed in European terms. This represents the Euclidean distance between the starting and ending points of the vehicle. ⑥ Collision penalty If the vehicle is involved in a collision, then ; Finally, a vectorized reward function considering three objectives—efficiency, collaboration, and security—is constructed. : Formula 35: ; In the formula, , , These represent the efficiency reward component, the collaboration reward component, and the security reward component, respectively.

[0015] As a preferred embodiment of the present invention, step S4 further includes the following step: Step S4-5: Construct a neural network containing an input layer, hidden layers, and an output layer; The multi-objective Actor neural network and Critic neural network for autonomous driving horizontal and vertical decision-making agents, as well as the adversarial agent Actor neural network, are all constructed using two fully connected hidden layers, with 256 and 128 neurons in each layer, respectively; all hidden layers use the ReLU activation function. The input to the Actor neural network, an autonomous driving lateral decision-making agent, is the perturbation state and a mixed preference vector; the output is a 3D lateral lane-changing action. The input to the Critic neural network, also an autonomous driving lateral decision-making agent, is the perturbation state and a mixed preference vector; the output is... indivual Value, that is, the value corresponding to each lateral movement. Vector, in the formula Represents the lateral movement space dimension. The number of targets is represented by the input of the Actor neural network for autonomous driving longitudinal decision-making, which is the perturbation state and the mixed preference vector, and the output is the longitudinal acceleration of the vehicle. The input of the Critic neural network for autonomous driving longitudinal decision-making is the perturbation state, the longitudinal action, and the mixed preference vector, and the output is... indivual Value, that is, the value corresponding to the vertical movement. Vector; The input to the adversarial agent Actor neural network is the original state and the mixed preference vector, which are then processed by the tanh function to output the best observation perturbation and the best action perturbation at the current moment.

[0016] As a preferred embodiment of the present invention, step S4 further includes the following step: Steps S4-6: Adversarial Agent: Based on the state space output of the autonomous driving agent, the agent counters disturbances. Including the best observation perturbation and optimal motion perturbation : Equation 36: ; In the formula, To counter agent policy networks, Network parameters for adversarial agent policy; Step S4-7: The autonomous driving lateral decision-making agent makes decisions based on the hybrid preference vector. and the state after resisting agent perturbation Output lateral lane change action and longitudinal following motion ,in: Equation 37: ; Equation 38: ; In the formula, , These represent the policy networks for the lateral decision-making agents and the longitudinal decision-making agents in autonomous driving, respectively. , These are the parameters of the Actor neural network for the horizontal and vertical decision-making intelligent agents in autonomous driving. As the weight of the motion disturbance; Step S4-8: The vehicle performs a lateral lane change maneuver in response to the disturbance. and longitudinal following motion Traffic conditions Based on the state transition, proceed to the next traffic state. And generate the current time step reward. Subsequently, the autonomous driving decision-making agents store the experience data into their respective experience replay pools. The state transition equation is as follows: Formula 39: ; In the formula, This is the state transition function. This indicates whether the round has ended. The round termination conditions are: ① The autonomous vehicle collides with another vehicle; ② The maximum number of simulation steps is reached; ③ The autonomous vehicle safely leaves the road segment. When the termination conditions are met... The value is 1; otherwise, =0; Step S4-9: In obtaining After obtaining a set of empirical data, samples are taken from the empirical replay pool. The adversarial agent policy network and the autonomous driving horizontal and vertical decision-making agent network are updated using alternating training methods; then the dual variables and the target network are updated. The goal of the adversarial agent is to minimize the expected utility of the autonomous driving agent and amplify policy bias and preference bias. This is achieved by updating the adversarial network by minimizing the loss function. Formula 40: ; In the formula, To address the network loss function of the adversarial agent policy, and As dual variables; The goal of an autonomous driving agent is to maximize expected utility while minimizing nonlinear constraints. The optimal policy is solved and updated by maximizing the objective function of the Actor neural network. Equation 41: ; In the formula, For the policy network of autonomous driving decision-making agents in both the horizontal and vertical directions; The dual variable is updated by minimizing the loss function: Equation 42: ; In the formula, The loss function is the dual variable. After completing the construction of the hybrid preference vector and the output of the driving strategy for the current moment, the vehicle control system enters the next moment and makes adaptive and robust driving actions according to the new traffic scenario.

[0017] The beneficial effects of this invention are as follows: 1. This invention introduces adversarial agents to model observation and action disturbances in stochastic dynamic traffic scenarios. The interaction model between the autonomous driving agent and the adversarial agent is designed as a two-person zero-sum Markov game. The model trained by the CMOR-AC reinforcement learning algorithm can minimize policy bias in adversarial uncertainty environments, effectively cope with various types of disturbance factors such as sensor noise and environmental changes, avoid the vehicle from making unwanted or dangerous driving decisions, and significantly improve the decision robustness of autonomous vehicles in complex traffic scenarios.

[0018] 2. This invention constructs a traffic situation-driven scenario demand weight optimization framework based on the PSO algorithm. It can dynamically adjust the weights of efficiency, cooperation, and safety sub-objectives according to real-time situational characteristics such as traffic pressure, smoothness, and conflict risk. It also integrates user personalized preferences to construct a hybrid preference space, overcoming the limitations of traditional static weighting methods. This enables autonomous vehicles to dynamically weigh multiple competing or conflicting objectives according to time-varying traffic scenarios and user needs, achieving adaptive and personalized human-like decision control.

[0019] 3. The CMOR-AC algorithm proposed in this invention can cover Pareto robust optimal policies in all possible mixed preference spaces in a discrete-continuous mixed action space with a single model. It does not require retraining the model for different traffic scenarios or user preferences, effectively solving the problems of poor generalization and low computational efficiency of traditional multi-objective reinforcement learning methods. At the same time, it ensures efficient matching between policy and preference through nonlinear constraints, and can find Pareto optimal solutions on non-convex surfaces, adapting to complex and ever-changing traffic scenarios. Attached Figure Description

[0020] Figure 1 A schematic diagram of the adaptive robust decision control method framework for autonomous vehicles constructed in this invention; Figure 2 This is a schematic diagram of the scene requirement weight adaptive adjustment framework based on the PSO algorithm constructed in this invention. Figure 3 A schematic diagram of the state space of an autonomous vehicle designed for this invention; Figure 4 This is a schematic diagram illustrating the perturbation process of the adversarial agent during adversarial training according to the present invention. Detailed Implementation

[0021] This embodiment provides an adaptive decision control method for autonomous vehicles that considers disturbance factors, comprising the following steps: Step S1: Based on the current traffic flow situation around the vehicle, output the scene demand weight vector using the PSO (Particle Swarm Optimization) algorithm. : Formula 1: ; Formula 2: ; In the formula, , , These are weights for scenario requirements, efficiency, collaboration, and security, with their values ​​all constrained within a certain range. ; This embodiment uses a scenario demand weight optimization framework based on the Particle Swarm Optimization (PSO) algorithm. Its core is to guide the iterative optimization process of the PSO algorithm through traffic situation assessment, achieving adaptive adjustment and smooth output of weights. Figure 2 As shown, the PSO algorithm mainly includes traffic situation feature extraction, historical weight reuse judgment, PSO weight iterative optimization, weight smooth transition, and situation and weight cache update; specifically, it includes the following steps: Step S1-1: In At any given time, initialize an empty state and weight cache pool. To reduce the computational cost of models under similar traffic conditions, this embodiment constructs a model with a capacity of [missing information]. Status and weighted cache pool This is used to store matching pairs of historical states and their corresponding optimal weights. Formula 3: ; In the formula, Representing the A set of historical traffic situation feature vectors, This represents the optimal weight combination of efficiency, cooperation, and security objectives obtained through the PSO algorithm under the corresponding situation; when the situation and weight cache pool Reaching maximum capacity At that time, the first-in-first-out (FIFO) strategy is used to eliminate the earliest stored position and weight pair; Step S1-2: Construct the traffic situation feature vector at the current moment ; Traffic situation feature vectors are the core basis for guiding weight optimization. In this embodiment, four core features are extracted from the vehicle observation spatial data to construct the traffic situation feature vector at the current moment. All core features The range of values ​​is constrained to This ensures that all core features have a uniform scale, preventing any single feature from dominating the entire iterative optimization process due to excessive scale differences. The physical meaning and calculation method are as follows: ① Traffic pressure characteristics The distance is represented by the mean of the Euclidean distance between the vehicle and other vehicles within the observation range. Formula 4: ; In the formula, This represents the number of other vehicles within the observation range. Representing the The European-style distance between the car and the driver. Represents the vehicle's safety threshold. The larger the size, the more space the vehicle has to move freely, and the less traffic pressure there is. The smaller the size, the less space the vehicle has to move freely, and the greater the traffic pressure. When there are no other vehicles within the observation range, it leads to... =1; ② Traffic flow characteristics The speed is represented by the average relative speed between the vehicle and other vehicles within the observation range. Formula 5: ; Formula 6: ; In the formula, This represents the average relative speed between the observed vehicle and other vehicles within the observation range. Represents the first within the observation range The speed of the car For the vehicle's speed, when When the value is positive, vehicles should increase their speed to meet the overall traffic flow efficiency requirements. When the value is negative, the weight of the vehicle in the efficiency index should be appropriately reduced. When there are no other vehicles within the observation range, let... =1; ③ Conflict risk characteristics The distance between the vehicle and other vehicles within the observation range is expressed as the minimum Euclidean distance. Formula 7: ; The larger the [velocity], the higher the risk of conflict during lane changing / lane following, especially when there are no other vehicles within the observation range. =0; ④ Characteristics of behavioral influence To prevent vehicles from pursuing excessive efficiency and engaging in aggressive driving behaviors that could decrease traffic flow safety, this embodiment uses the speed changes of following vehicles to represent the impact of a vehicle's driving behavior on traffic flow. This is because following vehicles are most easily affected by a vehicle's lane changes or overtaking maneuvers, and the impact of a vehicle's driving behavior on traffic flow is primarily transmitted through these following vehicles. The expression is as follows: Formula 8: ; In the formula, This represents the speed difference between two adjacent time steps of a vehicle following behind the first vehicle. The larger the value, the greater the impact of the driver's behavior on traffic flow. More attention should be paid to cooperation with other vehicles. When there are no following vehicles within the observation range, [the driver should]... It is 0.

[0022] Steps S1-3: Determine the similarity between the current situation and the historical situation, retrieve the historical situation and the weight cache pool, and if the current traffic situation feature vector... Historical traffic situation feature vector The L1 distance of the core feature value is less than the threshold, i.e., the Manhattan distance. If the value is less than the threshold, it is determined to be a similar situation, and the historical best weight is directly reused to reduce computational overhead, proceeding to step S1-10. If no similar situation exists, proceed to step S1-4, where: Formula 9: ; In the formula, The first feature vector of the traffic situation at the current moment. One portion, The first of the historical traffic situation feature vectors One component; Step S1-4: Initialize the particle swarm based on the characteristic distribution of the current situation and set constraints; To improve the optimization efficiency of the PSO algorithm and avoid meaningless global searches, traffic pressure features are used as a basis for optimization. Three traffic scenarios were divided into three categories, and a differentiated weight initialization range was set for each category. The weight initialization range under different traffic pressures is shown in Table 1.

[0023] Table 1 shows the weight initialization range under different traffic pressures:

[0024] Traffic pressure characteristic range Traffic pressure level Efficiency weight Collaboration weight Safety weight Traffic pressure characteristics <0.2 High pressure (0.05, 0.2) (0.3, 0.5) (0.4, 0.6) 0.2 ≤ Traffic pressure characteristic < 0.5 moderate pressure (0.3, 0.5) (0.3, 0.4) (0.2, 0.4) Traffic pressure characteristics ≥ 0.5 low pressure (0.5, 0.7) (0.1, 0.2) (0.2, 0.3) During the initialization process, unnormalized weight values ​​are first randomly generated within the corresponding range, and then the weights are summed to 1 through particle constraints to ensure that the distribution of the initial particle swarm matches the current traffic situation. The particle's dimension corresponds to the three target weights to be optimized; that is, a single particle is encoded as... To ensure the physical meaning of the weights, the following constraints must be satisfied: Formula 10:

[0025] The constraint processing method is as follows: the result after updating the particle position is first taken as non-negative, and then normalized. Formula 11: ; In the formula, To determine the final weights that satisfy the constraints after correction, if the sum of the weights before normalization is less than 10... -6 Then assign default weights. In other words, autonomous vehicles prioritize safety first, followed by efficiency and collaboration; Step S1-5: Determine if the maximum number of iterations has been reached. If it has, proceed to step S1-9; otherwise, proceed to step S1-6.

[0026] Steps S1-6: Calculate the fitness of all particles; The fitness function is the core criterion for optimization in the PSO algorithm. This embodiment designs a fitness function that balances situational matching and short-term gains. The calculation formula is: Formula 12: ; In the formula: ① Situational matching degree The weights represent the degree of matching between the current traffic situation and the current traffic conditions. This is calculated by combining the situation characteristics with the weights. Formula 13: ; In the formula, Prioritizing efficiency Prioritize collaboration. Prioritize safety; ② Short-term gains The weighted average of the short-term rewards obtained by the autonomous driving agent through reinforcement learning after interacting with the environment represents the effectiveness of the weighted average in a real traffic environment. Formula 14: ; In the formula, For short-term reward sample size, For the first The multi-objective reward vector of each sample. Transpose of the scenario requirement weight vector for each time step; ③ As for the situational matching weight, priority is given to ensuring the matching between the weight and the traffic situation, while also taking into account the benefits of the actual environment. Steps S1-7: Update the individual optimal solution and the global optimal solution for each particle; The iterative update of particles includes two stages: velocity update and position update. The core principle is to dynamically adjust the inertial weight to balance global exploration and local utilization capabilities. This embodiment employs a linear decreasing method, using a large weight to enhance global search in the early stages of iteration and gradually decreasing the inertial weight to enhance local utilization in the later stages. The calculation formula is as follows: Formula 15: ; In the formula, For the first Inertia weights in the next iteration This represents the current iteration number. The maximum number of iterations, and These represent the minimum and maximum values ​​of the inertia weight, respectively. Step S1-8: Update particle velocity and position and return to step S1-5.

[0027] Steps S1-9: Smooth transition of scenario requirement weights; To avoid instability in the vehicle control system caused by sudden weight changes, a weight smoothing strategy is designed based on L1 distance, specifically using Manhattan distance. Design a weight smoothing strategy: Formula 16: ; In the formula, The weighting of the scenario requirements for the final adoption of the vehicle The globally optimal weights obtained by the PSO algorithm. The weights output at the previous time step; Step S1-10: Output the scene requirement weight vector at the current moment. And save the current situation vector and optimal weights to the situation and weight cache pool. .

[0028] Step S2: Collect the user preference vector at the current moment. : Formula 17: ; Formula 18: ; In the formula, , , These are user-defined efficiency weights, collaboration weights, and security weights, with their value ranges constrained within a certain range. If the user does not specify a preference, a default value will be set. That is, safety should be the top priority, followed by efficiency and collaboration.

[0029] Step S3: Construct the mixture preference vector at the current time step ,in Based on user preference vectors With the scene requirement weight vector Weighted sum design: Formula 19: ; In the formula, These are preference weight coefficients, used to balance user preferences with the weights of scenario requirements, forming a hybrid preference vector. This will serve as the basis for decision-making in subsequent CMOR-AC reinforcement learning algorithms that drive autonomous driving agents.

[0030] Step S4: Based on the CMOR-AC reinforcement learning algorithm (Constrained Multi-Objective Robust Actor-Critic), output adaptive robust driving actions for autonomous vehicles; In this embodiment, an adversarial agent is introduced to model disturbance factors under different traffic scenarios and output the optimal observed disturbance. and optimal motion perturbation The goal of an autonomous driving agent is to maximize expected utility in an adversarial environment, while the goal of an adversarial agent is to minimize expected utility. Therefore, a two-player zero-sum Markov game is used to model the interaction between the two. Nash equilibrium is a method for defining the solution of a non-cooperative game with two or more participants. A two-player zero-sum Markov game has a Nash equilibrium solution when the following relationship is satisfied: Formula 20: ; In the formula, It is the value function of the optimal equilibrium state in the game. and These represent the optimal strategies of the autonomous driving agent and the adversarial agent when satisfying Nash equilibrium, respectively. To maximize the operator, the autonomous driving agent selects the optimal strategy with the goal of maximizing its long-term cumulative utility. To minimize the operator, representing the adversarial agent choosing the optimal adversarial strategy with the objective of minimizing the utility of the autonomous driving agent, For the maximum time step, For strategic utility, for Real-time traffic conditions , Represent At any time, autonomous driving agents and adversarial agents follow the optimal strategy. and The actions taken for At time 1, the mixed preference vectors satisfy Nash equilibrium, and neither party can increase their respective utility by unilaterally changing their strategies.

[0031] Furthermore, to solve the Nash equilibrium solution in the aforementioned zero-sum game, this embodiment constructs the CMOR-AC reinforcement learning algorithm. Under the premise of satisfying nonlinear constraints, it aims to cover the Pareto robust optimal policy under all possible mixed preference spaces using a single model. By alternately training the adversarial agent and the autonomous driving agent, it optimizes the policies of both sides, thereby efficiently approximating the Nash equilibrium solution. The specific steps are as follows: Step S4-1: Setting the optimization objectives and constraints for the CMOR-AC reinforcement learning algorithm; (1) Optimization objective: The CMOR-AC reinforcement learning algorithm aims to solve the following constrained optimization problem: Equation 21: ; ; ; In the formula, , These represent the preference constraint threshold and the policy deviation constraint threshold, respectively. In its original state, Given a mixed preference vector, the policy utility function is defined as: Equation 22: ; Equation 23: ; In the formula, For a multi-objective action value function vector, To account for the mixed actions of adversarial agents and autonomous driving agents, The original actions output by the autonomous driving agent policy network. For the action perturbation weights, It is the transpose symbol; (2) Constraints: ① Prefer nonlinear constraints Through the mixed preference vector With multi-objective action value function vector The modulo representation of the cross product ensures the diversity of learned decisions and achieves efficient matching between Pareto robust optimal policies and mixed preference spaces. Its expression is: Formula 24: ; In the formula, for and The angle between them The larger the value, the more the strategy deviates from the mixed preference direction, and the worse the strategy is. ②Strategy bias nonlinear constraint The JS (Jensen–Shannon) divergence is used to measure the difference between two probability distributions. Here, it represents the policy bias of an autonomous driving agent after being disturbed by observations. The larger the JS divergence, the worse the policy robustness, and vice versa. Its expression is: Formula 25: ; Equation 26: ; Equation 27: ; In the formula, Let JS divergence be the metric. Let KL (Kullback–Leibler) divergence be the divergence. Distribution of actions in the real state Distribution of actions in disturbance state The average mixed distribution, To observe the perturbation weights, In its original state, This is a disturbance state. For autonomous driving agents based on disturbance states The perturbation action performed.

[0032] Step S4-2: Design the state space; like Figure 3 As shown, select the Euclidean distance between the vehicle and the destination. ,speed acceleration Yaw angle Current lane index And, taking the vehicle as the center, the speeds and Euclidean distances of the six closest HDVs (Human-Driven Vehicles) within a 200-meter radius of the vehicle itself. The state space of autonomous driving decision-making agents (both horizontal and vertical) and adversarial agents; Step S4-3: Design the motion space; The autonomous driving lateral decision-making agent outputs discrete lane-changing actions. This includes left lane changing, lane keeping, and right lane changing; the autonomous driving longitudinal decision-making agent outputs the vehicle's continuous longitudinal acceleration. ; Adversarial agent outputs optimal observation perturbation and optimal motion perturbation ; Step S4-4: Design the reward function; The reward function is a core element in reinforcement learning that evaluates the quality of an agent's behavior and guides the agent to learn the optimal strategy. This embodiment designs competing or conflicting reward function vectors for three objectives: efficiency, cooperation, and safety. It also considers comfort constraints, trip completion rate, and collision penalties. When the autonomous driving agent performs rapid acceleration or deceleration, or a collision occurs, a penalty term is added to each sub-objective in its reward function vector. The specific settings are as follows: ①Efficiency sub-goal rewards : Formula 28: ; In the formula, For the vehicle's speed, and These represent the minimum and maximum speed limits for the road segment, respectively. ② Collaborative Sub-goal Rewards : To prevent vehicles from excessively pursuing their own efficiency and engaging in aggressive driving behaviors, which could lead to a decline in overall traffic flow efficiency and safety, a cooperative sub-objective reward function is designed as follows: Equation 29: ; In the formula, This represents the speed of vehicles following behind the vehicle before the vehicle performs a lane-changing or overtaking maneuver. This represents the speed of vehicles following behind a vehicle after the first vehicle performs a lane change or overtaking maneuver. The expected speed of the following vehicle is represented by the cooperative sub-objective reward function designed in this embodiment to encourage the autonomous vehicle to take normal driving behavior to promote the efficiency of the entire traffic flow. When the speed difference between the following vehicles is positive, the autonomous vehicle is rewarded, thereby guiding the autonomous driving agent to learn a cooperative driving strategy. ③ Safety sub-target reward : Formula 30: ; In the formula, This is the distance between the front of your vehicle and the vehicle in front of you in the current lane. This is a preset headway threshold; therefore, when the headway is less than... At that time, the vehicle will be penalized; only if the front distance of the vehicle is greater than [a certain value]. The vehicle will only receive a reward at that time; this embodiment will Set to 1.2 seconds; ④ Comfort constraints When the acceleration exceeds 4 m / s² 2 The jerk exceeds 1.5 m / s². 3 At times, this will affect driving comfort; Equation 31: ; Equation 32: ; Equation 33: ; In the formula, For acceleration comfort penalty, As a penalty for aggression, For the vehicle's acceleration, This is the maximum acceleration of the vehicle. This represents the change in acceleration between two adjacent time steps; ⑤ Trip completion rate To guide the vehicle in completing a long-term driving task, the following endpoint-driven rewards are designed: Equation 34: ; In the formula, The distance between the vehicle and the destination is expressed in European terms. This represents the Euclidean distance between the starting and ending points of the vehicle. ⑥ Collision penalty If the vehicle is involved in a collision, then ; Finally, a vectorized reward function considering three objectives—efficiency, collaboration, and security—is constructed. : Formula 35: ; In the formula, , , These represent the efficiency reward component, the collaboration reward component, and the security reward component, respectively.

[0033] Step S4-5: Construct a neural network containing an input layer, hidden layers, and an output layer; The multi-objective Actor neural network and Critic neural network for autonomous driving horizontal and vertical decision-making intelligent agents, as well as the adversarial intelligent agent Actor neural network, are all constructed using two fully connected hidden layers, with 256 and 128 neurons in each layer, respectively; all hidden layers use the ReLU activation function. The input to the Actor neural network, an autonomous driving lateral decision-making agent, is the perturbation state and a mixed preference vector; the output is a 3D lateral lane-changing action. The input to the Critic neural network, also an autonomous driving lateral decision-making agent, is the perturbation state and a mixed preference vector; the output is... indivual Value, that is, the value corresponding to each lateral movement. Vector, in the formula Represents the lateral movement space dimension. The number of targets is represented by the input of the Actor neural network for autonomous driving longitudinal decision-making, which is the perturbation state and the mixed preference vector, and the output is the longitudinal acceleration of the vehicle. The input of the Critic neural network for autonomous driving longitudinal decision-making is the perturbation state, the longitudinal action, and the mixed preference vector, and the output is... indivual Value, that is, the value corresponding to the vertical movement. Vector; The input to the adversarial agent Actor neural network is the original state and the mixed preference vector, which is passed through the tanh function (hyperbolic tangent function) to output the best observation perturbation and the best action perturbation at the current time.

[0034] Steps S4-6: Adversarial Agent: Based on the state space output of the autonomous driving agent, the agent counters disturbances. Including the best observation perturbation and optimal motion perturbation : Equation 36: ; In the formula, To counter agent policy networks, Network parameters for adversarial agent policy; like Figure 4 As shown, in the adversarial training process, the adversarial agent simultaneously interferes with the magnitude of the acceleration of both the state input and output of the autonomous driving longitudinal decision agent, while for the autonomous driving lateral decision agent, it only affects the state input of the autonomous driving lateral decision agent. Step S4-7: The autonomous driving lateral decision-making agent bases its decisions on the hybrid preference vector (hybrid weights). and the state after resisting agent perturbation Output lateral lane change action and longitudinal following motion ,in: Equation 37: ; Equation 38: ; In the formula, , These represent the policy networks for the lateral decision-making agents and the longitudinal decision-making agents in autonomous driving, respectively. , These are the parameters of the Actor neural network for the horizontal and vertical decision-making intelligent agents in autonomous driving. As the weight of the motion disturbance; Step S4-8: The vehicle performs a lateral lane change maneuver in response to the disturbance. and longitudinal following motion Traffic conditions (environment) Based on the state transition, proceed to the next traffic state. And generate the current time step reward. Subsequently, the autonomous driving decision-making agents store the experience data into their respective experience replay pools. The state transition equation is as follows: Formula 39: ; In the formula, This is the state transition function. This indicates whether the round has ended. The round termination conditions are: ① The autonomous vehicle collides with another vehicle; ② The maximum number of simulation steps is reached; ③ The autonomous vehicle safely leaves the road segment. When the termination conditions are met... The value is 1; otherwise, =0; Step S4-9: In obtaining After obtaining a set of empirical data, samples are taken from the empirical replay pool. The adversarial agent policy network and the autonomous driving horizontal and vertical decision-making agent network are updated using alternating training methods; then the dual variables and the target network are updated. The goal of the adversarial agent is to minimize the expected utility of the autonomous driving agent and amplify policy bias and preference bias. This is achieved by updating the adversarial network by minimizing the loss function. Formula 40: ; In the formula, To address the network loss function of the adversarial agent policy, and As dual variables; The goal of an autonomous driving agent is to maximize expected utility while minimizing nonlinear constraints. The optimal policy is solved and updated by maximizing the objective function of the Actor neural network. Equation 41: ; In the formula, For the policy network of autonomous driving decision-making agents in both the horizontal and vertical directions; The dual variable is updated by minimizing the loss function: Equation 42: ; In the formula, The loss function is the dual variable. After constructing the mixed preference vector (mixed weights) and outputting the driving strategy for the current moment, the autonomous vehicle control system moves to the next moment and makes adaptive robust driving actions (adaptive robust decision-making) based on the new traffic scenario. Specifically, in each time step, steps S1 to S4 are repeated, sequentially performing scenario demand weight output, user preference collection, mixed preference weight construction, adversarial disturbance environment modeling, robust strategy output, and parameter adjustment and update, until the model converges.

Claims

1. An adaptive decision-making control method for autonomous vehicles that considers disturbance factors, characterized in that, Includes the following steps: Step S1: Based on the current traffic flow situation around the vehicle, output the scene demand weight vector using the PSO algorithm. : Formula 1: ; Formula 2: ; In the formula, , , These are weights for scenario requirements, efficiency, collaboration, and security, with their values ​​all constrained within a certain range. ; Step S2: Collect the user preference vector at the current moment. : Equation 17: ; Formula 18: ; In the formula, , , These are user-defined efficiency weights, collaboration weights, and security weights, with their value ranges constrained within a certain range. If the user does not specify a preference, a default value will be set. That is, safety should be the top priority, followed by efficiency and collaboration; Step S3: Construct the mixture preference vector at the current time step ,in Based on user preference vectors With the scene requirement weight vector Weighted sum design: Formula 19: ; In the formula, These are preference weight coefficients, used to balance user preferences with the weights of scenario requirements, forming a hybrid preference vector. This will serve as the basis for decision-making in subsequent CMOR-AC reinforcement learning algorithms that drive autonomous driving agents. Step S4: Based on the CMOR-AC reinforcement learning algorithm, output the adaptive and robust driving actions of the autonomous vehicle.

2. The adaptive decision control method for autonomous vehicles considering disturbance factors according to claim 1, characterized in that, Step S1 also includes the following steps: Step S1-1: In At any given time, initialize an empty state and weight cache pool. Build capacity is Status and weighted cache pool This is used to store matching pairs of historical states and their corresponding optimal weights. Formula 3: ; In the formula, Representing the A set of historical traffic situation feature vectors, This represents the optimal weight combination of efficiency, cooperation, and security objectives obtained through the PSO algorithm under the corresponding situation; when the situation and weight cache pool Reaching maximum capacity At that time, the first-in-first-out (FIFO) strategy is used to eliminate the earliest-entered position and weight pair; Step S1-2: Construct the traffic situation feature vector at the current moment ; A traffic situation feature vector for the current moment is constructed by extracting four core features from the vehicle's observation space data. All core feature values ​​are constrained within a certain range. Ensure that all core features are scaled uniformly; ① Traffic pressure characteristics The distance is represented by the mean of the Euclidean distance between the vehicle and other vehicles within the observation range. Formula 4: ; In the formula, This represents the number of other vehicles within the observation range. Representing the The European-style distance between the car and the driver. Represents the vehicle's safety threshold. The larger the size, the more space the vehicle has to move freely, and the less traffic pressure there is. The smaller the size, the less space the vehicle has to move freely, and the greater the traffic pressure. When there are no other vehicles within the observation range, it leads to... =1; ② Traffic flow characteristics The speed is represented by the average relative speed between the vehicle and other vehicles within the observation range. Formula 5: ; Formula 6: ; In the formula, This represents the average relative speed between the observed vehicle and other vehicles within the observation range. Represents the first within the observation range The speed of the car For the vehicle's speed, when When the value is positive, vehicles should increase their speed to meet the overall traffic flow efficiency requirements. When the value is negative, the weight of the vehicle in the efficiency index should be reduced. When there are no other vehicles within the observation range, let... =1; ③ Characteristics of conflict risk The distance between the vehicle and other vehicles within the observation range is expressed as the minimum Euclidean distance. Formula 7: ; The larger the value, the higher the risk of conflict during lane changing / lane following, especially when there are no other vehicles within the observation range. =0; ④ Behavioral Influence Characteristics The impact of a driver's behavior on traffic flow is represented by the speed changes of vehicles following behind the vehicle. The expression is as follows: Formula 8: ; In the formula, This represents the speed difference between two adjacent time steps of a vehicle following behind the first vehicle. The larger the value, the greater the impact of the driver's behavior on traffic flow. More attention should be paid to cooperation with other vehicles. When there are no following vehicles within the observation range, [the driver should]... It is 0.

3. The adaptive decision control method for autonomous vehicles considering disturbance factors according to claim 2, characterized in that, Step S1 also includes the following steps: Steps S1-3: Determine the similarity between the current situation and the historical situation, retrieve the historical situation and the weight cache pool, and if the current traffic situation feature vector... Historical traffic situation feature vector The L1 distance of the core feature value is less than the threshold, i.e., the Manhattan distance. If the value is less than the threshold, it is determined to be a similar situation, and the historical best weight is directly reused to reduce computational overhead, proceeding to step S1-10. If no similar situation exists, proceed to step S1-4, where: Formula 9: ; In the formula, The first feature vector of the traffic situation at the current moment. One portion, The first of the historical traffic situation feature vectors One component; Step S1-4: Initialize the particle swarm based on the characteristic distribution of the current situation and set constraints; Based on traffic pressure characteristics The traffic scenarios are divided into three categories, and a differentiated weight initialization range is set for each category. During the initialization process, unnormalized weight values ​​are first randomly generated within the corresponding range, and then the weights are summed to 1 through particle constraints to ensure that the distribution of the initial particle swarm matches the current traffic situation. The particle's dimension corresponds to the three target weights to be optimized; that is, a single particle is encoded as... To ensure the physical meaning of the weights, the following constraints must be satisfied: Formula 10: ; The constraint processing method is as follows: the result after updating the particle position is first taken as non-negative, and then normalized. Formula 11: ; In the formula, To determine the final weights that satisfy the constraints after correction, if the sum of the weights before normalization is less than 10... -6 Then assign default weights. In other words, autonomous vehicles prioritize safety first, followed by efficiency and collaboration; Step S1-5: Determine if the maximum number of iterations has been reached. If it has, proceed to step S1-9; otherwise, proceed to step S1-6.

4. The adaptive decision control method for autonomous vehicles considering disturbance factors according to claim 3, characterized in that, Step S1 also includes the following steps: Steps S1-6: Calculate the fitness of all particles; Design a fitness function that balances situational fit with short-term gains. The calculation formula is: Formula 12: ; In the formula: ① Situational matching degree The weights represent the degree of matching between the current traffic situation and the current traffic conditions. This is calculated by combining the situation characteristics with the weights. Formula 13: ; In the formula, Prioritizing efficiency Prioritize collaboration. Prioritize safety; ② Short-term gains The weighted average of the short-term rewards obtained by the autonomous driving agent through reinforcement learning after interacting with the environment represents the effectiveness of the weighted average in a real traffic environment. Formula 14: ; In the formula, For short-term reward sample size, For the first The multi-objective reward vector of each sample. Transpose of the scenario requirement weight vector for each time step; ③ Weights for situational matching degree; Steps S1-7: Update the individual optimal solution and the global optimal solution for each particle; The iterative update of particles includes two stages: velocity update and position update. The inertial weight is dynamically adjusted to balance global exploration and local utilization capabilities, employing a linear decreasing approach. In the early stages of iteration, a large weight is used to enhance global search, while in the later stages, the inertial weight is gradually reduced to enhance local utilization. The calculation formula is as follows: Formula 15: ; In the formula, For the first Inertia weights in the next iteration This represents the current iteration number. The maximum number of iterations, and These represent the minimum and maximum values ​​of the inertia weight, respectively. Step S1-8: Update particle velocity and position and return to step S1-5.

5. The adaptive decision control method for autonomous vehicles considering disturbance factors according to claim 4, characterized in that, Step S1 also includes the following steps: Steps S1-9: Smooth transition of scenario requirement weights; A weight smoothing strategy is designed based on L1 distance, that is, using Manhattan distance. Design a weight smoothing strategy: Formula 16: ; In the formula, The weighting of the scenario requirements for the final adoption of the vehicle The globally optimal weights obtained by the PSO algorithm. The weights output at the previous time step; Step S1-10: Output the scene requirement weight vector at the current moment. And save the current situation vector and optimal weights to the situation and weight cache pool. .

6. The adaptive decision control method for autonomous vehicles considering disturbance factors according to claim 1, characterized in that, Step S4 also includes the following steps: Introducing adversarial agents to model disturbance factors under different traffic scenarios, and outputting the optimal observed disturbance. and optimal motion perturbation The goal of an autonomous driving agent is to maximize expected utility in an adversarial environment, while the goal of an adversarial agent is to minimize expected utility. Therefore, a two-person zero-sum Markov game is used to model the interaction between the two agents. A Nash equilibrium solution exists in the two-person zero-sum Markov game. Formula 20: ； In the formula, It is the value function of the optimal equilibrium state in the game. and These represent the optimal strategies of the autonomous driving agent and the adversarial agent when satisfying Nash equilibrium, respectively. To maximize the operator, the autonomous driving agent selects the optimal strategy with the goal of maximizing its long-term cumulative utility. To minimize the operator, representing the adversarial agent choosing the optimal adversarial strategy with the objective of minimizing the utility of the autonomous driving agent, For the maximum time step, For strategic utility, for Real-time traffic conditions , Represent At any time, autonomous driving agents and adversarial agents follow the optimal strategy. and The actions taken for At time 1, the mixed preference vectors satisfy Nash equilibrium, and neither party can increase their respective utility by unilaterally changing their strategies.

7. The adaptive decision control method for autonomous vehicles considering disturbance factors according to claim 6, characterized in that, Step S4 also includes the following steps: Construct the CMOR-AC reinforcement learning algorithm; Step S4-1: Setting the optimization objectives and constraints for the CMOR-AC reinforcement learning algorithm; (1) Optimization objective: The CMOR-AC reinforcement learning algorithm solves constrained optimization problems. Equation 21: ; ； ； In the formula, , These represent the preference constraint threshold and the policy deviation constraint threshold, respectively. In its original state, Given a mixed preference vector, the policy utility function is defined as: Equation 22: ; Equation 23: ; In the formula, For a multi-objective action value function vector, To account for the mixed actions of adversarial agents and autonomous driving agents, The original actions output by the autonomous driving agent policy network. For the action perturbation weights, It is the transpose symbol; (2) Constraints: ① Prefer nonlinear constraints Through the mixed preference vector With multi-objective action value function vector The modulo representation of the cross product ensures the diversity of learned decisions and achieves efficient matching between Pareto robust optimal policies and mixed preference spaces. Its expression is: Formula 24: ； In the formula, for and The angle between them The larger the value, the more the strategy deviates from the mixed preference direction, and the worse the strategy is. ②Strategy bias nonlinear constraint The JS divergence is used to represent the difference between two probability distributions. Here, it represents the policy bias of an autonomous driving agent after being disturbed by observations. The larger the JS divergence, the worse the policy robustness, and vice versa. Its expression is: Formula 25: ； Equation 26: ; Equation 27: ; In the formula, Let JS divergence be the metric. Let KL divergence be the KL divergence. Distribution of actions in the real state Distribution of actions in disturbance state The average mixed distribution, To observe the perturbation weights, In its original state, It is in a perturbation state. For autonomous driving agents based on disturbance states The perturbation action performed.

8. The adaptive decision control method for autonomous vehicles considering disturbance factors according to claim 7, characterized in that, Step S4 also includes the following steps: Step S4-2: Design the state space; Select the European distance between the vehicle and the destination. ,speed acceleration Yaw angle Current lane index And, taking the vehicle as the center, the speeds and Euclidean distances of the six closest HDVs within a 200-meter radius of the vehicle itself. The state space of autonomous driving decision-making agents (both horizontal and vertical) and adversarial agents; Step S4-3: Design the motion space; The autonomous driving lateral decision-making agent outputs discrete lane-changing actions. This includes left lane changing, lane keeping, and right lane changing; the autonomous driving longitudinal decision-making agent outputs the vehicle's continuous longitudinal acceleration. ; Adversarial agent outputs optimal observation perturbation and optimal motion perturbation ; Step S4-4: Design the reward function; To address the three objectives of efficiency, collaboration, and safety, competitive or conflicting reward function vectors are designed, while simultaneously considering comfort constraints, trip completion rate, and collision penalties. When the autonomous driving agent engages in rapid acceleration, rapid deceleration, or a collision occurs, a penalty term is added to each sub-objective in its reward function vector; the specific settings are as follows: ①Efficiency sub-goal rewards : Formula 28: ; In the formula, For the vehicle's speed, and These represent the minimum and maximum speed limits for the road segment, respectively. ② Collaborative Sub-goal Rewards : The reward function for the collaborative sub-objective is designed as follows: Equation 29: ; In the formula, This represents the speed of vehicles following behind the vehicle before the vehicle performs a lane-changing or overtaking maneuver. This represents the speed of vehicles following behind a vehicle after the first vehicle performs a lane change or overtaking maneuver. This represents the expected speed of the following vehicle; when the speed difference between the following vehicles is positive, the autonomous vehicle is rewarded, guiding the autonomous driving agent to learn a collaborative driving strategy. ③ Safety sub-target reward : Formula 30: ; In the formula, This is the distance between the front of your vehicle and the vehicle in front of you in the current lane. This is a preset headway threshold; therefore, when the headway is less than... At that time, the vehicle will be penalized; only if the front distance of the vehicle is greater than [a certain value]. At that time, the vehicle will receive a reward; ④ Comfort constraints When the acceleration exceeds 4 m / s² 2 The jerk exceeds 1.5 m / s². 3 At times, this will affect driving comfort; Equation 31: ; Equation 32: ; Equation 33: ; In the formula, For acceleration comfort penalty, As a penalty for aggression, For the vehicle's acceleration, This is the maximum acceleration of the vehicle. This represents the change in acceleration between two adjacent time steps; ⑤ Trip completion rate The design endpoint-driven rewards are as follows: Equation 34: ; In the formula, The distance between the vehicle and the destination is expressed in European terms. This represents the Euclidean distance between the starting and ending points of the vehicle. ⑥ Collision penalty If the vehicle is involved in a collision, then ; Finally, a vectorized reward function considering three objectives—efficiency, collaboration, and security—is constructed. : Formula 35: ; In the formula, , , These represent the efficiency reward component, the collaboration reward component, and the security reward component, respectively.

9. The adaptive decision control method for autonomous vehicles considering disturbance factors according to claim 8, characterized in that, Step S4 also includes the following steps: Step S4-5: Construct a neural network containing an input layer, hidden layers, and an output layer; The multi-objective Actor neural network and Critic neural network for autonomous driving horizontal and vertical decision-making agents, as well as the adversarial agent Actor neural network, are all constructed using two fully connected hidden layers, with 256 and 128 neurons in each layer, respectively; all hidden layers use the ReLU activation function. The input to the Actor neural network, an autonomous driving lateral decision-making agent, is the perturbation state and a mixed preference vector; the output is a 3D lateral lane-changing action. The input to the Critic neural network, also an autonomous driving lateral decision-making agent, is the perturbation state and a mixed preference vector; the output is... indivual Value, that is, the value corresponding to each lateral movement. Vector, in the formula Represents the lateral movement space dimension. The number of targets is represented by the input of the Actor neural network for autonomous driving longitudinal decision-making, which is the perturbation state and the mixed preference vector, and the output is the longitudinal acceleration of the vehicle. The input of the Critic neural network for autonomous driving longitudinal decision-making is the perturbation state, the longitudinal action, and the mixed preference vector, and the output is... indivual Value, that is, the value corresponding to the vertical movement. Vector; The input to the adversarial agent Actor neural network is the original state and the mixed preference vector, which are then processed by the tanh function to output the best observation perturbation and the best action perturbation at the current moment.

10. The adaptive decision control method for autonomous vehicles considering disturbance factors according to claim 9, characterized in that, Step S4 also includes the following steps: Steps S4-6: Adversarial Agent: Based on the state space output of the autonomous driving agent, the agent counters disturbances. Including the best observation perturbation and optimal motion perturbation : Equation 36: ; In the formula, To counter agent policy networks, Network parameters for adversarial agent policy; Step S4-7: The autonomous driving lateral decision-making agent makes decisions based on the hybrid preference vector. and the state after resisting agent perturbation Output lateral lane change action and longitudinal following motion ,in: Equation 37: ; Equation 38: ; In the formula, , These represent the policy networks for the lateral decision-making agents and the longitudinal decision-making agents in autonomous driving, respectively. , These are the parameters of the Actor neural network for the horizontal and vertical decision-making intelligent agents in autonomous driving. As the weight of the motion disturbance; Step S4-8: The vehicle performs a lateral lane change maneuver in response to the disturbance. and longitudinal following motion Traffic conditions Based on the state transition, proceed to the next traffic state. And generate the current time step reward. Subsequently, the autonomous driving decision-making agents store the experience data into their respective experience replay pools. The state transition equation is as follows: Formula 39: ; In the formula, This is the state transition function. This indicates whether the round has ended. The round termination conditions are: ① The autonomous vehicle collides with another vehicle; ② The maximum number of simulation steps is reached; ③ The autonomous vehicle safely leaves the road segment. When the termination conditions are met... The value is 1; otherwise, =0; Step S4-9: In obtaining After obtaining a set of empirical data, samples are taken from the empirical replay pool. The adversarial agent policy network and the autonomous driving horizontal and vertical decision-making agent network are updated using alternating training methods; then the dual variables and the target network are updated. The goal of the adversarial agent is to minimize the expected utility of the autonomous driving agent and amplify policy bias and preference bias. This is achieved by updating the adversarial network by minimizing the loss function. Formula 40: ； In the formula, To address the network loss function of the adversarial agent policy, and As dual variables; The goal of an autonomous driving agent is to maximize expected utility while minimizing nonlinear constraints. The optimal policy is solved and updated by maximizing the objective function of the Actor neural network. Equation 41: ； In the formula, For the policy network of autonomous driving decision-making agents in both the horizontal and vertical directions; The dual variable is updated by minimizing the loss function: Equation 42: ； In the formula, The loss function is the dual variable. After completing the construction of the hybrid preference vector and the output of the driving strategy for the current moment, the vehicle control system enters the next moment and makes adaptive and robust driving actions according to the new traffic scenario.

Images