Interconnected autonomous driving decision-making method based on cooperative perception and adaptive information fusion

Abstract

The application discloses an interconnected automatic driving decision-making method based on cooperative perception and adaptive information fusion, and mainly solves the problem that the existing automatic driving decision-making is less applicable under the condition of complex road structure and traffic light information. The method considers a multi-lane traffic environment under a world coordinate system, wherein a mixed traffic flow composed of interconnected automatic driving vehicles and human-driven vehicles is formed. Each CAV can obtain surrounding multi-modal environment features (such as lane information, HDV vehicle information and red-green traffic light information) through a vehicle-mounted sensor and an offline high-precision map. With the help of vehicle-to-vehicle communication, the CAVs can share their information and make decisions within a specified time step t. The goal of the method is to generate speed decisions and steering angle decisions for the CAVs. With such action decisions, the automatic driving vehicles can safely and effectively travel along a specific route, while maximizing the comfort of passengers and minimizing the impact on surrounding HDVs.

Description

Technical Field

[0001] This invention belongs to the field of autonomous driving technology, specifically, it relates to an interconnected autonomous driving decision-making method based on collaborative perception and adaptive information fusion. Background Technology

[0002] With rapid urbanization and vehicle growth, traffic congestion has become a major problem in most of the world's major cities, leading to severe inefficiencies, fuel waste, and air pollution. Traffic congestion is often caused by road conditions or improper driving behavior, and environmental variables such as road construction and narrow road sections can also contribute. Furthermore, poor driving habits (such as sudden braking and forced lane changes) can cause long-term instability in traffic flow. This typically occurs during periods of high traffic density, so even slight fluctuations can create a domino effect, resulting in severe congestion. To avoid this, vehicles need to maintain consistently good driving behavior, which is extremely challenging for human drivers.

[0003] With the rapid development of automotive automation technology, a significant portion of future road vehicles will be autonomous, thus achieving the aforementioned goals. In recent years, reinforcement learning (RL) has garnered widespread attention in the field of autonomous driving due to its adaptability and problem-solving capabilities. Many studies have explored the use of RL in single-agent settings, where autonomous vehicles learn to make decisions through trial-and-error interactions with dynamic traffic flow. While single-agent settings are fully scalable, they can lead to unsatisfactory performance in traffic environments requiring close cooperation between vehicles, such as when vehicles are driving on narrow roads or roundabouts. Therefore, researchers have extended the single-agent RL framework to multi-agent settings to collaboratively complete autonomous driving tasks. It supports information-sharing mechanisms between multiple autonomous vehicles, known as connected autonomous vehicles (CAVs). Benefiting from this information-sharing mechanism, CAVs can gain a comprehensive understanding of the surrounding traffic flow to improve collaborative decision-making performance. However, we have found that existing multi-agent methods still have the following limitations:

[0004] (1) They primarily focus on how CAVs respond to the surrounding traffic environment, neglecting their impact on human-driven vehicles (HDVs). For example, in pursuit of high driving efficiency, algorithms may prompt CAVs to take aggressive actions (e.g., sudden lane changes), negatively impacting HDV traffic flow. In high-density traffic flows, this impact will be amplified into continuous speed fluctuations in traffic flow, leading to long-term traffic instability or even congestion. (2) Although multimodal fusion is a development trend in autonomous driving, existing multi-agent methods cannot fully utilize multimodal information from different sensors. Specifically, they are typically tested in simple scenarios (e.g., highways) that do not consider complex road structures and traffic light information. Furthermore, their systems assume that CAVs can accurately obtain HDV vehicle information in advance, resulting in low applicability in practical applications. Summary of the Invention

[0005] The purpose of this invention is to provide an interconnected autonomous driving decision-making method based on collaborative perception and adaptive information fusion, which mainly solves the problem that existing autonomous driving decision-making methods have low applicability under complex road structures and traffic light information conditions.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0007] A connected autonomous driving decision-making method based on collaborative perception and adaptive information fusion includes the following steps:

[0008] S1, collects vehicle dynamics information of the CAV through onboard GNSS and IMU sensors. A And the vehicle dynamics information of the HDV is detected by the vehicle-mounted radar of the CAV. H Among them, CAV stands for Connected Autonomous Vehicle, and HDV stands for Human-Driven Vehicle;

[0009] S2 accurately locates the position and direction of the CAV and identifies roads and traffic lights near the CAV to obtain real-time road information for the CAV's preset driving route;

[0010] S3, each CAV transmits its own vehicle dynamics information f A and the perceived vehicle dynamics information f of the HDV H The results are passed to an MLP and then concatenated to form a vehicle code h. i ; and use a graph attention layer to aggregate the vehicle codes of CAV based on the CAV communication link matrix M to obtain traffic flow information of the CAV preset driving route; where MLP represents multilayer perceptron;

[0011] S4. The parameterized CAV action structure is trained using the multi-agent reinforcement learning algorithm MAPAC to obtain the optimal action policy, and a stochastic Gaussian policy is used to improve the algorithm's exploration ability, thereby realizing the action decision of the CAV.

[0012] S5 optimizes CAV action decisions by setting an egocentric reward function that improves the safety, efficiency, and comfort of the CAV and a social impact reward function that reduces the negative impact on surrounding HDVs.

[0013] Furthermore, in step S1, the CAV's vehicle dynamics information f A Includes vehicle speed, direction, length, width, lane ID, and offset from the lane centerline; HDV vehicle dynamics information f H This includes the relative distance, speed, and direction of the HDV compared to the CAV, as well as the lane markings and lane centerline offset of the HDV.

[0014] Furthermore, in step S2, the CAV's preset driving route consists of a multi-lane road; the road's characteristics f L Lane trajectory points represent the data, where each trajectory point includes the lane horizontal elevation (i.e., lane slope change), direction, and lane ID. Traffic light information uses camera-based detection technology to detect its real-time state and distance from vehicles; its real-time state feature f... TL Represented using one-hot encoding; where red is [1,0,0], green is [0,1,0], and yellow is [0,0,1].

[0015] Features of the road f L Traffic light status f TL If a road encoder is used for encoding, then:

[0016] E r,i =σ(φ(f) L,L ,f TL,L )||φ(f L,C ,f TL,C )||φ(f L,R ,f TL,R ))

[0017] Among them, E r,i This represents the road encoding, φ represents the MLP layer, σ is the ReLU activation function, and f L,L ,f TL,L This indicates the left lane code and traffic light code of the lane the vehicle is in, f L,C ,f TL,C This indicates the lane code and traffic light code of the lane the vehicle is in, f L,E ,f TL,RThis represents the right lane code and traffic light code of the vehicle's current lane; || is the connection operation; for each agent, the focus is limited to the lane features and traffic light features of the current lane and the two adjacent lanes; finally, the traffic flow code E is connected. t,i and road code E r,i The final road information encoding E is obtained for subsequent module operation. s,i .

[0018] Furthermore, in step S3, in the vehicle coding of CAV based on the CAV communication link matrix M, according to the attention mechanism, each agent i in the vehicle communication network calculates a query vector q. i Key vector k i Sum vector v i The following are examples:

[0019] q i =W q h i k i =W k h i v i =W v h i

[0020] In the formula, W q Represents the query matrix, W v W represents a value matrix. k Represents the bond matrix, h i Vehicle coding;

[0021] Assume agent i has N i There are 1 neighboring agents, and the attention score α of the agent to its neighboring agent j is α. ij It can be calculated as follows:

[0022]

[0023] Where σ is the ReLU activation function; LeakyReLU represents the LeakyReLU activation function; exp represents the exponential operation; and l represents N. i One of the neighboring agents among the neighboring agents;

[0024] Due to changes in the traffic environment, agents that have lost communication with the system in the current time step are filtered out. The final vehicle flow code E is calculated by combining the CAV link matrix. t,i The following are examples:

[0025]

[0026] Where φ represents the MLP layer, M i,j It is the value of the link matrix, Mi,j =0 indicates that there is no connection between agent i and agent j in the current time step, and vice versa; where the agent is CAV.

[0027] Further, in step S4, the multi-agent reinforcement learning algorithm MAPAC uses an actor-critic structure from reinforcement learning, where the actor network computes actions and the critic network evaluates actions through an estimation function; and replaces the original deep Q-network with a stochastic Gaussian network, which outputs a Gaussian distribution from which the agent samples to form parameterized continuous actions; wherein, during the model training process using the multi-agent reinforcement learning algorithm MAPAC, the actor network π of agent i... i Update the network by minimizing the following objectives:

[0028]

[0029] Among them, the experience buffer D is used to store the states and actions of all interconnected agents, Q β The network parameters represent the critics' parameters; λ is the regularization coefficient that controls the algorithm's exploration performance. and These represent the state information and action information of agent i at time step t, respectively. Represents the actor network with parameters for agent i;

[0030] The combination of states and actions of the interconnected agent is used as input to the critic network, which then outputs the Q-value of the action taken by agent i at time step t.

[0031] The critic network minimizes the Bellman error J as follows: Q To be updated:

[0032]

[0033]

[0034] Where γ is the reward discount factor, r i It is the instant reward for t time steps; two target commentator networks To stabilize the training process, during execution, each agent runs its own copy of the actor network and the critic network, i.e., distributed execution;

[0035] Agent i only needs to acquire the observed traffic environment information and enhance the information through shared information from interconnected agents, and then calculate the final parameterized action based on the fused information; finally, the action with the largest Q value is selected as the actual action to be executed; all CAVs in the interconnected network follow the above process to generate their respective action decisions.

[0036] Furthermore, in step S5, the egocentric reward function and the social influence reward function constitute a hybrid reward function; its expression is:

[0037]

[0038] In the formula, r ego Rewards that are self-centered, r soc Rewards are given for their social impact. This is a cooperation metric used to quantify the degree of cooperation between autonomous vehicle connectivity and human-driven vehicles.

[0039] Furthermore, the expression for the social impact reward function is:

[0040] r soc =r soc,1 +r soc,2

[0041] In the formula, r soc,1 The expression used to quantify the occurrence rate of sudden stopping or sudden lane-changing behavior of CAVs is:

[0042]

[0043] In the formula, The velocity of HDV at time step t is represented by thr. vel It is a threshold for speed change used to determine whether CAV causes HDV to brake suddenly. acc *Δt refers to the velocity change threshold between two consecutive time steps; this reward is only given if the HDV deceleration is greater than thr. vel It is only effective at certain times;

[0044] r soc,2 The expression used to quantify the occurrence of CAV adjusting its speed or position to reserve lane change space for HDV is:

[0045]

[0046] in Let i be the neighboring HDVs of agent i at time step t; when a vehicle in the adjacent lane safely changes lanes in front of or behind the CAV, Set to 1; in other cases, Set to 0.

[0047] Furthermore, the expression for the egocentric reward function is:

[0048] r ego =r saf +r eff +rcom

[0049] In the formula, r saf As a security reward, r eff For efficiency-based rewards, r com Reward passengers for their comfort;

[0050] The safety bonus is the sum of bonuses for unsafe CAV behavior and traffic rule compliance; following safety uses the estimated time to collision (TTC) to ensure the CAV maintains a safe following distance; the TTC calculation formula is:

[0051]

[0052] In the formula, f A .vel and f H .vel represents the speed of CAV and HDV respectively, dis(A,H) represents the Euclidean distance between A and H; following safety bonus r saf,1 The calculation is as follows:

[0053]

[0054] in, The first threshold is for TTC; the second is the lane keeping safety bonus for keeping the CAV centered in the lane, calculated as follows:

[0055]

[0056] Where dis(wp, A) measures the current distance of the CAV from the center point of the lane, and d is half the width of the lane; emergency safety is the penalty for CAV collision, lane departure, or violation of traffic signals, and is 0 in other cases;

[0057] Efficiency-based reward: The efficiency of a multi-lane task is the sum of speed control efficiency and lane-changing efficiency; speed control efficiency r eff,1 To help autonomous vehicles maintain a safe driving speed, the calculation is as follows:

[0058]

[0059] Where f A .vel indicates the speed of an autonomous vehicle. max Maximum speed set for the vehicle; lane change bonus r eff,2 Encourages vehicles to overtake and avoid obstacles; it is calculated after the lane change is completed.

[0060]

[0061] Where dis(H) tar ,A) and dis(H)prev A) represents the distance between the vehicle and the obstacle or the vehicle in front in the target lane and the lane ahead, respectively;

[0062] Passenger comfort bonus: Measured by the Jerk rate of change of vehicle acceleration; Jerk is calculated as follows:

[0063]

[0064] Where acc t Let r be the vehicle's acceleration at time step t, where Δt is the duration of each time step; com It was calculated by Jerk:

[0065]

[0066] Where thr acc This is the maximum allowed Jerk value.

[0067] Compared with the prior art, the present invention has the following beneficial effects:

[0068] (1) The multi-agent reinforcement learning collaborative decision-making framework of the present invention enables interconnected autonomous vehicles to drive safely, efficiently and comfortably on a designated route in a traffic flow that mixes autonomous vehicles and human-driven vehicles, while exerting minimal negative impact on human-driven vehicles in the traffic flow.

[0069] (2) The present invention utilizes graph attention mechanism to aggregate shared information between vehicles based on constantly changing vehicle interconnection relationships. Attached Figure Description

[0070] Figure 1 This is a system overall framework diagram of the method of the present invention.

[0071] Figure 2 The network structure diagram of the information aggregation module in the method of this invention. Detailed Implementation

[0072] The present invention will be further described below with reference to the accompanying drawings and embodiments. The embodiments of the present invention include, but are not limited to, the following embodiments.

[0073] Example

[0074] This invention discloses an interconnected autonomous driving decision-making method based on cooperative perception and adaptive information fusion. The method considers a multi-lane traffic environment in a world coordinate system, comprising a mixed traffic flow of interconnected autonomous vehicles (CAVs) and human-driven vehicles (HDVs). Each CAV can acquire multimodal environmental features (such as lane information, HDV vehicle information, and traffic light information) from its surroundings via onboard sensors (e.g., cameras) and offline high-precision maps. With the aid of vehicle-to-vehicle communication, CAVs can share their information and make decisions within a specified time step t (a period of time). The goal of this method is to generate speed and steering angle decisions for the CAVs. With such action decisions, autonomous vehicles can travel safely and efficiently along specific routes while maximizing passenger comfort and minimizing their impact on surrounding HDVs.

[0075] like Figure 1 As shown, the framework of this method mainly includes three modules: state representation, action decision-making, and model optimization.

[0076] State characterization module: This module consists of two sub-modules: feature extraction and information fusion. For the feature extraction sub-module, vehicle dynamics information of the CAV is collected through onboard GNSS and IMU sensors. A Specifically, f A Includes vehicle speed, direction, length, width, lane ID, and offset from the lane centerline; characteristics of an HDV. H Detection is performed using lidar detection technology. H This includes the relative distance, speed, and direction of the HDV compared to the CAV, as well as the lane markings and lane centerline offset of the HDV. CAV stands for Connected Autonomous Vehicle, while HDV refers to a human-driven vehicle.

[0077] For road information, an extended Kalman filter is used to accurately locate the position and orientation of the CAV and identify nearby roads and traffic lights. Each CAV has a preset route consisting of multi-lane roads. Road characteristics f L Lane trajectory points represent the data, where each trajectory point includes the lane horizontal elevation (i.e., lane slope change), direction, and lane ID. Traffic light information uses camera-based detection technology to detect its real-time state and distance from vehicles; its real-time state feature f... TL Represented using one-hot encoding; where red is [1,0,0], green is [0,1,0], and yellow is [0,0,1].

[0078] Features of the road f L Traffic light status f TL If a road encoder is used for encoding, then:

[0079] E r,i =σ(φ(f) L,L ,f TL,L )||φ(f L,C ,f TL,C )||φ(f L,R ,f TL,R ))

[0080] Among them, E r,i This represents the road encoding, φ represents the MLP layer, σ is the ReLU activation function, and f L,L ,f TL,L This indicates the left lane code and traffic light code of the lane the vehicle is in, f L,C ,f TL,C This indicates the lane code and traffic light code of the lane the vehicle is in, f L,R ,f TL,R This represents the right lane code and traffic light code of the vehicle's current lane; || is the connection operation; for each agent, the focus is limited to the lane features and traffic light features of the current lane and the two adjacent lanes; finally, the traffic flow code E is connected. t,i and road code E r,i The final road information encoding E is obtained for subsequent module operation. s,i After obtaining the multimodal environmental features, they are fused into a single state code.

[0081] like Figure 2 As shown, for traffic flow information, each CAV will display its own vehicle dynamics information f A and the perceived vehicle dynamics information f of the HDV H The results are transmitted separately to an MLP (Multilayer Perceptron), and the encoded results are concatenated to form a vehicle code h. i A graph attention layer is used to aggregate vehicle codes of CAVs based on the CAV communication link matrix M to obtain traffic flow information for the pre-defined CAV driving routes. According to the attention mechanism, each agent i in the vehicle communication network calculates a query vector q. i Key vector k i Sum vector v i The following are examples:

[0082] q i =W q h i k i =W k h i v i =W v h i

[0083] In the formula, W qRepresents the query matrix, W v W represents a value matrix. k Represents the bond matrix, h i Vehicle coding;

[0084] Assume agent i has N i There are 1 neighboring agents, and the attention score α of the agent to its neighboring agent j is α. ij It can be calculated as follows:

[0085]

[0086] Where σ is the ReLU activation function; LeakyReLU represents the LeakyReLU activation function; exp represents the exponential operation; and l represents N. i One of the neighboring agents in a network of neighboring agents; through an attention mechanism, each agent can selectively utilize information from its neighbors to achieve more effective cooperation in subsequent maneuver decision-making modules. Furthermore, as the traffic environment changes, the CAV link matrix is ​​updated at each time step, representing the communication state between CAVs. For example, an agent might have communicated with other agents in the previous time step but lost connection in the current time step. Therefore, these agents need to be filtered out to prevent their attention scores from interfering. Thus, the final traffic flow code E is calculated by combining the CAV link matrix. t,i The following are examples:

[0087]

[0088] Where φ represents the MLP layer, M i,j It is the value of the link matrix, M i,j =0 indicates that there is no connection between agent i and agent j in the current time step, and vice versa; where the agent is CAV.

[0089] To make action decisions, the agent needs both top-level decisions (determining whether a lane change is necessary) and bottom-level action decisions (executed by adjusting steering angle and acceleration / braking rate). Therefore, this embodiment introduces a parameterized action structure, including left lane change, lane keeping, and right lane change actions. To train an optimal action policy based on this structure, a multi-agent reinforcement learning algorithm, MAPAC, is proposed. This algorithm extends the parameterized deep Q-network (P-DQN) to multi-agent scenarios and employs a stochastic Gaussian policy to enhance the algorithm's exploration capabilities.

[0090] The MAPAC algorithm utilizes an actor-critic structure from reinforcement learning, where the actor network (policy network) computes actions, and the critic network evaluates actions using an estimated value function. Because deterministic policies from the original P-DQN easily converge to local optima, a stochastic Gaussian network replaces the original deep Q-network. Specifically, the stochastic Gaussian network outputs a Gaussian distribution from which the agent samples parameterized continuous actions, significantly improving the algorithm's exploration capability. During model training, MAPAC not only maximizes the agent's action reward but also maximizes the action entropy, ensuring the agent learns behaviors that maximize cumulative rewards and preventing the policy from prematurely converging to local optima. Assuming there are N interconnected agents in a traffic flow, distributed execution is used to improve the algorithm's scalability, while centralized training encourages cooperation among agents. Specifically, agent i's actor network π... i Update the network by minimizing the following objectives:

[0091]

[0092] Among them, the experience buffer D is used to store the states and actions of all interconnected agents, Q β The network parameters represent the critics' parameters; λ is the regularization coefficient that controls the algorithm's exploration performance. and These represent the state information and action information of agent i at time step t, respectively. Let represent the parameterized actor network for agent i. During training, all agents use the same critic network (centralized training). The set of combinations of states and actions of all interconnected agents serves as the input to the critic network, which then outputs the Q-value of the action taken by agent i at time step t. The critic network then minimizes the Bellman error J as follows: Q To be updated:

[0093]

[0094]

[0095] Where γ is the reward discount factor, r i It is the instant reward for t time steps; two target commentator networks To stabilize the training process, during execution, each agent runs a copy of its own actor network and critic network, i.e., distributed execution. Agent i only needs to acquire observed traffic environment information and augment it with information shared from interconnected agents, then computes the final parameterized action based on the fused information. Finally, the action with the highest Q-value is selected as the actual action to be executed. All CAVs in the interconnected network follow the above process to generate their respective action decisions.

[0096] Model Optimization Module: This module guides policy optimization through a reward function. Specifically, the reward function combines a self-centered functional reward to maximize the safety, efficiency, and comfort of the autonomous vehicle, with a social impact reward to minimize negative impacts on surrounding human-driven vehicles. To accelerate training, this embodiment also employs a distributed computing multi-node training setup. The collector gathers experiential data (trajectories) of the agent's interaction with the environment; the trainer uses the collected trajectories to optimize the training algorithm; and the evaluator evaluates the trained algorithm parameters, determines the algorithm's convergence, and decides when to terminate training (scheduling). The trained algorithm parameters are shared and propagated in real time among the three entities.

[0097] Hybrid reward function: The reward function acts as a guiding signal, directing the agent to learn the optimal action strategy through exploration. To encourage the agent to consider cooperative behavior with human drivers while optimizing its own driving behavior, a cooperation index is incorporated into the reward function. It quantifies the degree of cooperation between autonomous vehicle connectivity and human-driven vehicles:

[0098]

[0099] Where r ego Rewards that are self-centered, r soc The impact of autonomous vehicles on surrounding human drivers is considered (social impact reward). Their specific calculation method is as follows:

[0100] Social Impact Reward: To master strategies that minimize the impact on other road users, autonomous vehicles need to accurately assess the impact of their interactions with human-driven vehicles on the broader traffic flow. Agents can measure the number of times an HDV brakes suddenly due to their lane-changing behavior; however, this approach ignores the proactive yielding actions of CAVs when encountering HDVs cutting into their lanes, reflecting their cooperative nature. Therefore, a social impact reward r is designed. soc =r soc,1 +r soc,2 This is used to comprehensively measure the impact of CAV lane changing and yielding actions on surrounding HDVs. The higher the reward value, the smaller the social impact.

[0101] (1) If the CAV issues a lane change signal to prepare to change lanes, and the HDV subsequently yields, the sudden stop or sudden cut into the corresponding lane by the CAV may affect the driving efficiency and comfort of the HDV. soc,1 Quantify the incidence of this type of behavior:

[0102]

[0103] in, The velocity of HDV at time step t is represented by thr. vel It is a speed change threshold (0.1 m / s) used to determine whether the CAV causes the HDV to brake suddenly. acc *Δt refers to the velocity change threshold (3m / s) between two consecutive time steps. 2 This reward only applies when the HDV deceleration is greater than thr. vel It is only effective at certain times.

[0104] (2) When an HDV signals its intention to change lanes, a CAV may understand this intention and adjust its speed or position to allow the HDV space to change lanes. While this behavior may impede the CAV's own driving efficiency, it contributes to the smooth flow of traffic, especially when the road ahead narrows and traffic is required to merge. soc,2 Used to reward this type of behavior:

[0105]

[0106] in Let HDV be the neighboring HDV of agent i at time step t. After a vehicle in the adjacent lane safely changes lanes in front of or behind CAV, Set to 1. In other cases, Set to 0.

[0107] Egocentric reward: This refers to the agent's security reward r. saf Efficiency reward r eff and passenger comfort reward r com The combination results in a final egocentric reward r ego .

[0108] (1) Safety Bonus: The safety bonus is the sum of the bonuses from unsafe CAV behavior and traffic rule compliance. First, following safety uses the predicted time to collision (TTC) to ensure the CAV maintains a safe following distance. The TTC calculation formula is:

[0109]

[0110] Where f A.vel and f H .vel represents the speed of CAV and HDV respectively, and dis(A,H) represents the Euclidean distance between A and H. Following safety bonus r saf,1 The calculation is as follows:

[0111]

[0112] in, A threshold for TTC is set to 6 seconds. Secondly, the lane-keeping safety bonus for keeping the CAV centered in the lane is calculated as follows:

[0113]

[0114] Where dis(wp, A) measures the current distance of the CAV from the center point of the lane, and d is half the lane width. Third, emergency safety is assigned a penalty of -30 for CAV collision, lane departure, or violation of traffic signals, and 0 for other situations.

[0115] (2) Efficiency-based reward: The efficiency of a multi-lane task is the sum of speed control efficiency and lane-changing efficiency. Speed ​​control efficiency r eff,1 To help autonomous vehicles maintain a safe driving speed, the calculation is as follows:

[0116]

[0117] Where f A .vel indicates the speed of an autonomous vehicle. max The maximum speed set for the vehicle (80 km / h). Lane change bonus r eff,2 Vehicles are encouraged to overtake and avoid obstacles. It is calculated after the lane change is completed as follows:

[0118]

[0119] Where dis(H) tar ,A) and dis(H) prev A) represents the distance between the vehicle and the obstacle or the vehicle in front in the target lane and the previous lane, respectively.

[0120] Passenger comfort reward: The vehicle acceleration change rate (Jerk) is crucial to passenger comfort and vehicle energy efficiency. A comfort reward r has been incorporated. com As a metric for Jerk. Jerk is calculated as follows:

[0121]

[0122] Where acc t Let r be the vehicle's acceleration at time step t, where Δt is the length of each time step.com It was calculated by Jerk:

[0123]

[0124] Where thr acc The maximum allowed Jerk value (3m / s) 2 ).

[0125] Through the above design, the multi-agent reinforcement learning collaborative decision-making framework of the present invention enables connected autonomous vehicles to drive safely, efficiently, and comfortably along designated routes in traffic flow that mixes autonomous vehicles and human-driven vehicles, while exerting minimal negative impact on human-driven vehicles in the traffic flow.

[0126] The above embodiments are merely one of the preferred embodiments of the present invention and should not be used to limit the scope of protection of the present invention. Any modifications or refinements made to the main design concept and spirit of the present invention that are not of substantial significance, but solve the same technical problem as the present invention, should be included within the scope of protection of the present invention.

Claims

1. A connected autonomous driving decision-making method based on collaborative perception and adaptive information fusion, characterized in that, Includes the following steps: S1 collects vehicle dynamics information of the CAV through onboard GNSS and IMU sensors. And the vehicle dynamics information of the HDV is detected by the vehicle-mounted radar of the CAV. Among them, CAV stands for Connected Autonomous Vehicle, and HDV stands for Human-Driven Vehicle; S2 accurately locates the position and direction of the CAV and identifies roads and traffic lights near the CAV to obtain real-time road information for the CAV's preset driving route; S3, each CAV transmits its own vehicle dynamics information and perceived vehicle dynamics information of HDV The results are transmitted separately to an MLP, and the encoded results are concatenated to form a vehicle code. ; and use a graph attention layer to aggregate the vehicle codes of CAV based on the CAV communication link matrix M to obtain traffic flow information of the CAV preset driving route; where MLP represents multilayer perceptron; S4 employs the multi-agent reinforcement learning algorithm MAPAC to train a parameterized CAV action structure to obtain the optimal action policy, and uses a stochastic Gaussian policy to improve the algorithm's exploration ability, thereby realizing the CAV's action decision-making. Specifically, MAPAC utilizes the actor-critic structure from reinforcement learning, where the actor network computes actions, and the critic network evaluates actions through an estimation function. The original deep Q-network is replaced with a stochastic Gaussian network, which outputs a Gaussian distribution from which the agent samples to form parameterized continuous actions. During the model training process using MAPAC, the actor network π of agent i... i Update the network by minimizing the following objectives: The experience buffer D is used to store the states and actions of all interconnected agents. The network parameters represent the critics' parameters; λ is the regularization coefficient that controls the algorithm's exploration performance. These represent the state information and action information of agent i at time step t, respectively. Represents the actor network with parameters for agent i; The combination of states and actions of the interconnected agent is used as input to the critic network, which then outputs the Q-value of the action taken by agent i at time step t. The critic network minimizes the Bellman error J as follows: Q To be updated: in It is the reward discount factor, r i It is the instant reward for t time steps; two target commentator networks , To stabilize the training process, during execution, each agent runs its own copy of the actor network and the critic network, i.e., distributed execution; Agent i only needs to acquire the observed traffic environment information and enhance the information through shared information from interconnected agents, and then calculate the final parameterized action based on the fused information; finally, the action with the largest Q value is selected as the actual action to be executed; all CAVs in the interconnected network follow the above process to generate their respective action decisions; S5 optimizes CAV action decisions by setting an egocentric reward function that improves the safety, efficiency, and comfort of the CAV and a social impact reward function that reduces the negative impact on surrounding HDVs.

2. The interconnected autonomous driving decision-making method based on collaborative perception and adaptive information fusion according to claim 1, characterized in that, In step S1, the vehicle dynamics information of the CAV Includes vehicle speed, direction, length, width, lane ID, and offset from the lane centerline; HDV vehicle dynamics information. This includes the relative distance, speed, and direction of the HDV compared to the CAV, as well as the lane markings and lane centerline offset of the HDV.

3. The interconnected autonomous driving decision-making method based on collaborative perception and adaptive information fusion according to claim 2, characterized in that, In step S2, the CAV's preset driving route consists of a multi-lane road; the road's characteristics f L Lane trajectory points represent the data, where each trajectory point includes the lane horizontal elevation (i.e., lane slope change), direction, and lane ID. Traffic light information uses camera-based detection technology to detect its real-time state and distance from vehicles; its real-time state feature f... TL Represented using one-hot encoding; where red is [1,0,0], green is [0,1,0], and yellow is [0,0,1]. Features of the road f L Traffic light status f TL If a road encoder is used for encoding, then: in, This represents the road encoding, ϕ represents the MLP layer, and σ is the ReLU activation function. This indicates the left lane code and traffic light code of the lane the vehicle is in. This indicates the lane code and traffic light code of the lane the vehicle is in. This indicates the right lane code and traffic light code of the lane the vehicle is in. For the connection operation; for each agent, the focus is limited to the lane features and traffic light features of the current lane and the two adjacent lanes; finally, the traffic flow code E is connected. t,i and road code E r,i The final road information encoding E is obtained for subsequent module operation. s,i .

4. The interconnected autonomous driving decision-making method based on collaborative perception and adaptive information fusion according to claim 3, characterized in that, In step S3, in the vehicle coding of CAV based on the CAV communication link matrix M, according to the attention mechanism, each agent i in the vehicle communication network calculates a query vector q. i Key vector k i Sum vector v i The following are examples: In the formula, Represents the query matrix. Represents a value matrix, Represents the key matrix. Vehicle coding; Assume agent i has N i There are 1 neighboring agents, and the attention score α of the agent to its neighboring agent j is α. ij It can be calculated as follows: Where σ is the activation function ReLU; This represents the LeakyReLU activation function. This represents the exponentiation operation, where l represents N. i One of the neighboring agents among the neighboring agents; Due to changes in the traffic environment, agents that have lost communication with the system in the current time step are filtered out. The final vehicle flow code E is calculated by combining the CAV link matrix. t,i The following are examples: Where ϕ represents the MLP layer, M i,j It is the value of the link matrix, M i,j =0 indicates that there is no connection between agent i and agent j in the current time step, and vice versa; where agent i is CAV.

5. The interconnected autonomous driving decision-making method based on collaborative perception and adaptive information fusion according to claim 4, characterized in that, In step S5, the egocentric reward function and the social influence reward function constitute a hybrid reward function; its expression is: In the formula, Rewards that are self-centered. Rewards are given for their social impact. This is a cooperation metric used to quantify the degree of cooperation between autonomous vehicle connectivity and human-driven vehicles.

6. The interconnected autonomous driving decision-making method based on collaborative perception and adaptive information fusion according to claim 5, characterized in that, The expression for the social impact reward function is: In the formula, The incidence rate of sudden stopping or sudden lane change by a CAV is expressed as follows: In the formula, This represents the speed of HDV at time step t. It is a threshold for speed change used to determine whether CAV causes HDV to brake suddenly. * This refers to the threshold of velocity change between two consecutive time steps; this reward is only given if the HDV deceleration is greater than [a certain threshold]. It is only effective at certain times; The expression used to quantify the occurrence of CAV adjusting its speed or position to reserve lane change space for HDV is: in For agent i at time step t, the adjacent HDVs; when a vehicle in the adjacent lane safely changes lanes in front of or behind the CAV, Set to 1; otherwise, Set to 0.

7. The interconnected autonomous driving decision-making method based on collaborative perception and adaptive information fusion according to claim 6, characterized in that, The expression for the egocentric reward function is: In the formula, As a security reward, For efficiency-based rewards, Reward passengers for their comfort; The safety bonus is the sum of bonuses for unsafe CAV behavior and traffic rule compliance; following safety uses the estimated time to collision (TTC) to ensure the CAV maintains a safe following distance; the TTC calculation formula is: In the formula, f A .vel and f H .vel represents the speed of CAV and HDV respectively, dis(A,H) represents the Euclidean distance between A and H; following safety bonus r saf,1 The calculation is as follows: in, The first threshold is for TTC; the second is the lane keeping safety bonus for keeping the CAV centered in the lane, calculated as follows: Where dis(wp, A) measures the current distance of the CAV from the center point of the lane, and d is half the width of the lane; emergency safety is the penalty for CAV collision, lane departure, or violation of traffic signals, and is 0 in other cases; Efficiency-based reward: The efficiency of a multi-lane task is the sum of speed control efficiency and lane-changing efficiency; speed control efficiency r eff,1 To help autonomous vehicles maintain a safe driving speed, the calculation is as follows: in Vel indicates the speed of an autonomous vehicle. max Maximum speed set for the vehicle; lane change bonus r eff,2 Encourages vehicles to overtake and avoid obstacles; it is calculated after the lane change is completed. in and These indicate the distances of the vehicle to obstacles or vehicles ahead in the target lane and the preceding lane, respectively. Passenger comfort bonus: Measured by the Jerk rate of change of vehicle acceleration; Jerk is calculated as follows: in Let Δt be the acceleration of the vehicle at time step t, where Δt is the length of each time step. It was calculated by Jerk: in This is the maximum allowed Jerk value.

Images