Intelligent networked automobile collaborative decision-making method based on double interactive perception

Abstract

The invention discloses an intelligent network connection automobile collaborative decision-making method based on double interactive perception, and the method comprises the following steps: designing a double interactive perception collaborative control strategy DIACC through employing a centralized training and distributed execution multi-agent reinforcement learning framework; using a distributed interactive adaptive decision module D-IADM to learn interactive characteristics with surrounding vehicles and traffic environment information; a centralized interaction enhancement evaluator (C-IEC) module is utilized to capture a relationship between global vehicle interaction and global traffic dynamics. According to the intelligent connected automobile collaborative decision-making method based on double interactive perception, the behaviors of the intelligent connected automobile are actively adjusted in advance, transverse conflicts between a pre-confluence area and other vehicles are reduced, on the other hand, the behaviors of the intelligent connected automobile actively guide and coordinate manual driving of the automobile, and the driving safety of the automobile is improved. Therefore, the overall traffic performance is improved.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving technology, and in particular to a collaborative decision-making method for an intelligent networked vehicle based on dual interactive perception. Background Art

[0002] With the rapid increase in the number of vehicles worldwide, traffic congestion has become one of the constraints that hinder the rapid development of cities. Therefore, improving traffic efficiency has become an urgent and highly concerned research topic. Studies have shown that intelligent connected vehicles have great potential in improving traffic performance, and human-driven vehicles will coexist with intelligent connected vehicles in the foreseeable future. Human drivers tend to make decisions dynamically and randomly based on the reactions of surrounding vehicles, and generally lack a sense of collaboration. Therefore, human-driven vehicles often exhibit different driving styles, which poses a major challenge to intelligent connected vehicles, which must adapt to these uncertain and diverse behaviors and make collaborative decisions that contribute to overall traffic optimization.

[0003] In the prior art, the solutions proposed to improve the efficiency of mixed traffic systems can be roughly divided into two categories. The first category includes rule-based or optimization methods, and the second category is single-agent reinforcement learning-based methods. Existing research has the following limitations: 1. Rule-based or optimization methods rely on modeling of traffic environments and vehicle behaviors, and the centralized optimization methods that can obtain optimal solutions have a very high computational cost; 2. Single-agent reinforcement learning methods are more suitable for solving local control problems, but face challenges in global traffic optimization. This is because they require a lot of interactive experience to learn effective decision-making strategies, which makes training costs high and difficult to deploy on a large scale; 3. Single-agent reinforcement learning methods have difficulty dealing with coordination problems between multiple vehicles, and usually require additional rule guidance or manually designed coordinated interaction settings. Summary of the invention

[0004] The purpose of the present invention is to provide a collaborative decision-making method for intelligent connected vehicles based on dual interactive perception, learn the interaction characteristics and traffic environment information with surrounding vehicles to adapt to different types of vehicles and driving styles; better understand the impact of vehicle interaction on traffic evolution from a global perspective, and thus guide the update of collaborative strategies for intelligent connected vehicles.

[0005] To achieve the above object, the present invention provides a collaborative decision-making method for intelligent connected vehicles based on dual interactive perception, comprising the following steps:

[0006] Using the centralized training and distributed execution multi-agent reinforcement learning framework, a dual mutual perception collaborative control strategy DIACC is designed;

[0007] Use the distributed interactive adaptive decision module D-IADM to learn the interaction characteristics and traffic environment information with surrounding vehicles;

[0008] The relationship between global vehicle interactions and global traffic dynamics is captured using a centralized interaction enhanced evaluator module (C-IEC).

[0009] Preferably, the dual mutual perception collaborative control strategy DIACC includes a centralized critic network and several distributed actor networks;

[0010] The centralized critic network assists the training phase by accessing the global information of the mixed traffic system, and the distributed worker networks maximize their individual rewards by leveraging the global information provided by the centralized critic network.

[0011] Preferably, the state space, action space and reward function under the multi-agent reinforcement learning framework are designed according to the evolution characteristics of the hybrid traffic system and the vehicle interaction requirements;

[0012] The state space designed under the multi-agent reinforcement learning framework includes the motion state of each vehicle, traffic flow statistics of each lane, and static road structure information; the discrete quantities of the action space designed include: hold, left lane change, right lane change, acceleration, and deceleration; the reward function designed is:

[0013]

[0014] Among them, r e 、r g and r tp They represent the vehicle reward, global reward and lazy behavior penalty, or time penalty, respectively, and the coefficient w e =ρ、w g =1-ρ and w tp =n-1+ρ is the weight of each reward component related to the penetration rate of intelligent connected vehicles; r done To complete the reward, the vehicle reward r e Contains several sub-rewards:

[0015] r e =w e,v r e,v +w e,w r e,w +w e,c r e,c ;

[0016]

[0017]

[0018]

[0019] Among them, r e,v is the speed reward; re,w For warning distance penalty; r e,c is the collision penalty, w e,v 、w e,w and w e,c r e,v 、r e,w and r e,c The weight parameter v max is the maximum speed allowed by road traffic regulations; Indicates vehicles within the warning range; assembly Indicates the vehicles within the actual collision range; is the relative longitudinal distance between vehicles; is the relative lateral distance between vehicles, j is or Elements in a collection; and The distance thresholds are d th,w and d th,c ;p e,v Adjust parameters for speed bonus; p e,c Adjust parameters for collision penalty.

[0020] Global Rewards g Reflects the overall performance of all intelligent connected vehicles in terms of average speed, which is as follows:

[0021]

[0022] Among them, p g,v Adjust parameters for global speed bonus;

[0023] r tp Reflects the impact of the speed efficiency of intelligent connected vehicles on the overall traffic flow:

[0024] r tp =10×sigmoid(v i -v th );

[0025] Among them, v th is the speed threshold used to adjust the penalty level at different speeds, and sigmoid(·) is the Sigmoid function.

[0026] Preferably, D-IADM includes a trajectory-aware interaction encoder TAIE, which obtains the interaction features between the own vehicle and surrounding vehicles through TAIE, and then combines the previous decision output with the gated recurrent unit GRU and the decision layer to generate a learnable action output, and inputs the action output into the active safety-based action filter PSAF.

[0027] Preferably, an interaction graph between the own vehicle and surrounding human-driven cars is established, and the current state information and historical trajectory information of the own vehicle and surrounding human-driven cars are used as input, and the interaction features with surrounding human-driven cars are extracted using a multi-head graph attention mechanism;

[0028] Establish an interaction graph between the vehicle and surrounding intelligent connected vehicles, use the current state information of the vehicle and surrounding intelligent connected vehicles and the decision output at the last moment as input, and use the multi-head graph attention mechanism to extract the interaction features with surrounding intelligent connected vehicles;

[0029] Taking the traffic flow statistics of adjacent lanes and the static road structure information as input, a multi-layer perceptron is used to extract the traffic environment context information.

[0030] Preferably, the interaction features of surrounding manually driven cars, the interaction features of surrounding intelligent networked cars, the traffic environment context information and the decision instructions at the previous moment are integrated to obtain comprehensive features, and the comprehensive features are input into the gated recurrent unit GRU and the decision layer to obtain a learnable action output;

[0031] The active safety action filter PSAF uses the collision time TTC as an active safety evaluation indicator, optimizes actions when the TTC is lower than the set threshold, and outputs the longitudinal and lateral decisions of the intelligent connected vehicle.

[0032] Preferably, the centralized interaction enhancement evaluator module C-IEC receives all state information from the environment, and C-IEC includes an integrated traffic dynamic representation module ITDR, which captures traffic dynamics and vehicle interaction characteristics and outputs a global state value function.

[0033] Preferably, a multi-layer perceptron is used to obtain the traffic flow characteristics of each lane in the mixed traffic system, the movement characteristics of manually driven vehicles, and the movement characteristics of intelligent connected vehicles, and then combined with the static road structure information to obtain the comprehensive traffic flow dynamic characteristics.

[0034] Preferably, an interaction graph of all vehicles in the mixed traffic system is established, the current motion state of each vehicle is used as input, and a graph attention mechanism is used to obtain global vehicle interaction features.

[0035] Preferably, a multi-head cross-attention mechanism is used, with comprehensive traffic flow dynamic features as query objects and global vehicle interaction features as keys and values ​​to capture the characteristics of traffic dynamics and vehicle interactions.

[0036] Therefore, the present invention adopts the above-mentioned intelligent connected vehicle collaborative decision-making method based on dual interactive perception, and the beneficial effects are as follows:

[0037] (1) The present invention utilizes a distributed interactive adaptive module D-IADM to learn the interaction characteristics and traffic environment information with surrounding vehicles, so that intelligent connected vehicles can make reasonable collaborative decisions, plan merging routes in advance, and give way to surrounding vehicles at appropriate times, thereby improving overall traffic efficiency.

[0038] (2) The present invention utilizes the centralized interaction enhancement evaluator module C-IEC to better understand the impact of vehicle interaction on traffic evolution from a global perspective, thereby guiding the update of the collaborative strategy of intelligent connected vehicles. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 It is an overall scheme framework diagram of an embodiment of a collaborative decision-making method for intelligent networked vehicles based on dual interactive perception according to the present invention;

[0040] Figure 2 Schematic diagram of a mixed traffic scenario of an embodiment of a collaborative decision-making method for intelligent networked vehicles based on dual interactive perception according to the present invention, wherein (a) is a schematic diagram of a traffic scenario of a purely manually driven vehicle, and (b) is a schematic diagram of a mixed traffic scenario of an intelligent networked vehicle and a manually driven vehicle using a collaborative strategy;

[0041] Figure 3 It is a schematic diagram of static road structure division according to an embodiment of a collaborative decision-making method for intelligent networked vehicles based on dual interactive perception of the present invention;

[0042] Figure 4 It is a network framework diagram of a trajectory perception interactive encoder TAIE of an embodiment of an intelligent networked vehicle collaborative decision-making method based on dual interactive perception according to the present invention;

[0043] Figure 5 1 is a schematic diagram of a collision time adjustment decision output of an embodiment of a collaborative decision method for intelligent networked vehicles based on dual interactive perception according to the present invention, wherein (a) is a schematic diagram of lane change safety, and (b) is a schematic diagram of following vehicle safety judgment;

[0044] Figure 6 It is a network framework diagram of an integrated traffic dynamics representation module of an embodiment of an intelligent networked vehicle collaborative decision-making method based on dual interactive perception according to the present invention;

[0045] Figure 7 It is a schematic diagram of average speed comparison of 15 vehicles merging through a bottleneck area in an embodiment of an intelligent connected vehicle collaborative decision-making method based on dual interactive perception of the present invention;

[0046] Figure 8 It is a schematic diagram of average speed comparison of 25 vehicles merging through a bottleneck area in an embodiment of an intelligent connected vehicle collaborative decision-making method based on dual interactive perception of the present invention;

[0047] Fig. 9 The present invention is a schematic diagram of the comparison of vehicle trajectories generated by merging 15 vehicles in a bottleneck area in an embodiment of a collaborative decision-making method for intelligent connected vehicles based on dual interactive perception, wherein (a) is a schematic diagram of a mixed traffic flow using a dual interactive perception collaborative control strategy with an intelligent connected vehicle penetration rate of 0.4; (b) is a mixed traffic flow using a multi-agent reinforcement learning algorithm with an intelligent connected vehicle penetration rate of 0.4; and (c) is a purely manually driven traffic flow generated by a SUMO simulator. DETAILED DESCRIPTION

[0048] The technical solution of the present invention is further described below through the accompanying drawings and embodiments.

[0049] Unless otherwise defined, technical or scientific terms used in the present invention shall have the common meanings understood by one having ordinary skills in the field to which the present invention belongs.

[0050] Example

[0051] like Figure 1 As shown, a collaborative decision-making method for intelligent connected vehicles based on dual interactive perception includes the following steps:

[0052] A centralized training and distributed execution multi-agent reinforcement learning framework is proposed to design a dual mutual perception collaborative control strategy DIACC; the dual mutual perception collaborative control strategy DIACC includes a centralized critic network and several distributed executor networks.

[0053] The centralized critic network assists the training phase by accessing the global information of the mixed traffic system, and the distributed worker networks maximize their individual rewards by leveraging the global observations provided by the centralized critic network.

[0054] like Figure 2-Figure 3 As shown in the figure, first, the optimization problem of the mixed traffic system is modeled and characterized as a distributed partially observable Markov decision process. The multi-agent reinforcement learning model can be represented as a seven-tuple {N, S, A, O, R, P, γ}, where N is the set of intelligent connected vehicles in the mixed traffic environment, involving n intelligent connected vehicles; S represents the global state space of the environment; A is the joint action space of all intelligent connected vehicles; O represents the joint observation space of all intelligent connected vehicles; R is the joint reward function; P is the state transition probability function; γ is the discount factor, which is used to balance the importance of immediate rewards and future rewards.

[0055] In the setting, each intelligent connected car i observes its local state at any time t According to its strategy Select control decision instruction The policy is characterized by the parameter θ. The collective goal of all connected vehicles is to optimize the collective policy To maximize the expected accumulated reward within L time steps.

[0056] The state space, action space and reward function under the multi-agent reinforcement learning framework are designed according to the evolution characteristics of the hybrid traffic system and the vehicle interaction requirements.

[0057] The state space includes the position (x, y), velocity v, and heading angle of each vehicle. Lane index l and vehicle type c. Based on this, the roadside unit can obtain information about a specific lane, including the number of vehicles n l , density D l , average speed And the penetration rate of intelligent connected vehicles ρ l In addition, high-precision maps provide detailed road structure information, which constitutes the state information that the centralized enhanced evaluator module can read from the environment.

[0058] The bottleneck scenario is divided into three areas: multi-lane area, pre-merge area and post-merge area. In the multi-lane area and post-merge area, vehicles are evenly distributed in each lane and the speed is stable; in the pre-merge area, vehicles are concentrated in the center lane, and lateral conflicts increase, resulting in more frequent lane changes and greater speed fluctuations. The difference in behavior stems from the reduction in the number of lanes and traffic capacity in the bottleneck merging area, which leads to intensified vehicle competition and affects driving behavior. The above changes are driven by static road structure. Therefore, the local observation state of the intelligent connected vehicle is designed to include: the state information of the own vehicle, the state of adjacent vehicles, the statistical information of traffic flow in adjacent lanes, and the static road structure information.

[0059] The action space is a discrete decision-making behavior involving the lateral and longitudinal control of the vehicle. Specifically, there are five decision-making behaviors: maintaining the current lane and speed (denoted as "keep"), changing lanes to the left or right, and accelerating or decelerating.

[0060] The reward function is:

[0061]

[0062] Among them, r e 、r g and r tp Respectively represent the vehicle reward, global reward and lazy behavior penalty (or time penalty). e =ρ、w g =1-ρ and w tp =n-1+ρ is the weight of each reward component related to the penetration rate of intelligent connected vehicles. doneTo complete the reward, it is used to encourage smart connected cars to quickly pass through bottleneck areas, thereby improving traffic capacity. The vehicle reward itself includes several sub-rewards:

[0063] r e =w e,v r e,v +w e,w r e,w +w e,c r e,c ;

[0064]

[0065]

[0066]

[0067] Among them, r e,v is the speed reward; r e,w For warning distance penalty; r e,c is the collision penalty, w e,v 、w e,w and w e,c r e,v 、r e,w and r e,c The weight parameter v max is the maximum speed allowed by road traffic regulations; Indicates vehicles within the warning range; assembly Indicates the vehicles within the actual collision range; is the relative longitudinal distance between vehicles; is the relative lateral distance between vehicles, j is or Elements in a collection; and The distance thresholds are d th,w and d th,c ;p e,v Adjust parameters for speed bonus; p e,c Adjust parameters for collision penalty.

[0068] Global Rewards g It reflects the overall performance of all intelligent connected vehicles in terms of average speed, and its specific form is:

[0069]

[0070] Among them, p g,v Adjust parameters for the global speed bonus.

[0071] r tp Emphasizes the impact of the speed efficiency of intelligent connected vehicles on overall traffic flow:

[0072] r tp =10×sigmoid(v i -v th );

[0073] Among them, v th is the speed threshold used to adjust the penalty level at different speeds, and sigmoid(·) is the Sigmoid function. When dealing with global traffic coordination issues, some intelligent connected vehicles choose to slow down or even stop in multi-lane areas, which is likely to cause conflicts and congestion in the merging area. This design can avoid the inertial behavior of intelligent connected vehicles when facing congestion.

[0074] The distributed interactive adaptive decision module D-IADM is used to learn the interaction characteristics with surrounding vehicles and traffic environment information.

[0075] like Figure 4 As shown in the figure, the intelligent connected vehicle obtains local observations from the environment at each moment and outputs its control actions through a distributed interactive adaptive decision module D-IADM. D-IADM first obtains the interaction features between the vehicle itself and surrounding vehicles through the trajectory perception interaction encoder TAIE.

[0076] The local observation state of an intelligent connected vehicle includes its own state information, neighboring vehicle motion information, lane statistics information, and static road structure information. The six nearest vehicles in the same and adjacent lanes are included. Considering the behavioral differences between human-driven cars and intelligent connected cars, we classify neighboring vehicles into the human-driven car set and smart connected car collection Among them, H stands for human-driven vehicle (HDV); C stands for intelligent connected vehicle (CAV).

[0077] Taking the extraction of interaction features of human-driven cars as an example, an interaction graph is constructed between intelligent connected cars and surrounding human-driven cars, connecting each intelligent connected car with neighboring human-driven cars within its specified range. The interaction graph of human-driven cars is represented as in, Represents a node, are the edges connecting these nodes. It should be noted that these edges only connect the intelligent connected car with its neighboring human-driven cars, and there is no connection between human-driven cars.

[0078] The raw input consists of the current and historical states of the connected car and its surrounding human-driven cars: in, represents the number of human-driven cars around, and T is the control period. The state information m of each car includes position, speed and heading angle. The state information of the intelligent connected vehicle i at different times in the past; The adjacent manually driven cars Status information at different times in the past; The current state information of the intelligent connected vehicle i; Driving a car for a neighbor State information at the current moment. First, connect the historical information of each vehicle. First, connect the historical information of each vehicle and then embed it into the node features:

[0079]

[0080] Among them, h is the motion feature corresponding to each node; S H is the size of the hidden layer of the part of the network that handles the interaction with the human-driven vehicle. These features are then fed into a multi-head graph attention network (MH-GAT) to extract the human-driven car interaction features:

[0081]

[0082] in, is the interaction feature corresponding to each node; It is the interaction feature of the manually driven vehicle at the current moment.

[0083] A similar process is used to generate the intelligent connected car interaction graph. Unlike the human-driven car interaction graph, the input of this layer includes the current state and the last decision information of the intelligent connected car i and its surrounding intelligent connected cars:

[0084]

[0085] in, Represents the number of smart connected cars around; It is the decision information of the intelligent connected car at the last moment. Finally, the interactive features of the intelligent connected car are generated:

[0086]

[0087] in, is the interaction characteristics of the intelligent connected vehicles at the current moment; S C is the size of the hidden layer that handles the interaction with the intelligent connected car.

[0088] The self-attention mechanism quantifies the relative importance of each surrounding human-driven car to the intelligent connected vehicle i. These processes produce rich node feature representations and enhance the decision-making ability of the intelligent connected vehicle i.

[0089] Extract local traffic context information. The present invention uses traffic flow statistics from the own vehicle lane and adjacent lanes. The lane information of the own vehicle is expressed as The left lane information is represented as The right lane information is represented as And static road structure information F static This information is processed by a multi-layer perceptron (MLP) to obtain local traffic context features.

[0090] Then, the interaction features of human-driven cars, intelligent connected cars and local traffic context features are combined to form a comprehensive interaction environment. All embedded features are then integrated in a fusion layer based on a multi-layer perceptron to form a comprehensive observation embedding feature of the time step.

[0091] Then, the synthetic observations are embedded into the features and the last control decision instruction Input to the Gated Recurrent Unit (GRU) and decision layer to generate learnable decision instructions GRU captures the temporal dependency of observation features between different moments and enhances the continuity of decision making. The input is then a PSAF based on active safety, which evaluates the safety of actions based on the time to collision (TTC) criterion, filters out actions with higher collision risks, and provides safer decisions under current conditions. At the same time, PSAF maps the decision output to specific lateral and longitudinal control decisions. The lateral control decision involves a lane change action, which is expressed as:

[0092]

[0093] The longitudinal control decision involves a velocity action, expressed as Among them, v max It is the maximum speed allowed by road traffic regulations.

[0094] like Figure 5 As shown, when When changing lanes, PSAF first evaluates the vehicles in front and behind the target lane. The relevant index of the vehicle in front of the target lane is expressed as in, is the time distance between the vehicle and the preceding vehicle; is the relative distance between the vehicle and the vehicle in front. The index of the vehicle behind is expressed as is the inter-vehicle time distance between the vehicle and the following vehicle; is the relative distance between the vehicle and the following vehicle. As shown in (a), if the TTC or relative distance of the vehicle in the target lane is lower than a certain threshold, that is, the area within the blue frame, PSAF will refuse to change lanes and keep the vehicle in the current lane. If the TTC or relative distance to the vehicle in front is lower than a certain threshold, that is, the area within the orange frame, PSAF will recommend slowing down and changing lanes. The deceleration calculation formula is:

[0095]

[0096] in, is the relative speed between the smart connected car and the vehicle in front. In other cases, the vehicle will change lanes at a constant speed. When maintaining the lane, PSAF evaluates the TTC and relative distance between the current lane and the vehicle in front, as shown in (b). If the TTC or relative distance is lower than a certain threshold (dark blue area), PSAF will output a deceleration action. Its calculation method is similar to that of the lane change scenario. The light blue area indicates that the vehicle can maintain the current speed; the gray area indicates that the vehicle follows the decision instruction. The acceleration or deceleration is set to 2m / s. The next speed control action is determined based on the current speed of the intelligent connected vehicle and the calculated acceleration or deceleration. This action becomes the control decision instruction output by the D-IADM module. Used to control smart connected cars.

[0097] The relationship between global vehicle interactions and global traffic dynamics is captured using a centralized interaction enhanced evaluator module (C-IEC).

[0098] like Figure 6 As shown in the figure, the C-IEC module first obtains global traffic dynamic features through the integrated traffic dynamic representation module (ITDR). In the ITDR module, traffic information is divided into three layers: static road structure information, dynamic traffic information, and vehicle interaction information. The information of each layer is processed separately, among which the static road structure information and dynamic traffic information are processed by the multi-layer perceptron (MLP). The static road information is represented by F static ,The dynamic traffic information includes global lane statistics, all intelligent connected vehicles status information, and all manually driven vehicles status information, which are processed by MLP to obtain the corresponding embedding features, which are traffic flow features Intelligent connected car motion characteristics and the motion characteristics of manually driven cars These features are then concatenated to form dynamic embedding features

[0099] The vehicle interaction information is processed by the Graph Attention Network (GAT) to extract the global vehicle interaction features. Here, a global vehicle interaction graph is constructed, represented as Gglobal =(V,E), where V represents all vehicle nodes and E is the edge connecting these nodes, where only adjacent vehicles are connected, i.e. The information of each vehicle node includes position, speed and heading angle. These data are first passed through the embedding layer to obtain node features. Among them, h represents the characteristics corresponding to each node; N represents the total number of vehicles in the mixed traffic environment; S N is the hidden layer size of the network part that handles all vehicle interactions. The node features are then fed into GAT to extract interaction features is the interaction feature corresponding to each node.

[0100] The present invention uses dynamic traffic information embedding features as query objects, global vehicle interaction features as key values, and uses a multi-head cross attention mechanism to calculate global vehicle interaction features. Key Projection Matrix Sum Projection Matrix The query matrix Q can be obtained h , key matrix K h , value matrix V h :

[0101]

[0102] Then, the scaled dot-product attention is calculated, and the concatenation and final linear transformation are applied. By using multi-head criss-cross attention, we are able to capture and map the relationship between traffic dynamics and vehicle interactions, thus gaining a comprehensive understanding of traffic evolution.

[0103] These features are then fed into the GRU module to capture the temporal dependencies between global states. Finally, the fully connected layer outputs the global state value function and advantage function evaluation at the current time step t to assess the overall state quality and guide the parameter update in D-IADM.

[0104] In order to evaluate the effectiveness and superiority of the method proposed in the present invention, a mixed traffic system simulation environment was built based on the SUMO simulator and the effect of the method of the present invention was tested. In the test scenario, the penetration rate of intelligent connected vehicles is 0.4. At the same time, three different driving styles of manually driven vehicles are set based on the vehicle control model provided by SUMO: aggressive, normal and cautious. Aggressive driving is characterized by shorter following distances and more frequent lane changes, while cautious driving is characterized by longer following distances and fewer lane changes. Normal driving uses the default parameters of the SUMO model.

[0105] We evaluated the average vehicle speed in the 15-vehicle and 25-vehicle scenarios using three settings: pure manual driving traffic using the IDM+LC2013 model (SUMO’s built-in vehicle control model); mixed traffic using the MAPPO-IADM model (a multi-agent reinforcement learning algorithm that integrates the interactive adaptive decision module D-IADM designed in this paper) when the ICV penetration rate is 0.4; and mixed traffic using the DIACC model when the ICV penetration rate is 0.4.

[0106] like Figure 7-Figure 8 As shown in the figure, the performance of all vehicles in three areas is evaluated: multi-lane area, pre-merge area, and post-merge area. The mixed traffic scenarios using MAPPO-IADM and DIACC models show higher average speeds than the pure human-driven car scenario using the SUMO default model. The initial speed of all vehicles is 10m / s. In the multi-lane area, the vehicles mainly accelerate, and the performance of the three traffic settings is similar. However, in the pre-merge area, the pure human-driven car traffic shows a lower average speed, while the mixed traffic using MAPPO-IADM and DIACC models maintains a higher average speed, especially in the 25-vehicle scenario. In addition, the distribution of vehicle average speeds is more concentrated in the mixed traffic scenarios using these two models, while the distribution of vehicle average speeds is wider in the pure human-driven car traffic scenario. This shows that when using MAPPO-IADM and DIACC models, the intelligent connected vehicles better coordinate the traffic flow in the pre-merge area, which can not only achieve its own efficient driving, but also guide the surrounding human-driven cars to maintain a higher speed, thereby improving the overall traffic efficiency. In addition, the mixed traffic using the MAPPO-IADM model has a wider speed distribution in the pre-merge area and the post-merge area, while the mixed traffic using the DIACC model has a more concentrated speed distribution.

[0107] In addition, if Fig. 9 As shown, the present invention evaluates vehicle trajectories using different models in a 15-vehicle scenario. In the pure manually driven car traffic scenario, congestion occurred in the pre-merge area, which was manifested by intensified vehicle conflicts and multiple lane changes. In addition, there were two vehicles stationary in this area, and the resulting waiting time undoubtedly increased the completion time of the vehicle merge. In contrast, the mixed traffic system using the MAPPO-IADM model performed better overall, and the intelligent connected vehicles began to adjust their behavior in the multi-lane area. In the pre-merge area, vehicle conflicts were significantly reduced, and lane changes were also reduced, although one vehicle still experienced waiting time. The mixed traffic system using the DIACC model performed best, with reduced lane changes in the pre-merge area, no vehicles experiencing waiting time, smoother vehicle trajectories, and higher behavioral consistency.

[0108] Therefore, the present invention adopts the above-mentioned intelligent networked vehicle collaborative decision-making method based on dual interactive perception, so that the intelligent networked vehicle plays a vital role in coordinating traffic in bottleneck scenarios. On the one hand, the intelligent networked vehicle proactively adjusts its behavior in advance, reducing the lateral conflict with other vehicles in the pre-merge area. On the other hand, the behavior of the intelligent networked vehicle actively guides and coordinates the manually driven vehicle, thereby improving the overall traffic performance.

[0109] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solution of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solution to deviate from the spirit and scope of the technical solution of the present invention.

Claims

1. A collaborative decision-making method for intelligent connected vehicles based on dual interactive perception, characterized in that: The following steps are involved: Using the centralized training and distributed execution multi-agent reinforcement learning framework, a dual mutual perception collaborative control strategy DIACC is designed; Use the distributed interactive adaptive decision module D-IADM to learn the interaction characteristics and traffic environment information with surrounding vehicles; The relationship between global vehicle interactions and global traffic dynamics is captured using a centralized interaction enhanced evaluator module (C-IEC).

2. The intelligent connected vehicle collaborative decision-making method based on dual interactive perception according to claim 1 is characterized by: The dual mutual perception collaborative control strategy DIACC includes a centralized critic network and several distributed executor networks; The centralized critic network assists the training phase by accessing the global information of the mixed traffic system, and the distributed worker networks maximize their individual rewards by leveraging the global information provided by the centralized critic network.

3. The intelligent connected vehicle collaborative decision-making method based on dual interactive perception according to claim 2 is characterized by: Design the state space, action space and reward function in the multi-agent reinforcement learning framework according to the evolution characteristics of the hybrid traffic system and the vehicle interaction requirements; Designing the state space under the multi-agent reinforcement learning framework includes the motion state of each vehicle, traffic flow statistics of each lane, and static road structure information; The discrete quantities in the designed action space include: hold, left lane change, right lane change, acceleration, and deceleration; the designed reward function is: Among them, r e 、r g and r tp They represent the vehicle reward, global reward and lazy behavior penalty, or time penalty, respectively, and the coefficient w e =ρ、w g =1-ρ and w tp =n-1+ρ is the weight of each reward component related to the penetration rate of intelligent connected vehicles; r done To complete the reward, the vehicle reward r e Contains several sub-rewards: r e =w e,v r e,v +w e,w r e,w +w e,c r e,c ; Among them, r e,v is the speed reward; r e,w For warning distance penalty; r e,c is the collision penalty, w e,v 、w e,w and w e,c r e,v 、r e,w and r e,c The weight parameter v max is the maximum speed allowed by road traffic regulations; Indicates vehicles within the warning range; assembly Indicates the vehicles within the actual collision range; is the relative longitudinal distance between vehicles; is the relative lateral distance between vehicles, j is or Elements in a collection; and The distance thresholds are d th,w and d th,c ;p e,v Adjust parameters for speed bonus; p e,c Adjust parameters for collision penalties; Global Rewards g Reflects the overall performance of all intelligent connected vehicles in terms of average speed, which is as follows: Among them, p g,v Adjust parameters for global speed bonus; r tp Reflects the impact of the speed efficiency of intelligent connected vehicles on the overall traffic flow: r tp =10×sigmoid(v i -v th ); Among them, v th is the speed threshold used to adjust the penalty level at different speeds, and sigmoid(·) is the Sigmoid function.

4. The intelligent connected vehicle collaborative decision-making method based on dual interactive perception according to claim 1 is characterized by: D-IADM contains a trajectory-aware interaction encoder TAIE, which obtains the interaction features between the vehicle itself and surrounding vehicles. Then, it combines the previous decision output with the gated recurrent unit GRU and the decision layer to generate a learnable action output, which is then input into the active safety-based action filter PSAF.

5. The intelligent connected vehicle collaborative decision-making method based on dual interactive perception according to claim 4 is characterized by: Establish an interaction graph between the vehicle and surrounding human-driven cars, use the current state information and historical trajectory information of the vehicle and surrounding human-driven cars as input, and use the multi-head graph attention mechanism to extract the interaction features with surrounding human-driven cars; Establish an interaction graph between the vehicle and surrounding intelligent connected vehicles, use the current state information of the vehicle and surrounding intelligent connected vehicles and the decision output at the last moment as input, and use the multi-head graph attention mechanism to extract the interaction features with surrounding intelligent connected vehicles; Taking the traffic flow statistics of adjacent lanes and the static road structure information as input, a multi-layer perceptron is used to extract the traffic environment context information.

6. The intelligent connected vehicle collaborative decision-making method based on dual interactive perception according to claim 5 is characterized by: The interaction features of surrounding human-driven cars, the interaction features of surrounding intelligent connected cars, the traffic environment context information and the decision instructions at the previous moment are integrated to obtain comprehensive features, which are then input into the gated recurrent unit GRU and the decision layer to obtain learnable action outputs; The active safety action filter PSAF uses the collision time TTC as an active safety evaluation indicator, optimizes actions when the TTC is lower than the set threshold, and outputs the longitudinal and lateral decisions of the intelligent connected vehicle.

7. The intelligent connected vehicle collaborative decision-making method based on dual interactive perception according to claim 1 is characterized by: The centralized interaction enhancement evaluator module C-IEC receives all state information from the environment. C-IEC includes an integrated traffic dynamic representation module ITDR, which captures traffic dynamics and vehicle interaction characteristics and outputs a global state value function.

8. The intelligent connected vehicle collaborative decision-making method based on dual interactive perception according to claim 7 is characterized by: The multi-layer perceptron is used to obtain the traffic flow characteristics of each lane in the mixed traffic system, the movement characteristics of manually driven cars, and the movement characteristics of intelligent connected cars, and then the dynamic characteristics of the comprehensive traffic flow are obtained by combining the static road structure information.

9. The intelligent connected vehicle collaborative decision-making method based on dual interactive perception according to claim 8 is characterized by: An interaction graph of all vehicles in a mixed traffic system is established. The current motion state of each vehicle is used as input, and the graph attention mechanism is used to obtain the global vehicle interaction features.

10. The intelligent connected vehicle collaborative decision-making method based on dual interactive perception according to claim 9 is characterized by: The multi-head cross-attention mechanism is used to capture the characteristics of traffic dynamics and vehicle interactions by taking comprehensive traffic flow dynamic features as query objects and global vehicle interaction features as keys and values.

Images