Cross-attention-based autonomous driving cooperative collision avoidance method

Abstract

The present application relates to the technical field of traffic control and automatic driving, and provides an automatic driving cooperative risk avoidance method based on cross attention, acquires a local observation feature vector of each automatic driving intelligent agent and constructs a global mask matrix, inputs the local observation feature vector into a policy network and masks the states of non-activated or having left automatic driving intelligent agents, fuses the spatio-temporal interaction features of the current automatic driving intelligent agent and other surviving automatic driving intelligent agents, outputs a control action probability distribution, samples a cooperative risk avoidance control action, issues the control action to a traffic simulation environment, updates a simulation step and triggers a dynamic accident state machine, acquires the next time state of each surviving automatic driving intelligent agent and updates the global mask matrix, calculates a multi-level risk avoidance comprehensive reward value corresponding to each surviving automatic driving intelligent agent, stores the data into an experience replay pool to calculate a global state evaluation value, and updates parameters until a preset convergence condition is met.

Description

Technical Field

[0001] This invention relates to the technical fields of traffic control and autonomous driving, and in particular to a collaborative risk avoidance method for autonomous driving based on cross-attention. Background Technology

[0002] With the rapid evolution of autonomous driving technology, future urban roads and highways will be in a long-term stage of mixed traffic flow consisting of manually driven vehicles (HDVs) and autonomous vehicles (CAVs). In mixed traffic flow scenarios, sudden vehicle mechanical failures on main roads (such as breakdowns or tire blowouts) and high-density merging at ramps can easily trigger micro-shock waves in the traffic flow, causing vehicles to accelerate and decelerate frequently, which in turn can lead to secondary rear-end collisions and widespread traffic congestion.

[0003] In recent years, multi-agent reinforcement learning (MARL) technology has been introduced into the field of autonomous driving cooperative control. Typical methods such as multi-agent proximal policy optimization (MAPPO) and multi-agent deep deterministic policy gradient (MADDPG) have enabled autonomous vehicles to achieve collaborative decision-making among multiple vehicles to a certain extent by giving them autonomous learning capabilities, thereby improving local traffic efficiency.

[0004] However, existing neural network architectures typically require fixed-dimensional state tensor inputs and usually employ "zero-padding" to align dimensions. Real traffic flow is highly non-stationary, with vehicles frequently entering and leaving the control network, causing the number of agents in the collaborative control area to dynamically increase. When calculating attention weights, meaningless padding vectors are easily included in the calculation, leading to vanishing gradients or numerical computational failures.

[0005] Furthermore, existing simulation control and training methods often simplify sudden accidents into a model of a suddenly appearing static road obstacle, failing to realistically reflect the continuous physical evolution of a malfunctioning vehicle in reality from "power failure - slow deceleration - lateral sliding towards the emergency lane - complete stabilization". This results in the trained model lacking the ability to defend against dynamic hazards. Due to the lack of a forward-looking prevention mechanism for the area affected by the hazard, autonomous vehicles may blindly enter the high-risk lane affected by the accident in order to pursue local traffic efficiency during the hazard avoidance process, which may lead to serious secondary collisions. Summary of the Invention

[0006] To overcome the shortcomings of the prior art, this invention provides an autonomous driving cooperative risk avoidance method based on cross-attention, which solves the problem of feature dimension conflict in traditional fixed-dimensional neural networks caused by the frequent entry and exit of autonomous driving agents in mixed traffic flow. It effectively avoids the risk of invalid gradient calculation and Softmax (flexible maximum transfer function) numerical overflow caused by the traditional "zero-fill" strategy, and realizes adaptive beyond-line-of-sight cooperative perception of any number of surrounding vehicles.

[0007] This invention provides a cooperative hazard avoidance method for autonomous driving based on cross-attention, comprising: Step S1: Obtain the local observation feature vector of each autonomous driving agent in the road network, and construct a global mask matrix containing the survival status of all autonomous driving agents. Step S2: Each autonomous driving agent inputs its local observation feature vector into the policy network, uses the global mask matrix to dynamically mask the state of inactive or departing autonomous driving agents, fuses the spatiotemporal interaction features of the current autonomous driving agent with other surviving autonomous driving agents, outputs the probability distribution of control actions, and samples to generate collaborative risk avoidance control actions. Step S3: Send the collaborative risk avoidance control action to the traffic simulation environment for execution, update the simulation step and trigger the dynamic accident state machine, obtain the next moment state of each surviving autonomous driving agent and update the global mask matrix according to the next moment state, and independently calculate and feed back the corresponding multi-level risk avoidance comprehensive reward value according to the state of each surviving autonomous driving agent. Step S4: Store the data generated in steps S1 to S3 into the experience replay pool, calculate the global state evaluation value through the centralized value network, and update the parameters of the policy network and the centralized value network until the preset convergence condition is met. The preset convergence conditions include: the update magnitude of the policy network parameters is less than the set threshold within multiple consecutive training cycles, and the cumulative multi-level risk avoidance comprehensive reward value, collision rate and traffic efficiency evaluation indicators of the autonomous driving agent tend to stabilize. Step S5: Generate collaborative risk avoidance control actions using the trained policy network to achieve collaborative risk avoidance for autonomous driving.

[0008] As a further technical solution, step S3 also includes: Step S31: Send the collaborative risk avoidance control action to the traffic simulation environment for execution, advance the simulation steps, and detect whether the pre-set dynamic accident state machine in the simulation environment has traveled to the preset accident section and crossed the starting longitudinal coordinate. If so, it is determined that a mechanical failure has occurred and the dynamic accident state machine is activated. Step S32: Release the initial speed limit constraint of the dynamic accident state machine and set the fault low speed limit threshold. The fault low speed limit threshold is used to limit the maximum driving speed of the dynamic accident state machine in the process of moving towards the emergency lane after the fault occurs. At the same time, issue a forced lane change command to move towards the emergency lane and record the initial accident lane when the fault occurs. Step S33: During the process of the dynamic accident state machine transferring to the emergency lane, the actual lane where the center point of the vehicle body of the dynamic accident state machine is located is tracked in real time, and the accident lane index is dynamically updated. Step S34: When the dynamic accident state machine is found to have completely merged into the emergency lane and its longitudinal driving speed has decreased to below the set stopping threshold, the lateral and longitudinal motion control permissions of the dynamic accident state machine are cancelled, and the dynamic accident state machine is marked as a permanently stopped static obstacle in the traffic simulation environment. Step S35: Obtain the next moment state of each surviving autonomous driving agent, update the global mask matrix according to the next moment state, and independently calculate and feed back the corresponding multi-level risk avoidance comprehensive reward value for each surviving autonomous driving agent based on its own state.

[0009] As a further technical solution, the method for obtaining the multi-level risk avoidance comprehensive reward value includes: If the autonomous driving agent triggers a physical collision during the current control cycle, a fixed negative hard collision penalty will be directly assigned, and the multi-level risk avoidance comprehensive reward value will no longer be calculated. If the autonomous driving agent does not trigger a physical collision during the current control cycle, then the multi-level risk avoidance comprehensive reward value is calculated. ; ; ; ; ; in, This is a multi-level risk avoidance comprehensive reward value. To assign a weighting coefficient to the speed reward, To prevent rear-end collisions, a weighting coefficient is set for safety penalties. To set a weighting coefficient for the merging conflict penalty, A weighting coefficient is set for the penalty for staying in the accident lane. Set a weighting coefficient for lane-changing penalties in high-risk areas. This represents the speed bonus calculated based on the difference between the current vehicle speed and the expected ideal vehicle speed on the road. This indicates a penalty for failing to prevent rear-end collisions. Indicates punishment for merging conflicts. This indicates a penalty for being stranded in the accident lane. This indicates a penalty for changing lanes in a high-risk area. The vehicle's current speed. This refers to the collision time between the autonomous driving agent and the vehicle in front. To accelerate the total length of the lane, This is the current vertical position. To achieve the desired ideal speed on the road, This represents the relative longitudinal distance between the autonomous driving agent and the dynamic accident site. , To fix the calculation parameters, The critical point of the interval, , This is the set spatial attenuation scale constant.

[0010] As a further technical solution, the cooperative risk avoidance control action includes discrete control commands in multiple dimensions: Maintain your current lane and speed; After verifying the safety status of the adjacent lane on the left, a left lane change control command is triggered; Provided that the safety status of the adjacent lane on the right has been verified, a right lane change control command is triggered; Trigger a longitudinal acceleration command to control the current speed to increase in preset step increments without exceeding the global desired upper limit speed; Trigger a longitudinal deceleration command to control the current speed to decrease in preset steps and not fall below the set safety lower limit speed.

[0011] As a further technical solution, the process of integrating the spatiotemporal interaction characteristics of the currently described autonomous driving intelligent agent with other surviving autonomous driving intelligent agents includes: Step Z1: Encode the local observation feature vector of the current autonomous driving agent into its own dense feature vector, and generate a query vector by linear mapping the own dense feature vector; Step Z2: Map the local observation feature vectors of all the autonomous driving agents in the road network to generate a key matrix and a value matrix; Step Z3: Extract the survival status bits of all other autonomous driving agents except the current autonomous driving agent from the global mask matrix and invert them to construct an attention-filled masking matrix; if it is detected that all other autonomous driving agents are in an ungenerated or departed state, then activate the full masking protection mechanism. Step Z4: If no other autonomous driving agents are detected to be in an ungenerated or departed state, then calculate the context interaction feature vector based on the attention-filled occlusion matrix, concatenate the context interaction feature vector with the self-dense feature vector, input it to the action decision layer, and output the control action probability distribution.

[0012] As a further technical solution, methods for calculating context interaction feature vectors include: Based on the attention-filled masking matrix, a weighted sum is calculated between the key matrix and the value matrix; ; ; in, For autonomous driving intelligent agents Contextual interaction feature vectors For autonomous driving intelligent agents The linear mapping generates the query vector. For autonomous driving intelligent agents Its own dense feature vector, The query projection matrix parameters are learnable in the policy network. The scaling factor for the dimensions of the attention feature vector is set. Fill the occlusion matrix with attention. V is the key matrix, and V is the value matrix. Represents the key matrix Perform matrix transpose.

[0013] As a further technical solution, the method for constructing the key matrix and the value matrix includes: ; ; in, The key matrix, It is a set of locally observed feature vectors. These are the learnable key projection matrix parameters in the policy network. For value matrices, The parameters are the projection matrix parameters for the learnable values ​​in the policy network.

[0014] As a further technical solution, the local observation feature vector includes road topology coding, vehicle motion state, surrounding interactive traffic flow, and dynamic accident zone situational awareness information; The road topology coding includes the one-hot coding of the road segment where the current autonomous driving agent is located, the normalized lane index, the normalized longitudinal position coordinates, and the normalized current driving speed; The vehicle motion state includes the interaction features of the leading vehicle in front and the following vehicle behind the current autonomous driving agent. The surrounding interactive traffic flow includes lateral safety prediction status positions, which include the left adjacent lane safety status position based on spatiotemporal safety distance prediction and the right adjacent lane safety status position based on spatiotemporal safety distance prediction. The dynamic accident zone situational awareness information includes the normalized feature of the remaining distance in the merging zone, the normalized feature of the relative longitudinal distance of the dynamic accident point, and the marker position of the danger zone.

[0015] The present invention provides an electronic device including a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements an autonomous driving cooperative risk avoidance method based on cross-attention.

[0016] The present invention provides a computer program product comprising a computer program / instruction that, when executed by a processor, implements a cross-attention-based cooperative risk avoidance method for autonomous driving.

[0017] In summary, the present invention has at least one of the following beneficial technical effects: 1. This invention solves the problem of feature dimension conflict in traditional fixed-dimensional neural networks caused by the frequent entry and exit of autonomous driving agents in mixed traffic flow by constructing a global mask matrix (AliveMask) and introducing a cross-attention mechanism. It effectively avoids the risk of invalid gradient calculation and Softmax (flexible maximum transfer function) numerical overflow caused by the traditional "zero-fill" strategy, and realizes adaptive beyond-line-of-sight collaborative perception of any number of surrounding vehicles.

[0018] 2. This invention extracts and inverts the survival status bits of other intelligent agents besides the current intelligent agent in the global mask matrix to construct an attention-filled masking matrix. Combined with a full masking protection mechanism, it forces the context feature vector to be cleared to zero in extreme empty scenarios, avoiding numerical calculation anomalies. It abandons the traditional single linear penalty mode and innovatively designs a multi-level reward function, effectively cutting off the negative behavior path of vehicles blindly cutting in and overtaking. This enables mixed traffic flow to spontaneously form an advanced collaborative risk avoidance strategy of "early warning from a distance and staggered left lane changes", significantly reducing the risk of secondary collisions in sudden accident scenarios. Attached Figure Description

[0019] Figure 1 This is a flowchart of the workflow for a collaborative risk avoidance method for autonomous driving based on cross-attention.

[0020] Figure 2 This is a schematic diagram of the strategy network and value network architecture.

[0021] Figure 3 It is a schematic diagram of the evolution of the dynamic accident state machine and the topology of the traffic network.

[0022] Figure 4 This is a comparison convergence curve of the average reward under different numbers of agents.

[0023] Figure 5 This is a convergence curve comparing the average speed of autonomous vehicles with different numbers of intelligent agents.

[0024] Figure 6 It is a curve comparing the average speed of the system under different traffic volumes. Detailed Implementation

[0025] The embodiments of the present invention are described in detail below, and examples of the embodiments are shown in the accompanying drawings.

[0026] In the description of this specification, the references to "certain embodiments," "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples" indicate that a specific feature, structure, material, or characteristic described in connection with the described embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0027] Please refer to Figure 1 As shown, this invention discloses a cooperative hazard avoidance method for autonomous driving based on cross-attention, comprising the following steps: Step S1: Obtain the local observation feature vector of each autonomous driving agent in the road network, and construct a global mask matrix containing the survival status of all autonomous driving agents. The local observation feature vector includes road topology coding, vehicle motion status, surrounding interactive traffic flow, and dynamic accident zone situational awareness information. In this embodiment, a mixed traffic flow network is constructed using microscopic traffic simulation software. The specific composition and physical meaning of the local observation feature vectors are shown in Table 1: Table 1. Explanation of the composition of local observation feature vectors (16 dimensions) of autonomous driving intelligent agents.

[0028] Meanwhile, a global mask matrix is ​​generated in each control cycle. For autonomous driving agents that are currently alive in the road network, the mask value is set to 1.0; for autonomous driving agents that have left the road network or have not yet been generated, the mask value is set to 0.0. By embedding a multi-head cross-attention mechanism into the action decision-making process of the policy network (Actor network) and combining it with a global mask matrix (Alive Mask) extracted from the simulation environment, the interference features of non-live agents can be dynamically filtered out, realizing adaptive beyond-line-of-sight cooperative perception for any number of surrounding vehicles.

[0029] Step S2, as follows Figure 2 The policy network diagram shown in subgraph (a) is as follows: Each autonomous driving agent inputs its own local observation feature vector into the policy network, uses the global mask matrix to dynamically mask the state of inactive or departed autonomous driving agents, fuses the spatiotemporal interaction features of the current autonomous driving agent with other surviving autonomous driving agents, outputs the probability distribution of control actions, and samples to generate collaborative risk avoidance control actions. In this embodiment, the collaborative risk avoidance control action includes discrete control commands in multiple dimensions: Maintain your current lane and speed; After verifying the safety status of the adjacent lane on the left, a left lane change control command is triggered; Provided that the safety status of the adjacent lane on the right has been verified, a right lane change control command is triggered; Trigger a longitudinal acceleration command to control the current speed to increase in preset step increments without exceeding the global desired upper limit speed; Trigger a longitudinal deceleration command to control the current speed to decrease in preset steps and not fall below the set safety lower limit speed.

[0030] In this embodiment, the process of fusing the spatiotemporal interaction features of the currently described autonomous driving agent with those of other surviving autonomous driving agents includes: Step Z1: Encode the local observation feature vector of the current autonomous driving agent into its own dense feature vector, and generate a query vector by linear mapping the own dense feature vector; ; For autonomous driving intelligent agents The linear mapping generates the query vector. For autonomous driving intelligent agents Its own dense feature vector, These are the parameters of the query projection matrix that can be learned in the policy network; Step Z2: Map the local observation feature vectors of all the autonomous driving agents in the road network to generate a key matrix and a value matrix; ; ; in, The key matrix, It is a set of locally observed feature vectors. These are the learnable key projection matrix parameters in the policy network. For value matrices, The parameters are the projection matrix parameters for the learnable values ​​in the policy network; Step Z3: Extract the survival status bits of all autonomous driving agents other than the current autonomous driving agent in the global mask matrix and invert them to construct the attention-filled masking matrix; Step Z4: If none of the other autonomous driving agents are detected to be in an ungenerated or departed state, calculate a valid context interaction feature vector based on the attention-filled occlusion matrix; otherwise, activate the full occlusion protection mechanism to force the context feature vector to be... Reset to zero; Methods for computing effective contextual interaction feature vectors include: Based on the attention-filled masking matrix, a weighted sum is calculated between the key matrix and the value matrix; ; in, For autonomous driving intelligent agents Context feature vectors, This is a scaling factor for the dimensions of the attention feature vector, used to prevent gradient vanishing due to excessively large dot product values. Fill the occlusion matrix with attention. Represents the key matrix Perform matrix transpose.

[0031] The context interaction feature vector With its own dense feature vector The data is concatenated and input into the action decision layer, where it is output as a 5-dimensional control action probability distribution via the Softmax function.

[0032] Step S3: Send the collaborative risk avoidance control action to the traffic simulation environment for execution, update the simulation step and trigger the dynamic accident state machine, obtain the next moment state of each surviving autonomous driving agent, update the global mask matrix according to the next moment state, and independently calculate and feed back the corresponding multi-level risk avoidance comprehensive reward value for each surviving autonomous driving agent according to its own state. like Figure 3 The dynamic accident state machine evolution and traffic network topology diagram shown below, the specific working process of step S3 includes: Step S31: Send the collaborative risk avoidance control action to the traffic simulation environment for execution, advance the simulation steps, and detect whether the pre-set dynamic accident state machine in the simulation environment has traveled to the preset accident section and crossed the starting longitudinal coordinate. If so, it is determined that a mechanical failure has occurred and the dynamic accident state machine is activated. Step S32: Release the initial speed limit constraint of the dynamic accident state machine and set the fault low speed limit threshold. The fault low speed limit threshold is used to limit the maximum driving speed of the dynamic accident state machine in the process of moving towards the emergency lane after the fault occurs. At the same time, issue a forced lane change command to move towards the emergency lane and record the initial accident lane when the fault occurs. Step S33: During the process of the dynamic accident state machine transferring to the emergency lane, the actual lane where the center point of the vehicle body of the dynamic accident state machine is located is tracked in real time, and the accident lane index is dynamically updated. Step S34: When the dynamic accident state machine is found to have completely merged into the emergency lane and its longitudinal driving speed has decreased to below the set stopping threshold, the lateral and longitudinal motion control permissions of the dynamic accident state machine are cancelled, and the dynamic accident state machine is marked as a permanently stopped static obstacle in the traffic simulation environment. Step S35: Obtain the next moment state of each surviving autonomous driving agent, update the global mask matrix according to the next moment state, and independently calculate and feed back the corresponding multi-level risk avoidance comprehensive reward value for each surviving autonomous driving agent based on its own state.

[0033] By constructing a high-precision dynamic accident physical state machine in a microscopic simulation environment, it is possible to track the real lane where the physical center point of the faulty vehicle is located in real time at high frequency, perfectly reproducing the dangerous evolution process of the faulty vehicle slowly approaching the emergency lane across multiple lanes.

[0034] The methods for obtaining the multi-level risk aversion comprehensive reward value include: If the autonomous driving agent triggers a physical collision during the current control cycle, a fixed negative hard collision penalty will be directly assigned, and the multi-level risk avoidance comprehensive reward value will no longer be calculated. If the autonomous driving agent does not trigger a physical collision during the current control cycle, then the multi-level risk avoidance comprehensive reward value is calculated. ; ; ; ; ; in, This is a multi-level risk avoidance comprehensive reward value. To assign a weighting coefficient to the speed reward, To prevent rear-end collisions, a weighting coefficient is set for safety penalties. To set a weighting coefficient for the merging conflict penalty, A weighting coefficient is set for the penalty for staying in the accident lane. Set a weighting coefficient for lane-changing penalties in high-risk areas. This represents the speed bonus calculated based on the difference between the current vehicle speed and the expected ideal vehicle speed on the road. This indicates a penalty for failing to prevent rear-end collisions. Indicates punishment for merging conflicts. This indicates a penalty for being stranded in the accident lane. This indicates a penalty for changing lanes in a high-risk area. The vehicle's current speed. This refers to the collision time between the autonomous driving agent and the vehicle in front. To accelerate the total length of the lane, This is the current vertical position. To achieve the desired ideal speed on the road, This represents the relative longitudinal distance between the autonomous driving agent and the dynamic accident site. , To fix the calculation parameters, The critical point of the interval, , The set spatial attenuation scale constant; For example, the weighting coefficients for each reward / penalty are set as follows: Speed ​​weight Safety weight Merging weight Accident Delay Weight Change lane weight And set the spatial attenuation scale constant. , Fixed calculation parameters , Interval critical point ; Assuming that at the current simulation time step, the controlled autonomous driving agent is in the acceleration lane preparing to merge, and there is an accident vehicle on the main road ahead, the system obtains the following real-time status data: Ideal speed on the road (Approximately 120 km / h), current speed of the autonomous driving intelligent vehicle. ; An autonomous driving agent is approaching the vehicle in front, whose speed is [missing information]. And the distance between the car and the car in front The calculated collision time ; The autonomous driving agent is currently positioned longitudinally in the acceleration lane. The total length of the acceleration lane The relative longitudinal distance between the autonomous driving agent and the dynamic accident site ahead. An autonomous driving agent attempts a dangerous lane change into the accident lane in a hazardous area, triggering a high-risk lane change penalty. .

[0035] Based on the above scenario data, the specific calculation process of the control decision module of this invention is as follows: Efficiency Rewards Calculate and substitute the current speed and the ideal speed: ; Rear-end collision safety penalties calculate, ; Merging Conflict Punishment calculate: ; Penalty for being stranded in the accident lane calculate: ; Total Reward calculate: ; In the above embodiment, although the autonomous driving agent maintained a relatively reasonable speed (earning a positive reward of 0.36), it also faced the risk of rear-end collision, was in the forced lane change pressure zone at the end of a merging lane, and mistakenly initiated a dangerous lane change in the lane ahead of the accident lane. The cumulative effect of these penalties led to a decrease in the final comprehensive reward. It is a relatively small negative value (-2.4795); This negative reward signal will be fed back to the reinforcement learning network, updating the network parameters through backpropagation. This will effectively suppress the autonomous driving agent from taking such high-risk actions in similar future conditions, guiding it to learn to make safe and collaborative risk avoidance decisions in complex road sections with frequent accidents and merging traffic.

[0036] Step S4: Store the data generated in steps S1 to S3 into the experience replay pool, calculate the global state evaluation value through a centralized value network, and then... Figure 2 The policy network shown in subgraph (a) and Figure 2 The value network shown in subgraph (b) is updated with parameters until the model converges.

[0037] Specifically, this embodiment uses the Proximal Policy Optimization (MAPPO) algorithm based on a centralized training and distributed execution architecture for iteration: The observation data from the experience replay pool is spliced ​​together and then input into the centralized value network to evaluate the global state value. The system reads the local observation information, actions, and environmental feedback rewards of all autonomous driving agents within each control cycle stored in the experience replay pool, and concatenates the local observation vectors of each autonomous driving agent at the same time step to form the global state input. : ; in, For global state input, Indicates the first An autonomous driving agent at time step Local observation information, This represents the total number of autonomous driving agents; The global state The input is a centralized value network, which undergoes forward propagation through multiple fully connected layers to output the global state value estimate for the corresponding time step. ; ; in, For the mapping function of the centralized value network, A set of learnable parameters for a centralized value network; Calculating the action advantage function using generalized advantage estimation ; Based on environmental feedback rewards and state value estimation results at adjacent time steps, the action advantage function is calculated using the generalized advantage estimation method: ; in, For the action advantage function, As a discount factor, These are parameters used to measure deviation. For the summation index, The attenuation coefficient; in: ; in, In time step The rewards given to all autonomous driving agents after they perform actions. This represents the global state at the next time step. Finally, based on the aforementioned action advantage function Construct a truncated agent objective function and use an optimizer to update the policy network parameters, so that the probability distribution of the policy network output actions is gradually optimized; ; in, To truncate the proxy objective function, Indicates a time step within a control cycle Expected value based on experience The importance sampling ratio, This is a truncation function. The set truncation hyperparameter; By limiting the step size of policy updates to a defined range, it is possible to effectively prevent a single iteration from causing destructive updates to the policy network. Step S5: Generate collaborative risk avoidance control actions using the trained policy network to achieve collaborative risk avoidance for autonomous driving.

[0038] To further verify the effectiveness and practicality of the above-mentioned autonomous driving cooperative risk avoidance method based on cross-attention multi-agent reinforcement learning, this embodiment built a high-fidelity mixed traffic flow simulation test environment based on the microscopic traffic simulation platform SUMO, and carried out multi-agent scalability analysis and robustness analysis under different traffic flow conditions.

[0039] The physical modeling parameters such as the micro-road network size and traffic flow boundary constraints in the simulation experiment strictly refer to the relevant standard system. The specific hyperparameters and reward function weights of the reinforcement learning model in this embodiment are shown in Table 2. Table 2. Hyperparameters and Reward Weights Configuration for Multi-Agent Reinforcement Learning Models

[0040] To verify the effectiveness of the proposed method, this paper sets up a baseline method for comparative experiments. The baseline method represents the traditional traffic control method that does not use reinforcement learning strategy to optimize the model, that is, directly using the default vehicle behavior model provided by the simulation platform to simulate vehicle operation. At the same time, this embodiment compares and verifies the proposed algorithm (CA-MAPPO) with the traditional baseline algorithm (MAPPO) in environments with 10, 15, 20 and 25 autonomous driving agents.

[0041] like Figure 4 As shown, Figure 4 Subgraph (a) in the figure shows the convergence curves of the average reward for 10 agents under the traditional MAPPO algorithm and the CA-MAPPO algorithm; Figure 4 Subgraph (b) in the figure shows the convergence curves of the average reward for 15 agents under the traditional MAPPO algorithm and the CA-MAPPO algorithm; Figure 4Subgraph (c) in the figure shows the convergence curves of the average reward for 20 agents under the traditional MAPPO algorithm and the CA-MAPPO algorithm; Figure 4 Subgraph (d) in the figure is the convergence curve of the average reward of 25 agents under the traditional MAPPO algorithm and the CA-MAPPO algorithm; As the number of agents in the control network increases, the traditional MAPPO algorithm suffers from the explosion of state space dimensions, which limits its feature extraction capability. When the number of agents reaches 20 and 25, the average reward curve of the traditional MAPPO algorithm shows a significant degradation trend, and the overall traffic performance of autonomous vehicles is significantly suppressed. In contrast, this invention utilizes a cross-attention mechanism and a global mask matrix to dynamically filter redundant states of vehicles that have left or have not yet been generated, effectively overcoming the curse of dimensionality caused by variable-length agents. The CA-MAPPO algorithm exhibits strong anti-interference capabilities under all test scales and robustly converges to a higher average reward level. To quantitatively verify the scalability of this method, this embodiment conducted a targeted analysis of the traffic performance of mixed traffic flow in complex merging and accident impact scenarios, and obtained the core performance indicators of each algorithm in the convergence stage (i.e. the last fifty rounds). Detailed comparison data is shown in Table 3. Table 3. Comparison of the overall collaborative risk avoidance performance of various algorithms under different agent sizes.

[0042] like Figure 5 As shown, Figure 5 Subgraph (a) in the figure shows the convergence curves of the average speed of the 10 agents under different algorithms; Figure 5 Subgraph (b) in the figure shows the convergence curves of the average speed of the 15 agents under different algorithms; Figure 5 Subgraph (c) in the figure is a convergence curve of the average speed of the 20 agents under different algorithms; Figure 5 Subgraph (d) in the figure is a convergence curve of the average speed of the 25 agents under different algorithms; Based on Table 3 Figure 5As shown, in complex merging and accident-affected scenarios where congestion and collisions are highly likely to occur, the traditional MAPPO algorithm shows a significant decline in average speed and system reward when the number of agents increases to twenty and twenty-five. However, the cooperative vehicle fleet controlled by the method of this invention maintains extremely high safety and traffic efficiency at various agent scales. The average speed of the autonomous vehicles remains stable at a high level of 23.7 meters to 25.1 meters per second, achieving perfect passage through the core conflict area. This fully demonstrates that the method has absolute technical advantages and scalability in handling variable-length agent cooperative risk avoidance tasks.

[0043] Secondly, the robustness analysis of the system under different traffic flow conditions is carried out. In this embodiment, under the tiered conditions of traffic flow of 1000, 2000 and 3000 vehicles per hour in the road network background, the traffic efficiency of the algorithm of this invention, the traditional MAPPO algorithm and the environment without autonomous driving control baseline is comprehensively compared. like Figure 6 The graph shows a comparison of the system's average speed under different traffic volumes. Figure 6 Subgraph (a) in the figure is a schematic diagram of the average velocity of the system under the algorithm of this invention. Figure 6 Subgraph (b) in the diagram is a schematic diagram of the system's average velocity under the traditional MAPPO algorithm. Figure 6 Subgraph (c) in the diagram is a schematic diagram of the system's average speed under conditions without autonomous driving control baseline; The average speed of the road network, the average speed of autonomous vehicles, and the average speed of manually driven vehicles under various traffic flow levels were extracted for comprehensive evaluation. Simulation data shows that the CA-MAPPO algorithm of this invention achieved the best traffic efficiency index under all traffic flow levels. Especially under the high-density traffic flow of 3,000 vehicles per hour and the high-risk bottleneck condition with sudden accidents, compared with the baseline environment where vehicles pile up behind accident vehicles and frequently trigger extreme braking, the algorithm of this invention not only greatly improves the driving efficiency of autonomous vehicles themselves, but also effectively smooths the micro traffic flow shock waves through forward-looking off-peak lane changing and beyond-line-of-sight active risk avoidance behavior. This high-level micro-cooperative behavior significantly improves the average speed of manually driven vehicles in the background flow and the average speed of the road network as a whole, achieving zero secondary collision accidents. This fully verifies the excellent robustness and macro-optimization capability of this cooperative risk avoidance method in dealing with complex and highly dynamic mixed traffic flow environments.

[0044] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A collaborative risk avoidance method for autonomous driving based on cross-attention, characterized in that, include: Step S1: Obtain the local observation feature vector of each autonomous driving agent in the road network, and construct a global mask matrix containing the survival status of all autonomous driving agents. Step S2: Each autonomous driving agent inputs its local observation feature vector into the policy network, uses the global mask matrix to dynamically mask the state of inactive or departing autonomous driving agents, fuses the spatiotemporal interaction features of the current autonomous driving agent with other surviving autonomous driving agents, outputs the probability distribution of control actions, and samples to generate collaborative risk avoidance control actions. Step S3: Send the collaborative risk avoidance control action to the traffic simulation environment for execution, update the simulation step and trigger the dynamic accident state machine, obtain the next moment state of each surviving autonomous driving agent and update the global mask matrix according to the next moment state, and independently calculate and feed back the corresponding multi-level risk avoidance comprehensive reward value according to the state of each surviving autonomous driving agent. Step S4: Store the data generated in steps S1 to S3 into the experience replay pool, calculate the global state evaluation value through the centralized value network, and update the parameters of the policy network and the centralized value network until the preset convergence condition is met. The preset convergence conditions include: the update magnitude of the policy network parameters is less than the set threshold within multiple consecutive training cycles, and the cumulative multi-level risk avoidance comprehensive reward value, collision rate and traffic efficiency evaluation indicators of the autonomous driving agent tend to stabilize. Step S5: Generate collaborative risk avoidance control actions using the trained policy network to achieve collaborative risk avoidance for autonomous driving.

2. The autonomous driving cooperative risk avoidance method based on cross-attention according to claim 1, characterized in that, Step S3 further includes: Step S31: Send the collaborative risk avoidance control action to the traffic simulation environment for execution, advance the simulation steps, and detect whether the pre-set dynamic accident state machine in the simulation environment has traveled to the preset accident section and crossed the starting longitudinal coordinate. If so, it is determined that a mechanical failure has occurred and the dynamic accident state machine is activated. Step S32: Release the initial speed limit constraint of the dynamic accident state machine and set the fault low speed limit threshold. The fault low speed limit threshold is used to limit the maximum driving speed of the dynamic accident state machine in the process of moving towards the emergency lane after the fault occurs. At the same time, issue a forced lane change command to move towards the emergency lane and record the initial accident lane when the fault occurs. Step S33: During the process of the dynamic accident state machine transferring to the emergency lane, the actual lane where the center point of the vehicle body of the dynamic accident state machine is located is tracked in real time, and the accident lane index is dynamically updated. Step S34: When the dynamic accident state machine is found to have completely merged into the emergency lane and its longitudinal driving speed has decreased to below the set stopping threshold, the lateral and longitudinal motion control permissions of the dynamic accident state machine are cancelled, and the dynamic accident state machine is marked as a permanently stopped static obstacle in the traffic simulation environment. Step S35: Obtain the next moment state of each surviving autonomous driving agent, update the global mask matrix according to the next moment state, and independently calculate and feed back the corresponding multi-level risk avoidance comprehensive reward value for each surviving autonomous driving agent based on its own state.

3. The autonomous driving cooperative risk avoidance method based on cross-attention according to claim 2, characterized in that, The methods for obtaining the multi-level risk aversion comprehensive reward value include: If the autonomous driving agent triggers a physical collision during the current control cycle, a fixed negative hard collision penalty will be directly assigned, and the multi-level risk avoidance comprehensive reward value will no longer be calculated. If the autonomous driving agent does not trigger a physical collision during the current control cycle, then the multi-level risk avoidance comprehensive reward value is calculated. in, This is a multi-level risk avoidance comprehensive reward value. To assign a weighting coefficient to the speed reward, To prevent rear-end collisions, a weighting coefficient is set for safety penalties. To set a weighting coefficient for the merging conflict penalty, A weighting coefficient is set for the penalty for staying in the accident lane. Set a weighting coefficient for lane-changing penalties in high-risk areas. This represents the speed bonus calculated based on the difference between the current vehicle speed and the expected ideal vehicle speed on the road. This indicates a penalty for failing to prevent rear-end collisions. Indicates punishment for merging conflicts. This indicates a penalty for being stranded in the accident lane. This indicates a penalty for changing lanes in a high-risk area. The vehicle's current speed. This refers to the collision time between the autonomous driving agent and the vehicle in front. To accelerate the total length of the lane, This is the current vertical position. To achieve the desired ideal speed on the road, This represents the relative longitudinal distance between the autonomous driving agent and the dynamic accident site. , To fix the calculation parameters, The critical point of the interval, , This is the set spatial attenuation scale constant.

4. The autonomous driving cooperative risk avoidance method based on cross-attention according to claim 2, characterized in that, The collaborative risk avoidance control actions include discrete control commands in multiple dimensions: Maintain your current lane and speed; After verifying the safety status of the adjacent lane on the left, a left lane change control command is triggered; Provided that the safety status of the adjacent lane on the right has been verified, a right lane change control command is triggered; Trigger a longitudinal acceleration command to control the current speed to increase in preset step increments without exceeding the global desired upper limit speed; Trigger a longitudinal deceleration command to control the current speed to decrease in preset steps and not fall below the set safety lower limit speed.

5. The autonomous driving cooperative hazard avoidance method based on cross-attention according to claim 1, characterized in that, The process of integrating the spatiotemporal interaction features of the currently described autonomous driving agent with those of other surviving autonomous driving agents includes: Step Z1: Encode the local observation feature vector of the current autonomous driving agent into its own dense feature vector, and generate a query vector by linear mapping the own dense feature vector; Step Z2: Map the local observation feature vectors of all the autonomous driving agents in the road network to generate a key matrix and a value matrix; Step Z3: Extract the survival status bits of all other autonomous driving agents except the current autonomous driving agent from the global mask matrix and invert them to construct an attention-filled masking matrix; if it is detected that all other autonomous driving agents are in an ungenerated or departed state, then activate the full masking protection mechanism. Step Z4: If no other autonomous driving agents are detected to be in an ungenerated or departed state, then calculate the context interaction feature vector based on the attention-filled occlusion matrix, concatenate the context interaction feature vector with the self-dense feature vector, input it to the action decision layer, and output the control action probability distribution.

6. The autonomous driving cooperative risk avoidance method based on cross-attention according to claim 5, characterized in that, Methods for calculating contextual interaction feature vectors include: Based on the attention-filled masking matrix, a weighted sum is calculated between the key matrix and the value matrix; in, For autonomous driving intelligent agents Contextual interaction feature vectors For autonomous driving intelligent agents The linear mapping generates the query vector. For autonomous driving intelligent agents Its own dense feature vector, The query projection matrix parameters are learnable in the policy network. The scaling factor for the dimensions of the attention feature vector is set. Fill the occlusion matrix with attention. V is the key matrix, and V is the value matrix. Represents the key matrix Perform matrix transpose.

7. The autonomous driving cooperative risk avoidance method based on cross-attention according to claim 6, characterized in that, The method for constructing the key matrix and the value matrix includes: in, The key matrix, It is a set of locally observed feature vectors. These are the learnable key projection matrix parameters in the policy network. For value matrices, The parameters are the projection matrix parameters for the learnable values ​​in the policy network.

8. The autonomous driving cooperative risk avoidance method based on cross-attention according to claim 1, characterized in that, The local observation feature vector includes road topology coding, vehicle motion status, surrounding interactive traffic flow, and dynamic accident zone situational awareness information. The road topology coding includes the one-hot coding of the road segment where the current autonomous driving agent is located, the normalized lane index, the normalized longitudinal position coordinates, and the normalized current driving speed; The vehicle motion state includes the interaction features of the leading vehicle in front and the following vehicle behind the current autonomous driving agent. The surrounding interactive traffic flow includes lateral safety prediction status positions, which include the left adjacent lane safety status position based on spatiotemporal safety distance prediction and the right adjacent lane safety status position based on spatiotemporal safety distance prediction. The dynamic accident zone situational awareness information includes the normalized feature of the remaining distance in the merging zone, the normalized feature of the relative longitudinal distance of the dynamic accident point, and the marker position of the danger zone.

9. An electronic device, characterized in that, It includes a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the autonomous driving cooperative risk avoidance method based on cross-attention as described in any one of claims 1 to 8.

10. A computer program product, characterized in that, It includes a computer program / instruction that, when executed by a processor, implements the autonomous driving cooperative risk avoidance method based on cross-attention as described in any one of claims 1 to 8.

Images