A cooperative risk avoidance method and system for autonomous driving based on scene deconstruction and imitation learning

Abstract

This invention relates to a collaborative risk avoidance method and system for autonomous driving based on scenario deconstruction and imitation learning. The method includes: collecting traffic datasets and preprocessing the data; identifying single-vehicle conflict events and linking them into a chain of conflicts based on alternative safety indicators; extracting root conflict and traffic flow features and full-process evolution features; constructing a scenario-based dataset based on root conflict and traffic flow features and full-process evolution features using a two-stage clustering method; selecting all driver risk avoidance response behavior data with excellent response quality levels to form an expert demonstration dataset and performing data augmentation; training an initial driving strategy network using a behavior cloning algorithm; using the initial driving strategy network as a pre-trained model and employing a multi-agent reinforcement learning algorithm to collaboratively optimize and train all autonomous vehicles to achieve collaborative risk avoidance for autonomous driving. Compared with existing technologies, this invention improves the safety, comfort, and generalization ability of autonomous driving system decision-making.

Description

Technical Field

[0001] This invention relates to the fields of intelligent transportation and autonomous driving technology, and in particular to an autonomous driving cooperative risk avoidance method and system based on scene deconstruction and imitation learning. Background Technology

[0002] With the continuous growth of motor vehicle ownership, chain collisions involving multiple vehicles caused by rear-end collisions have become a major threat to road traffic safety, especially on highways and urban expressways. While existing Advanced Driver Assistance Systems (ADAS) and autonomous driving technologies have improved driving safety to some extent, they still have limitations in dealing with such sudden and highly contagious chain collisions.

[0003] Current research on vehicle control strategies suffers from the following shortcomings: 1) Traditional rule-based or model-based controllers, such as model predictive control or car-following models, can handle single-vehicle car-following problems, but they are difficult to adapt to complex dynamic interactions involving multiple vehicles, heterogeneous (human-machine mixed driving), and non-cooperative behavior in chain conflicts.

[0004] 2) Existing multi-vehicle cooperative control technologies, such as Cooperative Adaptive Cruise Control (CACC), mostly rely on pre-grouped, communication-stable vehicle platoons. However, the essence of chain conflicts is a series of disordered and sudden emergency avoidance behaviors, which are difficult to apply with existing technologies.

[0005] 3) Although reinforcement learning-based control methods have adaptive capabilities, they often suffer from problems such as low sample efficiency due to "learning from scratch", slow training convergence, and "black box" decision-making that makes the behavior difficult to interpret or inconsistent with human driving habits, thus limiting their application in safety-critical scenarios.

[0006] For example, patent application CN114407931A discloses a highly human-like method for safe driving decisions in automated commercial vehicles based on Generative Adversarial Imitation Learning (GAIL). This method constructs an expert trajectory dataset by collecting multi-source sensor data from skilled drivers, uses a proximal policy optimization algorithm (PPO) and a deep neural network to build a generator and discriminator respectively, and designs a reward function that comprehensively considers factors such as forward collisions, rearward collisions, side collisions, vehicle roll stability, and driving comfort, thereby outputting driving decisions such as steering, acceleration, and deceleration. However, this method is only applicable to safety decisions for a single vehicle in a steady-state traffic environment and cannot handle propagating conflict accidents involving multiple vehicles caused by rear-end collisions, nor can it adapt to the complex dynamic interactions of heterogeneous traffic flows. Summary of the Invention

[0007] The purpose of this invention is to overcome the shortcomings of the existing technology by providing a collaborative risk avoidance method and system for autonomous driving based on scene deconstruction and imitation learning, thereby improving the safety, comfort and generalization ability of autonomous driving system decision-making.

[0008] The objective of this invention can be achieved through the following technical solutions: An autonomous driving cooperative risk avoidance method based on scene deconstruction and imitation learning includes the following steps: Collect traffic datasets containing high-precision motion trajectories of multiple vehicles and perform data preprocessing; Based on alternative safety indicators, single-vehicle conflict events are identified from the preprocessed traffic dataset, and multiple single-vehicle conflict events are linked into a chain of conflict events using a spatiotemporal correlation matching algorithm. Extract the root conflict and traffic flow characteristics and the whole process evolution characteristics of the chain of conflict; A two-stage clustering method is used to divide the chain of conflict into a scenario-based dataset with dual labels of initial disturbance level and response quality level, based on the root conflict and traffic flow characteristics and the whole process evolution characteristics. All driver avoidance response behavior data with excellent response quality levels are selected from the scenario-based dataset to form an expert demonstration dataset, and data augmentation is performed on the expert demonstration dataset. An initial driving policy network was obtained by training a behavior cloning algorithm using a data-enhanced expert demonstration dataset. The weights of the initial driving policy network are loaded into the actor network of all autonomous vehicles as a pre-trained model, and a multi-agent reinforcement learning algorithm is used to perform collaborative optimization training on all autonomous vehicles to achieve autonomous collaborative risk avoidance.

[0009] Furthermore, the alternative safety metric is a corrected collision time, used to dynamically identify and quantify the risk level of single-vehicle collision events from the high-precision motion trajectory of autonomous vehicles.

[0010] Furthermore, the root conflict and traffic flow features are used to characterize the initial disturbance intensity of the event from the source of a single-vehicle conflict, including conflict time, speed change, mean deceleration, variance deceleration, mean MTTC, variance MTTC, mean vehicle spacing, variance vehicle spacing, mean inter-vehicle speed, variance inter-vehicle speed, mean inter-vehicle acceleration, variance inter-vehicle acceleration, flow rate, traffic flow speed, average headway, and average headway time.

[0011] Furthermore, the entire process evolution characteristics are used to evaluate the comprehensive effectiveness and quality of the driver's avoidance behavior response from a global perspective throughout the entire process from the occurrence to the complete end of a single-vehicle conflict. The calculation is based on a potential conflict index, the formula for which is: In the formula, In order to be in The potential conflict index at any given moment The threshold for severe conflict. In order to be in The collision time is adjusted in real time.

[0012] Furthermore, the evolutionary characteristics of the entire process include the conflict risk intensity dimension, the conflict propagation dimension, and the driver's risk avoidance behavior response dimension. The characteristics of the conflict risk intensity dimension include the chain conflict risk intensity, chain conflict risk fluctuation, and chain conflict risk trend. The characteristics of the conflict propagation dimension include the chain conflict propagation speed, the number of chain conflicts, and the chain conflict propagation time. The characteristics of the driver's risk avoidance behavior response dimension include the chain conflict braking intensity. The chain reaction risk intensity is used to simultaneously measure the maximum risk of the most dangerous vehicle in the chain of conflict and the average risk level of all vehicles. Its calculation formula is as follows: In the formula, For a chain of conflicts The intensity of the risk of chain conflicts This is the sequence number of the chain of conflicts. For a chain of conflicts The vehicle involved in the incident A time-varying sequence of potential conflict indices To obtain the maximum value, To take the average value; The volatility of the chain conflict risk is reflected by the standard deviation of the potential conflict index, which indicates the short-term volatility of the risk level during the conflict. The calculation formula is as follows: In the formula, For a chain of conflicts The risk of chain conflicts fluctuates. For standard deviation calculation; The trend of cascading conflict risk is measured by the difference between the start and end values ​​of the potential conflict index, which represents the cumulative trend of risk change. The calculation is as follows: In the formula, For a chain of conflicts The trend of chain conflict risks For the chain of conflict The potential conflict index value at the end of the time. For the chain of conflict The potential conflict index value at the initial moment; The chain reaction propagation speed is used to reflect the speed at which the chain of conflicts propagates from front to back in space, and its calculation formula is as follows: In the formula, For a chain of conflicts The speed at which chain conflicts spread For a chain of conflicts Spatial span, For a chain of conflicts The total number of vehicles involved, For a chain of conflicts The length of time it takes for the virus to spread; The number of chain reaction propagations reflects the spatial extent of the conflict propagation, using the total number of vehicles affected in the chain reaction. The calculation formula is as follows: In the formula, For a chain of conflicts The number of chain reactions that spread; The chain reaction propagation time represents the average time a single vehicle remains in a conflict state, reflecting the intensity of interference in a single vehicle conflict. Its calculation formula is as follows: In the formula, For a chain of conflicts The time it takes for the chain of conflicts to spread For a chain of conflicts The vehicle involved in the incident The duration of the conflict; The cascading braking intensity is used to characterize the overall braking intensity applied by the driver in a cascading conflict chain, and its calculation formula is as follows: In the formula, For a chain of conflicts The intensity of chain-reaction braking, To obtain the minimum value, For a chain of conflicts The vehicle involved in the incident Instantaneous braking deceleration.

[0013] Furthermore, employing a two-stage clustering method, based on the root conflict and traffic flow characteristics and the entire process evolution characteristics, the specific steps for dividing the chain of conflict into a scenario-based dataset with dual labels of initial disturbance level and response quality level include: Based on the root conflict and traffic flow characteristics, the self-organizing map network algorithm and K-Means method are used to cluster the chain conflict events in the chain conflict chain into multiple main scenarios with different initial disturbance levels. Based on the relevant indicators of the entire process evolution characteristics, the K-Means method is used to cluster the driver's risk avoidance behavior response in each of the main scenarios to obtain the response quality level; Each chain of conflict events in the chain is assigned an initial perturbation level and a response quality level, forming a scenario-based dataset.

[0014] Furthermore, the specific steps for training the initial driving policy network using the behavior cloning algorithm on a data-enhanced expert demonstration dataset include: The policy function is learned by taking the vehicle states in the data-enhanced expert demonstration dataset as input and the expert demonstration actions as output. Using a negative log-likelihood loss function, the network parameters are iteratively updated through a gradient descent algorithm to make the network's output action probability distribution approximate the expert distribution, thereby achieving human-like decision-making. The negative log-likelihood loss function is: In the formula, It is the negative log-likelihood loss function. For conditional probability distribution, Vehicle status. To demonstrate the movements to the experts, For network parameters, This represents the total number of network parameters.

[0015] Furthermore, the specific steps for loading the weights of the initial driving policy network into the actor network of all autonomous vehicles as a pre-trained model, and employing a multi-agent reinforcement learning algorithm to collaboratively optimize and train all autonomous vehicles include: Construct a multi-agent reinforcement learning simulation environment that supports a centralized training-decentralized execution paradigm, and load the weights of the initial driving policy network into the actor network of all autonomous vehicles as pre-training initial weights. A multi-agent reinforcement learning algorithm supporting centralized training and decentralized execution is adopted, and a comprehensive reward function that takes into account driving safety, comfort and efficiency is used to conduct collaborative optimization training for all autonomous vehicles. After training convergence, the actor network of all autonomous vehicles together constitutes the autonomous driving cooperative risk avoidance control model.

[0016] Furthermore, the comprehensive reward function that takes into account driving safety, comfort, and efficiency is as follows: In the formula, For the comprehensive reward function, The weighting coefficient for security reward items. For security rewards, The weighting coefficient for comfort reward items. For comfort bonus items, The weighting coefficient for efficiency reward items. For efficiency rewards, The collection of vehicles surrounding an autonomous vehicle. For autonomous vehicles and the first The time of collision with surrounding vehicles. The time scale constant, The weighting coefficient for the distance penalty. For indicator functions, The actual distance between the autonomous vehicle and the nearest vehicle. The minimum safe distance threshold, The weighting coefficient for acceleration penalty, For the longitudinal acceleration of autonomous vehicles, The weighting coefficient for the agitation penalty. The swiftness of autonomous vehicles. The weighting coefficient for maintaining the reward based on speed. The current speed of the autonomous vehicle. For the desired speed of autonomous vehicles, The weighting coefficient for the reward based on the distance traveled. This refers to the distance an autonomous vehicle travels within a time step.

[0017] According to another aspect of the present invention, an autonomous driving cooperative risk avoidance system based on scene deconstruction and imitation learning is provided, comprising: The traffic data acquisition module is used to collect traffic datasets containing high-precision motion trajectories of multiple vehicles and to perform data preprocessing. The conflict event association module is used to identify single-vehicle conflict events from the preprocessed traffic dataset based on alternative safety indicators, and to link multiple single-vehicle conflict events into a chain of conflict events through a spatiotemporal association matching algorithm. The feature extraction module is used to extract the root conflict and traffic flow features and the whole process evolution features of the chain of conflict. The scenario-based dataset construction module is used to divide the chain of conflict into scenario-based datasets with dual labels of initial disturbance level and response quality level based on the root conflict and traffic flow characteristics and the whole process evolution characteristics using a two-stage clustering method. The expert demonstration dataset construction module is used to select all driver avoidance response behavior data with excellent response quality level from the scenario-based dataset to form the expert demonstration dataset, and to perform data augmentation on the expert demonstration dataset. The initial driving strategy network training module is used to train the initial driving strategy network using a behavior cloning algorithm on a data-enhanced expert demonstration dataset. The autonomous vehicle cooperative optimization training module is used to load the weights of the initial driving policy network into the actor network of all autonomous vehicles as a pre-trained model, and to use a multi-agent reinforcement learning algorithm to perform cooperative optimization training on all autonomous vehicles to achieve autonomous cooperative risk avoidance.

[0018] Compared with the prior art, the present invention has the following beneficial effects: 1. This invention introduces a two-stage scenario clustering and full-process evolution feature quantification mechanism. First, based on root conflict and traffic flow features, chain conflict events are clustered into main scenarios with different initial disturbance levels. Then, based on the full-process evolution features, the quality of driver response is graded and quantified. This overcomes the limitations of existing technologies in dynamically adapting to complex conflict scenarios and lacking adaptability to human driving habits in decision-making. It solves the problems of insufficient safety and poor human-likeness of risk avoidance behavior in chain conflicts. Through fine judgment of conflict state and optimization of comprehensive reward function, the safety, comfort and generalization ability of autonomous driving system decision-making are improved.

[0019] 2. This invention adopts a paradigm of imitation learning pre-training and multi-agent reinforcement learning collaborative optimization. It trains an initial driving policy network from an expert demonstration dataset using a behavior cloning algorithm and utilizes centralized training-decentralized execution multi-agent reinforcement learning for collaborative optimization. This overcomes the limitations of existing technologies, such as low sample efficiency and slow training convergence caused by reinforcement learning starting from scratch. It solves the problems of low collaborative policy learning efficiency and difficulty in real-time adaptation to dynamic interactions in autonomous vehicles in chain conflict scenarios. It improves the automation level, iteration speed, and real-time performance and robustness of multi-vehicle collaborative risk avoidance in the model training process. Attached Figure Description

[0020] Figure 1 A flowchart illustrating an autonomous driving cooperative risk avoidance method based on scene deconstruction and imitation learning provided by this invention; Figure 2 This is a diagram illustrating the stable recovery of a vehicle after it leaves a state of chained conflict. Figure 3 This is a diagram illustrating the radical recovery of a vehicle leaving a chain of conflict states. Figure 4 This is a schematic diagram illustrating the conservative recovery process when a vehicle leaves a chain of conflict states. Figure 5 This is a diagram illustrating the default timeout state when a vehicle leaves a chain conflict state. Detailed Implementation

[0021] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. These embodiments are based on the technical solution of the present invention and provide detailed implementation methods and specific operating procedures. However, the scope of protection of the present invention is not limited to the following embodiments.

[0022] The following English abbreviations are involved: Self-Organizing Map (SOM) Modified Time-to-Collision (MTTC) Conflict Potentiality Index (CPI) Chain Conflict Risk Intensity (CCRI) Chain Conflict Risk Volatility (CCRV) Chain Conflict Risk Trend (CCRT) Chain Conflict Transaction Velocity (CCTV) Chain Conflict Transaction Number (CCTN) Chain Conflict Transaction Time (CCTT) Chain Conflict Deceleration Intensity (CCDI) Behavioral Cloning (BC) algorithm Multi-Agent Reinforcement Learning (MARL) Centralized Training with Decentralized Execution (CTDE) Example 1 This embodiment provides a collaborative risk avoidance method for autonomous driving based on scene deconstruction and imitation learning, such as Figure 1 As shown, it includes the following steps: S1. Collect a traffic dataset containing high-precision motion trajectories of multiple vehicles and perform data preprocessing.

[0023] First, obtain a high-precision vehicle trajectory dataset, such as the NGSIM dataset or the MAGIC dataset, and then perform data preprocessing.

[0024] The specific steps of data preprocessing include: Data cleaning and quality verification involves identifying and processing missing values ​​in data records, identifying and removing obviously erroneous data based on physical laws, and applying filtering algorithms (such as Kalman filtering and low-pass filtering) to smooth the position and velocity sequences in order to restore the true motion state of the vehicle and avoid misjudgment of micro-scale spatial changes due to noise.

[0025] Data formatting and coordinate unification convert data from different coordinate systems (such as latitude and longitude, local planar coordinates) to a unified coordinate system, ensuring consistency in spatial calculations and normalizing or standardizing feature data.

[0026] S2. Identify single-vehicle conflict events from the preprocessed traffic dataset based on alternative safety indicators, and link multiple single-vehicle conflict events into a chain of conflicts through a spatiotemporal correlation matching algorithm.

[0027] The alternative safety metric is the corrected collision time, which is used to dynamically identify and quantify the risk level of single-vehicle collision events from the high-precision motion trajectory of autonomous vehicles.

[0028] The alternative safety indicator is the corrected collision time, which is calculated using the following formula: In the formula, To correct for collision time, and This refers to the potential time of a rear-end collision when a vehicle behind takes specific deceleration measures in response to the deceleration of the vehicle in front. This refers to the potential time of a rear-end collision if the following vehicle fails to take any avoidance measures or takes inappropriate countermeasures (i.e., assuming the relative motion remains constant). This is the difference between the acceleration of the vehicle behind and the acceleration of the vehicle in front. This refers to the distance between the vehicle behind and the vehicle in front. This is the difference between the speed of the vehicle behind and the speed of the vehicle in front.

[0029] To accurately determine when each vehicle exits the conflict state, this embodiment employs a four-condition rule set based on driving behavior. This rule set can distinguish between four scenarios—stable recovery, aggressive recovery, conservative recovery, and default timeout—based on the vehicle's acceleration change curve. Stable recovery is as follows: Figure 2 As shown, the acceleration smoothly returns to near zero, indicating that the conflict has been smoothly resolved. Radical recovery is as follows: Figure 3 As shown, the acceleration exhibits a large positive fluctuation before reverting, indicating that the driver took relatively aggressive remedial measures. Conservative recovery is as follows: Figure 4 As shown, the acceleration remains at a low negative value, indicating that the driver is maintaining a cautious low-speed state. The default timeout is as follows: Figure 5 As shown, when the duration of the conflict exceeds a preset threshold, a forced disengagement is initiated. This four-condition rule set based on driving behavior allows for a more precise definition of the conflict state and its duration compared to a single threshold method.

[0030] This step allows for the extraction of thousands of well-structured chain conflicts and their associated data from the raw data.

[0031] S3. Extract the root conflict and traffic flow characteristics and the evolution characteristics of the entire process of the chain of conflict.

[0032] For each chain conflict event in the chain of conflicts, two sets of independent but logically related features need to be extracted for quantitative description, including root conflict and traffic flow features and whole-process evolution features.

[0033] The root conflict and traffic flow characteristics are used to characterize the initial disturbance intensity of an event from the source of a single-vehicle conflict. These characteristics include conflict time, speed variation, mean deceleration, variance deceleration, mean MTTC, variance MTTC, mean vehicle spacing, variance vehicle spacing, mean inter-vehicle speed, variance inter-vehicle speed, mean inter-vehicle acceleration, variance inter-vehicle acceleration, flow rate, traffic flow speed, average headway, and average headway, as shown in the table below: Table 1 Explanation of Root Conflict and Traffic Flow Characteristic Parameters The whole-process evolution characteristics are used to evaluate the overall effectiveness and quality of the driver's avoidance behavior response from a global perspective throughout the entire process from the occurrence to the complete end of a single-vehicle conflict. Its calculation is based on the potential conflict index, and the formula for calculating the potential conflict index is as follows: In the formula, In order to be in The potential conflict index at any given moment The threshold for severe conflict. In order to be in The collision time is adjusted in real time.

[0034] The evolutionary characteristics of the entire process include the conflict risk intensity dimension, the conflict propagation dimension, and the driver's risk avoidance behavior response dimension. The characteristics of the conflict risk intensity dimension include the chain conflict risk intensity, chain conflict risk fluctuation, and chain conflict risk trend. The characteristics of the conflict propagation dimension include the chain conflict propagation speed, the number of chain conflicts propagated, and the chain conflict propagation time. The characteristics of the driver's risk avoidance behavior response dimension include the chain conflict braking intensity. The cascading conflict risk intensity is used to simultaneously measure the maximum risk of the most dangerous vehicle in a cascading conflict chain and the average risk level of all vehicles. Its calculation formula is as follows: In the formula, For a chain of conflicts The intensity of the risk of chain conflicts This is the sequence number of the chain of conflicts. For a chain of conflicts The vehicle involved in the incident A time-varying sequence of potential conflict indices To obtain the maximum value, To take the average value; The volatility of cascading conflict risk is reflected by the standard deviation of the potential conflict index, which indicates the short-term volatility of risk levels during a conflict. The formula for its calculation is as follows: In the formula, For a chain of conflicts The risk of chain conflicts fluctuates. For standard deviation calculation; The cascading conflict risk trend is measured by the difference between the start and end values ​​of the potential conflict index, which represents the cumulative trend of risk change. The calculation is as follows: In the formula, For a chain of conflicts The trend of chain conflict risks For the chain of conflict The potential conflict index value at the end of the time. For the chain of conflict The potential conflict index value at the initial moment; The chain reaction propagation speed reflects the speed at which a chain of conflicts propagates from front to back in space. Its calculation formula is as follows: In the formula, For a chain of conflicts The speed at which chain conflicts spread For a chain of conflicts Spatial span, For a chain of conflicts The total number of vehicles involved, For a chain of conflicts The length of time it takes for the virus to spread; The number of cascading conflict propagations reflects the spatial extent of the conflict propagation by the total number of vehicles affected in the cascading conflict chain. The formula for calculation is as follows: In the formula, For a chain of conflicts The number of chain reactions that spread; The cascading conflict propagation time represents the average time a single vehicle remains in a conflict state, reflecting the intensity of the interference in a single vehicle conflict. Its calculation formula is as follows: In the formula, For a chain of conflicts The time it takes for the chain of conflicts to spread For a chain of conflicts The vehicle involved in the incident The duration of the conflict; Chain reaction braking intensity is used to characterize the overall braking intensity applied by the driver in a chain reaction of conflict. Its calculation formula is as follows: In the formula, For a chain of conflicts The intensity of chain-reaction braking, To obtain the minimum value, For a chain of conflicts The vehicle involved in the incident Instantaneous braking deceleration.

[0035] S4. Using a two-stage clustering method, based on root conflict and traffic flow characteristics and full-process evolution characteristics, the chain of conflict is divided into a scenario-based dataset with dual labels of initial disturbance level and response quality level.

[0036] The specific steps of using a two-stage clustering method to divide the chain of conflict into a scenario-based dataset with dual labels of initial disturbance level and response quality level, based on root conflict and traffic flow characteristics and full-process evolution characteristics, include: In the first stage, based on root conflict and traffic flow characteristics, the self-organizing map network algorithm and the K-Means method are used to cluster chain conflict events in the chain of conflicts into multiple main scenarios with different initial disturbance levels. The optimal number of clusters is determined using the elbow rule. On both the NGSIM and MAGIC datasets, the sum of squared errors curves show a clear inflection point when the number of clusters is 3, thus determining that the initial disturbances should be divided into three levels. To verify the superiority of the algorithm, the performance of the self-organizing map network algorithm and K-Means method of this invention is compared with other clustering methods, as shown in the table below.

[0037] Table 2 Performance Comparison of Different Algorithms The comparison results in Table 2 show that the method used in this invention performs best in key indicators such as the profile coefficient. After clustering, three main scenarios with different initial perturbation levels are obtained, which can be named slight perturbation, moderate perturbation, and severe perturbation, respectively.

[0038] In the second stage, based on relevant indicators of the whole-process evolution characteristics, the K-Means method is used to cluster the driver's risk avoidance behavior response in each main scenario to obtain the response quality level. In each main scenario with different initial disturbance levels, based on the above seven whole-process evolution characteristic indicators, the K-Means method is used to cluster the driver's response behavior into two levels. One category is significantly better in indicators such as Conflict Risk Intensity (CCRI) and Braking Intensity (CCDI), and is therefore marked as an excellent response, while the other category is marked as a general response.

[0039] Each chain of conflict events in the chain is assigned an initial perturbation level and a response quality level, forming a scenario-based dataset.

[0040] S5. Select all driver avoidance response behavior data with excellent response quality level from the scenario-based dataset to form an expert demonstration dataset, and perform data augmentation on the expert demonstration dataset.

[0041] Based on imitation learning, expert policy extraction is performed to select all driver avoidance response behavior data with excellent response quality levels from the scenario-based dataset. To address the data imbalance problem, data augmentation techniques, such as Synthetic Minority Oversampling Technique (SMOTE) or Generative Adversarial Networks (GANs), are applied to the expert demonstration dataset for expansion and balancing.

[0042] S6. The initial driving policy network is obtained by training the behavior cloning algorithm using the data-enhanced expert demonstration dataset.

[0043] The specific steps for training the initial driving policy network using a behavior cloning algorithm on a data-enhanced expert demonstration dataset include: The policy function is learned by taking the vehicle states in the data-enhanced expert demonstration dataset as input and the expert demonstration actions as output. Using the negative log-likelihood loss function, the network parameters are iteratively updated through the gradient descent algorithm, making the network's output action probability distribution approximate the expert distribution, thus achieving human-like decision-making. The negative log-likelihood loss function is: In the formula, It is the negative log-likelihood loss function. For conditional probability distribution, Vehicle status. To demonstrate the movements to the experts, For network parameters, This represents the total number of network parameters.

[0044] The impact of different data augmentation strategies on model training performance is shown in the table below.

[0045] Table 3 Comparison of training effects of imitation learning models under different data augmentation strategies By comparing the impact of different data augmentation strategies on model training performance, it can be found that the model trained using a combination of data augmentation strategies has a significantly lower action error on the test set than the model trained using only the original data, demonstrating the effectiveness of data augmentation. The resulting initial policy network can effectively reproduce the risk avoidance decisions of expert drivers in different scenarios.

[0046] S7. Load the weights of the initial driving policy network into the actor network of all autonomous vehicles as a pre-trained model, and use a multi-agent reinforcement learning algorithm to perform collaborative optimization training on all autonomous vehicles to achieve autonomous collaborative risk avoidance.

[0047] The specific steps for loading the weights of the initial driving policy network into the actor network of all autonomous vehicles as a pre-trained model, and using a multi-agent reinforcement learning algorithm to collaboratively optimize and train all autonomous vehicles include: Construct a multi-agent reinforcement learning simulation environment that supports a centralized training-decentralized execution paradigm, and load the weights of the initial driving policy network into the actor networks of all autonomous vehicles as pre-training initial weights. A multi-agent reinforcement learning algorithm supporting centralized training and decentralized execution is adopted, and a comprehensive reward function that takes into account driving safety, comfort and efficiency is used to conduct collaborative optimization training for all autonomous vehicles. After training convergence, the actor network of all autonomous vehicles together constitutes the autonomous driving cooperative risk avoidance control model.

[0048] The comprehensive reward function, taking into account driving safety, comfort, and efficiency, is as follows: In the formula, For the comprehensive reward function, The weighting coefficient for security reward items. For security rewards, The weighting coefficient for comfort reward items. For comfort bonus items, The weighting coefficient for efficiency reward items. For efficiency rewards, The collection of vehicles surrounding an autonomous vehicle. For autonomous vehicles and the first The time of collision with surrounding vehicles. The time scale constant, The weighting coefficient for the distance penalty. For indicator functions, The actual distance between the autonomous vehicle and the nearest vehicle. The minimum safe distance threshold, The weighting coefficient for acceleration penalty, For the longitudinal acceleration of autonomous vehicles, The weighting coefficient for the agitation penalty. The swiftness of autonomous vehicles. The weighting coefficient for maintaining the reward based on speed. The current speed of the autonomous vehicle. For the desired speed of autonomous vehicles, The weighting coefficient for the reward based on the distance traveled. This refers to the distance an autonomous vehicle travels within a time step.

[0049] The MADDPG model used in this invention is compared with other MARL architectures (such as Independent Q-Learning MADQN-IQL and Fully Centralized MADQN-FC), as shown in the table below.

[0050] Table 4 Comparison of training performance of different multi-agent reinforcement learning (MARL) architectures The results show that MADDPG achieves the best balance between training efficiency and final performance.

[0051] To further verify the effectiveness of the core idea of ​​this invention, namely, using excellent human experience as prior knowledge, an ablation experiment was conducted to compare the training effects of initialization using three different prior knowledge: excellent experience, general experience, and no experience, as shown in the table below.

[0052] Table 5. Comparison of average reward values ​​for the impact of different human prior knowledge on the training effect of MARL models. As shown in Table 5, the model initialized using excellent experience converges the fastest and achieves the highest final performance. This fully demonstrates the superiority of the technical approach of imitation learning pre-training and reinforcement learning optimization in this invention. After training, the final actor network obtained is the cooperative driving control model to be generated by this invention.

[0053] Example 2 This embodiment provides an autonomous driving cooperative risk avoidance system based on scene deconstruction and imitation learning, including: The traffic data acquisition module is used to collect traffic datasets containing high-precision motion trajectories of multiple vehicles and to perform data preprocessing. The conflict event association module is used to identify single-vehicle conflict events from the preprocessed traffic dataset based on alternative safety indicators, and to link multiple single-vehicle conflict events into a chain of conflict events through a spatiotemporal association matching algorithm. The feature extraction module is used to extract the root conflict and traffic flow features and the whole process evolution features of the chain of conflict. The scenario-based dataset construction module is used to divide the chain of conflict into scenario-based datasets with dual labels of initial disturbance level and response quality level based on the root conflict and traffic flow characteristics and the whole process evolution characteristics using a two-stage clustering method. The expert demonstration dataset construction module is used to select all driver avoidance response behavior data with excellent response quality levels from the scenario-based dataset to form the expert demonstration dataset, and to perform data augmentation on the expert demonstration dataset. The initial driving strategy network training module is used to train the initial driving strategy network using a behavior cloning algorithm on a data-enhanced expert demonstration dataset. The autonomous vehicle cooperative optimization training module is used to load the weights of the initial driving policy network into the actor network of all autonomous vehicles as a pre-trained model, and to use a multi-agent reinforcement learning algorithm to perform cooperative optimization training on all autonomous vehicles to achieve autonomous cooperative risk avoidance.

[0054] The rest is the same as in Example 1.

[0055] The preferred embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make numerous modifications and variations based on the concept of the present invention without creative effort. Therefore, all technical solutions that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning, or limited experimentation on the basis of existing technology should be within the scope of protection defined by the claims.

Claims

1. A collaborative risk avoidance method for autonomous driving based on scene deconstruction and imitation learning, characterized in that, Includes the following steps: Collect traffic datasets containing high-precision motion trajectories of multiple vehicles and perform data preprocessing; Based on alternative safety indicators, single-vehicle conflict events are identified from the preprocessed traffic dataset, and multiple single-vehicle conflict events are linked into a chain of conflict events using a spatiotemporal correlation matching algorithm. Extract the root conflict and traffic flow characteristics and the whole process evolution characteristics of the chain of conflict; A two-stage clustering method is used to divide the chain of conflict into a scenario-based dataset with dual labels of initial disturbance level and response quality level, based on the root conflict and traffic flow characteristics and the whole process evolution characteristics. All driver avoidance response behavior data with excellent response quality levels are selected from the scenario-based dataset to form an expert demonstration dataset, and data augmentation is performed on the expert demonstration dataset. An initial driving policy network was obtained by training a behavior cloning algorithm using a data-enhanced expert demonstration dataset. The weights of the initial driving policy network are loaded into the actor network of all autonomous vehicles as a pre-trained model, and a multi-agent reinforcement learning algorithm is used to perform collaborative optimization training on all autonomous vehicles to achieve autonomous collaborative risk avoidance.

2. The autonomous driving cooperative risk avoidance method based on scene deconstruction and imitation learning according to claim 1, characterized in that, The alternative safety metric is the corrected collision time, which is used to dynamically identify and quantify the risk level of single-vehicle collision events from the high-precision motion trajectory of autonomous vehicles.

3. The autonomous driving cooperative risk avoidance method based on scene deconstruction and imitation learning according to claim 1, characterized in that, The root conflict and traffic flow features are used to characterize the initial disturbance intensity of an event from the source of a single-vehicle conflict, including conflict time, speed change, mean deceleration, variance deceleration, mean MTTC, variance MTTC, mean vehicle spacing, variance vehicle spacing, mean inter-vehicle speed, variance inter-vehicle speed, mean inter-vehicle acceleration, variance inter-vehicle acceleration, flow rate, traffic flow speed, average headway, and average headway.

4. The autonomous driving cooperative risk avoidance method based on scene deconstruction and imitation learning according to claim 1, characterized in that, The entire process evolution characteristics are used to evaluate the comprehensive effectiveness and quality of the driver's avoidance behavior response from a global perspective throughout the entire process from the occurrence to the complete end of a single-vehicle conflict. The calculation is based on a potential conflict index, the formula for which is: In the formula, In order to be in The potential conflict index at any given moment The threshold for severe conflict. In order to be in The collision time is adjusted in real time.

5. The autonomous driving cooperative risk avoidance method based on scene deconstruction and imitation learning according to claim 4, characterized in that, The evolutionary characteristics of the entire process include the conflict risk intensity dimension, the conflict propagation dimension, and the driver's risk avoidance behavior response dimension. The characteristics of the conflict risk intensity dimension include the chain conflict risk intensity, chain conflict risk fluctuation, and chain conflict risk trend. The characteristics of the conflict propagation dimension include the chain conflict propagation speed, the number of chain conflicts, and the chain conflict propagation time. The characteristics of the driver's risk avoidance behavior response dimension include the chain conflict braking intensity. The chain reaction risk intensity is used to simultaneously measure the maximum risk of the most dangerous vehicle in the chain of conflict and the average risk level of all vehicles. Its calculation formula is as follows: In the formula, For a chain of conflicts The intensity of the risk of chain conflicts This is the sequence number of the chain of conflicts. For a chain of conflicts The vehicle involved in the incident A time-varying sequence of potential conflict indices To obtain the maximum value, To take the average value; The volatility of the chain conflict risk is reflected by the standard deviation of the potential conflict index, which indicates the short-term volatility of the risk level during the conflict. The calculation formula is as follows: In the formula, For a chain of conflicts The risk of chain conflicts fluctuates. For standard deviation calculation; The trend of cascading conflict risk is measured by the difference between the start and end values ​​of the potential conflict index, which represents the cumulative trend of risk change. The calculation is as follows: In the formula, For a chain of conflicts The trend of chain conflict risks For the chain of conflict The potential conflict index value at the end of the time. For the chain of conflict The potential conflict index value at the initial moment; The chain reaction propagation speed is used to reflect the speed at which the chain of conflicts propagates from front to back in space, and its calculation formula is as follows: In the formula, For a chain of conflicts The speed at which chain conflicts spread For a chain of conflicts Spatial span, For a chain of conflicts The total number of vehicles involved, For a chain of conflicts The length of time it takes for the virus to spread; The number of chain reaction propagations reflects the spatial extent of the conflict propagation, using the total number of vehicles affected in the chain reaction. The calculation formula is as follows: In the formula, For a chain of conflicts The number of chain reactions that spread; The chain reaction propagation time represents the average time a single vehicle remains in a conflict state, reflecting the intensity of interference in a single vehicle conflict. Its calculation formula is as follows: In the formula, For a chain of conflicts The time it takes for the chain of conflicts to spread For a chain of conflicts The vehicle involved in the incident The duration of the conflict; The cascading braking intensity is used to characterize the overall braking intensity applied by the driver in a cascading conflict chain, and its calculation formula is as follows: In the formula, For a chain of conflicts The intensity of chain-reaction braking, To obtain the minimum value, For a chain of conflicts The vehicle involved in the incident Instantaneous braking deceleration.

6. The autonomous driving cooperative risk avoidance method based on scene deconstruction and imitation learning according to claim 1, characterized in that, The specific steps of using a two-stage clustering method to divide the chain of conflict into a scenario-based dataset with dual labels of initial disturbance level and response quality level, based on the root conflict and traffic flow characteristics and the whole process evolution characteristics, include: Based on the root conflict and traffic flow characteristics, the self-organizing map network algorithm and K-Means method are used to cluster the chain conflict events in the chain conflict chain into multiple main scenarios with different initial disturbance levels. Based on the relevant indicators of the entire process evolution characteristics, the K-Means method is used to cluster the driver's risk avoidance behavior response in each of the main scenarios to obtain the response quality level; Each chain of conflict events in the chain is assigned an initial perturbation level and a response quality level, forming a scenario-based dataset.

7. The autonomous driving cooperative risk avoidance method based on scene deconstruction and imitation learning according to claim 1, characterized in that, The specific steps for training the initial driving policy network using a behavior cloning algorithm on a data-enhanced expert demonstration dataset include: The policy function is learned by taking the vehicle states in the data-enhanced expert demonstration dataset as input and the expert demonstration actions as output. Using a negative log-likelihood loss function, the network parameters are iteratively updated through a gradient descent algorithm to make the network's output action probability distribution approximate the expert distribution, thereby achieving human-like decision-making. The negative log-likelihood loss function is: In the formula, It is the negative log-likelihood loss function. For conditional probability distribution, Vehicle status. To demonstrate the movements to the experts, For network parameters, This represents the total number of network parameters.

8. The autonomous driving cooperative risk avoidance method based on scene deconstruction and imitation learning according to claim 7, characterized in that, The specific steps for loading the weights of the initial driving policy network into the actor network of all autonomous vehicles as a pre-trained model, and using a multi-agent reinforcement learning algorithm to perform collaborative optimization training on all autonomous vehicles include: Construct a multi-agent reinforcement learning simulation environment that supports a centralized training-decentralized execution paradigm, and load the weights of the initial driving policy network into the actor network of all autonomous vehicles as pre-training initial weights. A multi-agent reinforcement learning algorithm supporting centralized training and decentralized execution is adopted, and a comprehensive reward function that takes into account driving safety, comfort and efficiency is used to conduct collaborative optimization training for all autonomous vehicles. After training convergence, the actor network of all autonomous vehicles together constitutes the autonomous driving cooperative risk avoidance control model.

9. The autonomous driving cooperative risk avoidance method based on scene deconstruction and imitation learning according to claim 8, characterized in that, The comprehensive reward function that takes into account driving safety, comfort, and efficiency is as follows: In the formula, For the comprehensive reward function, The weighting coefficient for security reward items. For security rewards, The weighting coefficient for comfort reward items. For comfort bonus items, The weighting coefficient for efficiency reward items. For efficiency rewards, The collection of vehicles surrounding an autonomous vehicle. For autonomous vehicles and the first The time of collision with surrounding vehicles. The time scale constant, The weighting coefficient for the distance penalty. For indicator functions, The actual distance between the autonomous vehicle and the nearest vehicle. The minimum safe distance threshold, The weighting coefficient for acceleration penalty, For the longitudinal acceleration of autonomous vehicles, The weighting coefficient for the agitation penalty. The swiftness of autonomous vehicles. The weighting coefficient for maintaining the reward based on speed. The current speed of the autonomous vehicle. For the desired speed of autonomous vehicles, The weighting coefficient for the reward based on the distance traveled. This refers to the distance an autonomous vehicle travels within a time step.

10. An autonomous driving cooperative risk avoidance system based on scene deconstruction and imitation learning, characterized in that, include: The traffic data acquisition module is used to collect traffic datasets containing high-precision motion trajectories of multiple vehicles and to perform data preprocessing. The conflict event association module is used to identify single-vehicle conflict events from the preprocessed traffic dataset based on alternative safety indicators, and to link multiple single-vehicle conflict events into a chain of conflict events through a spatiotemporal association matching algorithm. The feature extraction module is used to extract the root conflict and traffic flow features and the whole process evolution features of the chain of conflict. The scenario-based dataset construction module is used to divide the chain of conflict into scenario-based datasets with dual labels of initial disturbance level and response quality level based on the root conflict and traffic flow characteristics and the whole process evolution characteristics using a two-stage clustering method. The expert demonstration dataset construction module is used to select all driver avoidance response behavior data with excellent response quality level from the scenario-based dataset to form the expert demonstration dataset, and to perform data augmentation on the expert demonstration dataset. The initial driving strategy network training module is used to train the initial driving strategy network using a behavior cloning algorithm on a data-enhanced expert demonstration dataset. The autonomous vehicle cooperative optimization training module is used to load the weights of the initial driving policy network into the actor network of all autonomous vehicles as a pre-trained model, and to use a multi-agent reinforcement learning algorithm to perform cooperative optimization training on all autonomous vehicles to achieve autonomous cooperative risk avoidance.

Images