Fair perception overtaking decision optimization method for networked and non-networked vehicle unstructured mixed scene

Abstract

The invention provides a fair perception overtaking decision optimization method for a networked and non-networked vehicle unstructured mixed scene, and the method comprises the steps: enabling a networked vehicle to sense the state information of surrounding vehicles in real time through employing a carried sensor device; dynamically evaluating the availability of overtaking execution of the left lane by using a left lane availability detection mechanism; dynamically evaluating the availability of overtaking execution of a left lane based on a left lane availability detection mechanism of adaptive maneuvering tree search; a fast lane speed recommendation mechanism based on traffic flow analysis provides a real-time fast lane recommendation speed according to traffic density and flow; by using a K-nearest neighbor enhanced deep reinforcement learning method, combining a parameterized breach depth recursive Q network with KNN enhanced experience playback, and combining fairness probability, left lane availability, lane recommendation speed and other efficiency and safety indexes to construct a lane changing overtaking model; and the networked vehicle executes a lane changing overtaking decision according to a decision result provided by the lane changing overtaking model.

Description

Technical Field

[0001] The present invention belongs to the field of vehicle networking communication technology, and in particular relates to a fairness-aware overtaking decision optimization method for an unstructured mixed scenario of networked and non-networked vehicles. Background Art

[0002] Traffic systems have greatly improved the efficiency of traffic flow and road safety by enabling real-time information sharing and cooperative driving through vehicle network technology. However, the high deployment cost and inconsistent communication standards between manufacturers and regions have hindered the widespread application of this technology. Therefore, there may be both connected and unconnected vehicles on the road in the future, forming a mixed connected and unconnected vehicle (MCCV) environment. In this case, lane changing and overtaking operations are crucial to vehicle driving and require precise consideration of speed, location, road conditions, and traffic regulations. Although reasonable strategies can reduce traffic congestion and accidents, the uncertainty of unconnected vehicles, especially in congested or high-speed environments, increases safety risks and makes the decision-making process more complicated. To cope with these challenges in the MCCV environment, lane changing and overtaking strategies need to prioritize safety and efficiency, and also ensure fairness, that is, fair treatment of all types of vehicles. This fairness is crucial to the safety, efficiency, and shared road resources of connected and unconnected vehicles, and is of key significance to the sustainable development of intelligent transportation systems. Therefore, ensuring that connected vehicles perform safe and efficient overtaking decisions in the MCCV environment and minimize interference with surrounding vehicles has become an important research direction.

[0003] At present, the known technologies in the field of lane change and overtaking of networked autonomous vehicles have made some progress. However, most of these known technologies focus on optimizing the decision of a single vehicle to improve efficiency and traffic flow, often ignoring the possible unfair impact on surrounding vehicles. The fairness of vehicle decision-making is reflected in minimizing interference with other vehicles while maximizing its own performance in terms of speed and safety. In the process of lane change and overtaking, fairness not only affects the efficiency of individuals, but also affects the coordination between vehicles, which is crucial for road safety and traffic capacity. Current known technologies usually use globally available data to measure fairness, such as comparing the average waiting time of vehicles at intersections in different directions. However, in the MCCV scenario, the lack of real-time data of non-networked vehicles limits the effectiveness of these methods. The main challenge is how to infer and quantify fairness based on limited data of networked vehicles and their surroundings. Bayesian networks provide a solution to deal with uncertainty and quantify fairness through probabilistic reasoning, but due to the multidimensionality and complex conditional dependencies of vehicle state data, how to build these networks in complex traffic environments is still challenging. In addition, the process of discretizing continuous data into intervals introduces subjectivity and reduces the accuracy of fairness evaluation. Therefore, measuring fairness under incomplete data conditions, optimizing model complexity, and meeting real-time decision-making needs remain major challenges.

[0004] Incorporating traffic regulations into driving decisions helps improve coordination between vehicles and complies with the principle of fairness. However, strategies in existing known technologies generally do not fully consider the differences in traffic regulations in different countries. For example, in left-hand drive countries, vehicles need to overtake in the left lane and quickly return to the right lane to reduce traffic congestion, while in right-hand drive countries, the opposite is true. The present invention is mainly aimed at left-hand drive scenarios, giving priority to overtaking in the left lane to improve efficiency and reduce interference with traffic in the right lane. However, the priority of the left lane depends on traffic density, road conditions, and the status of the vehicle in front. When the lane is unavailable or congested, forced left turns increase the risk. Therefore, it is crucial for connected vehicles to evaluate the availability of the left lane in real time and dynamically adjust the overtaking strategy. Decisions after overtaking, such as whether to return to the right lane, usually depend on vehicle speed and the recommended minimum acceptable speed, which is based on traffic density, lane speed changes, and real-time infrastructure data. In the MCCV environment, when there is a lack of information from non-connected vehicles, road sensors can be used to monitor lane flow density. Modeling the relationship between lane speed and traffic flow density remains a challenge for providing speed recommendations in real time.

[0005] In addition, the scarcity of data in extreme driving scenarios poses difficulties for algorithm performance, especially when dealing with long-tail distributions. The long-tail phenomenon in traffic scenarios is very common, especially in MCCV environments, where common decisions such as lane changes or overtaking are often concentrated in high-frequency situations, while very few emergencies, such as sudden deceleration of the vehicle in front or abnormal driving behavior, have a greater impact on safety and efficiency. Traditional algorithms usually rely on conventional data sets and ignore these rare but critical events, resulting in poor performance in high-risk situations. Therefore, how to effectively deal with long-tail scenarios in extreme situations and improve the reliability of the model has become the key to achieving safe driving decisions in MCCV environments.

[0006] In the prior art, there are multiple challenges in achieving lane change and overtaking in MCCV scenarios with connected and unconnected vehicles: First, most of the related public known technologies on mixed traffic scenarios focus on macro traffic decisions, lack in-depth analysis of individual vehicle behavior, and may ignore the behavior details of individual vehicles during lane change and overtaking, thereby increasing safety risks. Second, many related public known technologies focus on optimizing the efficiency of connected autonomous driving vehicles, but do not consider the impact on unconnected vehicles driven by humans, which may have an adverse impact on traditional vehicles. Third, traffic regulations are often ignored in existing lane change and overtaking models, which assume that vehicles can change lanes at any time, but the legal restrictions and safety risks in reality should be fully considered. Finally, real-time decisions on lane changes and overtaking require extremely high response speeds, and the dynamic traffic environment requires vehicles to make safe and effective decisions in a very short time, which places higher requirements on decision-making efficiency. Summary of the invention

[0007] In view of this, considering the increased decision complexity and safety risks caused by the communication differences between connected and non-connected vehicles in the MCCV environment, this paper proposes a fairness-aware lane-changing and overtaking decision optimization method. This method incorporates fairness and traffic regulations into the lane-changing and overtaking decisions of connected vehicles in the MCCV environment, aiming to achieve efficient, safe and fair driving decisions to cope with the challenges of complexity and safety risks of lane-changing and overtaking decisions caused by differences in vehicle communication capabilities.

[0008] The present invention provides a fairness-aware overtaking decision optimization method for an unstructured mixed scenario of networked and non-networked vehicles, comprising the following steps:

[0009] S1: The connected vehicle monitors and records the traffic information of surrounding vehicles in real time through its own sensor equipment. The traffic information includes the vehicle's position, speed, acceleration, lane, and lane density;

[0010] S2: Based on traffic information, the fairness evaluation method of Gauss's Bayesian network is used to model the relationship between vehicle behavior and fairness results, and the fairness probability is obtained to quantify the fairness of vehicle decision-making under limited data conditions;

[0011] S3: Based on traffic information, the left lane availability detection mechanism of adaptive maneuver tree search is used to obtain a feasibility index for dynamically evaluating the left lane for overtaking, which is used to evaluate the availability of the left lane for overtaking.

[0012] S4: Based on traffic information, a fast lane speed recommendation mechanism based on traffic flow analysis is used to provide a real-time fast lane recommended speed according to the lane density in the traffic information;

[0013] S5: Based on traffic information, fairness probability, feasibility index and recommended speed, a deep reinforcement learning method enhanced by K-nearest neighbor (KNN) is used to combine the parameterized dueling deep recursive Q network with KNN-enhanced experience playback to construct a lane changing and overtaking decision model to optimize the fairness, safety and efficiency of driving operations; the connected vehicle executes driving decisions based on the decision results provided by the lane changing and overtaking model.

[0014] Furthermore, in step S2, the lane change state L is calculated based on the traffic information. t , relative speed V r , the distance D from the target vehicle r , the deceleration rate DRAC to avoid collision and the state change S of the target vehicle t , and combined with the speed V in the traffic information s , acceleration A s As variables of each node in the Bayesian network;

[0015] Use Gaussian mixture model GMM to discretize the V in Bayesian network nodes s , A s 、V r , D r The five continuous variables DRAC are discretized using formula (1), where K is the number of Gaussian components, π K is the weight of each component, is the Gaussian distribution of the Kth component;

[0016]

[0017] The sand cat swarm intelligent optimization algorithm based on K2 scoring is used to represent the probabilistic relationship between different nodes, so as to evaluate the fairness in the vehicle decision-making process; the optimization process of this structure is carried out through global search and local fine-tuning by the sand cat swarm intelligent optimization algorithm to obtain the optimized Bayesian network structure.

[0018] Furthermore, in S3, the left lane availability detection mechanism of the adaptive maneuver tree search is as follows:

[0019] A maneuver tree structure is designed, where each node represents the state of the vehicle, including the current lane, position, speed, and acceleration; the maneuvering behavior covers three lateral actions, including changing lanes to the left, keeping lanes, and changing lanes to the right, and three longitudinal actions, including acceleration, constant speed, and deceleration; these behaviors are combined to form a 3×3 decision matrix;

[0020] Starting from the initial state of the decision tree, each node is recursively expanded to generate potential maneuvers based on the current lane and calculate the corresponding child node states; the state of each node is updated according to physical formulas, including vehicle position, speed, and lane change behavior; an adaptive beam search algorithm is used to find the optimal path after optimization in the maneuver tree;

[0021] After finding the optimal path through adaptive beam search, the system will dynamically evaluate the safety of each step; if each step meets the safety requirements, the vehicle can overtake in the left lane; otherwise, the system will recommend other action plans based on the current environmental conditions, such as overtaking in the right lane.

[0022] Furthermore, in S4, the vehicle density data of each lane can be obtained using traffic information;

[0023] The fast lane speed recommendation mechanism introduces a density-speed based traffic flow model, which calculates the recommended speed by determining the following key traffic parameters:

[0024] The coverage area of ​​the roadside unit RSU is divided into n blocks of length R L / n road section, where R L Indicates the length of the road section covered by the current RSU, and then selects the road section with the lowest density in the left lane for analysis;

[0025] First, calculate the free flow speed v of each road segment f , i.e., the velocity under low-density conditions;

[0026] Secondly, the critical density k is determined by analyzing the rate of change of the velocity-density curve. c and the corresponding critical speed v c , that is, find the point where the speed difference changes the least, expressed as:

[0027]

[0028] in, represents the rate of change of the average velocity relative to the density k under low-density conditions, argmin(·) represents the minimum value, v c=v(k c ) represents the speed at the critical density; this critical point reflects the transition from free flow to congested traffic state; in order to further characterize the relationship between speed and density, the smoothness parameter is defined The recommended speed v for the left lane can be calculated based on the density-speed traffic flow model r , as shown below:

[0029]

[0030] Where k is the current vehicle density on the lane; if the speed of the connected vehicle in the leftmost lane exceeds v r , you can stay in that lane; otherwise, it is recommended to return to the right lane to optimize the overall traffic flow.

[0031] Furthermore, in S5, KNN is used to select batch data in the experience pool in deep reinforcement learning to ensure that the experience pool contains both samples closest to the current state and relatively different samples.

[0032] Furthermore, in S5, the dueling deep Q network DDQN and the deep deterministic policy gradient DDPG are combined to capture the temporal features through the long short-term memory network LSTM, estimate the state value and action advantage respectively, and adapt to the mixed decision space of discrete and continuous actions.

[0033] Furthermore, in S5, comprehensive efficiency, safety, fairness, compliance with traffic rules and penalties for unreasonable operations are combined through a weighted combination of various indicators to ensure that vehicles can make decisions efficiently, safely and fairly when performing lane change and overtaking.

[0034] The fairness-aware overtaking decision optimization method for unstructured mixed scenarios of networked and non-networked vehicles provided by the present invention aims to improve the safety, efficiency and fairness of lane-changing overtaking operations in MCCV scenarios. The method introduces a fairness evaluation method based on a Gaussian Bayesian network, which quantifies fairness under limited data conditions by modeling the probabilistic relationship between vehicle behavior and fairness results; develops a left lane availability detection mechanism based on adaptive maneuvering tree search and a fast lane speed recommendation mechanism based on traffic flow analysis. These mechanisms enhance compliance with traffic regulations and improve efficiency by dynamically evaluating lane conditions and providing real-time speed recommendations based on traffic density and flow; introduces a KNN-enhanced deep reinforcement learning method that combines a parameterized dueling deep recursive Q network with KNN-enhanced experience playback. This method effectively copes with rare but critical driving conditions, such as unpredictable vehicle behavior or sudden traffic dynamics, and improves decision reliability in various traffic scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0036] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0037] Figure 1 is a flow chart of a fairness-aware overtaking decision optimization method for an unstructured mixed scenario of networked and non-networked vehicles provided by the present invention;

[0038] Figure 2 It is the overall framework diagram of the fairness-aware overtaking decision optimization method composed of multiple modules in the MCCV scenario. DETAILED DESCRIPTION

[0039] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present invention. Instead, they are merely examples of systems consistent with some aspects of the present invention as detailed in the appended claims.

[0040] In MCCV scenarios involving connected and non-connected vehicles, there are multiple challenges, including increased decision-making complexity and safety risks brought about by changes in communication capabilities between connected and non-connected vehicles, the need to evaluate the fairness of vehicle decisions when data is limited to reduce the impact of decisions on surrounding vehicles, and rare but critical traffic scenarios (such as sudden traffic changes or unpredictable vehicle behavior), which require decision-making algorithms to have strong adaptability.

[0041] like Figure 1 As shown, the present invention provides a fairness-aware overtaking decision optimization method for unstructured mixed scenarios of networked and non-networked vehicles, aiming to improve the safety, efficiency and fairness of lane-changing overtaking operations in MCCV scenarios. The method comprises the following steps:

[0042] S1: A networked vehicle equipped with an onboard unit monitors and records the traffic information of surrounding vehicles in real time through its own onboard sensor equipment, wherein the traffic information includes the vehicle's position, speed, acceleration, lane, and lane density;

[0043] S2: Based on traffic information, a fairness evaluation method based on Gaussian Bayesian network is used to model the relationship between vehicle behavior and fairness results, and the fairness probability is obtained to quantify the fairness of vehicle decision-making under limited data conditions;

[0044] S3: Based on traffic information, a left lane availability detection mechanism based on adaptive maneuvering tree search is used to obtain a feasibility index for dynamically evaluating the left lane for overtaking, which is used to evaluate the availability of the left lane for overtaking.

[0045] S4: Based on the traffic information, a fast lane speed recommendation mechanism based on traffic flow analysis is used to provide a real-time fast lane recommended speed according to the lane density in the traffic information;

[0046] S5: Based on the traffic information, fairness probability, feasibility index and recommended speed obtained in the previous steps, a deep reinforcement learning method enhanced by K-nearest neighbor (KNN) is used, that is, a parameterized duel deep recursive Q network is combined with KNN-enhanced experience playback to construct a lane changing and overtaking decision model to optimize the fairness, safety and efficiency of driving operations; the connected vehicle executes safe, efficient and fair driving decisions according to the decision results provided by the lane changing and overtaking model.

[0047] The overall structure of the present invention is as follows Figure 2 As shown. The present invention aims to control the connected vehicle to make safe and efficient lane change and overtaking decisions in the MCCV scenario while minimizing the interference to surrounding vehicles. In order to explain in detail how to control the connected vehicle to perform lane change and overtaking actions through various modules, a specific scenario is described below. In a highway with three lanes, four vehicles and one roadside unit (RSU) are selected: one connected vehicle (ECV) equipped with an OBU; two non-connected vehicles (NCV), marked as NCV 1 and NCV 2 ; and an affected NCV (INCV), whose status may be affected by the ECV decision.

[0048] The invented framework first analyzes a specific lane-changing overtaking scenario and extracts fairness features based on the state changes of surrounding vehicles. Then, it combines road condition information to evaluate several factors that affect the decision. These factors are processed in the decision layer, which outputs specific driving decisions for the ECV to guide it to complete the maneuver. At the road segment layer, two cases of unfair decisions are illustrated. In the left figure, the ECV moves to the middle lane to increase speed, forcing the INCV to change lanes to avoid a collision. In the right figure, the ECV requests a left lane change, causing the INCV in the leftmost lane to brake suddenly to maintain a safe distance. At the fairness layer, the state changes of each vehicle are evaluated to identify unfair effects. The figure shows how lane position and vehicle speed change over time, highlighting how the ECV lane change decision forces the INCV to make significant adjustments, such as changing lanes or slowing down. At the analysis layer, driving decisions are evaluated through three key modules: a Bayesian fairness measurement module, a tree search module that detects the availability of the left lane, and a recommended speed module for the leftmost fast lane. The outputs of these modules are fed to the decision layer, where a neural network algorithm processes them, enabling the ECV to make safe, efficient, and fair decisions. Then, the ECV executes the lane-changing and overtaking action output by the decision layer. The framework aims to ensure the fairness of ECV decision-making in MCCV scenarios, emphasizing safety, efficiency, and minimizing the negative impact on surrounding vehicles.

[0049] S2: Based on traffic information, a fairness evaluation method based on a Gaussian Bayesian network is used to model the relationship between vehicle behavior and fairness results, and the fairness probability is obtained to quantify the fairness of vehicle decisions under limited data conditions. First, the deceleration rate for collision avoidance (DRAC) and the maximum available deceleration rate (MADR) are used to determine whether the vehicle decision is fair. After three steps of node selection, Gaussian mixture model and structure learning, an optimized Bayesian network model is constructed, and the fairness probability is obtained using this model and traffic information.

[0050] 1) Fairness Definition: Assessing potential traffic conflicts between vehicles is critical to ensuring road safety. DRAC quantifies the risk of a potential collision between the front and rear vehicles by calculating the deceleration required to avoid a collision. DRAC helps assess whether the rear vehicle can safely continue driving without colliding with the front vehicle. DRAC is calculated as follows:

[0051]

[0052] where v i ,y i and l iRepresents the speed, longitudinal position and length of vehicle i (i=m for the rear vehicle and n for the front vehicle). A DRAC value of zero means that the rear vehicle is traveling at a speed equal to or lower than the speed of the front vehicle, which means there is no risk of collision. A positive DRAC value means that the rear vehicle must slow down to avoid a collision. To further evaluate the competing conflicts, the maximum available deceleration rate (MADR) is used, which represents the maximum deceleration that the vehicle can achieve in an emergency situation. In this invention, the MADR value generally adopted is 1.4m / s 2 . When the calculated DRAC exceeds the MADR, it indicates that there is a competitive conflict between the front and rear vehicles. If the vehicle needs a deceleration rate greater than the MADR to avoid a collision, it means that the decision of the front vehicle disrupts the driving state of the rear vehicle. In this case, if the vehicle's decision forces the surrounding vehicles into an emergency situation that requires a drastic change in speed, it constitutes unfair interference. By integrating DRAC and MADR, the risk of conflict between vehicles can be quantitatively evaluated and it can be determined whether the decision has caused unfair interference to the surrounding vehicles. After completing the fairness judgment of the decision, it is necessary to construct an optimized Bayesian network model to measure the fairness probability.

[0053] 2) Node selection: Selecting appropriate node variables is crucial to building a Bayesian network that effectively measures fairness. Select variables that can capture system characteristics and causal relationships: vehicle speed (V s ), acceleration (A s ), Lane change status (L t ), relative speed (V r ) and the distance to the target vehicle (D r ), DRAC and target vehicle status changes (s t ). These seven variables are collectively referred to as X={V s , A s , L t , V r , D r ,DRAC,s t}, reflecting the multi-dimensional impact of ECV decision-making. Specifically, V s and A s This can affect vehicle dynamics and affect the reaction time of surrounding vehicles. Sudden changes in acceleration may force nearby vehicles to perform emergency maneuvers. t Indicates lane change behavior. Sudden or unpredictable lane changes can destabilize surrounding vehicles, especially at high speeds or in heavy traffic. r and D r Evaluate vehicle interaction. High V r or small D rThe DRAC evaluates the dynamic risk between the two vehicles. A high DRAC value indicates a need for significant deceleration, reflecting a high risk and potentially unfair decision; for S t ,If the target vehicle has to adjust its behavior to maintain safety, it indicates that the ECV’s decision has an adverse impact on the surrounding traffic. These variables provide a comprehensive representation of vehicle dynamics, interactions, and risks, forming the core nodes in the Bayesian network for evaluating fairness.

[0054] 3) Gaussian mixture model: Since the above node selection includes continuous data as nodes, processing continuous data in Bayesian networks will increase the computational complexity, especially in real-time vehicle control tasks. Continuous variables may introduce delays and mismatches with the training node probabilities. To solve this problem, Gaussian mixture models (GMMs) are used to discretize continuous variables. The continuous variable set is discretized using equation (2). Convert to discrete variable X GMM , where K is the number of Gaussian components, π K is the weight of each component, is the Gaussian distribution of the Kth component. In this invention, GMM is used to classify the continuous variable V s , A s , P r , D r and DRAC are trained to effectively capture their distribution. The number of components K varies across variables. This discretization provides effective data support for subsequent Bayesian network modeling.

[0055]

[0056] 4) Structural learning: After selecting node variables and discretizing the data, continue to build a Bayesian network, which involves parameter learning and structural learning. Parameter learning uses Bayesian estimation to combine prior information with observed data to estimate the conditional probability distribution of each node given the structure.

[0057] For structure learning, the challenge is to find the optimal network structure that represents the probability distribution of the data. The K2 score metric is used to evaluate different network structures. For a directed acyclic graph (DAG) G constructed from a dataset D, equation (3) is used to represent each node X i Calculate the K2 score f(G:D), where n is the number of nodes and q i is node X i The number of parent node combinations, r i Yes X i The number of states, n ij Yes X i The count of the j-th parent node combination, n ijkis the X given the j-th parent node combination i Since the log probability is negative, f(G:D) returns a negative K2 score, so the goal is to maximize f(G:D).

[0058]

[0059] Given that structure learning is an NP-hard problem, traditional algorithms are inefficient for large networks. This invention uses the Sand Cat Swarm Optimization (SCSO) algorithm, inspired by the hunting behavior of sand cats, to optimize the network structure. The SCSO algorithm is divided into two phases: search and attack. In the search phase, the algorithm explores the solution space through random mutations to ensure diversity and avoid local optimality. Each candidate DAG G i The probability P calculated based on equation (4) add Modify the parent node to undergo mutation, where λ is the control parameter, Δ f is the change in K2 score due to mutation. Let the mutant population be {G 1 ′,G 2 ′, ..., G N ′}, and then calculate the K2 score f(G i ′: D).

[0060]

[0061] During the attack phase, the algorithm fine-tunes the best performing individuals, focusing on small adjustments such as changing edge directions or reconfiguring parent nodes. While maintaining population diversity by introducing new individuals, the best performing individuals are iteratively updated. best Adjustments are as follows:

[0062] X′ i =X i +α·(X i -X best ) (5)

[0063] Among them, X′ i is the adjusted node, α is the control parameter, X best is the current best solution. Let the best individual after adjustment be G best ′, and calculate its K2 score f(G best Then, according to the mutation and the adjusted score, the individual with the highest K2 score is selected to update the population, forming a new population, denoted as {G 1 , G 2 , ..., G N The search and attack phases are repeated until the convergence or termination conditions are met, and the DAGG with the highest K2 score is selected. optAs the optimized Bayesian network structure. Finally, the traffic information is input into the optimized Bayesian network structure to obtain the fairness probability.

[0064] In summary, the fairness evaluation method proposed in the present invention constructs a Bayesian network based on selected variables (such as vehicle speed, acceleration, lane change status, relative speed, relative distance, DRAC and target vehicle state change). GMM is used to discretize continuous variables to reduce complexity. Bayesian estimation is used for parameter learning, and the SCSO algorithm is used for efficient structure learning. This enables the Bayesian network to accurately evaluate the fairness of ECV decisions in a dynamic traffic environment.

[0065] In S3, the present invention incorporates the traffic rule of giving priority to left lane changes when changing lanes to overtake. However, it is not always possible to perform left-hand overtaking at any time; when the vehicle is in the leftmost lane or the distance to the left front vehicle is insufficient, it may be necessary to overtake on the right. Therefore, it is crucial to detect the availability of the left lane in a dynamic environment. The present invention proposes a method using a maneuver tree structure combined with an adaptive beam search to identify an optimized maneuver sequence within a specified depth. This method helps to optimize path planning under the constraint of performing left lane changes.

[0066] First, vehicle and maneuver classes representing vehicle states and potential actions are defined. The vehicle state includes its lane, position, velocity, and acceleration, while the maneuver includes three lateral actions (left lane change, lane hold, right lane change) and three longitudinal actions (acceleration, constant speed, deceleration), forming a matrix of nine (3×3) possible maneuver decisions. To reduce unnecessary computations, the maneuver tree is pruned according to the vehicle's state: ① If the vehicle is in the leftmost or rightmost lane, the decision of "left lane change" or "right lane change" is canceled respectively; ② At the root node, the decision of "right lane change" is deleted because the left lane is given priority for overtaking.

[0067] Then, a maneuver tree is constructed, representing all possible paths from the root node (current state) to the leaf nodes (future states). The depth of the tree corresponds to the number of search steps. Starting from the initial state, each node is recursively expanded by generating potential maneuvers based on the current lane and calculating the corresponding child node states. This ensures that all feasible paths are considered in subsequent searches. During the node expansion process, a new state is calculated for each possible maneuver, and the vehicle's position, velocity, and lane change are updated according to the physical formulas. This module assumes that the surrounding vehicles maintain a constant speed. The initial state of the ECV is defined by its detected velocity and acceleration, where |a 0 |Indicates the magnitude of acceleration or deceleration.

[0068] The speed v of the vehicle at time t t , by v t =v t-1 +a0 ·Δ t Calculated, its position x t Calculated as

[0069] It should be noted that in order to accurately evaluate the safety of ECV execution decisions, the extended time to collision (ETTC) metric is adopted, which combines longitudinal and lateral dynamics. The ETTC of vehicle m at time t is calculated as Equation (6), where D mn is the center distance between vehicles m and n, l i , O i and v i represents the length, center position and speed of vehicle i (i=m,n).

[0070]

[0071] During a lane change and overtaking maneuver, the target vehicle needs to calculate the time to collision (TTC) of the vehicles in front and behind it in the current lane, as well as the vehicles in the adjacent lane. These collision indicators are divided into the TTC of the vehicle in front (fTTC), the TTC of the vehicle behind (rTTC), the ETTC of the vehicle in front of the adjacent lane (fETTC), and the ETTC of the vehicle behind the adjacent lane (rETTC). The smallest value indicates the most imminent collision risk and can more accurately assess the highest potential risk. Define the risk factor (RF) as the minimum value of these four attributes:

[0072] RF=min(fTTC, rTTC, fETTC, rETTC) (7)

[0073] A smaller RF indicates a higher probability of an imminent collision and therefore a greater risk. In this scheme, the minimum safety threshold of RF (denoted as RF safe ) is set to 3s.

[0074] In order to find the optimal path in the maneuvering tree, an adaptive beam search algorithm is used. The key steps include calculating the cumulative weight of each node, sorting the nodes in descending order, and selecting high-confidence nodes. Unlike the traditional beam search that uses a fixed candidate width, the adaptive method proposed in this invention selects high-confidence nodes based on the cumulative weight of each layer. Specifically, the node with the highest weight W max In comparison, nodes whose weight difference is less than the threshold ∈ are selected as high confidence nodes, defined as {i|W max -W i <∈}, where W i is the weight of node i. This method retains the most likely path while reducing computational complexity. The cumulative weight of each node includes the speed benefit (benefit V =Vt / V max ) and safety benefits R ), where V t and V max They represent the speed of the vehicle at time t and the maximum speed allowed for the vehicle, benefit R According to the RF calculation defined in equation (7), once the optimization path is selected, each step can be evaluated to see whether it meets the safety condition. Specifically, the benefit of each node is R Must be greater than RF safe If all nodes in the optimized path meet the safety requirements, then the feasibility index checkFlag = 1, and the vehicle can safely perform a left lane change; otherwise, if the safety conditions are not met, the feasibility index checkFlag = 0, and it is determined that left overtaking is not available, and the vehicle can choose to perform a right lane change.

[0075] In S4, in order to improve the traffic efficiency of the leftmost lane of the highway, lane speed recommendation is performed based on traffic flow analysis in this scheme. This method helps ECV optimize lane selection and speed. After completing the overtaking maneuver in the leftmost lane, maintaining a high speed ensures efficient traffic flow. However, staying in this lane at a lower speed will reduce the overall efficiency. Therefore, a real-time dynamic speed recommendation is provided for the leftmost lane to help ECV decide whether to return to the right lane. In MCCV scenarios, it is usually impractical to obtain real-time data from all vehicles. However, vehicle density data for each lane can be collected through vehicle-to-infrastructure systems such as inductive loops or visual sensors. Due to different road conditions and traffic volumes, the vehicle status in different RSU coverage areas may be different. By analyzing the lane density within the coverage area of ​​each RSU, speed recommendations can be provided for the leftmost lane of each road segment.

[0076] The present invention introduces an algorithm based on the density-speed model to analyze the speed-density relationship by determining the following key traffic parameters.

[0077] First, calculate the free flow speed v of each road segment f , which represents the speed under low density conditions. The RSU coverage area is divided into blocks of length R L / n segment, where R L Indicates the length of the road section covered by the current RSU, and then selects the leftmost lane section with the lowest density for analysis. Free flow speed v f It can be calculated as Where N is the number of data points under low density conditions, u n is the velocity at data point n. Next, the critical density k is determined by analyzing the rate of change of the velocity-density curvec and the corresponding speed v c , that is, find the point where the speed difference changes the least, expressed as:

[0078]

[0079] in, represents the rate of change of the average velocity relative to the density k under low-density conditions, argmin(·) represents the minimum value, v c =v(k c ) represents the speed at the critical density. This critical point reflects the transition from free flow to congested traffic state. To further characterize the speed-density relationship, the smoothness parameter m is defined as:

[0080]

[0081] Using these parameters, the S3 model calculates the recommended speed v for the leftmost lane r , as shown below:

[0082]

[0083] Where k is the current vehicle density on the lane. The model calculates the recommended speed v based on the current traffic density. r If the speed of the ECV in the leftmost lane exceeds v r , it can stay in that lane; otherwise, it is recommended to return to the right lane to optimize the overall traffic flow.

[0084] In S5, a lane-changing overtaking model is constructed to control connected vehicles to perform safe, efficient, and fair decisions.

[0085] 1) Preliminary knowledge: To solve the complex lane-changing and overtaking problem involving discrete lane-changing decisions and continuous speed adjustment, a parameterized action Markov decision process (PAMDP) is used. PAMDP integrates discrete and continuous action types to achieve precise and intelligent decision making. In PAMDP, the state space S includes all possible states of the vehicle and its environment. Decision making involves selecting a discrete action a d and continuous control parameters θ, forming a parameterized action space a(θ). Each action is represented by a(θ)=(a d , a θ ). The state transition function P(s′|s, a(θ)) describes how an action leads to a new state. To evaluate the effectiveness of a(θ), the reward function R(s, a(θ)) assigns an immediate reward to a state-action pair. The discount factor γ∈[0,1] balances the immediate and future rewards. Therefore, PAMDP is represented as a five-tuple (S, A(θ), P, R, γ).

[0086] 2) Action space: The state of the vehicle consists of its own information and its interaction with the surrounding environment. The self-state includes the position d, lane L, speed v, acceleration a, and the previous acceleration a t-1 For environmental interaction, the area around the ECV is divided into six regions: left front, left rear, straight ahead, straight behind, right front, and right rear. In the MCCV scenario with NCV, the ECV can detect the nearest vehicle in each region using onboard sensors. If there is no vehicle in region m, the distance Δd m and speed v m Set to -1. In addition, the distance d of the target vehicle is also included T and speed v T to ensure effective overtaking.

[0087] Therefore, the state S of the ECV at time t t is defined as

[0088] 3) Action space: In lane change overtaking, the action space includes discrete lateral lane change actions and continuous longitudinal speed control actions. This combined action space accurately reflects the decision-making requirements in complex traffic environments. Discrete actions include lane keeping, left lane change, and right lane change, represented by a d ∈{0, -1, 1}. Continuous actions involve speed control, denoted as a θ , where a θ ∈(a min ,a max ). Therefore, each action in A(θ) is represented by a(θ)=(a d , a θ ).

[0089] 4) Reward function: In order to guide the ECV to make safe, efficient and fair driving decisions, a comprehensive reward function is designed, which includes five aspects: efficiency, safety, fairness, compliance with traffic rules and punishment for unreasonable behavior.

[0090] ① Efficiency: In lane-changing overtaking maneuvers, maintaining high speed and comfort is crucial to improving overall efficiency. The speed bonus is quantified as where v,v min and,v max The acceleration reward is r. ea =-|a| / a max , penalizes high acceleration, where a and a max Represent the current acceleration and maximum acceleration of the vehicle respectively. The acceleration reward is r ej =-|a t -at-1 | / (a max ·Δt), to penalize sudden acceleration changes, a t 、a t-1 and Δt represent the acceleration of the vehicle at time t and t-1 and the time interval between each decision step. The overall efficiency reward is r e =r ev +(r ea +r ej ) / 2.

[0091] ② Safety: Ensuring safety can prevent accidents and protect road users. Safety is quantified using RF calculated using equation (7), expressed as r s =RF.

[0092] ③ Fairness: To ensure that the decision of the ECV does not have a negative impact on other vehicles, the fairP calculated in S2 is used as the fairness reward, denoted as r f = fairP.

[0093] ④ Traffic rules: Complying with traffic rules ensures legality and compliance. Use the feasibility index of the left lane detection designed in S3 to determine whether the ECV can perform a left lane change:

[0094]

[0095] ⑤ Penalties for unreasonable actions: Penalties for unreasonable actions help prevent vehicles from making dangerous decisions. Penalties are divided into two areas: impracticable maneuvers and lane speed. For impracticable maneuvers, vehicles will be penalized for attempting to change lanes to the left or right when already in the leftmost or rightmost lane, or for changing lanes when no vehicle is detected in the forward sensing range. In these cases, r will be assigned ue = -1 penalty. For lane speed, if the vehicle is in the leftmost lane and its speed v 0 Below recommended speed v r , it will be punished and encouraged to leave the lane, represented by r pv =v θ -v r , the total penalty is denoted as r p =r ue +r pv .

[0096] Therefore, combining all of the above, the total reward is calculated using equation (12), where ω e ,ω s ,ω f ,ω r and ω p are the weights assigned to each component.

[0097] r=ω e r e +ω s r s +ω f r f +ω r r r +ω p r p (12)

[0098] 5) KPDDRQN algorithm: In order to handle mixed discrete and continuous action spaces, this paper proposes the KPDDRQN algorithm, which combines the dueling deep Q network (DDQN) and the deep deterministic policy gradient (DDPG). DDQN performs well in discrete action spaces by separating state values ​​and action advantages, while DDPG effectively handles continuous actions. KPDDRQN integrates these methods to improve decision accuracy in mixed action spaces.

[0099] In the KPDDRQN network structure, long short-term memory (LSTM) layers and fully connected (FC) layers are used to capture temporal features and accurately evaluate actions. The input is the state vector and the action parameter vector Where n s and n a Represent the dimensions of the state vector and action parameters respectively, and are combined into LSTM processes the input sequence and outputs the hidden state {h 1 ,h 2 , ..., h T}. For simplicity, only h T =LSTM(x) selects the last hidden state h L . This is followed by the FC layer, which is used to extract high-dimensional features h L The network is divided into two parts: one estimates the state value function V(s), and the other estimates the advantage function A(s, a). The output of V(s) is transformed by the linear transformation V(s) = W V h L +b V Given, where the weight and is the bias term. The output of A(s,a) is A(s,a)=W A h L +b A , where the weight is the bias term. The final Q value is calculated as follows:

[0100]

[0101] in is a set of possible actions. Q-value optimization is based on the Bellman equation. State s t and action d t (discrete) and c t The (continuous) Q ​​value is:

[0102]

[0103] Among them, r t is the immediate reward, γ is the discount factor, D and C d are the set of discrete actions and the set of continuous actions respectively.

[0104] In order to improve the effectiveness of experience playback, the invention introduces the KNN enhanced experience playback (KNN-ER) method, which extracts the experience pools from two different groups. Select samples. First, define a central state s c =s t , and use the Euclidean distance d(s c ,s i )=|s c -s i | as the i-th state s i The distance measure between them is from the experience pool Retrieve the δ most similar samples from These samples represent states that are closely related to the current state, providing a more direct context for the agent’s decision-making process. In order to maintain diversity, m-δ most dissimilar samples are also selected. where m is the total batch size. These samples introduce variability, allowing the model to effectively capture the diversity of distributions present in complex driving environments. This approach mitigates the bias towards frequent driving conditions, enabling the reinforcement learning agent to better predict and handle low-frequency but critical events, such as sudden deceleration or unpredictable maneuvers of surrounding vehicles. Finally, the selected training batch data is represented as

[0105] Use KNN enhanced batching during actor network update Calculate the loss function. At this stage, the parameters of the critic network remain unchanged while the parameters of the actor network are optimized with the goal of minimizing the following loss:

[0106]

[0107] where Q(s, d, π(s; θ π );θ Q ) represents the critic network θ Q Estimate the Q value for state s and action d. By minimizing the negative Q value, the parameters θ of the actor network πUpdate in the direction of maximizing the Q value, resulting in an optimized action. When updating the critic network, the goal is to minimize the temporal difference error. The target Q value y t Defined in Equation (16), where s′ is the next state, d′ represents the possible action parameters, and r t Represents the immediate reward at time t.

[0108]

[0109] Therefore, the loss function is expressed as:

[0110]

[0111] By minimizing the loss in Equation (17), the critic network can estimate the Q-value more accurately. During the KPDDRQN update process, the actor network and the critic network are optimized alternately, ensuring continuous improvement of the policy and value estimation in the mixed action space.

[0112] This paper proposes a new fairness-aware overtaking decision optimization method for MCCV scenarios with connected and non-connected vehicles, which significantly improves the safety, efficiency and fairness of vehicle decision-making. A fairness evaluation method based on a Gaussian Bayesian network is introduced to model the probabilistic relationship between vehicle behavior and fairness results, thereby quantifying the fairness of vehicle decision-making under limited data conditions; at the same time, a left lane availability detection mechanism based on adaptive maneuvering tree search and a fast lane speed recommendation mechanism based on traffic flow analysis are designed. These mechanisms enhance compliance with traffic regulations and improve efficiency by dynamically evaluating lane conditions and providing real-time speed recommendations based on traffic density and flow; in addition, a deep reinforcement learning method based on KNN enhancement is designed, which combines a parameterized dueling deep recursive Q network with KNN-enhanced experience playback. This method effectively copes with rare but critical driving conditions, such as unpredictable vehicle behavior or sudden traffic dynamics, and improves decision reliability in various traffic scenarios.

[0113] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These changes and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A fairness-aware overtaking decision optimization method for unstructured mixed scenarios of networked and non-networked vehicles, characterized in that: The steps include: S1: The connected vehicle monitors and records the traffic information of surrounding vehicles in real time through its own sensor equipment. The traffic information includes the vehicle's position, speed, acceleration, lane, and lane density; S2: Based on traffic information, the fairness evaluation method of Gauss's Bayesian network is used to model the relationship between vehicle behavior and fairness results, and the fairness probability is obtained to quantify the fairness of vehicle decision-making under limited data conditions; S3: Based on traffic information, the left lane availability detection mechanism of adaptive maneuver tree search is used to obtain a feasibility index for dynamically evaluating the left lane for overtaking, which is used to evaluate the availability of the left lane for overtaking. S4: Based on traffic information, a fast lane speed recommendation mechanism based on traffic flow analysis is used to provide a real-time fast lane recommended speed according to the lane density in the traffic information; S5: Based on traffic information, fairness probability, feasibility index and recommended speed, a deep reinforcement learning method enhanced by K-nearest neighbor (KNN) is used to combine the parameterized dueling deep recursive Q network with KNN-enhanced experience playback to build a lane-changing overtaking decision model to optimize the fairness, safety and efficiency of driving operations; The connected vehicle executes driving decisions according to the decision results provided by the lane change and overtaking model.

2. The fairness-aware overtaking decision optimization method for an unstructured mixed scenario of networked and non-networked vehicles according to claim 1, characterized in that: In step S2, the lane change state L is calculated based on the traffic information. t , relative speed V r , the distance D from the target vehicle r , the deceleration rate DRAC to avoid collision and the state change S of the target vehicle t , and combined with the speed V in the traffic information s , acceleration A s As variables of each node in the Bayesian network; Use Gaussian mixture model GMM to discretize the V in Bayesian network nodes s , A s 、V r , D r The five continuous variables DRAC are discretized using formula (1), where K is the number of Gaussian components, π K is the weight of each component, is the Gaussian distribution of the Kth component; The sand cat swarm intelligent optimization algorithm based on K2 scoring is used to represent the probabilistic relationship between different nodes, so as to evaluate the fairness in the vehicle decision-making process; the optimization process of this structure is carried out through global search and local fine-tuning by the sand cat swarm intelligent optimization algorithm to obtain the optimized Bayesian network structure.

3. The fairness-aware overtaking decision optimization method for an unstructured mixed scenario of networked and non-networked vehicles according to claim 1, characterized in that: In S3, the left lane availability detection mechanism of the adaptive maneuver tree search is as follows: A maneuver tree structure is designed, where each node represents the state of the vehicle, including the current lane, position, speed, and acceleration; the maneuvering behavior covers three lateral actions, including changing lanes to the left, keeping lanes, and changing lanes to the right, and three longitudinal actions, including acceleration, constant speed, and deceleration; these behaviors are combined to form a 3×3 decision matrix; Starting from the initial state of the decision tree, each node is recursively expanded to generate potential maneuvers based on the current lane and calculate the corresponding child node states; the state of each node is updated according to physical formulas, including vehicle position, speed, and lane change behavior; an adaptive beam search algorithm is used to find the optimal path after optimization in the maneuver tree; After finding the optimal path through adaptive beam search, the system will dynamically evaluate the safety of each step; if each step meets the safety requirements, the vehicle can overtake in the left lane; otherwise, the system will recommend other action plans based on the current environmental conditions, such as overtaking in the right lane.

4. The fairness-aware overtaking decision optimization method for an unstructured mixed scenario of networked and non-networked vehicles according to claim 1, characterized in that: In S4, the vehicle density data of each lane can be obtained using traffic information; The fast lane speed recommendation mechanism introduces a density-speed based traffic flow model, which calculates the recommended speed by determining the following key traffic parameters: The coverage area of ​​the roadside unit RSU is divided into n blocks of length R L / n road section, where R L Indicates the length of the road section covered by the current RSU, and then selects the road section with the lowest density in the left lane for analysis; First, calculate the free flow speed v of each road segment f , i.e., the velocity under low-density conditions; Secondly, the critical density k is determined by analyzing the rate of change of the velocity-density curve. c and the corresponding critical speed v c , that is, find the point where the speed difference changes the least, expressed as: in, represents the rate of change of the average velocity relative to the density k under low-density conditions, argmin(·) represents the minimum value, v c =v(k c ) represents the speed at the critical density; this critical point reflects the transition from free flow to congested traffic state; in order to further characterize the relationship between speed and density, the smoothness parameter is defined The recommended speed v for the left lane can be calculated based on the density-speed traffic flow model r , as shown below: Where k is the current vehicle density on the lane; if the speed of the connected vehicle in the leftmost lane exceeds v r , you can stay in that lane; otherwise, it is recommended to return to the right lane to optimize the overall traffic flow.

5. The fairness-aware overtaking decision optimization method for unstructured mixed scenarios of networked and non-networked vehicles according to claim 1, characterized in that: In S5, KNN is used to select batch data in the experience pool in deep reinforcement learning to ensure that the experience pool contains both samples that are closest to the current state and relatively different samples.

6. The fairness-aware overtaking decision optimization method for an unstructured mixed scenario of networked and non-networked vehicles according to claim 1, characterized in that: In S5, the dueling deep Q network DDQN and the deep deterministic policy gradient DDPG are combined. The long short-term memory network LSTM is used to capture the timing characteristics, estimate the state value and action advantage respectively, and adapt to the mixed decision space of discrete and continuous actions.

7. The fairness-aware overtaking decision optimization method for an unstructured mixed scenario of networked and non-networked vehicles according to claim 1, characterized in that: In S5, comprehensive efficiency, safety, fairness, compliance with traffic rules and penalties for unreasonable operations are taken into account through a weighted combination of various indicators to ensure that vehicles can make decisions efficiently, safely and fairly when performing lane changes and overtaking.

Images