A lane reduction area-oriented vehicle lane-changing depth reinforcement learning decision method

Abstract

The present application relates to a kind of vehicle lane-changing depth reinforcement learning decision method for lane reduction area, belong to intelligent network connected vehicle decision-making technical field.The method aims to solve the problem that the existing lane-changing decision lacks macro traffic flow coordination cognition in lane reduction area, and easily leads to congestion spread.The technical scheme is to construct the state space that fuses vehicle motion state, lane macro traffic situation and lane-changing urgency, design multi-dimensional reward function, and use the multi-branch feature coding network based on double deep Q network to make decisions;After model training converges, it is deployed online, and the output action is judged and safety checked to ensure safety by introducing strategy rollback mechanism.The method can significantly improve the cognitive ability of vehicle to congestion propagation law and the quality of lane-changing decision, so as to enhance the overall traffic efficiency of road section, decision safety and model convergence speed.

Description

Technical Field

[0001] This invention belongs to the field of intelligent connected vehicle decision-making technology, and relates to a deep reinforcement learning decision-making method for lane changing in lane reduction areas. Background Technology

[0002] With the deep application of cutting-edge technologies such as 5G communication, artificial intelligence, high-precision positioning, and vehicle sensor fusion, intelligent connected vehicles have gradually moved from proof-of-concept to large-scale practice, becoming a core driving force for the intelligent transformation of transportation systems. Deep reinforcement learning, as a cutting-edge direction in the field of artificial intelligence, has become an important method for researching driving strategies for intelligent connected vehicles due to its unique advantage of continuously optimizing decision-making strategies through autonomous interaction in complex dynamic environments. This technology can effectively address challenges such as the uncertainty of vehicle behavior, the dynamic evolution of environmental states, and multi-objective optimization conflicts in traffic scenarios, autonomously generating optimal or near-optimal driving decisions through trial and error iteration.

[0003] However, existing lane-changing decision-making methods have significant limitations in lane-reduction scenarios. These methods primarily focus on micro-level optimization of individual vehicle decisions, emphasizing the efficiency of individual vehicles while neglecting consideration of the overall coordination of traffic flow. In traffic bottleneck areas such as lane reduction, improper deceleration or lane-changing behavior by individual vehicles in densely mixed traffic flow can easily induce phantom traffic jams and stop-and-go oscillations, causing congestion shockwaves to spread upstream. Simultaneously, the lack of perception of the macro-level traffic flow status makes it impossible to quantify the propagation trend of the congestion shockwave ahead, resulting in a lack of foresight in lane-changing decisions and making it difficult to achieve a coordinated balance between individual efficiency and systemic efficiency in complex bottleneck scenarios.

[0004] To address the aforementioned issues, this invention proposes a deep reinforcement learning-based decision-making method for lane-changing in lane-reduction areas. It introduces a traffic wave theory model to achieve real-time perception of the macro-level congestion spread, deeply integrating this with a micro-level individual vehicle decision-making framework. By constructing a composite reward function incorporating multi-dimensional indicators such as individual efficiency, driving safety, and traffic flow stability, the agent can proactively perceive and quantify the propagation trend of congestion waves caused by lane reduction, thereby achieving forward-looking lane-changing decisions and avoiding chain reactions caused by blind lane changes. This method overcomes the limitations of local perception and the disconnect between micro-level decision-making and macro-level situation in traditional lane-changing strategies. It provides a robust and forward-looking decision-making solution for lane-reduction bottleneck scenarios, significantly improving individual vehicle speed gain, driving safety, and traffic flow efficiency, effectively suppressing congestion spread, reducing traffic delays, and demonstrating excellent scenario adaptability and broad engineering application prospects. Summary of the Invention

[0005] In view of this, the purpose of the present invention is to provide a deep reinforcement learning decision-making method for vehicle lane changing in lane reduction areas.

[0006] To achieve the above objectives, the present invention provides the following technical solution: A deep reinforcement learning decision-making method for lane changing in lane-shortening regions includes the following steps: S1: Introducing the LWR (Lighthill-Whitham-Richards) traffic flow model, a state space integrating macroscopic traffic flow state and microscopic vehicle information is designed, and an action space satisfying safety constraints and driving feasibility is constructed. A composite reward function that balances traffic efficiency and driving safety is proposed. Furthermore, a decision network is designed by combining graph convolutional networks and the DDQN algorithm to construct a vehicle lane-changing decision model for lane reduction regions. S2: Build a lane reduction traffic simulation scenario, train the vehicle lane-changing decision model until the vehicle lane-changing decision model converges, save the model weights, and obtain the trained vehicle lane-changing decision model. S3: Deploy the trained model into the control unit, acquire environmental information and construct a state representation that adapts to the model input, then use the pre-trained DDQN model to perform online state evaluation and output actions to achieve real-time autonomous decision-making for the vehicle. S4: Introducing a strategy rollback mechanism, the system first checks the actions output by DDQN for exceeding limits. If the check passes, it then performs dual verification using TTC and safe following distance. If the action is compliant, it is executed; if any step fails, the system will intercept the action and automatically trigger a safety rollback, downgrading to the lane keeping baseline strategy.

[0007] Furthermore, S1 specifically includes the following steps: S11: State space establishment: Status characteristics include: the micro-motion status of surrounding vehicles, vehicle location information, macro-traffic situation of the lane, and the urgency of lane changing.

[0008] Characteristics of motion state: The movement status of surrounding vehicles Modeled as a The matrix, where, The maximum number of observable vehicles within the vehicle's perception range, including the vehicle itself. For a single vehicle's feature dimension, the single vehicle feature consists of the following 6 state variables: (1) These six features represent the presence of surrounding vehicles as observed by the vehicle, its lateral and longitudinal positions relative to the vehicle, its absolute lateral and longitudinal speeds, and the lane number it is currently in. Vehicle positional relationship characteristics: Vehicle relative position characteristics Represented as a The matrix, where each row corresponds to the relative position of an observed vehicle. , This represents the maximum number of vehicles observable within the sensing range.

[0009] Macroscopic traffic situation characteristics of lanes: Lane macro situation characteristics Represented as a This is a matrix where the rows correspond to the left lane, the current lane, and the right lane where the vehicle is currently located, respectively. The three columns correspond to the feature set of a single lane. , respectively represent: Physical connectivity : Indicates whether the lane exists (when the vehicle is in the leftmost or rightmost lane, the corresponding left or right lane does not exist); average lane speed This indicates the average speed of vehicles traveling on the road section ahead of this lane. Traffic wave propagation speed : Represents the propagation speed of traffic waves in that lane, with the vehicle's position as the origin of the coordinate system, and two lengths defined as . Featured region: Upstream vehicle adjacent region: Interval range is Downstream area ahead: The interval range is ( Taking positive values), according to the LWR model, the formula for calculating the traffic wave velocity can be expressed as: (2) In the formula, , These represent the average vehicle speed and average headway in the upstream region, respectively. , These represent the average vehicle speed and average headway in the downstream area, respectively.

[0010] Lane change urgency characteristics: Lane change urgency characteristics It consists of the following two parts: Remaining distance This indicates the distance from the vehicle's current position to the lane reduction point, used to quantify the urgency of the vehicle approaching the bottleneck area; area marker. : Boolean state variable used to indicate whether the vehicle has entered the detention penalty zone. This zone is set on the outermost lane (i.e., the lane that shrinks), extending upstream from the lane shrinking point as the endpoint (which can be adjusted according to actual needs), to clearly indicate the current location of the vehicle in the zone.

[0011] S12: Action space establishment: The lane-changing decision of a vehicle is defined as a strategy selection based on a discrete set of actions. This set of actions includes three possible actions: changing lanes to the adjacent left lane, maintaining the current lane, and changing lanes to the adjacent right lane (the action set is...). ).

[0012] S13: Reward Function The reward function adopts a multi-dimensional weighted summation form, including basic speed reward, basic lane reward, collision penalty, etc., to quantify the merits of lane-changing decision actions.

[0013] Base speed bonus: (3) In the formula, For the speed of this vehicle, This is the maximum speed limit.

[0014] Basic lane bonus: (4) In the formula, This indicates the lane number of the vehicle's current lane. The outermost lane (i.e., the lane to be reduced) is defined as number 0, and the numbers increase sequentially inwards from the lane. This indicates the lane number of the innermost lane.

[0015] Delay Penalty: A penalty will be imposed on vehicles that remain in the delay penalty area (near the lane reduction point and have not yet left the lane) and are subject to the following penalties: (5) In the formula, Indicates the vehicle's current location. Indicates the starting position of the detention penalty zone. This indicates the end point of the penalty zone (i.e., the location where the lanes narrow).

[0016] Collision Penalty: To ensure driving safety, when a vehicle collides with surrounding vehicles, a corresponding negative penalty value is assigned: (6) Action Penalty: When an agent performs an invalid action, a corresponding penalty value is applied. Invalid actions include two situations: first, the action will cause the vehicle to cross the road boundary; second, the action does not meet preset safety constraints. The corresponding penalty terms are defined as follows: (7) Traffic flow guidance rewards: (8) In the formula, , , These represent the traffic wave velocities of the vehicle's left lane, current lane, and right lane, respectively. If there are no adjacent lanes (e.g., the vehicle is in the leftmost or rightmost lane), the corresponding wave velocity is 0.

[0017] Lane change speed benefits: (9) In the formula, , These represent the speeds of the vehicle in front in the new lane after the lane change and the speeds of the vehicle in front in the original lane before the lane change, respectively.

[0018] The total reward is expressed as follows: (10) In the formula, The weight parameters are adjusted according to the importance of each reward and penalty item, thereby positively guiding the agent to make lane-changing decisions.

[0019] S14: Network Design To fully extract traffic environment information, a multi-branch feature fusion network is designed. By encoding and fusing different types of state information, the Q-value prediction of discrete actions is achieved. The network divides the input into four independent branches: a graph convolutional network (GCN) is used to extract the positional interaction features between vehicles in the local space, while multiple independent multilayer perceptrons (MLPs) are used to extract vehicle motion state, lane macro situation, and lane change urgency features, respectively. The extracted features from the four branches are concatenated into a high-dimensional joint feature vector, which is then linearly transformed into an output vector with the same dimension as the action space. Each element of the vector corresponds to the Q-value estimate of each discrete action in the current state.

[0020] Furthermore, S2 specifically includes the following steps: S21: Set up the lane reduction simulation environment and initialize the online network parameters. and target network parameters .

[0021] S22: Obtain the environmental state described in S1 from the environment. Input the state into the network, and the network outputs the action. ; Regarding actions Perform a security check; if the check passes, execute the action within the environment. Conversely, if the action is not performed, a penalty is imposed; then the state information for the next moment is obtained. Calculate the reward value of the action. and termination mark .

[0022] S23: Use the series of data obtained in the previous step as a set of experiences. The data is stored in the experience replay pool D. During each subsequent training round, a batch of data of size D will be randomly drawn from this buffer. We use a small portion of the data for training.

[0023] S24: Parameter Update: Select actions using an online network: (11) In the formula, Actions selected for online networks For parameters of the online network, Candidate actions refer to various possible actions in the action space that are waiting to be evaluated by the network. This is the action value function.

[0024] Calculate the target Q value: (12) In the formula, For the target Q value, It is the end-of-round marker. Discount factor, These are the parameters of the target network.

[0025] Calculate the prediction Q-value and loss: (13) In the formula, For batch size, This is the loss value. To predict the Q-value, this is an online network (with parameters...). In a given state and actions The output is an estimated value of the expected return.

[0026] Online network parameter updates: (14) In the formula, For learning rate, For the Q function of network parameters The gradient.

[0027] Target network parameter update: (15) In the formula, These are the current parameters of the target network. This is the soft update coefficient.

[0028] S25: Repeat S22 to S24 until training is complete and an intelligent agent capable of online applications is obtained.

[0029] Furthermore, S3 specifically includes the following steps: S31: Environmental perception and structured feature extraction, constructing network state input. : The system utilizes onboard sensors or V2X communication to acquire raw environmental information around the vehicle and performs structured processing to construct an environmental state vector composed of multi-dimensional features. These multi-dimensional features include: motion state features, vehicle position features, lane macro-traffic situation features, and lane change urgency features.

[0030] Motion state characteristics : Fill dimension is The motion state matrix, This represents the maximum number of observable vehicles within the perception range, including the vehicle itself. For each vehicle's feature dimension, the first row of the matrix is ​​predefined as the vehicle's own features, and subsequent rows are filled in ascending order of distance from surrounding vehicles to the vehicle. For each vehicle, its feature vector is filled in as follows: The feature vector sequentially represents: the target vehicle's presence status, its lateral and longitudinal relative position to the vehicle, its lateral and longitudinal speeds, and its current lane number. When the number of vehicles within the perception range is insufficient... When vehicles are involved, the remaining dimensions of the matrix are padded with 0.

[0031] Vehicle location features : Extract the relative position coordinates of each vehicle from the above motion state matrix. Composition dimension is The matrix. If the number of effective targets is insufficient. For each vehicle, zero-padding is also performed to maintain data dimension consistency.

[0032] Macroscopic traffic situation characteristics of lanes : Fill dimension is The macroscopic traffic situation feature matrix of the lanes is used to extract feature vectors for the left, middle, and right lanes respectively. , respectively, are lane presence sign, average lane speed, and traffic wave speed.

[0033] Using the vehicle's position as the origin of the coordinate system, obtain the distance to the front of each lane. Vehicle information within the range is used to calculate the average speed of each lane. Simultaneously, two lengths are defined. Featured region: Upstream vehicle adjacent region: Interval range is Downstream area ahead: The interval range is Information on vehicles in the two areas mentioned above is extracted for each lane, and the average vehicle speed in the upstream area of ​​each lane is calculated. and average headway and the average vehicle speed in the downstream area and average headway The traffic wave propagation speed is calculated according to formula (2). If a lane does not exist, The value is 0, corresponding to , All values ​​are filled with 0. The obtained wave velocity values ​​are used to determine and quantify the evolution trend of traffic congestion. A negative value indicates that the traffic wave is propagating upstream, meaning that congestion is spreading. A positive value indicates that the traffic wave is spreading downstream, meaning that the congested area is dissipating and traffic conditions are gradually recovering.

[0034] Lane change urgency characteristics : This feature consists of two-dimensional components. The system is designed to characterize the spatial relationship between a vehicle and a lane-closing point. It acquires the vehicle's current position in real time, calculates its physical distance to the lane-closing point ahead, and maps this distance to a normalized value. Depending on whether the vehicle has entered a pre-defined penalty detention zone, if the vehicle has entered the zone, then... The value is 1 if it is not 0 otherwise.

[0035] S32: Motion Value Prediction The heterogeneous environmental observation status in S31 above The pre-trained online network is input, and the network processes information of different dimensions through four parallel branches to extract features. These four sets of features are then concatenated into a high-dimensional state vector. The network then performs a final linear transformation to map this vector to the action value space, completing the mapping from the "state space" to the "action value space." This vector has the same dimension as the action space, and its components represent the Q-value estimate of the agent performing the corresponding action in the current state.

[0036] S33: Greedy Action Selection: The discrete action with the highest predicted Q value is directly selected and sent as the decision instruction to the verification module.

[0037] (17) In the formula, Actions selected for online networks For parameters of the online network, Candidate actions refer to various possible actions in the action space that are waiting to be evaluated by the network. This is the action value function.

[0038] Furthermore, S4 specifically includes the following steps: S41 Action Verification: Receive actions output from the intelligent agent network and perform boundary violation judgment and security verification; Boundary crossing determination: Check whether the action exceeds the physical feasible boundary of the road; if the vehicle receives a left lane change instruction while in the leftmost lane, the action is immediately intercepted and the vehicle returns to the preset safe baseline strategy - that is, to maintain the current lane. Safety verification: If the action meets the boundary constraints, further safety checks are performed, including two indicators: safe distance and time to collision (TTC). The action is considered to have passed the verification only if both TTC and distance to the vehicle are greater than the corresponding safety threshold. Minimum safe distance threshold from the vehicle in front : (18) (19) (20) In the formula, , These are the speeds of this vehicle and the vehicle in front, respectively. This is the vehicle's reaction time. This is the maximum deceleration of the vehicle. This is a stationary safety distance; Minimum safe distance threshold from the vehicle behind : (twenty one) (twenty two) (twenty three) In the formula, , These are the speeds of the vehicle and the vehicle behind it, respectively. This is the vehicle's reaction time. This is the maximum deceleration of the following vehicle. This is a stationary safety distance; TTC calculation: (twenty four) (25) In the formula , These are the positional differences with the vehicle in front and the vehicle behind, respectively. When performing TTC safety checks, the calculation premise is that the speed of the vehicle behind is strictly greater than that of the vehicle in front; otherwise, it is assumed that a rear-end collision will not occur and the TTC check is skipped. S42 Action Interception and Strategy Rollback: When a pre-selected action goes out of bounds or fails the safety check, the system will intercept the issuance of the dangerous instruction and revert to the lane keeping baseline strategy. If the out-of-bounds check and safety check pass, the original action instruction will be executed normally.

[0039] The beneficial effects of this invention are as follows: (1) Significantly enhances the vehicle's ability to perceive traffic conditions and make forward-looking decisions. The traffic wave speed of the LWR macro traffic flow model is introduced as a guiding signal to construct a macro-micro fusion state input. In addition to acquiring the micro state such as vehicle position and speed, the traffic wave speed of each lane is also acquired in real time, realizing the dynamic quantitative representation of the congestion shock wave spread trend in the bottleneck area, and significantly improving the situational awareness and forward-looking decision-making in complex traffic environments.

[0040] (2) It improves the overall traffic efficiency and single-vehicle decision-making performance of bottleneck road sections with lane reduction, reduces the spread of congestion caused by blindly crowding vehicles and late-stage forced lane changes, guides vehicles to distribute evenly in advance, maximizes the utilization rate of road network time and space resources, smooths macro traffic flow, alleviates the backward transmission of congestion waves, and achieves coordinated optimization of individual vehicles and global traffic flow.

[0041] (3) It enhances the convergence speed and decision safety of deep reinforcement learning models. By transforming traffic wave features into reward signals with clear physical meaning, it provides neural networks with more stable and directional gradient update information, reducing the exploration and trial-and-error costs of the model in complex dynamic traffic environments. At the same time, a safety redundancy module is set in the execution layer to impose safety constraints on the actions output by reinforcement learning. While maintaining decision flexibility, it ensures that vehicle behavior conforms to physical laws, thereby improving the rationality of vehicle decision-making and operational stability in complex traffic environments.

[0042] Other advantages, objectives, and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination, or may be learned from practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description

[0043] To make the objectives, technical solutions, and advantages of the present invention clearer, the preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, wherein: Figure 1 This is a flowchart illustrating the specific implementation of the method of the present invention; Figure 2 This is the reward curve of the DDQN algorithm in this invention example; Figure 3 This is a schematic diagram of a lane reduction scenario; Figure 4This is a curve showing the change in average vehicle speed with the penetration rate of intelligent connected vehicles in an example of the present invention. Figure 5 This is a curve showing the variation of the average maximum queue length of vehicles with the penetration rate of intelligent connected vehicles in an example of the present invention. Figure 6 This is a curve showing the change in average vehicle delay time as a function of the penetration rate of intelligent connected vehicles in an example of the present invention. Detailed Implementation

[0044] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0045] The accompanying drawings are for illustrative purposes only and are schematic diagrams, not actual pictures. They should not be construed as limiting the invention. To better illustrate the embodiments of the invention, some parts in the drawings may be omitted, enlarged, or reduced, and do not represent the actual product dimensions. It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.

[0046] In the accompanying drawings of the embodiments of the present invention, the same or similar reference numerals correspond to the same or similar components. In the description of the present invention, it should be understood that if terms such as "upper," "lower," "left," "right," "front," and "rear" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, they are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, the terms used to describe positional relationships in the drawings are only for illustrative purposes and should not be construed as limiting the present invention. For those skilled in the art, the specific meaning of the above terms can be understood according to the specific circumstances.

[0047] Please see Figures 1-6 This invention proposes a lane-changing decision-making method for vehicles in lane-reduction sections based on deep reinforcement learning, aiming to improve the traffic efficiency and lane-changing decision-making quality of intelligent connected vehicles in lane-reduction areas and alleviate road congestion. The method includes the following steps: S1: Introducing the LWR traffic flow model, a state space integrating macroscopic traffic flow status and microscopic vehicle information is designed, and an action space satisfying safety constraints and driving feasibility is constructed. A composite reward function that balances traffic efficiency and driving safety is proposed. Furthermore, a decision network is designed by combining graph convolutional networks and the DDQN algorithm to construct a vehicle lane-changing decision model for lane reduction regions. S2: Build a lane reduction traffic simulation scenario, train the vehicle lane-changing decision model until the vehicle lane-changing decision model converges, save the model weights, and obtain the trained vehicle lane-changing decision model. S3: Deploy the trained model into the control unit, acquire environmental information and construct a state representation that adapts to the model input, then use the pre-trained DDQN model to perform online state evaluation and output actions to achieve real-time autonomous decision-making for the vehicle. S4: Introducing a strategy rollback mechanism, the system first checks the actions output by DDQN for exceeding limits. If the check passes, it then performs dual verification using TTC and safe following distance. If the action is compliant, it is executed; if any step fails, the system will intercept the action and automatically trigger a safety rollback, downgrading to the lane keeping baseline strategy.

[0048] Furthermore, S1 specifically includes the following steps: S11: State space establishment: Status characteristics include: the micro-motion status of surrounding vehicles, vehicle location information, macro-traffic situation of the lane, and the urgency of lane changing.

[0049] Characteristics of motion state: vehicle motion status Modeled as a The matrix. Where, Setting it to 10 indicates the maximum number of observable vehicles (including the driver's own vehicle) within the perception range. For a single vehicle's feature dimension, the single vehicle feature consists of the following 6 state variables: (1) These six features represent the presence of surrounding vehicles as observed by the vehicle, its lateral and longitudinal positions relative to the vehicle, its absolute lateral and longitudinal speeds, and the lane number it is currently in. Vehicle positional relationship characteristics: Vehicle relative position characteristics Represented as a The matrix, where each row corresponds to the relative position of an observed vehicle. , Setting it to 10 indicates the maximum number of vehicles that can be observed within the perception range.

[0050] Macroscopic traffic situation characteristics of lanes: Lane macro situation characteristics Represented as a This is a matrix where the rows correspond to the left lane, the current lane, and the right lane where the vehicle is currently located, respectively. The three columns correspond to the feature set of a single lane. , respectively represent: Physical connectivity : Indicates whether the lane exists (when the vehicle is in the leftmost or rightmost lane, the corresponding left or right lane does not exist). Fill with 1 to indicate existence, fill with 0 to indicate non-existence; average lane speed This indicates the average speed of vehicles traveling on the road section ahead of this lane. Traffic wave propagation speed This represents the propagation speed of the traffic wave in that lane. Using the vehicle's position as the origin, two characteristic regions of 100 meters each are defined: the upstream adjacent region (range [-50, 50], i.e., the range 50 meters in front of and behind the vehicle); and the downstream region (range [150, 250], i.e., the range 150 meters to 250 meters in front of the vehicle). According to the LWR model, the formula for calculating the traffic wave speed can be expressed as: (2) In the formula, , These represent the average vehicle speed and average headway in the upstream region, respectively. , These represent the average vehicle speed and average headway in the downstream area, respectively.

[0051] Lane change urgency characteristics: Lane change urgency characteristics It consists of the following two parts: Remaining distance This indicates the distance from the vehicle's current position to the lane reduction point, used to quantify the urgency of the vehicle approaching the bottleneck area; area marker. : Boolean state variable used to indicate whether the vehicle has entered the detention penalty area, which is the area on the lane reduction lane that extends 200 m upstream from the lane reduction point. This variable is used to distinguish the current state of the area where the vehicle is located.

[0052] S12: Action space establishment: The lane-changing decision of a vehicle is defined as a strategy selection based on a discrete set of actions. This set of actions includes three possible actions: changing lanes to the adjacent left lane, maintaining the current lane, and changing lanes to the adjacent right lane (the action set is...). ).

[0053] S13: Reward Function The reward function adopts a multi-dimensional weighted summation form, including basic speed reward, basic lane reward, collision penalty, etc., to quantify the merits of lane-changing decision actions.

[0054] Base speed bonus: (3) In the formula, For the speed of this vehicle, This is the maximum speed limit.

[0055] Basic lane bonus: (4) In the formula, This indicates the lane number of the vehicle's current lane. The outermost lane (i.e., the lane to be reduced) is defined as number 0, and the numbers increase sequentially inwards from the lane. This indicates the lane number of the innermost lane.

[0056] Delay Penalty: A penalty will be imposed on vehicles that remain in the delay penalty area (near the lane reduction point and have not yet left the lane) and are subject to the following penalties: (5) In the formula, Indicates the vehicle's current location. Indicates the starting position of the detention penalty zone. This indicates the end point of the penalty zone (i.e., the location where the lanes narrow).

[0057] Collision Penalty: To ensure driving safety, when a vehicle collides with surrounding vehicles, a corresponding negative penalty value is assigned: (6) Action Penalty: When an agent performs an invalid action, a corresponding penalty value is applied. Invalid actions include two situations: first, the action will cause the vehicle to cross the road boundary; second, the action does not meet preset safety constraints. The corresponding penalty terms are defined as follows: (7) Traffic flow guidance rewards: (8) In the formula, , , These represent the traffic wave velocities of the vehicle's left lane, current lane, and right lane, respectively. If there are no adjacent lanes (e.g., the vehicle is in the leftmost or rightmost lane), the corresponding wave velocity is 0.

[0058] Lane change speed benefits: (9) In the formula, , These represent the speeds of the vehicle in front in the new lane after the lane change and the speeds of the vehicle in front in the original lane before the lane change, respectively.

[0059] The total reward is expressed as follows: (10) In the formula, The weight parameters are adjusted according to the importance of each reward and penalty item, thereby positively guiding the agent to make lane-changing decisions.

[0060] S14: Network Design To fully extract traffic environment information, a multi-branch feature fusion network is designed. By encoding and fusing different types of state information, the Q-value prediction of discrete actions is achieved. The network divides the input into four independent branches: a graph convolutional network (GCN) is used to extract the positional interaction features between vehicles in the local space, while multiple independent multilayer perceptrons (MLPs) are used to extract vehicle motion state, lane macro situation, and lane change urgency features, respectively. The extracted features from the four branches are concatenated into a high-dimensional joint feature vector, which is then linearly transformed into an output vector with the same dimension as the action space. Each element of the vector corresponds to the Q-value estimate of each discrete action in the current state.

[0061] Furthermore, S2 specifically includes the following steps: S21: Set up the lane reduction simulation environment and initialize the online network parameters. and target network parameters .

[0062] S22: Obtain the environmental state described in S1 from the environment. Input the state into the network, and the network outputs the action. ; Regarding actions Perform a security check; if the check passes, execute the action within the environment. Conversely, if the action is not performed, a penalty is imposed; then the state information for the next moment is obtained. Calculate the reward value of the action. and termination mark .

[0063] S23: Use the series of data obtained in the previous step as a set of experiences. The data is stored in the experience replay pool D. During each subsequent training round, a batch of data of size D will be randomly drawn from this buffer. We use a small portion of the data for training.

[0064] S24: Parameter Update: Select actions using an online network: (11) In the formula, Actions selected for online networks For parameters of the online network, Candidate actions refer to various possible actions in the action space that are waiting to be evaluated by the network. This is the action value function.

[0065] Calculate the target Q value: (12) In the formula, For the target Q value, It is the end-of-round marker. Discount factor, These are the parameters of the target network.

[0066] Calculate the prediction Q-value and loss: (13) In the formula, For batch size, This is the loss value. To predict the Q-value, this is an online network (with parameters...). In a given state and actions The output is an estimated value of the expected return.

[0067] Online network parameter updates: (14) In the formula, For learning rate, For the Q function of network parameters The gradient.

[0068] Target network parameter update: (15) In the formula, These are the current parameters of the target network. This is the soft update coefficient.

[0069] S25: Repeat S22 to S24 until training is complete and an intelligent agent capable of online applications is obtained.

[0070] Furthermore, S3 specifically includes the following steps: S31: Environmental perception and structured feature extraction, constructing network state input. : The system utilizes onboard sensors or V2X communication to acquire raw environmental information around the vehicle and performs structured processing to construct an environmental state vector composed of multi-dimensional features. These multi-dimensional features include: motion state features, vehicle position features, lane macro-traffic situation features, and lane change urgency features.

[0071] Motion state characteristics : The motion state matrix is ​​filled with dimensions 10×6, where 10 represents the maximum number of observable vehicles within the perception range, including the vehicle itself, and 6 represents the feature dimension of a single vehicle. The first row of the matrix is ​​predefined as the vehicle's features, and subsequent rows are filled in ascending order of distance between the vehicle and surrounding vehicles. For each vehicle, its feature vector is filled: The feature vector sequentially represents: the presence status of the target vehicle, its lateral and longitudinal relative position to the vehicle, its lateral and longitudinal speeds, and its current lane number. When the number of vehicles within the perception range is less than 10, the remaining dimensions of the matrix are padded with 0.

[0072] Vehicle location features : Extract the relative position coordinates of each vehicle from the above motion state matrix. This forms a 10×2 matrix. If there are fewer than 10 surrounding vehicles, zero-padding is performed to maintain data dimensionality consistency.

[0073] Macroscopic traffic situation characteristics of lanes : Fill dimension is The macroscopic traffic situation feature matrix of the lanes is used to extract feature vectors for the left, middle, and right lanes respectively. , respectively, are lane presence sign, average lane speed, and traffic wave speed.

[0074] Using the vehicle's position as the origin, vehicle information within a 250m radius ahead of each lane is acquired to calculate the average speed for each lane. Simultaneously, two 100m-long feature regions are defined: the upstream adjacent region (50m in front of and behind the vehicle, totaling 100m) and the downstream forward region (150m to 250m ahead of the vehicle). Vehicle information within these two regions for each lane is extracted, and the average speed in the upstream region of each lane is calculated. and average headway and the average vehicle speed in the downstream area and average headway The traffic wave propagation speed is calculated according to formula (2). If a lane does not exist, The value is 0, corresponding to , All values ​​are filled with 0. The obtained wave velocity values ​​are used to determine and quantify the evolution trend of traffic congestion. A negative value indicates that the traffic wave is propagating upstream, meaning that congestion is spreading. A positive value indicates that the traffic wave is spreading downstream, meaning that the congested area is dissipating and traffic conditions are gradually recovering.

[0075] Lane change urgency characteristics : This feature consists of two-dimensional components. The system is designed to characterize the spatial relationship between a vehicle and a lane-closing point. It acquires the vehicle's current position in real time, calculates its physical distance to the lane-closing point ahead, and maps this distance to a normalized value. Depending on whether the vehicle has entered a pre-defined penalty detention zone, if the vehicle has entered the zone, then... The value is 1 if it is not 0 otherwise.

[0076] S32: Motion Value Prediction The heterogeneous environmental observation status in S31 above The pre-trained online network is input, and the network processes information of different dimensions through four parallel branches to extract features. These four sets of features are then concatenated into a high-dimensional state vector. The network then performs a final linear transformation to map this vector to the action value space, completing the mapping from the "state space" to the "action value space." This vector has the same dimension as the action space, and its components represent the Q-value estimate of the agent performing the corresponding action in the current state.

[0077] S33: Greedy Action Selection: The discrete action with the highest predicted Q value is directly selected and sent as the decision instruction to the verification module.

[0078] (17) In the formula, Actions selected for online networks For parameters of the online network, Candidate actions refer to various possible actions in the action space that are waiting to be evaluated by the network. This is the action value function.

[0079] Furthermore, S4 specifically includes the following steps: S41 Action Verification: Receive actions output from the intelligent agent network and perform boundary violation judgment and security verification; Boundary crossing determination: Check whether the action exceeds the physical feasible boundary of the road; if the vehicle receives a left lane change instruction while in the leftmost lane, the action is immediately intercepted and the vehicle returns to the preset safe baseline strategy - that is, to maintain the current lane. Safety Verification: If the action meets the boundary constraints, further safety checks are performed, including two indicators: safe distance and time to collision (TTC). The action is considered to have passed the verification only when both TTC and distance to the vehicle are greater than the corresponding safety thresholds. The TTC threshold is set at 3 seconds, and the distance to the vehicle threshold is calculated based on the vehicle speed. Minimum safe distance threshold from the vehicle in front : (18) (19) (20) In the formula, , These are the speeds of this vehicle and the vehicle in front, respectively. This is the vehicle's reaction time. This is the maximum deceleration of the vehicle. This is a stationary safety distance; Minimum safe distance threshold from the vehicle behind : (twenty one) (twenty two) (twenty three) In the formula, , These are the speeds of the vehicle and the vehicle behind it, respectively. This is the vehicle's reaction time. This is the maximum deceleration of the following vehicle. This is a stationary safety distance; TTC calculation: (twenty four) (25) In the formula , These are the positional differences with the vehicle in front and the vehicle behind, respectively. When performing TTC safety checks, the calculation premise is that the speed of the vehicle behind is strictly greater than that of the vehicle in front; otherwise, it is assumed that a rear-end collision will not occur and the TTC check is skipped. S42 Action Interception and Strategy Rollback: When a pre-selected action goes out of bounds or fails the safety check, the system will intercept the issuance of the dangerous instruction and revert to the lane keeping baseline strategy. If the out-of-bounds check and safety check pass, the original action instruction will be executed normally.

[0080] To verify the effectiveness of the proposed method, this embodiment first establishes a simulation environment for a lane reduction road segment in SUMO. The simulation segment is 5 kilometers long, with the lanes changing from three lanes to two lanes at kilometer 2.5. The speed limit for each lane is 20 m / s. Next, background traffic flow is set to simulate a real traffic environment. The background vehicle car-following model is SUMO's built-in IDM model, and the lane-changing model is SUMO's built-in LC2013 model. Then, a certain number of target vehicles are placed into the environment, and the online network parameters are initialized. and target network parameters The hyperparameters of DDQN during training are shown in Table 1.

[0081] Table 1

[0082] During training, the sampling interval was set to 0.5 seconds, each round lasted 500 seconds, and the entire training consisted of 1000 rounds.

[0083] To verify the effectiveness of the intelligent agent, this embodiment conducts simulation tests on a deep reinforcement learning-based vehicle lane-changing decision-making method for lane reduction scenarios. The test settings are as follows: Traffic flow conditions: Total vehicle volume is 3,000 vehicles / hour and 4,000 vehicles / hour respectively; Penetration rate settings: DDQN controls the penetration rates of vehicles at 0%, 20%, 40%, 60%, and 80% respectively; Test duration: Each simulation lasts 30 minutes, and the average value is taken from multiple tests; Evaluation metrics: average delay time of all vehicles within 2.5 kilometers before lane reduction, average maximum queue length of all vehicles within 2.5 kilometers before lane reduction, and average speed of all vehicles within 500 meters before lane reduction.

[0084] Figure 3 This is a schematic diagram of a lane reduction scenario. Indicates each vehicle in Actions at a given moment; Figure 4 The figure shows the curve of the average vehicle speed as a function of the vehicle penetration rate controlled by DDQN when the DDQN algorithm is used for simulation test in the example of this invention. Under the conditions of a total traffic flow of 3,000 vehicles per hour and 4,000 vehicles per hour, the average vehicle speed increases significantly with the increase of penetration rate, which verifies the effect of DDQN algorithm on improving traffic efficiency. Figure 5 The curves showing the variation of the average maximum queue length of vehicles with the penetration rate controlled by DDQN in simulation tests demonstrate that, under conditions of a total traffic flow of 3000 vehicles per hour and 4000 vehicles per hour, the maximum queue length exhibits a decreasing trend with increasing penetration rate. Figure 6 The curve shows the change in average vehicle delay time as a function of the penetration rate of DDQN-controlled vehicles. Under the conditions of a total traffic flow of 3,000 vehicles per hour and 4,000 vehicles per hour, the delay time also decreases as the penetration rate increases.

[0085] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A deep reinforcement learning decision-making method for vehicle lane changing in lane-shortening regions, characterized in that: Includes the following steps: S1: Introduce the LWR traffic flow model, design a state space that integrates macroscopic traffic flow status and microscopic vehicle information, construct an action space that satisfies safety constraints and driving feasibility, and propose a composite reward function that takes into account both traffic efficiency and driving safety; further, combine graph convolutional networks and DDQN algorithm to design a decision network and construct a vehicle lane-changing decision model in lane reduction areas. S2: Build a lane reduction traffic simulation scenario, train the vehicle lane-changing decision model until the vehicle lane-changing decision model converges, save the model weights, and obtain the trained vehicle lane-changing decision model. S3: Deploy the trained model into the control unit, acquire environmental information and construct a state representation that adapts to the model input, then use the pre-trained DDQN model to perform online state evaluation and output actions to achieve real-time autonomous decision-making for the vehicle. S4: Introducing a strategy rollback mechanism, firstly, the actions output by DDQN are judged to be out of bounds, and then TTC and safe distance are dually verified after passing the judgment; if the action is compliant, it is issued for execution; if any step fails, the system will intercept the action and automatically trigger a safety rollback, downgrading to the execution of the lane keeping baseline strategy.

2. The vehicle lane-changing deep reinforcement learning decision-making method for lane-shortening regions according to claim 1, characterized in that: S1 specifically includes the following steps: S11: State space establishment: Status characteristics include: the micro-motion status of surrounding vehicles, vehicle location information, macro-traffic situation of the lane, and the urgency of lane changing; Characteristics of motion state: The movement status of surrounding vehicles Modeled as a , where, The maximum number of observable vehicles within the vehicle's perception range, including the vehicle itself; For a single vehicle's feature dimension, the single vehicle feature consists of the following 6 state variables: （1） These six features represent the presence of surrounding vehicles as observed by the vehicle, its lateral and longitudinal positions relative to the vehicle, its absolute lateral and longitudinal speeds, and the lane number it is currently in. Vehicle positional relationship characteristics: Vehicle relative position characteristics Represented as a The matrix, where each row corresponds to the relative position of an observed vehicle. , The maximum number of vehicles observable within the sensing range; Macroscopic traffic situation characteristics of lanes: Lane macro situation characteristics Represented as a A matrix, whose rows correspond to the left lane, the current lane, and the right lane where the vehicle is currently located, respectively; the three columns correspond to the feature set of a single lane. , respectively represent: Physical connectivity : Indicates whether the lane exists. When a vehicle is in the leftmost or rightmost lane, the corresponding left or right lane does not exist. average lane speed This indicates the average speed of vehicles traveling on the road section ahead of this lane. Traffic wave propagation speed : Represents the propagation speed of traffic waves in that lane, with the vehicle's position as the origin of the coordinate system, and two lengths defined as . Featured region: Upstream vehicle adjacent region: Interval range is Downstream area ahead: The interval range is , Taking a positive value, according to the LWR model, the formula for calculating the traffic wave velocity is as follows: （2） In the formula, , These represent the average vehicle speed and average headway in the upstream region, respectively. , These represent the average vehicle speed and average headway in the downstream area, respectively. Lane change urgency characteristics: Lane change urgency characteristics It consists of the following two parts: Remaining distance This indicates the distance from the vehicle's current position to the lane reduction point, used to quantify the urgency of the vehicle approaching the bottleneck area; area marker. : Boolean state variable used to indicate whether the vehicle has entered the detention penalty zone; this zone is set on the outermost lane, i.e. the lane reduction lane, with the lane reduction point as the endpoint, extending upstream for a certain length to clarify the current location of the vehicle in the zone. S12: Action space establishment: The lane-changing decision of a vehicle is defined as a strategy selection based on a discrete set of actions; the set of actions includes three optional actions: changing lanes to the adjacent left lane, maintaining the current lane, and changing lanes to the adjacent right lane. ; S13: Reward Function The reward function adopts a multi-dimensional weighted summation form, including a base speed reward, a base lane reward, and a collision penalty, which is used to quantify the merits of lane-changing decisions. Base speed bonus: （3） In the formula, For the speed of this vehicle, Maximum speed limit; Basic lane bonus: （4） In the formula, This indicates the lane number of the vehicle's current lane. The outermost lane (i.e., the lane to be reduced) is defined as number 0, and the numbers increase sequentially inwards from the lane. This indicates the lane number of the innermost lane; Delay Penalty: A penalty will be imposed on vehicles that remain in the delay penalty area (near the lane reduction point and have not yet left the lane) and are subject to the following penalties: （5） In the formula, Indicates the vehicle's current location. Indicates the starting position of the detention penalty zone. This indicates the end point of the penalty zone for being detained, i.e., the location where the lanes narrow. Collision Penalty: To ensure driving safety, when a vehicle collides with surrounding vehicles, a corresponding negative penalty value is assigned: （6） Action Penalty: When an agent performs an invalid action, a corresponding penalty value is applied. Invalid actions include two situations: first, the action will cause the vehicle to exceed the road boundary; second, the action does not meet preset safety constraints. The corresponding penalty terms are defined as follows: （7） Traffic flow guidance rewards: （8） In the formula, , , These represent the traffic wave velocities of the vehicle's left lane, current lane, and right lane, respectively; if no adjacent lane exists, the corresponding wave velocity is 0. Lane change speed benefits: （9） In the formula, , These represent the speeds of the vehicle in the new lane after the lane change and the speeds of the vehicle in the original lane before the lane change, respectively. The total reward is expressed as follows: （10） In the formula, The weight parameters are adjusted according to the importance of each reward and penalty item, thereby positively guiding the agent to make lane-changing decisions. S14: Network Design To fully extract traffic environment information, a multi-branch feature fusion network is designed. By encoding and fusing different types of state information, the Q-value prediction of discrete actions is achieved. The network divides the input into four independent branches: a graph convolutional network (GCN) is used to extract the positional interaction features between vehicles in the local space, while multiple independent multilayer perceptrons (MLPs) are used to extract vehicle motion state, lane macro situation, and lane change urgency features, respectively. The extracted features from the four branches are concatenated into a high-dimensional joint feature vector, which is then linearly transformed into an output vector with the same dimension as the action space. Each element of the vector corresponds to the Q-value estimate of each discrete action in the current state.

3. The vehicle lane-changing deep reinforcement learning decision-making method for lane-shortening regions according to claim 1, characterized in that: S2 specifically includes the following steps: S21: Set up the lane reduction simulation environment and initialize the online network parameters. and target network parameters ; S22: Obtain the environmental state described in S1 from the environment. Input the state into the network, and the network outputs the action. ; Regarding actions Perform a security check; if the check passes, execute the action within the environment. Conversely, if the action is not performed, a penalty is imposed; then the state information for the next moment is obtained. Calculate the reward value of the action. and termination mark ; S23: Use the series of data obtained in the previous step as a set of experiences. The data is stored in the experience replay pool D. During each subsequent training round, a batch of data of size D will be randomly drawn from this buffer. Training is performed using a small portion of the data; S24: Parameter Update: Select actions using an online network: （11） In the formula, Actions selected for online networks For parameters of the online network, Candidate actions refer to various possible actions in the action space that are waiting to be evaluated by the network. The action value function; Calculate the target Q value: （12） In the formula, For the target Q value, It is the end-of-round marker. Discount factor, These are the parameters of the target network; Calculate the prediction Q-value and loss: （13） In the formula, For batch size, This is the loss value. To predict the Q-value, this is an online network (with parameters...). In a given state and actions The estimated expected return for the output; Online network parameter updates: （14） In the formula, For learning rate, For the Q function of network parameters The gradient; Target network parameter update: （15） In the formula, These are the current parameters of the target network. This is a soft update coefficient; S25: Repeat S22 to S24 until training is complete and an intelligent agent capable of online applications is obtained.

4. The vehicle lane-changing deep reinforcement learning decision-making method for lane-shortening regions according to claim 1, characterized in that: S3 specifically includes the following steps: S31: Environmental perception and structured feature extraction, constructing network state input. : The system utilizes onboard sensors or V2X communication to acquire raw environmental information around the vehicle and performs structured processing to construct an environmental state vector composed of multi-dimensional features. These multi-dimensional features include: motion state features, vehicle position features, lane macro-traffic situation features, and lane change urgency features. Motion state characteristics : Fill dimension is The motion state matrix, This represents the maximum number of observable vehicles within the perception range, including the vehicle itself. For each vehicle's feature dimension, the first row of the matrix is ​​predefined as the vehicle's own features, and subsequent rows are filled in ascending order of distance from surrounding vehicles to the vehicle; for each vehicle, its feature vector is filled in: The feature vector sequentially represents: the presence status of the target vehicle, its lateral and longitudinal relative position to the vehicle, its lateral and longitudinal speeds, and its current lane number; when the number of vehicles within the perception range is insufficient... When vehicles are involved, the remaining dimensions of the matrix are padded with 0; Vehicle location features : Extract the relative position coordinates of each vehicle from the above motion state matrix. Composition dimension is The matrix; if the number of effective targets is insufficient For each vehicle, zero-padding is also performed to maintain data dimension consistency; Macroscopic traffic situation characteristics of lanes : Fill dimension is The macroscopic traffic situation feature matrix of the lanes is used to extract feature vectors for the left, middle, and right lanes respectively. These are lane presence signs, average lane speed, and traffic wave speed, respectively. Using the vehicle's position as the origin of the coordinate system, obtain the distance to the front of each lane. Vehicle information within the range is used to calculate the average speed of each lane; simultaneously, two lengths are defined. Featured region: Upstream vehicle adjacent region: Interval range is Downstream area ahead: The interval range is Extract vehicle information from the two areas mentioned above for each lane, and calculate the average vehicle speed in the upstream area of ​​each lane. and average headway and the average vehicle speed in the downstream area and average headway The traffic wave propagation speed is calculated according to formula (2). If a lane does not exist, The value is 0, corresponding to , All values ​​are filled with 0; the obtained wave velocity values ​​are used to determine and quantify the evolution trend of traffic congestion. A negative value indicates that the traffic wave is propagating upstream, meaning that congestion is spreading. A positive value indicates that the traffic wave is propagating downstream, meaning that the congested area is dissipating and traffic conditions are gradually recovering. Lane change urgency characteristics : This feature consists of two-dimensional components. The system is designed to characterize the spatial relationship between a vehicle and a lane-closing point. It acquires the vehicle's current position in real time, calculates its physical distance to the lane-closing point ahead, and maps this distance to a normalized value. Depending on whether the vehicle has entered a pre-defined penalty detention zone, if the vehicle has entered the zone, then... The value is 1 if it is set to 1, and 0 otherwise. S32: Motion Value Prediction The heterogeneous environmental observation status in S31 above The pre-trained online network is input, and the network processes information of different dimensions through four parallel branches to extract features. These four sets of features are finally concatenated into a high-dimensional state vector. The network then maps this vector to the action value space through a final linear transformation, completing the mapping from the "state space" to the "action value space". The dimension of this vector is consistent with the size of the action space, and each component represents the Q-value estimate of the agent performing the corresponding action in the current state. S33: Greedy Action Selection: The discrete action with the highest predicted Q value is directly selected and sent as the decision instruction to the verification module; （17） In the formula, Actions selected for online networks For parameters of the online network, Candidate actions refer to various possible actions in the action space that are waiting to be evaluated by the network. This is the action value function.

5. The vehicle lane-changing deep reinforcement learning decision-making method for lane-shortening regions according to claim 1, characterized in that: S4 specifically includes the following steps: S41: Action Verification: Receive actions output from the intelligent agent network and perform boundary violation judgment and security verification; Boundary crossing determination: Check whether the action exceeds the physical feasible boundary of the road; if the vehicle receives a left lane change instruction while in the leftmost lane, the action is immediately intercepted and the vehicle returns to the preset safe baseline strategy - that is, to maintain the current lane. Safety verification: If the action meets the boundary constraints, further safety checks are performed, including two indicators: safe distance and time to collision (TTC). The action is considered to have passed the verification only if both TTC and distance to the vehicle are greater than the corresponding safety threshold. Minimum safe distance threshold from the vehicle in front : （18） （19） （20） In the formula, , These are the speeds of this vehicle and the vehicle in front, respectively. This is the vehicle's reaction time. This is the maximum deceleration of the vehicle. This is a stationary safety distance; Minimum safe distance threshold from the vehicle behind : （21） （22） （23） In the formula, , These are the speeds of the vehicle and the vehicle behind it, respectively. This is the vehicle's reaction time. This is the maximum deceleration of the following vehicle. This is a stationary safety distance; TTC calculation: （24） （25） In the formula , These are the positional differences with the vehicle in front and the vehicle behind, respectively. When performing TTC safety checks, the calculation premise is that the speed of the vehicle behind is strictly greater than that of the vehicle in front; otherwise, it is assumed that a rear-end collision will not occur and the TTC check is skipped. S42: Action Interception and Strategy Rollback: When a pre-selected action goes out of bounds or fails the safety check, the system will intercept the issuance of the dangerous instruction and revert to the lane keeping baseline strategy. If the out-of-bounds check and safety check pass, the original action instruction will be executed normally.

Images