An unmanned vehicle left-turn passing decision method and system in a perception occlusion scenario

Abstract

The application discloses a kind of perception under the scene of occlusion unmanned vehicle left turn traffic decision method and system, belong to unmanned vehicle decision control technical field.The application is by obtaining the observation state containing perception blind area information, input actor network output preliminary acceleration instruction as to-be-executed action;The return distribution of state-action pair is modeled using value distribution critic network based on completely parameterized quantile function, to explicitly depict the return uncertainty caused by perception occlusion;Based on the return distribution, the conditional risk value of variable risk level is calculated, and compared with the safety threshold, the to-be-executed action is replaced by safety brake action when the risk is too high, so as to safely bottom out the decision output;Finally, the acceleration instruction is sent to the lower controller for execution and generates steering wheel corner, completes left turn behavior.Based on the method, the corresponding system is also proposed.The application realizes the dynamic balance of left turn traffic safety and traffic efficiency of unmanned vehicle under the scene of perception occlusion.

Description

Technical Field

[0001] This invention belongs to the field of decision-making and control technology for autonomous vehicles, and specifically relates to a method and system for making left-turn traffic decisions for autonomous vehicles in scenarios with obstructed perception. Background Technology

[0002] When autonomous vehicles drive on real urban roads, they inevitably encounter perception blind spots caused by construction barriers, infrastructure walls, and parked vehicles. In these scenarios with impaired perception, especially when making left turns at T-junctions or intersections, the detection range of onboard sensors is severely limited, preventing the vehicle from obtaining complete status information of lateral vehicles in advance, posing a serious challenge to driving safety.

[0003] Existing autonomous driving decision-making methods, such as the TD3 algorithm in deep reinforcement learning, typically assume that perceived information is complete and deterministic. The critic network in these methods outputs a single scalar Q-value, which is the mathematical expectation of future cumulative rewards. However, in scenarios with occluded perception, the same observed state may correspond to completely different real-world traffic conditions within the occluded area (e.g., there may be cars or no cars behind the blind spot), leading to significant differences in rewards for performing the same action. Traditional expectation-based value networks cannot capture this randomness in rewards caused by perceptual uncertainty. Their decision-making strategies tend to polarize between "overly conservative" (long periods of waiting) and "overly aggressive" (reckless overtaking), making it difficult to achieve a reasonable balance between safety and efficiency. Furthermore, existing methods lack effective exploration and safety constraint mechanisms during training, resulting in low training efficiency or difficulty converging in the early stages due to excessive collisions.

[0004] Therefore, there is an urgent need for an unmanned vehicle left-turn decision-making method that can effectively handle the uncertainty of rewards caused by perception occlusion, and achieve efficient exploration and safety assurance coordination during the training process, so as to overcome the above-mentioned defects of existing technologies and achieve a dynamic balance between safety and efficiency in perception occlusion scenarios. Summary of the Invention

[0005] To address the aforementioned technical challenges, this invention proposes a method and system for making left-turn decisions for autonomous vehicles in scenarios with perceptual occlusion. By constructing a value distribution critic network based on a fully parameterized quantile function to explicitly model the uncertainty in rewards caused by perceptual occlusion, and combining conditional value-at-risk constraints with variable risk levels and a phased exploration constraint training strategy, a dynamic balance between safety and traffic efficiency is achieved in the left-turn decision-making process for autonomous vehicles in perceptual occlusion scenarios.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: In a first aspect, this invention proposes a method for making left-turn traffic decisions for autonomous vehicles in scenarios with perceived occlusion, comprising the following steps: The observation status of the vehicle is obtained, which includes: vehicle status information, status information of surrounding vehicles, and perception blind spot information in the current environment; the perception blind spot information includes the maximum perceptible distance of the vehicle to the left lane and the right lane respectively. The observed state is input into the decision network, which outputs preliminary continuous acceleration control commands as actions to be executed; the decision network is an actor network. By utilizing a value distribution critic network based on a fully parameterized quantile function, the future cumulative reward distribution of the state-action pair consisting of the current observed state and the action to be performed is modeled, and the reward distribution is output to assess the risk of the action to be performed. Based on the reward distribution, the conditional risk value of the variable risk level is calculated, and the conditional risk value is compared with a safety threshold. If it is lower than the safety threshold and the vehicle speed is greater than a preset speed threshold, the action to be executed is replaced with a preset safe braking action as the final acceleration command; otherwise, the action to be executed is used as the final acceleration command. The final acceleration command is sent to the lower-level controller for execution, and the steering wheel angle is generated by the trajectory tracking controller to control the vehicle to complete the left turn.

[0007] Furthermore, during the process of acquiring the observation status of the vehicle itself, the status information of environmental vehicles located in the perception blind zone is set to zero.

[0008] Furthermore, the observed state is input into the decision network, which outputs preliminary continuous acceleration control commands as actions to be executed; the decision network is an actor network, specifically: The observed state is used as the input feature vector of the input layer of the actor network. After nonlinear transformation by the hidden layer of the actor network, a continuous value is output by the output layer of the actor network as a preliminary continuous acceleration control command. The actor network adopts a multilayer perceptron structure.

[0009] Furthermore, the observation state is represented as ;in, Indicates the vehicle's status; Represents the environmental vehicle state vector; Represents the distance vector of the perception blind spot; in, Including the vehicle's position and speed; Includes location and speed information for up to four environmental vehicles; ; This indicates the maximum perceptible distance of the vehicle to the left lane. This indicates the maximum perceptible distance of the vehicle to the right lane.

[0010] Furthermore, the value distribution commentator network based on the fully parameterized quantile function includes: Quantile proposal network, used to output the positions of increasing quantiles. ,in, The number of quantiles, And satisfy ; Indicates the first quantile; Indicates the first quantiles; Quantile numerical networks are used to output the value corresponding to each quantile position. ,in, Indicates the current observation status; Indicates an action to be performed; ; Define the position of the boundary quantile and The return distribution Approximated as Mixing of weighted Dirac pulses: ; in, Indicates in Dirac impulse function at the location; For the first The probability quality of each quantile interval.

[0011] Furthermore, based on the aforementioned return distribution, the conditional value of risk for the variable risk level is calculated, specifically as follows: A lightweight MLP network is constructed, which takes the perception blind spot information, the vehicle state, and the observable vehicle state in the current observation state as input, and adaptively outputs a variable risk level. ;

[0012] ; Based on variable risk level Calculate the conditional value of risk for the current action to be performed. : ; in, Indicates the first quantiles, Indicates the first quantiles; express The corresponding value.

[0013] Furthermore, the variable risk level The MLP network maps to a preset interval using the Sigmoid function; it takes the distance to the perception blind zone in the observation state as input and outputs a value negatively correlated with the perception blind zone distance. value.

[0014] Furthermore, the training phase of the method employs a phased exploration-constraint training strategy, specifically as follows: In the early stages of training, the portion of the reward distribution above a preset quantile threshold is multiplied by a dynamic scaling factor greater than 1. To encourage exploration of high-risk actions, the adjusted expected distribution is: ; in, This represents the adjusted expected distribution; This indicates the preset quantile threshold. ; Indicates the current training step count; for abbreviation; In the later stages of training, the dynamic scaling factor will be... The risk level is reduced to a preset constant less than 1, and the conditional value of risk constraint of the variable risk level is enabled.

[0015] Furthermore, the dynamic scaling factor The value is set to a preset constant during the early stage of training. In the later stages of training from Linear decay to a preset constant And satisfy The attenuation method is as follows: ; in, This represents the total number of training steps; This indicates the number of steps separating the early and late stages of training; when hour, .

[0016] Secondly, this invention also proposes a left-turn decision-making system for unmanned vehicles in scenarios with sensor occlusion, comprising: The perception information acquisition module is used to acquire the observation status of the vehicle. The observation status includes: vehicle status information, status information of surrounding vehicles, and perception blind spot information in the current environment. The perception blind spot information includes the maximum perceptible distance of the vehicle to the left lane and the right lane, respectively. The decision output module is used to input the observation state into the decision network and output preliminary continuous acceleration control commands as actions to be executed; the decision network is an actor network. The reward distribution output module is used to model the future cumulative reward distribution of the state-action pair consisting of the current observed state and the action to be performed using a value distribution critic network based on a fully parameterized quantile function, and outputs the reward distribution to assess the risk of the action to be performed. The variable risk constraint module is used to calculate the conditional risk value of the variable risk level based on the reward distribution, and compare the conditional risk value with a safety threshold; if it is lower than the safety threshold and the vehicle speed is greater than a preset speed threshold, the action to be executed is replaced with a preset safe braking action as the final acceleration command; otherwise, the action to be executed is used as the final acceleration command. The left-turn behavior output module is used to send the final acceleration command to the lower-level controller for execution, and generate the steering wheel angle through the trajectory tracking controller to control the vehicle to complete the left-turn behavior.

[0017] The effects described in the invention are merely those of the embodiments, and not all the effects of the invention. One of the above technical solutions has the following advantages or beneficial effects: This invention proposes a method and system for left-turn decision-making for unmanned vehicles in scenarios with perceptual occlusion. The method includes the following steps: acquiring the observation state of the vehicle, which includes: vehicle state information, state information of surrounding vehicles, and perceptual blind spot information in the current environment; the perceptual blind spot information includes the maximum perceptible distance of the vehicle to the left lane and the right lane, respectively; inputting the observation state into a decision network, and outputting a preliminary continuous acceleration control command as an action to be executed; the decision network is an actor network; using a value distribution critic network based on a fully parameterized quantile function, modeling the future cumulative reward distribution of the state-action pair consisting of the current observation state and the action to be executed, and outputting a reward distribution to assess the risk of the action to be executed; based on the reward distribution, calculating the conditional risk value of a variable risk level, and comparing the conditional risk value with a safety threshold; if it is lower than the safety threshold and the vehicle speed is greater than a preset speed threshold, then replacing the action to be executed with a preset safe braking action as the final acceleration command; otherwise, using the action to be executed as the final acceleration command; sending the final acceleration command to the lower-level controller for execution, and generating a steering wheel angle through a trajectory tracking controller to control the vehicle to complete the left turn. Based on this method, a corresponding system is also proposed. This invention effectively solves the decision-making problem of autonomous vehicles turning left at narrow intersections with occluded perception by introducing a value distribution commentator network based on FQF, CVaR tail constraints with variable risk levels, and a phased exploration constraint training strategy. It achieves a dynamic balance between safety and traffic efficiency in the left-turn decision-making of autonomous vehicles in occluded perception scenarios.

[0018] This invention replaces the traditional critic network with a single expected value output with a value distribution network based on a fully parameterized quantile function. This allows for explicit modeling of the uncertainty in rewards caused by occlusion, enabling the decision model to perceive the potential downside risks of different actions when faced with the same observation state. Furthermore, a conditional risk value tail constraint with variable risk levels is introduced to provide a safety net for actions with pessimistic expectations worse than the task failure threshold. This effectively reduces the collision rate, improves decision security, and solves the problem of strategy polarization caused by incomplete perceptual information in existing technologies.

[0019] The variable risk level conditional risk value constraint of this invention adaptively outputs the risk level through a lightweight MLP network, achieving dynamic matching between risk sensitivity and scene occlusion. When the perception blind spot is large, a more conservative constraint is adopted; when the field of view is good, more efficient passage is allowed, thus avoiding the overly conservative problem caused by a fixed risk level. This invention maintains a reasonable passage speed while keeping the collision rate low, effectively reducing the timeout rate and achieving a good balance between safety and efficiency.

[0020] The proposed phased exploration-constrained training strategy encourages exploration of high-risk but important scenarios by multiplying the high quantile portion of the reward distribution by a dynamic scaling factor greater than 1 in the early stages of training. This accelerates the method's recognition of risk interaction patterns in perceptually occluded scenarios. In the later stages of training, the scaling factor is decayed to less than 1, and conditional value-at-risk (VAT) safety constraints are enabled, guiding the strategy to converge smoothly towards a conservative approach. This strategy effectively solves the problem of frequent collisions and difficulty in convergence caused by random exploration in existing deep reinforcement learning methods in perceptually occluded scenarios. It achieves a smooth transition from "exploration-first" to "safety-first" approaches, significantly improving training efficiency and the reliability of the final strategy.

[0021] This invention improves upon the value distribution network, the tail constraint of variable conditional value of risk, and the phased exploration constraint training strategy, each functioning independently yet collaboratively. The value distribution network provides an accurate risk quantification basis for conditional value of risk constraints, which transform the risk quantification results into an executable safety verification mechanism. The phased training strategy ensures that the model can fully explore key scenarios during training and form a safe and stable decision-making strategy in the later stages. The combination of these improvements achieves a technical enhancement across the entire "perception-evaluation-decision-training" chain, demonstrating superior overall performance compared to any single improvement or any combination of two improvements. Attached Figure Description

[0022] Figure 1 This is a flowchart of a method for making a left-turn decision for an unmanned vehicle in a scenario with perceived occlusion, as proposed in Embodiment 1 of the present invention. Figure 2This is a framework diagram of the implementation of a method for making left-turn traffic decisions for unmanned vehicles in a scenario with perceived occlusion, as proposed in Embodiment 1 of the present invention. Figure 3 This is a schematic diagram of a T-shaped single-lane intersection without traffic lights built based on the Gym framework, as proposed in Embodiment 1 of the present invention. Figure 4 This is a schematic diagram of a left-turn traffic decision system for an unmanned vehicle in a scenario with perceived occlusion, as proposed in Embodiment 2 of the present invention. Detailed Implementation

[0023] To clearly illustrate the technical features of this solution, the invention will be described in detail below through specific embodiments and in conjunction with the accompanying drawings. The following disclosure provides many different embodiments or examples for implementing different structures of the invention. To simplify the disclosure of the invention, components and arrangements of specific examples are described below. Furthermore, reference numerals and / or letters may be repeated in different examples. This repetition is for simplification and clarity and does not in itself indicate a relationship between the various embodiments and / or arrangements discussed. It should be noted that the components illustrated in the drawings are not necessarily drawn to scale. Descriptions of well-known components, processing techniques, and processes are omitted in this invention to avoid unnecessarily limiting the invention.

[0024] Example 1 Embodiment 1 of this invention proposes a left-turn decision-making method for autonomous vehicles in a perceptual occlusion scenario. This method addresses the technical problems of existing autonomous driving left-turn decision-making methods being unable to handle the uncertainty of rewards caused by perceptual occlusion, leading to a polarization of decision-making strategies between overly conservative and overly aggressive approaches, and the lack of effective exploration and safety constraint mechanisms and difficulty in convergence during the training process.

[0025] The detailed process of this invention includes: First, receiving the position and speed information of surrounding vehicles sensed by the upstream BEV, and simultaneously acquiring the perception blind spot information of the current environment (i.e., the maximum perceptible distance of the vehicle to the left and right lanes). Then, the state information is input into the decision network, which outputs a reasonable acceleration action. Based on this, the core improvement of this invention lies in replacing the traditional critic network that outputs a single expected value with a value distribution network based on a fully parameterized quantile function (FQF). This network can explicitly model the complete distribution of rewards, thereby quantifying the uncertainty caused by perception occlusion. The actor network outputs preliminary continuous acceleration commands based on the input state, which are then verified for safety by a CVaR tail constraint module with a variable risk level. If the risk of the current action under pessimistic expectations exceeds a threshold, it is forcibly replaced with a creeping, slow-moving action; otherwise, the original command is executed. Finally, longitudinal acceleration is executed by the lower-level controller, and lateral behavior is generated by the trajectory tracking controller to generate the steering wheel angle, jointly controlling the vehicle to complete a safe and efficient left turn.

[0026] Figure 1 This is a flowchart of a method for making a left-turn decision for an unmanned vehicle in a scenario with perceived occlusion, as proposed in Embodiment 1 of the present invention. Figure 2 This is a framework diagram of the implementation of a left-turn traffic decision-making method for unmanned vehicles in a perceptual occlusion scenario proposed in Embodiment 1 of the present invention; combined with Figure 1 and Figure 2 The implementation process of this invention will be explained together.

[0027] In step S1, the observation status of the vehicle is obtained, which includes: vehicle status information, status information of surrounding vehicles, and perception blind spot information in the current environment; the perception blind spot information includes the maximum perceptible distance of the vehicle to the left lane and the right lane respectively. Figure 3 This is a schematic diagram of a simulation environment for a T-shaped single-lane intersection without traffic lights, built using the Gym framework as proposed in Embodiment 1 of this invention. This invention models a left-turn scenario at a T-shaped intersection without traffic lights. The vehicle is located behind the stop line on the vertical road of the intersection and plans to make an unprotected left turn into the horizontal road. The intersection point of the vehicle's left-turn trajectory and the horizontal road is defined as the conflict point. There are oncoming vehicles traveling straight on the left and right sides of the horizontal road, denoted as the left-side and right-side oncoming vehicles, respectively. Both vehicles travel in opposite directions along the horizontal road, and both may conflict with the vehicle's left-turn trajectory. To simulate the perception obstruction environment in real urban roads, obstructions are placed to the left and right front of the vehicle, forming perception blind spots.

[0028] exist Figure 3 In the diagram, point a represents the current position of the vehicle, which is the center point of the rear axle; point b represents the end point of the vehicle's left-turn trajectory, which is the merging point where the vehicle enters the cross road after completing the left turn; point c represents the current position of the vehicle going straight on the left; and point d represents the current position of the vehicle going straight on the right. This indicates the maximum perceptible distance of the vehicle to the left lane. R represents the maximum perceptible distance of the vehicle to the right lane; R represents the turning radius of the vehicle, that is, the radius of the arc of the vehicle's left-turn trajectory; Y represents the distance parameter in the lateral road width direction.

[0029] Figure 3 In the simulation environment, there are 0 to 4 vehicles traveling straight on a transverse road, with the probability of the first vehicle appearing being 100% and the probability of the following vehicle appearing being 50%. To simulate perception occlusion, a perception blind zone variable is defined. and For any vehicle in the environment, if its lateral distance is greater than the corresponding perceptible distance, it is considered to be in the blind spot, and all state information (position, speed) of the vehicle in the state space is set to zero.

[0030] Ultimately, the observation state of this invention is designed as a 22-dimensional vector, represented as follows: ;in, Indicates the vehicle's status; Represents the environmental vehicle state vector; Represents the distance vector of the perception blind spot; in, This includes the vehicle's position and velocity in the planar coordinate system; Includes the position and speed information of up to four environmental vehicles (two on each side), and the information of vehicles located in the blind spot is set to zero; ; This indicates the maximum perceptible distance of the vehicle to the left lane. This indicates the maximum perceptible distance of the vehicle to the right lane. The longitudinal acceleration control range of the vehicle is... The negative values ​​represent deceleration, and the positive values ​​represent acceleration. Through this environmental modeling, this invention provides a simulation verification basis for the left-turn traffic decision-making method for autonomous vehicles in scenarios with occlusion.

[0031] In step S2, the observation state is input to the decision network, and a preliminary continuous acceleration control command is output as the action to be executed; the decision network is an actor network. The observation state obtained in step S1 is input into the decision network, which outputs preliminary continuous acceleration control commands as actions to be executed. The decision network is an actor network, employing a multilayer perceptron (MLP) structure. Specifically, the actor network uses the observation state vector constructed in step S1... The input feature vector, used as the input layer, contains information about the vehicle's state, the state of surrounding vehicles, and blind spot information. This input feature vector undergoes a non-linear transformation through one or more hidden layers of the actor network, extracting feature representations relevant to the current driving decision layer by layer. Each hidden layer employs a fully connected structure and is equipped with a non-linear activation function (such as ReLU) to enhance the network's expressive power. The outputs of the actor network's hidden layers are ultimately passed to the output layer, which outputs a continuous value as an initial continuous acceleration control command through a linear mapping. This acceleration control command is a scalar value with a range of [value missing]. The negative values ​​represent deceleration, and the positive values ​​represent acceleration. The network parameters of the actor network are updated during the training phase using a policy gradient method to maximize the expected cumulative reward evaluated by the critic network. Through this process, the actor network maps high-dimensional observation states into specific longitudinal acceleration control commands for subsequent risk assessment and safety verification.

[0032] In step S3, a value distribution critic network based on a fully parameterized quantile function is used to model the future cumulative reward distribution of the state-action pair consisting of the current observed state and the action to be performed, and output the reward distribution to assess the risk of the action to be performed. Traditional reinforcement learning critic networks output a scalar Q-value, which is the expected cumulative reward in the future. However, it is impossible to characterize the randomness of rewards caused by perceptual occlusion. Therefore, this invention introduces an FQF value distribution network.

[0033] The FQF network consists of two working sub-modules: the Fully Parameterized Quantile Function (FPN) and the Quantile Value Network (QVN).

[0034] Quantile Proposal Network (FPN) uses state-action pairs Given the input, output the increasing quantile positions. ,in, The number of quantiles, And satisfy ; Indicates the first quantile; Indicates the first quantiles; Quantile numerical networks (QVN) and a certain quantile position The input is used to output the value corresponding to each quantile position. ,in, Indicates the current observation status; Indicates an action to be performed; ; By minimizing the 1-Wasserstein distance between the predicted and target distributions, FPN learns to "deploy" a limited number of quantile points in the regions where the distribution changes most dramatically, while QVN learns the corresponding quantile values. The boundary quantile locations are defined. and The return distribution Approximated as Mixing of weighted Dirac pulses: ; in, Indicates in Dirac impulse function at the location; For the first The probability quality of each quantile interval.

[0035] In this way, the value distribution commentator network no longer outputs a single scalar Q-value (i.e., the mathematical expectation of future cumulative returns), but instead outputs a complete return distribution. This reward distribution explicitly characterizes the various rewards that may be obtained by performing a specific action under the current observation state and their corresponding probabilities, thus fully preserving the uncertainty information brought about by perceptual occlusion. Compared with traditional expectation-based value networks, this value distribution critic network can provide the actor network with richer feedback signals than a single mean, enabling the decision model to perceive the potential downside risks of different actions and providing an accurate basis for risk quantification and safety constraints in subsequent steps.

[0036] In step S4, based on the reward distribution, the conditional risk value of the variable risk level is calculated, and the conditional risk value is compared with a safety threshold. If it is lower than the safety threshold and the vehicle speed is greater than a preset speed threshold, the action to be executed is replaced with a preset safe braking action as the final acceleration command; otherwise, the action to be executed is used as the final acceleration command. In obtaining the distribution of returns Subsequently, this invention proposes a Conditional Value at Risk (CVaR) tail constraint method with variable risk levels to achieve an on-demand balance between safety and efficiency by quantifying the current actions to be executed. In the current observation state The downside risk is assessed, and the conditional risk value is compared with a safety threshold to determine whether a safe replacement should be made for the action to be performed.

[0037] Specifically, a lightweight MLP network is constructed, which takes the perception blind spot information, the vehicle state, and the observable vehicle state in the current observation state as input, and adaptively outputs a variable risk level. ;

[0038] ; This MLP network learns to output appropriate values ​​under different occlusion conditions through training. Value. When the perception blind zone distance is large, the network output is smaller. This triggers more conservative risk constraints; when the field of view is good, the network output is larger. This allows for more efficient communication operations.

[0039] Variable risk level The signal is mapped to the interval [0.05, 0.1] using the Sigmoid function. When the perceptual blind spot is large, the network tends to output smaller values. (e.g., 0.05) triggers more conservative constraints; when the field of view is good, the output is larger. (e.g., 0.1) allows for more efficient communication.

[0040] Based on variable risk level Calculate the conditional value of risk for the current action to be performed. : ; in, Indicates the first quantiles, Indicates the first quantiles; express The corresponding value. This formula calculates the worst-case scenario in the return distribution. The expected value of a portion, i.e. the average return under pessimistic expectations, the lower the value, the higher the risk of the current action under adverse conditions.

[0041] During the execution phase, if ( The safety threshold value of this invention is [value to be filled in]. If the pessimistic expectation is worse than mission failure and the vehicle speed v > 1.0 m / s, then the system will forcibly execute a preset safety braking action (e.g., This achieves a "safety safety net." MLP network By jointly training the main network with a dedicated loss function, it learns to output appropriate values ​​under different levels of occlusion. This maximizes traffic efficiency while ensuring safety.

[0042] In step S5, the final acceleration command is sent to the lower-level controller for execution, and the steering wheel angle is generated by the trajectory tracking controller to control the vehicle to complete the left turn.

[0043] The final acceleration command determined in step S4 is sent to the lower-level controller for execution. The final acceleration command is a longitudinal acceleration control quantity, with units of m / s². After receiving the acceleration command, the lower-level controller converts it into control signals for lower-level actuators such as throttle opening or braking pressure. Through the vehicle's drive-by-wire and braking system, precise control of longitudinal movement is achieved, enabling the vehicle to accelerate or decelerate according to the target acceleration.

[0044] Meanwhile, the trajectory tracking controller calculates the steering wheel angle control value in real time based on the vehicle's current pose information and the planned target path. The target path is the planned trajectory for the vehicle's left turn, containing a series of desired points, each with desired position and desired heading angle information. Based on the lateral and heading deviations between the vehicle's current actual pose and reference points on the target path, and combined with the vehicle's kinematic model, the trajectory tracking controller calculates the required steering wheel angle at the current moment using control algorithms such as proportional-integral-derivative (PID) control, model predictive control (MPC), or pure pursuit control. This steering wheel angle control value is executed via the vehicle's steer-by-wire system, ensuring that the vehicle's actual driving trajectory follows the target path.

[0045] The coordinated action of longitudinal acceleration control and lateral steering control works on the vehicle, adjusting its longitudinal speed according to the target acceleration command while simultaneously traveling laterally along the target left-turn path, ultimately completing a safe and efficient left turn. Through this decoupled longitudinal and lateral control, this invention transforms the acceleration command output by the upper-level decision-making module into specific vehicle motion control signals, achieving a complete closed loop from decision-making to execution.

[0046] Steps S1-S5 above describe the execution (deployment) phase of this method. During the training phase, this invention also employs a phased exploration constraint training strategy: In the early stages of training, the portion of the reward distribution above a preset quantile threshold is multiplied by a dynamic scaling factor greater than 1. To encourage exploration of high-risk actions, the adjusted expected distribution is: ; in, This represents the adjusted expected distribution; This indicates the preset quantile threshold. ; Indicates the current training step count; for abbreviation; In this embodiment This weighted expectation replaces the traditional expectation as the target for actor network updates. Because the contribution of the high quantile portion is amplified, the actor network tends to choose actions that may bring higher rewards (even if accompanied by high risks), thereby actively exploring blind spots and accelerating the recognition of risk interaction patterns.

[0047] In the later stages of training, the dynamic scaling factor will be... The risk level is reduced to a preset constant less than 1, and the conditional value-of-risk constraint of the variable risk level is enabled. In this embodiment, The value was set in the early stage of training. During the later stages of training, the value linearly decreased from 2.0 to .

[0048] The dynamic scaling factor The value is set to a preset constant during the early stage of training. In the later stages of training from Linear decay to a preset constant And satisfy The attenuation method is as follows: ; in, This represents the total number of training steps; This indicates the number of steps separating the early and late stages of training (in this embodiment). );when When the value is >1, a certain incentive for exploration is still retained. When the risk level is less than 1, high-risk actions are effectively "punished," guiding the strategy towards a more conservative convergence. Simultaneously, the variable risk level CVaR tail constraint is formally activated in the later stages of training as a safety fallback mechanism for the execution layer. In synergy with CVaR constraints, this invention achieves a smooth strategy transition of "enhancing exploration in the early stage and ensuring safety in the later stage", which significantly improves training efficiency and the safety of the final strategy.

[0049] The decision-making method and training strategy proposed in this invention constitute a complete algorithmic framework, denoted as the Exploration-Constrained Fully Parameterized Quantile Function (ECFQF) algorithm. This algorithmic name reflects the three core improvements of this invention: E (Exploration) corresponds to the phased exploration-constrained training strategy, C (Constrained) corresponds to the conditional risk value tail constraint for variable risk levels, and FQF (Fully Parameterized Quantile Function) corresponds to the value distribution commentator network based on the fully parameterized quantile function. These three components work together to achieve a balance between safety, traffic efficiency, and training stability in the left-turn decision-making of autonomous vehicles in perceptually occluded scenarios.

[0050] To verify the effectiveness of the ECFQF algorithm proposed in this invention, an ablation experiment was conducted in a simulated T-shaped intersection with perceived occlusion. The results were compared with the original TD3 baseline and variants with progressively added improvement modules. A total of 200 tests were conducted, and the experimental results are shown in Table 1.

[0051] Table 1: Comparative Test Results of Left-Turn Algorithms at Intersections with Obstructed Traffic Detection

[0052] Note: Improvement A replaces the standard MLP value function in the Critic network with an FQF value distribution network (the upper bound scaling factor is constant at 0.8, and the risk level parameter is...). (The value is kept constant at 0.08); Improvement B adds a CVaR tail constraint with a variable risk level to the value distribution network; Improvement C adopts a training strategy with a phased exploration-constraint mechanism. The complete scheme (Improvements B+C) is the ECFQF algorithm proposed in this invention.

[0053] The experimental results in Table 1 show that: Significantly improved safety: The original TD3 baseline had a collision rate as high as 27.5% and a success rate of only 72.5% in perceptually occluded scenarios. After introducing the FQF value distribution network (+A) alone, the collision rate dropped significantly to 7.5%, and the success rate increased to 90.5%, verifying the effectiveness of value distribution modeling for perceptual uncertainty. Adding a variable CVaR tail constraint (+B) further reduced the collision rate to 4.0% and increased the success rate to 94.0%. The complete solution of this invention, ECFQF (+B+C), while maintaining a low collision rate of 4.0%, reduced the timeout rate from 2.0% to 1.5% and increased the success rate to 94.5%. Compared to the original TD3 baseline, this represents a relative reduction of 85.5% in collision rate (from 27.5% to 4.0%) and an absolute increase of 22% in success rate.

[0054] A reasonable efficiency trade-off: While the original TD3 baseline has the highest average speed (3.53 m / s), this is an "aggressive rushing" strategy at the cost of extremely high collision risk. The ECFQF model of this invention achieves an average speed of 3.15 m / s while maintaining a low collision rate, an improvement of approximately 4.0% compared to the +B scheme (3.03 m / s) with the same low collision rate, while also having a lower timeout rate. This demonstrates that the synergistic effect of the CVaR constraint with variable risk levels and the phased training strategy can effectively restore traffic efficiency while ensuring safety, achieving a dynamic balance between safety and efficiency.

[0055] The contributions of each improvement module are clear: the progressive improvement from +A to +B and then to +B+C proves that the three core improvements proposed in this invention—value distribution network, variable CVaR tail constraint, and phased exploration constraint training strategy—all contribute positively to the model performance, and the combination of the three achieves the best results.

[0056] The ECFQF algorithm proposed in this invention significantly improves decision-making safety (reducing the collision rate by 85.5% and increasing the success rate to 94.5%) in the scenario of turning left at a narrow intersection with occlusion, while maintaining reasonable traffic efficiency. It solves the problem of policy polarization in existing technologies under uncertain perception scenarios and provides a reliable and interpretable solution for the safe passage of autonomous vehicles in occluded environments.

[0057] Example 2 Based on the method for determining left-turn passage of an autonomous vehicle in a perceptual occlusion scenario proposed in Embodiment 1 of the present invention, Embodiment 2 of the present invention further proposes a system for determining left-turn passage of an autonomous vehicle in a perceptual occlusion scenario. Figure 4 This is a schematic diagram of a left-turn traffic decision-making system for an unmanned vehicle in a perceptual occlusion scenario, as proposed in Embodiment 2 of the present invention; the system includes: The perception information acquisition module is used to acquire the observation status of the vehicle. The observation status includes: vehicle status information, status information of surrounding vehicles, and perception blind spot information in the current environment. The perception blind spot information includes the maximum perceptible distance of the vehicle to the left lane and the right lane, respectively. The decision output module is used to input the observation state into the decision network and output preliminary continuous acceleration control commands as actions to be executed; the decision network is an actor network. The reward distribution output module is used to model the future cumulative reward distribution of the state-action pair consisting of the current observed state and the action to be performed using a value distribution critic network based on a fully parameterized quantile function, and outputs the reward distribution to assess the risk of the action to be performed. The variable risk constraint module is used to calculate the conditional risk value of the variable risk level based on the reward distribution, and compare the conditional risk value with a safety threshold; if it is lower than the safety threshold and the vehicle speed is greater than a preset speed threshold, the action to be executed is replaced with a preset safe braking action as the final acceleration command; otherwise, the action to be executed is used as the final acceleration command. The left-turn behavior output module is used to send the final acceleration command to the lower-level controller for execution, and generate the steering wheel angle through the trajectory tracking controller to control the vehicle to complete the left-turn behavior.

[0058] Embodiment 2 of the present invention modularizes the method steps disclosed in Embodiment 1 of the present invention, and can achieve the same technical effects as Embodiment 1 of the present invention.

[0059] The description of the relevant parts of the autonomous vehicle left-turn traffic decision system under a perception occlusion scenario provided in Embodiment 2 of the present invention can be found in the detailed description of the corresponding parts of the autonomous vehicle left-turn traffic decision system under a perception occlusion scenario provided in Embodiment 1 of the present invention, and will not be repeated here.

[0060] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that the elements inherent in a process, method, article, or apparatus that includes a list of elements are included. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element. Additionally, portions of the technical solutions provided in the embodiments of this application that are consistent with the implementation principles of corresponding technical solutions in the prior art have not been described in detail to avoid excessive elaboration.

[0061] While specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art can make other modifications or variations based on the above description. It is neither necessary nor possible to exhaustively describe all embodiments here. Various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A method for making left-turn traffic decisions for autonomous vehicles in scenarios with perceptual occlusion, characterized in that, Includes the following steps: The observation state of the vehicle is acquired, including: vehicle status information, status information of surrounding vehicles, and blind spot information in the current environment; the blind spot information includes the maximum perceptible distance of the vehicle to the left lane and the right lane, respectively; the observation state is represented as... ;in, Indicates the vehicle's status; Represents the environmental vehicle state vector; Represents the distance vector of the perception blind spot; in, Including the vehicle's position and speed; Includes location and speed information for up to four environmental vehicles; ; This indicates the maximum perceptible distance of the vehicle to the left lane. This indicates the maximum perceptible distance of the vehicle to the right lane. The observed state is input into the decision network, which outputs preliminary continuous acceleration control commands as actions to be executed; the decision network is an actor network. By utilizing a value distribution critic network based on a fully parameterized quantile function, the future cumulative reward distribution of the state-action pair consisting of the current observed state and the action to be performed is modeled, and the reward distribution is output to assess the risk of the action to be performed. Based on the reward distribution, the conditional risk value for the variable risk level is calculated, and compared with a safety threshold. If the conditional risk value is lower than the safety threshold and the vehicle speed is greater than a preset speed threshold, the action to be executed is replaced with a preset safe braking action as the final acceleration command; otherwise, the action to be executed is used as the final acceleration command. The calculation of the conditional risk value for the variable risk level based on the reward distribution is as follows: A lightweight MLP network is constructed, which takes the perception blind spot information, the vehicle state, and the observable vehicle state in the current observation state as input, and adaptively outputs a variable risk level. ; ； Based on variable risk level Calculate the conditional value of risk for the current action to be performed. : ； in, Indicates the first quantiles, Indicates the first quantiles; express Corresponding value; The variable risk level The MLP network maps to a preset interval using the Sigmoid function; it takes the distance to the perception blind zone in the observation state as input and outputs a value negatively correlated with the perception blind zone distance. value; The final acceleration command is sent to the lower-level controller for execution, and the steering wheel angle is generated by the trajectory tracking controller to control the vehicle to complete the left turn.

2. The method for making a left-turn decision for an unmanned vehicle in a perceptual occlusion scenario according to claim 1, characterized in that, During the process of acquiring the observation status of the vehicle itself, the status information of environmental vehicles located in the perception blind zone is set to zero.

3. The method for making a left-turn decision for an unmanned vehicle in a perceptual occlusion scenario according to claim 1, characterized in that, The observed state is input into the decision network, which outputs preliminary continuous acceleration control commands as actions to be executed; the decision network is an actor network, specifically: The observed state is used as the input feature vector of the input layer of the actor network. After nonlinear transformation by the hidden layer of the actor network, a continuous value is output by the output layer of the actor network as a preliminary continuous acceleration control command. The actor network adopts a multilayer perceptron structure.

4. The method for making a left-turn decision for an unmanned vehicle in a perceptual occlusion scenario according to claim 1, characterized in that, The value distribution commentator network based on the fully parameterized quantile function includes: Quantile proposal network, used to output the positions of increasing quantiles. ,in, The number of quantiles, And satisfy ; Indicates the first quantile; Indicates the first quantiles; Quantile numerical networks are used to output the value corresponding to each quantile position. ,in, Indicates the current observation status; Indicates an action to be performed; ; Define the position of the boundary quantile and , No. The quantile intervals are The value corresponding to this quantile interval ;in, The return distribution Approximated as Mixing of weighted Dirac pulses: ； in, Indicates in Dirac impulse function at the location; For the first The probability quality of each quantile interval.

5. The method for making a left-turn decision for an unmanned vehicle in a perceptual occlusion scenario according to claim 1, characterized in that, The training phase of the method employs a phased exploration-constraint training strategy, specifically as follows: In the early stages of training, the portion of the reward distribution above a preset quantile threshold is multiplied by a dynamic scaling factor greater than 1. To encourage exploration of high-risk actions, the adjusted expected distribution is: ； in, This represents the adjusted expected distribution; This indicates the preset quantile threshold. ; Indicates the current training step count; for abbreviation; In the later stages of training, the dynamic scaling factor will be... The risk level is reduced to a preset constant less than 1, and the conditional value of risk constraint of the variable risk level is enabled.

6. The method for making a left-turn decision for an unmanned vehicle in a perceptual occlusion scenario according to claim 5, characterized in that, The dynamic scaling factor The value is set to a preset constant during the early stage of training. In the later stages of training from Linear decay to a preset constant And satisfy The attenuation method is as follows: ； in, This represents the total number of training steps; This indicates the number of steps separating the early and late stages of training; when hour, .

7. A left-turn decision-making system for an unmanned vehicle in a perceptually occluded scenario, used to execute the left-turn decision-making method for an unmanned vehicle in a perceptually occluded scenario as described in any one of claims 1 to 6, characterized in that, include: The perception information acquisition module is used to acquire the observation status of the vehicle. The observation status includes: vehicle status information, status information of surrounding vehicles, and perception blind spot information in the current environment. The perception blind spot information includes the maximum perceptible distance of the vehicle to the left lane and the right lane, respectively. The decision output module is used to input the observation state into the decision network and output preliminary continuous acceleration control commands as actions to be executed; the decision network is an actor network. The reward distribution output module is used to model the future cumulative reward distribution of the state-action pair consisting of the current observed state and the action to be performed using a value distribution critic network based on a fully parameterized quantile function, and outputs the reward distribution to assess the risk of the action to be performed. The variable risk constraint module is used to calculate the conditional risk value of the variable risk level based on the reward distribution, and compare the conditional risk value with a safety threshold; if it is lower than the safety threshold and the vehicle speed is greater than a preset speed threshold, the action to be executed is replaced with a preset safe braking action as the final acceleration command; otherwise, the action to be executed is used as the final acceleration command. The left-turn behavior output module is used to send the final acceleration command to the lower-level controller for execution, and generate the steering wheel angle through the trajectory tracking controller to control the vehicle to complete the left-turn behavior.

Images