Vehicle danger cut-in strategy construction method based on aerial view input

Abstract

The invention discloses a vehicle danger cut-in strategy construction method based on aerial view input, and belongs to the technical field of automatic driving, and the method comprises the following steps: 1, building a simulation scene needed for vehicle danger cut-in strategy model training; 2, processing the bird's-eye view input obtained from the simulation scene, and taking the processed bird's-eye view as the input of a vehicle danger cut-in strategy model; 3, defining a reward function, wherein the reward function comprises a danger cut-in instant reward, a guide danger cut-in reward and an improper behavior punishment; 4, constructing a vehicle danger cut-in strategy model, and constructing a strategy model based on PPO; 5, training a vehicle danger cut-in strategy model; and step 6, after completing the model training in the step 5, obtaining a vehicle danger cut-in strategy model, which is applied to the safety performance test of an AVs system, and has the advantages of reducing the time and effort required for designing model state input, improving the research and development efficiency and thus improving the accuracy of decision making.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving technology, and in particular to a method for constructing a vehicle danger cut-in strategy based on bird's-eye view input. Background Art

[0002] Autonomous Vehicles (AVs) provide new solutions to solve the problems of traffic congestion, frequent accidents and energy shortages existing in traditional vehicles, and have become the focus of research in recent years. In an autonomous driving environment, the driving subject of the vehicle changes from the driver to the machine. Therefore, before the actual deployment of AVs, it is crucial to conduct safety-oriented performance tests to ensure their safety.

[0003] Testing in an open road environment is a common method for AVs testing. Since dangerous scenarios that challenge the safety of AVs in an open road environment are extremely rare, AVs need to travel billions of miles to prove that their safety is no less than that of human drivers. Considering many aspects such as testing cost, efficiency, and safety, open road testing can no longer meet the requirements of autonomous vehicle technology research and development and iteration. Virtual testing has the advantages of flexible scenario configuration, low testing cost, and high repeatability. Under virtual conditions, dangerous scenarios with a small probability of occurring on open roads can be constructed. Virtual testing has become the main method for AVs testing.

[0004] If AVs can safely deal with dangerous situations, their safety can usually be guaranteed in other situations. Therefore, virtual testing of AVs based on dangerous scenarios is conducive to shortening the test cycle and realizing accelerated testing of AVs. In order to give full play to the advantages of virtual testing, constructing dangerous test scenarios that are challenging for AVs has become an important research topic.

[0005] The vehicle cut-in scenario is a typical potentially dangerous driving scenario. Improper vehicle cut-in can easily cause rear-end collisions with the following vehicles. In 2015, 4.9% of the traffic accidents in my country were caused by improper vehicle cut-in. Constructing a dangerous virtual vehicle cut-in scenario is of great significance for AVs safety testing. Traditional methods use methods such as clustering of natural driving data, Monte Carlo simulation or probability sampling to generate dangerous vehicle cut-in test scenarios. The traffic vehicles in the generated scenarios drive according to preset trajectories and do not have autonomous decision-making capabilities. They cannot respond to changes in the AVs system under test in a timely manner, which is different from the real environment. In order to make the traffic vehicles in the constructed dangerous scenarios interactive, the study began to use machine learning methods to construct a traffic vehicle model with continuous decision-making capabilities, and then generate dangerous test scenarios. The generated test scenarios can test the continuous decision-making capabilities of AVs.

[0006] In the research on building traffic control strategies, traditional methods often require a lot of time and effort to design and extract scene features as input to the strategy model. At the same time, due to the limitation of sensor acquisition range, scene features extracted by sensors such as radar often fail to fully capture the key information in the scene. A bird's-eye view is an image that projects a three-dimensional scene onto a two-dimensional plane. It contains key information such as the position, direction, and size of all relevant objects in the scene. A bird's-eye view can provide the vehicle with a global perception of the surrounding environment, so that the strategy model can have a more comprehensive understanding of the relative positions and dynamic changes of targets such as roads, obstacles, and other vehicles, enhancing the model's understanding and response capabilities to complex traffic scenes, and helping it make more effective decisions. In a simulation test environment, a bird's-eye view can be generated by pre-set rules or algorithms and used as the input to the traffic control strategy model to avoid the defects of traditional methods.

[0007] In order to generate a vehicle dangerous cut-in test scenario for AVs testing, the present invention provides a vehicle dangerous cut-in strategy construction method based on bird's-eye view input, and trains the strategy model through the PPO (Proximal Policy Optimization) algorithm. Compared with traditional methods, this method can reduce the time and effort required for designing model state input and improve research and development efficiency. At the same time, since the bird's-eye view provides a global perspective, the model can capture key information in the scene more comprehensively, thereby improving the accuracy of decision-making. This method can be used to construct a vehicle control strategy model with both dangerous cut-in capabilities and interactive capabilities. The model can be used to control traffic vehicles to dangerously cut in front of the tested vehicle and respond to the dynamic changes of the tested AVs system and the driving environment in a timely manner, so that the tested vehicle faces the risk of rear-end collision, thereby testing the safety of the tested AVs system. Summary of the invention

[0008] In view of the deficiencies in the prior art, an embodiment of the present invention aims to provide a method for constructing a vehicle danger cut-in strategy based on a bird's-eye view input, so as to solve the problems in the above-mentioned background technology.

[0009] To achieve the above object, the present invention provides the following technical solutions: A method for constructing a vehicle danger cut-in strategy based on a bird's-eye view input comprises the following steps: Step 1: Build a simulation scenario for training the vehicle dangerous entry strategy model; Step 2: Process the bird's-eye view input obtained from the simulation scene, and use the processed bird's-eye view as the input of the vehicle dangerous entry strategy model; Step 3: Define the reward function, which includes the immediate reward for dangerous entry, the reward for guiding dangerous entry, and the penalty for improper behavior; Step 4: Construct a vehicle danger cut-in strategy model based on the PPO algorithm; Step 5: Train the vehicle dangerous entry strategy model; Step 6: After completing the model training described in step 5, the vehicle danger entry strategy model is obtained and applied to the safety performance test of the AVs system.

[0010] As a further solution of the present invention, in the step 1, any one of the traffic vehicles in the simulation scene is selected as the self-vehicle, another traffic vehicle is selected as the target vehicle, the selected target vehicle is set to black, and the self-vehicle is set to white.

[0011] As a further solution of the present invention, the step 2 also includes calculating and obtaining the orientation, horizontal and vertical coordinate positions of the vehicle and the path points of the vehicle, and converting the path points into image pixel coordinates.

[0012] As a further solution of the present invention, the processing of the bird's-eye view includes replacing the colors of the valid path points in the image with green transparent color. If there is a target position, it is converted into image pixel coordinates, and the color of the corresponding target position in the image is replaced with purple. If it is necessary to remove unnecessary parts in the bird's-eye view, the image is cropped.

[0013] As a further solution of the present invention, the instant reward for the dangerous entry include: (1) In the formula, It refers to the time when the vehicle cuts in. The collision time with the target vehicle, It refers to the relative speed between the ego vehicle and the target vehicle at the moment of cut-in.

[0014] When the self-vehicle cuts into the lane where the target vehicle is located and is in front of the target vehicle, if , then give the model a larger reward; if If it does not fall into the above range, the relative speed between the self-vehicle and the target vehicle will be used. Determine the reward value, The larger it is, the higher the risk of the target car rear-ending the ego car, and the reward value increases accordingly.

[0015] As a further solution of the present invention, the guiding risk cut-in reward include: (2) (3) (4) In the formula, is the distance between the vehicle and the set target point, For The coefficient used for normalization, Distance reward; TT is the TTC discount factor, The coefficient used to normalize the collision time TTC, where , , is the distance between the ego vehicle and the target vehicle, is the longitudinal distance between the ego vehicle and the target vehicle, is the lateral distance between the ego vehicle and the target vehicle, is the relative speed between the ego vehicle and the target vehicle.

[0016] As a further solution of the present invention, the improper behavior penalty is to prevent the vehicle from having improper behaviors such as driving off the road, reversing or actively colliding with traffic vehicles during driving, and to punish the improper behavior of the vehicle by setting a negative reward value. .

[0017] As a further solution of the present invention, the reward value Belongs to the reward function , the reward function

[0018] (5) in, , and for , and The weight coefficients corresponding to the three rewards.

[0019] In summary, compared with the prior art, the embodiments of the present invention have the following beneficial effects: 1. The present invention uses a bird's-eye view as the input of the vehicle dangerous entry strategy model, which helps to reduce the time and effort required for model state input design and improve research and development efficiency. At the same time, the use of a bird's-eye view can provide a global perspective, which helps the strategy model to more comprehensively capture key information in the scene, thereby improving the accuracy of decision-making and making the trained model have better performance.

[0020] 2. The present invention adopts a deep reinforcement learning algorithm to train a vehicle dangerous cut-in strategy model with autonomous decision-making and interactive capabilities. By proposing a targeted reward function, the trained model has dangerous cut-in capabilities and interactive capabilities. It can start from any initial position and eventually cut dangerously to the front of the lane where the vehicle under test is located, creating a greater risk of rear-end collision for the vehicle under test, thereby playing a role in testing the safety of the AVs system.

[0021] 3. The present invention is trained in scenarios containing complex and diverse traffic flows. The trained model has strong generalization ability and can be applied to test scenarios with different traffic flows.

[0022] 4. The vehicle danger entry strategy model constructed by the present invention can be implanted into any autonomous driving virtual test scenario and system, and is easy to promote and use.

[0023] In order to more clearly illustrate the structural features and effects of the present invention, the present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 A vehicle cut-in environment based on SMARTS according to an embodiment of the invention.

[0025] Figure 2 This is a vehicle dangerous cut-in strategy model structure based on PPO according to an embodiment of the invention. DETAILED DESCRIPTION

[0026] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0027] The specific implementation of the present invention is described in detail below in conjunction with specific embodiments.

[0028] In one embodiment, a method for constructing a vehicle dangerous entry strategy based on a bird's-eye view input is described in detail in FIG. Figure 1 , including the following steps: Step 1: Build a simulation scenario for training the vehicle dangerous entry strategy model; Step 2: Process the bird's-eye view input obtained from the simulation scene, and use the processed bird's-eye view as the input of the vehicle dangerous entry strategy model; Step 3: Define the reward function, which includes the immediate reward for dangerous entry, the reward for guiding dangerous entry, and the penalty for improper behavior; Step 4: Construct a vehicle danger entry strategy model based on PPO; Step 5: Train the vehicle dangerous entry strategy model; Step 6: After completing the model training described in step 5, the vehicle danger entry strategy model is obtained and applied to the safety performance test of the AVs system.

[0029] For further information, see Figure 1 In the step 1, any traffic vehicle is selected as the self-vehicle in the simulation scene, and another traffic vehicle is selected as the target vehicle. The selected target vehicle is set to black, and the self-vehicle is set to white.

[0030] Furthermore, the step 2 also includes calculating and obtaining the orientation, horizontal and vertical coordinate positions of the vehicle and the path points of the vehicle, and converting the path points into image pixel coordinates.

[0031] Furthermore, the bird's-eye view is processed, including replacing the colors of valid path points in the image with green transparent color. If a target position exists, it is converted into image pixel coordinates, and the color of the corresponding target position in the image is replaced with purple. If it is necessary to remove unnecessary parts in the bird's-eye view, the image is cropped.

[0032] Furthermore, the danger cuts into the immediate reward include: (1) In the formula, It refers to the time when the vehicle cuts in. The collision time with the target vehicle, It refers to the relative speed between the ego vehicle and the target vehicle at the moment of cut-in.

[0033] When the self-vehicle cuts into the lane where the target vehicle is located and is in front of the target vehicle, if , then give the model a larger reward; if If it does not fall into the above range, the relative speed between the self-vehicle and the target vehicle will be used. Determine the reward value, The larger it is, the higher the risk of the target car rear-ending the ego car, and the reward value increases accordingly.

[0034] Furthermore, the said guiding dangerous entry reward include: (2) (3) (4) In the formula, is the distance between the vehicle and the set target point, For The coefficient used for normalization, Distance reward; TT is the TTC discount factor, The coefficient used to normalize the collision time TTC, where , , is the distance between the ego vehicle and the target vehicle, is the longitudinal distance between the ego vehicle and the target vehicle, is the lateral distance between the ego vehicle and the target vehicle, is the relative speed between the ego vehicle and the target vehicle.

[0035] The improper behavior penalty is to prevent the vehicle from engaging in improper behaviors such as driving off the road, reversing, or actively colliding with traffic vehicles during driving. The improper behavior of the vehicle is punished by setting a negative reward value. .

[0036] The reward value Belongs to the reward function , the reward function The details are as follows: (5) in, , and for , and The weight coefficients corresponding to the three rewards.

[0037] In this embodiment, step 1, this example builds a vehicle entry simulation environment based on the SMARTS (Scalable Multi-Agent Reinforcement learning Training School) platform. SMARTS is a virtual simulation platform for multi-agent reinforcement learning and autonomous driving research, developed by Huawei Noah's Ark Laboratory. This example builds the following based on the SMARTS platform. Figure 1 In the vehicle cut-in environment shown, the length of the lane is 200m. The speed of the background vehicle in the scene is controlled by SUMO (Simulation of Urban Mobility). Specifically, the background vehicle will combine the lane speed limit, the maximum expected speed, and the speed value output by the built-in following model of SUMO, and select the lowest value as the target speed. The ego vehicle is controlled by the vehicle dangerous cut-in strategy described above.

[0038] Take the following steps to improve the diversity of traffic flow in the scene: 1. In order to be closer to the ever-changing traffic flow density in the real traffic environment, three training scenarios with different flow levels were generated, corresponding to low, medium and high flow. The traffic flow on a single lane in these three scenarios was set to 1200veh / h, 1800veh / h and 2400veh / h respectively. The initial spacing, speed and position of the traffic vehicles in the scene were randomly generated, and the three scenarios contained a total of 1200 different initialized traffic flows.

[0039] 2. At the beginning of each round of interaction with the environment using the dangerous vehicle cut-in strategy, any vehicle in the traffic flow is randomly selected as the target vehicle. When the ego vehicle starts from the starting position, enters the lane where the target vehicle is located and is in front of the target vehicle, the ego vehicle is considered to have completed the cut-in. When the ego vehicle successfully cuts in front of the target vehicle and the collision time TTC value at the time of cut-in is 0~6s (TTC is the distance between the ego vehicle and the target vehicle divided by the relative speed of the two vehicles), the ego vehicle is considered to have successfully cut in.

[0040] 3. Since the vehicle dangerous entry strategy model uses a bird's-eye view as the model input, in order to enable the model to distinguish between the ego vehicle, the target vehicle and other traffic vehicles in the scene, the selected target vehicle is set to black, the ego vehicle is set to white, and other traffic vehicles are set to gray.

[0041] Step 2: Process the obtained bird's-eye view of the simulation scene and use the processed bird's-eye view as the input of the policy model. Calculate and obtain the direction, horizontal and vertical coordinate positions of the ego vehicle, and the path points of the ego vehicle. Convert the path points to image pixel coordinates, process the bird's-eye view, and replace the colors of the valid path points in the image with green transparent color. If there is a target position, convert it to image pixel coordinates and replace the color of the corresponding target position in the image with purple.

[0042] In order to remove the unnecessary parts of the bird's-eye view, such as the extra black background in the picture, the image is cropped. Define a crop array to store the image crop amount The original height of the bird's-eye view image is 200, the width is 200, and the number of color channels is 3. The height and width of the image are cropped separately, and the height direction of the image is Cut from The image is cropped, and the color channels remain unchanged. Next, the channel order of the image is converted from HWC (height, width, color channel) to CHW (color channel, height, width).

[0043] Since a single frame image can only provide instantaneous information, stacking continuous image frames can capture the temporal changes in dynamic scenes, thereby providing the model with additional temporal context information such as vehicle motion and speed changes. This information is very important for continuous time series decision making. Therefore, the processed bird's-eye view images are stacked. In this example, the number of stacks is , that is, stacking three consecutive frames of bird's-eye view images. The processed three consecutive frames of bird's-eye view images are spliced ​​in the color channel dimension.

[0044] The bird's-eye view after the above processing and stacking is used as the input of the policy model.

[0045] Step 3: Define the reward function. The reward function in this example includes three parts: immediate reward for entering a dangerous area, reward for guiding entering a dangerous area, and penalty for improper behavior.

[0046] Set a virtual target point 10m in front of the target vehicle lane. Extract state information for further reward calculation, including: the distance between the vehicle and the set target point , the longitudinal distance between the ego vehicle and the target vehicle , lateral distance , relative speed , collision time TTC. Among them, , .

[0047] 1. Dangerous entry instant reward The time when the self-vehicle cuts into the lane where the target vehicle is and is in front of the target vehicle is ,like , then give the model a larger reward; if If it does not fall into the above range, the relative speed between the self-vehicle and the target vehicle will be used. Determine the reward value, The larger the value, the higher the risk of the target car rear-ending the self-vehicle, and the reward value increases accordingly. Specifically, as shown in formula (6): (6) 2. Guide the danger into the reward In order to guide the ego vehicle to the front of the target vehicle, according to the distance between the ego vehicle and the target point Design the distance reward as shown in formula (7) ,along with Reduce, The closer the distance between the vehicle and the target point is, the higher the reward value is. It encourages the model to control the vehicle to drive to the target point and arrive in front of the target vehicle.

[0048] (7) In the formula, For The coefficient used for normalization is 50.

[0049] Since reinforcement learning training aims to maximize the cumulative reward, setting only the distance reward will cause the model to drive in front of the target vehicle at the highest possible speed to collect more rewards, which will cause the vehicle to be faster than the target vehicle when cutting in front of the target vehicle, and fail to meet the requirements of dangerous cutting. In order to solve this problem, the TTC discount coefficient is defined , calculate its value according to formula (8). Obviously, the smaller the TTC value, the When the distance between the vehicle and the target point is far (>10m), The value is set to 1.

[0050] (8) In the formula, is the coefficient for normalizing TTC, and its value is 20. If TTC is a negative value, 20 is uniformly substituted into the above formula for calculation.

[0051] Finally, the reward item for guiding dangerous entry is calculated by formula (9): .

[0052] (9) 3. Penalties for improper behavior In order to prevent the ego vehicle from engaging in improper behaviors such as driving off the road, reversing, or actively colliding with traffic vehicles during driving, the improper behaviors of the ego vehicle are punished by setting a negative reward value. Since the ego vehicle can easily cause the target vehicle to rear-end when it cuts in dangerously, in order not to affect the model's exploration of the environment, no penalty is given when the ego vehicle is rear-ended by a traffic vehicle. In this example, the penalty reward setting for the ego vehicle's improper behavior is shown in Table 1.

[0053] Since the viewing range of the bird’s-eye view is centered on the ego vehicle, when the target vehicle is far away from the ego vehicle, the target vehicle will not appear in the bird’s-eye view. In this case, the reward function should not include the immediate reward for dangerous entry. and guiding dangerous entry rewards Whether there is a target car in the bird's-eye view can be determined based on whether there is a target car color in the bird's-eye view. The reward function used in this example is as shown in formula (10).

[0054] (10) in, , and for , and The weight coefficients corresponding to the three rewards. In this example, the weight coefficients of the three rewards are all set to 1.

[0055] Table 1 Penalties for improper behavior of the vehicle

[0056] Step 4: Construct a vehicle danger entry strategy model. Use the Actor-Critic model structure with a shared feature extraction layer, such as Figure 2 The specific parameters of each layer in the model are shown in Table 2. The input of the model is the bird's-eye view after processing in step 2, denoted as After passing through the feature extraction layer, the Critic network on the left outputs the state Estimated value , the value estimate is used to calculate the advantage function, thereby guiding the Actor network to choose more advantageous actions. The Actor network on the right is the current strategy , is the parameter of the Actor-Critic model. The output of the last layer of the Actor network is converted into the probability value of the optional action category through the softmax layer, and the action is determined by sampling based on the probability. , As the control action of the ego vehicle, it enables the ego vehicle to interact with the vehicle cut-in simulation environment.

[0057] Table 2 Structural parameters of each layer of the model

[0058] This example uses a discrete action space, and the 8 action values ​​available are shown in Table 3.

[0059] Table 3 Action specific values

[0060] Step 5: Train the vehicle dangerous entry strategy model. First, initialize the Actor-Critic network parameters as Then, repeat the following steps for 1000 rounds:

[0061] (1) Repeat the following steps 1024 times: ① Start and run the simulation scenario described in step 1, process the bird's-eye view of the simulation scenario according to the method described in step 2, and use the processed bird's-eye view as the input of the strategy model, that is, The state at the moment is recorded as .

[0062] ②Get the current state of the Actor-Critic network The output value of , as The action selection at the moment, the reward is calculated according to the method described in step 3 .

[0063] ③Execute action , extract according to the method described in step 2 Status of the moment .

[0064] ④The current state Updated to .

[0065] ⑤Store the records generated by the interaction between the vehicle and the simulation environment during the whole process, that is ( ).

[0066] ⑥ When the vehicle encounters a termination state (such as a collision, leaving the lane, etc.), it starts again from step ①.

[0067] (2) Calculate the advantage value of all state actions in the interaction record according to the following formula: (11) (12) (13) In the formula, is the discount factor, which is taken as 0.99.

[0068] (3) Repeat the following steps 4 times, and perform the calculations in batches according to the following steps, with a batch size of 64.

[0069] ①Calculate the Actor’s loss (14) In the formula, , the clip function intercepts the calculation result so that it is range, The value is 0.2.

[0070] ②Calculate the loss of the Critic network (15) ③Calculate the total loss of the Actor-Critic network (16) In the formula, Take 0.5 and 0.01 respectively. It’s a strategy The entropy value of .

[0071] ④ Using the calculated loss, use the Adam optimizer to perform gradient updates to train the parameters of the Actor-Critic network, and the learning rate used is 0.0003.

[0072] Step 6: After completing the model training described in step 5, the vehicle dangerous entry strategy model is obtained, which can be further applied to the safety performance test of the AVs system.

[0073] Step seven, after completing the model training described in step five, obtain the vehicle dangerous cut-in strategy model. In order to use the vehicle dangerous cut-in strategy model constructed in this example for AVs safety performance testing, a three-lane simulation environment is established, in which a multi-agent test environment containing two agents is constructed, wherein one agent uses the intelligent driving strategy to be tested to control the vehicle to be tested, and its color is set to black, and the other agent controls the dangerously cut-in traffic vehicle through the vehicle dangerous cut-in strategy obtained through training, and its color is set to white, with the vehicle to be tested as the target vehicle. Other traffic vehicles in the environment are controlled by the built-in traffic vehicle model of SMARTS, and the color is gray. During the test, the traffic vehicle controlled by the vehicle dangerous cut-in strategy will actively and dangerously cut in front of the vehicle to be tested, causing the vehicle to be tested to have a rear-end collision risk, which can play a role in safety testing the control strategy of the vehicle to be tested.

[0074] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for constructing a vehicle danger entry strategy based on bird's-eye view input, characterized in that: The following steps are involved: Step 1: Build a simulation scenario for training the vehicle dangerous entry strategy model; Step 2: Process the bird's-eye view input obtained from the simulation scene, and use the processed bird's-eye view as the input of the vehicle dangerous entry strategy model; Step 3: Define the reward function, which includes the immediate reward for dangerous entry, the reward for guiding dangerous entry, and the penalty for improper behavior; Step 4: Construct a vehicle danger entry strategy model based on PPO; Step 5: Train the vehicle dangerous entry strategy model; Step 6: After completing the model training described in step 5, the vehicle danger entry strategy model is obtained and applied to the safety performance test of the AVs system.

2. The method for constructing a vehicle dangerous entry strategy based on a bird's-eye view input according to claim 1, characterized in that: In the step 1, a traffic vehicle is randomly selected as the self-vehicle in the simulation scene, another traffic vehicle is selected as the target vehicle, the selected target vehicle is set to black, and the self-vehicle is set to white.

3. The method for constructing a vehicle dangerous entry strategy based on a bird's-eye view input according to claim 2, characterized in that: The step 2 also includes calculating and obtaining the orientation, horizontal and vertical coordinate positions of the vehicle and the path points of the vehicle, and converting the path points into image pixel coordinates.

4. The method for constructing a vehicle dangerous entry strategy based on a bird's-eye view input according to claim 1, characterized in that: The processing of the bird's-eye view includes replacing the color of the valid path points in the image with green transparent color. If there is a target position, it is converted into image pixel coordinates, and the color of the corresponding target position in the image is replaced with purple. If it is necessary to remove unnecessary parts in the bird's-eye view, the image is cropped.

5. The method for constructing a vehicle dangerous entry strategy based on a bird's-eye view input according to claim 1, characterized in that: The danger cuts into instant rewards As shown in the following formula, It refers to the time when the vehicle cuts in. The collision time with the target vehicle, It refers to the relative speed between the ego vehicle and the target vehicle at the time of cut-in. （1） When the self-vehicle cuts into the lane where the target vehicle is located and is in front of the target vehicle, if , then give the model a larger reward; if If it does not fall into the above range, the relative speed between the self-vehicle and the target vehicle will be used. Determine the reward value, The larger it is, the higher the risk of the target car rear-ending the ego car, and the reward value increases accordingly.

6. The method for constructing a vehicle dangerous entry strategy based on a bird's-eye view input according to claim 5, characterized in that: The guide danger cut reward As follows: （2） (3) （4） In the formula, is the distance between the vehicle and the set target point, For The coefficient used for normalization, Distance reward; TT is the TTC discount factor, The coefficient used to normalize the collision time TTC, where , , is the distance between the ego vehicle and the target vehicle, is the longitudinal distance between the ego vehicle and the target vehicle, is the lateral distance between the ego vehicle and the target vehicle, is the relative speed between the ego vehicle and the target vehicle.

7. The method for constructing a vehicle dangerous entry strategy based on a bird's-eye view input according to claim 1, characterized in that: The improper behavior penalty is to prevent the vehicle from engaging in improper behaviors such as driving off the road, reversing, or actively colliding with traffic vehicles during driving. The improper behavior of the vehicle is punished by setting a negative reward value. .

8. The method for constructing a vehicle dangerous entry strategy based on a bird's-eye view input according to claim 7, characterized in that: The reward value Belongs to the reward function , the reward function for: （5） in, , , for , and The weight coefficients corresponding to the three rewards.

Images