Automatic driving lane change speed control method, computer device and storage medium

Abstract

The embodiment of the present invention discloses an automatic driving lane changing speed control method, a computer device and a storage medium, which obtain current driving information data of a lane changing vehicle, wherein the driving information data includes position information and speed information, and the lane changing vehicle is an automatic driving vehicle; based on the position information, the driving information data of surrounding vehicles of the lane changing vehicle in the original lane and the target lane are determined, wherein the surrounding vehicles include the front vehicle in the original lane of the lane changing vehicle and the front and rear vehicles in the target lane; based on the current driving information data of the lane changing vehicle and the driving information data of the surrounding vehicles, a trained deep reinforcement learning model is input, and the acceleration value and direction angle of the lane changing vehicle are output; in this way, in the mixed traffic flow of the future Internet of Vehicles, the automatic driving vehicle can fully perceive the driving status of surrounding vehicles in the mixed traffic flow, thereby improving the lane changing efficiency, thereby improving the road travel efficiency and safety.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving, and in particular to an autonomous driving lane change speed control method, a computer device and a storage medium. Background Art

[0002] Lane changing is a common driving behavior. Lane changing is complex and challenging for the driver because the lane changing vehicle needs to interact with other vehicles and he needs to complete lateral and longitudinal movements simultaneously when maneuvering. Once the driver makes an error in executing the lane change, it will lead to serious traffic accidents and have a great impact on the safety and stability of traffic flow.

[0003] Traditional lane-changing models include numerical lane-changing models and lane-changing models based on supervised learning. However, the traditional numerical lane-changing model uses a certain number of parameters to express lane-changing behavior, but these parameters are difficult to fully cover the diversity of driving styles and the complexity of driving scenarios. Therefore, the numerical lane-changing model is not suitable for complex environments. Secondly, the lane-changing model based on supervised learning lacks the ability to distinguish between good and bad behaviors, and can only imitate human behavior as perfectly as possible through supervised learning. Summary of the invention

[0004] In view of this, the present invention provides an automatic driving lane change speed control method, a computer device and a storage medium, so that an automatic driving vehicle can safely and efficiently complete lane changes between surrounding vehicles, so as to improve road driving safety and driving efficiency.

[0005] To achieve the above object, the technical solution of the present invention is achieved as follows:

[0006] In a first aspect, the present invention provides a method for controlling lane change speed of an automatic driving, the method comprising:

[0007] Acquiring current driving information data of the lane-changing vehicle, wherein the driving information data includes position information and speed information, and the lane-changing vehicle is an automatic driving vehicle;

[0008] Determine driving information data of surrounding vehicles of the lane-changing vehicle in the original lane and the target lane based on the position information, wherein the surrounding vehicles include the front vehicle in the original lane of the lane-changing vehicle and the front and rear vehicles in the target lane;

[0009] Based on the current driving information data of the lane-changing vehicle and the driving information data of the surrounding vehicles, a trained deep reinforcement learning model is input to output the acceleration value and direction angle of the lane-changing vehicle.

[0010] Wherein, the obtaining of current driving information data of the lane-changing vehicle includes:

[0011] The vehicle driving information data, including position information and speed information, is collected through cameras and road section detectors on the road sections in the target area, and the driving information data is transmitted to the lane-changing vehicle through a roadside communication unit.

[0012] Wherein, determining the driving information data of the surrounding vehicles of the lane-changing vehicle in the original lane and the target lane based on the position information includes:

[0013] determining lane position data of the lane-changing vehicle based on the position information of the lane-changing vehicle;

[0014] Determine an original lane and a target lane for lane change based on the lane position data and the lane change intention of the autonomous driving vehicle, and determine a starting point of the lane change process;

[0015] Obtain driving information data of the front vehicle in the current lane and the front and rear vehicles in the target lane at the starting time, and build a lane change simulation environment for autonomous driving vehicles.

[0016] The step of constructing a lane change simulation environment for an autonomous driving vehicle includes:

[0017] Determine that the intelligent agent in the deep reinforcement learning model is an autonomous vehicle, and the action space is the acceleration value and direction angle of the autonomous vehicle [a v , a d ], the state space is the relative longitudinal speed and longitudinal distance between the lane-changing vehicle and the surrounding vehicles, the lateral speed of the lane-changing vehicle, and the lateral moving distance of the lane-changing vehicle [Δv y_lcs , Δv y_lts , Δv y_fts , Δy lcs , Δy lts , Δy fts , v x_s , x s -x o ], the state is updated according to the kinematic principle, the formula is:

[0018] v x_s (t+1)=v x_s (t)+a v (t)*sin a d (t)*ΔT

[0019] v y_s (t+1)=v y_s (t)+a v (t)*cos a d (t)*ΔT

[0020]

[0021]

[0022] Among them, s, lc, lt, ft represent the autonomous driving vehicle, the front vehicle in the original lane, the front vehicle in the current lane, and the rear vehicle in the target lane, respectively, and ΔT is the measurement time step.

[0023] Among them, the network structure of the deep reinforcement learning model includes a policy network, two value function networks and two Q function networks.

[0024] Among them, it also includes:

[0025] The reward function is designed to guide the autonomous driving vehicle to complete acceleration and deceleration within a given lateral range during the lane change process, and its lateral speed is less than the given range when the lane change is completed. The given lateral range is determined according to the center line of the lane, and the given final lateral speed is determined according to the straight-line driving requirements and the road speed limit. The reward function designs corresponding reward values ​​for the lateral movement position and lateral speed or lateral acceleration at different stages of the lane change process.

[0026] Before inputting the current driving information data of the lane-changing vehicle and the driving information data of the surrounding vehicles into the trained deep reinforcement learning model, the method further includes:

[0027] Inputting the driving information data of the lane-changing vehicle and surrounding vehicles into an initial deep reinforcement learning model for training, and updating various network parameters;

[0028] The deep reinforcement learning model is iteratively updated until the set reward function meets the convergence condition, thereby obtaining a trained deep reinforcement learning model.

[0029] The deep reinforcement learning model trained based on the current driving information data of the target vehicle, the current driving information data of the lane-changing vehicle, and the driving information data of surrounding vehicles is input to output the acceleration value and direction angle of the lane-changing vehicle, including:

[0030] Determining a current target state based on driving information data of the lane-changing vehicle and surrounding vehicles;

[0031] The trained deep reinforcement learning model is input based on the target state, and the acceleration value and direction angle of the lane-changing vehicle are output.

[0032] In a second aspect, the present invention provides a computer device, comprising: a processor and a memory for storing a computer program that can be run on the processor;

[0033] Among them, when the processor is used to run the computer program, it implements the automatic driving lane change speed control method described above.

[0034] In a third aspect, the present invention provides a computing storage medium, wherein a computer program is stored in the computing storage medium, and the computer program is executed by a processor to implement any of the above-mentioned automatic driving lane change speed control methods.

[0035] The automatic driving lane changing speed control method, computer device and storage medium provided by the embodiments of the present invention include: taking the front vehicle in the original lane of the lane changing vehicle and the front and rear vehicles in the target lane as the surrounding vehicles in the lane changing process of the automatic driving vehicle, and constructing an automatic lane changing driving environment by acquiring position information and speed information; determining the action space as the value of acceleration and direction angle based on the deep reinforcement learning algorithm, and the state space as the relative longitudinal speed and longitudinal distance between the lane changing vehicle and the surrounding vehicles, the lateral speed of the lane changing vehicle and the lateral movement distance of the lane changing vehicle; guiding the automatic driving vehicle to complete the acceleration and deceleration process of lateral movement during the lane changing process by designing the reward function in the reinforcement learning algorithm, and introducing a collision avoidance strategy to ensure that the automatic driving vehicle will not collide with surrounding vehicles during the longitudinal movement; inputting the current driving information data of the automatic driving vehicle and the driving information data of the surrounding vehicles into the trained deep reinforcement learning model, and outputting the value of acceleration and the corresponding direction angle during the lane changing process, so as to control the lane changing speed of the automatic driving vehicle. In this way, in the mixed traffic flow conditions of the future Internet of Vehicles, autonomous driving vehicles can fully perceive the driving status of surrounding vehicles in the mixed traffic flow, thereby improving lane changing efficiency, thereby improving road travel efficiency and safety; the method of the present invention is simple in design and easy to calculate; the lane changing model of autonomous driving vehicles based on deep reinforcement learning can overcome the defects of existing lane changing models, improve the lane changing efficiency of autonomous driving vehicles, and effectively improve the travel efficiency of road traffic. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 A schematic flow chart of a method for controlling lane change speed of an automatic driving system provided by an embodiment of the present invention;

[0037] Figure 2 A schematic flow chart of another automatic driving lane change speed control method provided by an embodiment of the present invention;

[0038] Figure 3 A schematic diagram of the network structure of a deep reinforcement learning model provided by an embodiment of the present invention;

[0039] Figure 4 A schematic diagram of the structure of an automatic driving lane change speed control device provided by an embodiment of the present invention;

[0040] Figure 5 A schematic diagram of the structure of a computer device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0041] The technical solution of the present invention is further elaborated in detail below in conjunction with the accompanying drawings and specific embodiments of the specification. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those generally understood by those skilled in the art of the technical field of the present invention. The terms used in the specification of the present invention herein are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "and / or" used herein includes any and all combinations of one or more related listed items.

[0042] With the continuous iteration and update of artificial intelligence, the application of reinforcement learning methods to study the control strategy of autonomous driving has become a new hot spot. Compared with the control strategy based on supervised learning, reinforcement learning enables the intelligent agent to interact with the environment for trial and error learning, with the goal of maximizing rewards. The driving process of an autonomous vehicle is similar to the Markov process in reinforcement learning. The autonomous vehicle observes the surrounding vehicle environment to make decisions and executes the strategy to update the environment. Therefore, based on a large amount of natural driving data, the application of deep reinforcement learning methods has certain reference value for the lane change behavior control of autonomous vehicles.

[0043] See also Figure 1 , is an automatic driving lane change speed control method provided by an embodiment of the present invention, which can be applied to any vehicle terminal equipped with automatic driving. The method can be executed by a computer device, which can be a terminal or a server, etc. The terminal can be a desktop computer, a laptop computer, a smart phone, a personal digital assistant or a tablet computer, etc. The server can be a single server device or a server cluster, etc. The automatic driving lane change speed control method includes the following steps:

[0044] Step 101: Acquire current driving information data of a lane-changing vehicle, wherein the driving information data includes position information and speed information, and the lane-changing vehicle is an automatic driving vehicle. ;

[0045] In one embodiment, the step of obtaining the current driving information data of the target vehicle includes:

[0046] The vehicle driving information data, including position and speed information, is collected through cameras and road section detectors on the road sections in the target area, and the driving information data is transmitted to the lane-changing vehicle through a roadside communication unit.

[0047] Determine driving information data of surrounding vehicles of the lane-changing vehicle based on the position information, wherein the surrounding vehicles include a front vehicle in the original lane of the lane-changing vehicle and front and rear vehicles in the target lane;

[0048] determining lane position data of the lane-changing vehicle based on the position information of the lane-changing vehicle;

[0049] Determine an original lane and a target lane for lane change based on the lane position data and the lane change intention of the autonomous driving vehicle, and determine a starting point of the lane change process;

[0050] Obtain driving information data of the front vehicle in the current lane and the front and rear vehicles in the target lane at the starting time, and build a lane change simulation environment for autonomous driving vehicles.

[0051] Step 102: Determine driving information data of surrounding vehicles of the lane-changing vehicle in the original lane and the target lane based on the position information, wherein the surrounding vehicles include the front vehicle in the original lane of the lane-changing vehicle and the front and rear vehicles in the target lane.

[0052] In one embodiment, the step of constructing a lane change simulation environment for an autonomous driving vehicle includes:

[0053] Determine that the intelligent agent in the deep reinforcement learning model is an autonomous vehicle, and the action space is the acceleration value and direction angle of the autonomous vehicle [a v , a d ], the state space is the relative longitudinal speed and longitudinal distance between the lane-changing vehicle and the surrounding vehicles, the lateral speed of the lane-changing vehicle, and the lateral moving distance of the lane-changing vehicle [Δv y_lcs , Δv y_lts , Δv y_fts , Δy lcs , Δy lts , Δy fts , v x_s , x s -x o ], the state is updated according to the kinematic principle, the formula is:

[0054] v x_s (t+1)=v x_s (t)+a v (t)*sin a d (t)*ΔT

[0055] v y_s (t+1)=v y_s (t)+a v (t)*cos a d (t)*ΔT

[0056]

[0057]

[0058]

[0059] Among them, s, lc, lt, ft represent the autonomous driving vehicle, the front vehicle in the original lane, the front vehicle in the current lane, and the rear vehicle in the target lane, respectively, and ΔT is the measurement time step.

[0060] Step 103: Based on the current driving information data of the lane-changing vehicle and the driving information data of the surrounding vehicles, a trained deep reinforcement learning model is input to output the acceleration value and direction angle of the lane-changing vehicle.

[0061] In one embodiment, it further comprises:

[0062] The reward function is designed to guide the autonomous driving vehicle to complete the acceleration and deceleration movement within a given lateral range during the lane change process, and its lateral speed is less than the given range when the lane change is completed. The given lateral range is determined according to the center line of the lane, and the given final lateral speed is determined according to the straight-line driving requirements and the road speed limit. The reward function designs corresponding reward values ​​for the lateral movement position and lateral speed or lateral acceleration at different stages of the lane change process, thereby guiding the autonomous driving vehicle to learn how to complete the acceleration and deceleration process of lateral movement within a given range.

[0063] The determination of the reward value for each step in the lane change process is divided into four cases. First, during the lane change process, if the lateral position of the autonomous driving vehicle is less than the initial lateral position, a constant penalty value is given; when the lateral movement distance of the autonomous driving vehicle does not exceed half of the maximum lateral distance, the vehicle enters the acceleration process in the lateral direction, and a reward and penalty value r is given according to the comparison between the lateral speed and the final lateral speed range. vx =f(v x_s )=ω 1 *v x_s +b 1 , and r td =f(x s -x o ), the total reward value is r = r vx +r td When the lateral movement distance of the autonomous driving vehicle exceeds half of the maximum lateral distance but is less than the maximum lateral distance, the vehicle enters the deceleration process in lateral movement, and the reward and penalty value r=f(a x_s )=ω 2 *a x_s +b 2 , and r td =f(x s -x o ), the total reward value is r = r vx +r td ; If the lateral movement distance of the autonomous driving vehicle exceeds the maximum lateral distance, a corresponding penalty value will be given according to the excess range.

[0064] It also includes the introduction of a collision avoidance strategy to ensure that the autonomous vehicle does not collide with surrounding vehicles during longitudinal movement. The collision avoidance strategy compares the longitudinal distance of the vehicle with the safety distance and assigns corresponding acceleration, deceleration and angle values ​​to the autonomous vehicle according to the situation. The following situations are divided into the following situations. The remaining situations are assigned according to the input of the model:

[0065] (1)(Δy lcs <D sf_lcs orΔy lts <D sf_lts )andΔy fts >D sf_fts , a v =-3m / s 2 , a d =0°;

[0066] (2)(Δy lcs >D sf_lcs andΔy lts >D sf_lts )andΔy fts <D sf_fts , a v =3m / s 2 , a d =0°;

[0067] (3)(Δy lcs <D sf_lcs orΔy lts <D sf_lts )andΔy fts <D sf_fts ,

[0068] min(Δy lcs / Δv y_lcs , Δy lts / Δv y_lts )<Δy fts / Δv y_fts , a v =-3m / s 2 , ad=0°

[0069] min(Δy lcs / Δv y_lcs , Δy lts / Δv y_lts )≥Δy fts / Δv y_fts , a v =3m / s 2 , a d =0°

[0070] Step 104: Based on the current driving information data of the lane-changing vehicle and the driving information data of the surrounding vehicles, the trained deep reinforcement learning model is input to output the acceleration value and direction angle of the lane-changing vehicle;

[0071] In one embodiment, the network structure of the deep reinforcement learning model includes a policy network, two value function networks and two Q function networks.

[0072] In one embodiment, before inputting the current driving information data of the lane-changing vehicle and the driving information data of the surrounding vehicles into the trained deep reinforcement learning model, the method further includes:

[0073] Inputting the driving information data of the lane-changing vehicle and surrounding vehicles into an initial deep reinforcement learning model for training, and updating various network parameters;

[0074] The deep reinforcement learning model is iteratively updated until the set reward function meets the convergence condition, thereby obtaining a trained deep reinforcement learning model.

[0075] In one embodiment, the inputting of the trained deep reinforcement learning model based on the current driving information data of the lane-changing vehicle and the driving information data of surrounding vehicles to output the acceleration value and direction angle of the lane-changing vehicle includes:

[0076] Determining a current target state based on driving information data of the lane-changing vehicle and surrounding vehicles;

[0077] The trained deep reinforcement learning model is input based on the target state, and the acceleration value and direction angle of the lane-changing vehicle are output.

[0078] Through the above-mentioned embodiments of the present invention, in the mixed traffic flow conditions of the future Internet of Vehicles, the autonomous driving vehicle can fully perceive the driving status of the surrounding vehicles in the mixed traffic flow, thereby improving the lane changing efficiency, thereby improving the road travel efficiency and safety; the method of the present invention is simple in design and easy to calculate; the lane changing model of the autonomous driving vehicle based on deep reinforcement learning can overcome the defects of the existing lane changing model, can improve the lane changing efficiency of the autonomous driving vehicle, and effectively improve the travel efficiency of road traffic.

[0079] For example, see Figure 2 , described through a specific embodiment.

[0080] Step 201: using cameras and road section detectors on the road section of the target area to collect vehicle driving information data, including position and speed information, and transmitting the driving information data to the lane-changing vehicle through a roadside communication unit;

[0081] Step 202: Determine the lane position data of the lane-changing vehicle based on the position information of the lane-changing vehicle, determine the original lane and the target lane of the lane change based on the lane position data and the lane-changing intention of the autonomous driving vehicle, determine the starting point of the lane-changing process, and obtain the driving information data of the front vehicle in the current lane and the front and rear vehicles in the target lane at the starting point, and build a lane-changing simulation environment for the autonomous driving vehicle;

[0082] Step 203: According to the lane change simulation environment of the autonomous driving vehicle, determine that the intelligent agent in the deep reinforcement learning model is the autonomous driving vehicle, and the action space is the acceleration value and direction angle of the autonomous driving vehicle [a v , a d ], the state space is the relative longitudinal speed and longitudinal distance between the lane-changing vehicle and the surrounding vehicles, the lateral speed of the lane-changing vehicle, and the lateral moving distance of the lane-changing vehicle [Δv y_lcs , Δv y_lts , Δv y_fts , Δy lcs , Δy lts , Δy fts , v x_s , x s -x o ], the state is updated according to the kinematic principle, the formula is:

[0083] v x_s (t+1)=v x_s (t)+a v (t)*sin a d (t)*ΔT

[0084] v y_s (t+1)=v y_s (t)+a v (t)*cos a d (t)*ΔT

[0085]

[0086]

[0087]

[0088] Among them, s, lc, lt, ft represent the autonomous driving vehicle, the front vehicle in the original lane, the front vehicle in the current lane, and the rear vehicle in the target lane, respectively, and ΔT is the measurement time step.

[0089] Step 204: Based on the above states and corresponding actions, a reward function is designed to guide the autonomous driving vehicle to complete acceleration and deceleration within a given lateral range during the lane change process, and its lateral speed is less than the given range when the lane change is completed. The given lateral range is determined according to the center line of the lane, and the given final lateral speed is determined according to the straight-line driving requirements and the road speed limit. The reward function guides the autonomous driving vehicle to learn how to complete the acceleration and deceleration process of lateral movement within a given range by designing corresponding reward values ​​for the lateral movement position and lateral speed or lateral acceleration at different stages of the lane change process;

[0090] The determination of the reward value for each step in the lane change process is divided into four cases. First, during the lane change process, if the lateral position of the autonomous driving vehicle is less than the initial lateral position, a constant penalty value is given; when the lateral movement distance of the autonomous driving vehicle does not exceed half of the maximum lateral distance, the vehicle enters the acceleration process in the lateral direction, and a reward and penalty value r = f(v x_s ); When the lateral movement distance of the autonomous driving vehicle exceeds half of the maximum lateral distance but is less than the maximum lateral distance, the vehicle enters a deceleration process in lateral motion, and a reward and penalty value r=f(a x_s ); If the lateral movement distance of the autonomous driving vehicle exceeds the maximum lateral distance, a corresponding penalty value is given according to the excess range.

[0091] Step 205: Determine the network structure of the policy network, two value networks, and two Q networks in the deep reinforcement learning SAC algorithm. The network parameter of the policy network is φ, the input is the state, and the output is the mean and variance of the Gaussian distribution. The action and its corresponding entropy are sampled by the reparameterization method; the network parameter of the value function is ψ, the input is the state, and the output is the value function; the network parameter of the Q function is θ, the input is the state and action, and the output is the state-action value. The algorithm contains five networks, each of which has two hidden layers and 256 neurons in each hidden layer. The network structure of the automatic driving lane change model based on the SAC algorithm is as follows: Figure 3 .

[0092] Step 206: According to the automatic driving lane change simulation environment in step 202, the driving environment is initialized, and the state is input into the policy network, and the corresponding action and entropy are input, so that the state at the current moment is updated to the next moment, and the reward at the current moment is obtained, and the experience (s t , a t , r t ,s t+1 ) is stored in the experience replay buffer. The function of the experience replay buffer is to store the experience of the agent during exploration and to sample it with the minimum batch number during the algorithm learning process to update the network parameters;

[0093] Step 207: A lane change operation performed by the autonomous driving vehicle in the driving environment is a lane change round. When the autonomous driving vehicle collides, completes the lane change, or fails to change lanes, it is considered the termination condition of the round. When the experience of the agent's exploration is sufficient to update the parameters of each network, the process of network parameter learning begins. The value function network and the Q function network are both based on the mean square error between the value obtained by the optimal Bellman equation and the network input value to update the network. The policy network is a network that minimizes the KL divergence of the action. Therefore, the value function network calculates the gradient Update the network, and the target value function network is based on Update the network; the Q function network calculates the gradient Update the network; the policy network calculates the gradient

[0094] Step 208: Based on the completion of the lane change round, the lane change round is reinitialized, and iterative updates are continuously performed until the SAC algorithm reaches convergence.

[0095] The method uses traffic flow detection equipment to obtain vehicle driving information in the target area, and constructs a lane changing environment consisting of the front vehicle in the original lane of the lane changing vehicle and the front and rear vehicles in the target lane to simulate the mixed traffic flow of future autonomous driving vehicles and human-driven vehicles; the Soft Actor-Critic (SAC) algorithm in deep reinforcement learning is used as the speed control strategy algorithm of the autonomous driving vehicle in the lane changing process, and the action space in the SAC framework is defined as the value and direction angle of acceleration, and the state space is the state space as the relative longitudinal speed and longitudinal distance between the lane changing vehicle and the surrounding vehicles, the lateral speed of the lane changing vehicle, and the lateral movement distance of the lane changing vehicle; a reward function based on the state and the corresponding action is designed to guide the autonomous driving vehicle to complete the lateral movement of the lane change within a given range, and reduce the final lateral speed to a given range; at the same time, a collision avoidance strategy is introduced to ensure that the autonomous driving vehicle does not collide with the surrounding vehicles during the longitudinal movement; the operation data of the vehicles around the lane changing vehicle collected in the target area is used as the external input of the model, and the lane changing vehicle driven by humans is replaced by the autonomous driving vehicle, which is used as the training set data of the model. In the mixed traffic flow conditions of future Internet of Vehicles, autonomous driving vehicles will be able to fully perceive the driving status of surrounding vehicles in the mixed traffic flow, thereby improving lane changing efficiency, thereby improving road travel efficiency and safety; the method of the present invention is simple in design and easy to calculate; the lane changing model of autonomous driving vehicles based on deep reinforcement learning can overcome the defects of existing lane changing models, improve the lane changing efficiency of autonomous driving vehicles, and effectively improve the travel efficiency of road traffic.

[0096] The embodiment of the present invention also provides an automatic driving lane change speed control device, such as Figure 4 As shown, the device comprises:

[0097] An acquisition module 51 is used to acquire current driving information data of the lane-changing vehicle, wherein the driving information data includes position information and speed information, and the lane-changing vehicle is an automatic driving vehicle;

[0098] A determination module 52 determines driving information data of surrounding vehicles of the lane-changing vehicle in the original lane and the target lane based on the position information, wherein the surrounding vehicles include a front vehicle in the original lane of the lane-changing vehicle and front and rear vehicles in the target lane;

[0099] The output module 53 is used to output the acceleration value and direction angle of the lane-changing vehicle based on the current driving information data of the lane-changing vehicle and the driving information data of the surrounding vehicles input into the trained deep reinforcement learning model.

[0100] In an optional embodiment, the acquisition module 51 is further used to:

[0101] The vehicle driving information data, including position and speed information, is collected through cameras and road section detectors on the road sections in the target area, and the driving information data is transmitted to the lane-changing vehicle through a roadside communication unit.

[0102] In an optional embodiment, the determining module 52 is further configured to:

[0103] determining lane position data of the lane-changing vehicle based on the position information of the lane-changing vehicle;

[0104] Determine an original lane and a target lane for lane change based on the lane position data and the lane change intention of the autonomous driving vehicle, and determine a starting point of the lane change process;

[0105] Obtain driving information data of the front vehicle in the current lane and the front and rear vehicles in the target lane at the starting time, and build a lane change simulation environment for autonomous driving vehicles.

[0106] In an optional embodiment, the determining module 52 is further configured to:

[0107] Determine that the intelligent agent in the deep reinforcement learning model is an autonomous vehicle, and the action space is the acceleration value and direction angle of the autonomous vehicle [a v , a d ], the state space is the relative longitudinal speed and longitudinal distance between the lane-changing vehicle and the surrounding vehicles, the lateral speed of the lane-changing vehicle, and the lateral moving distance of the lane-changing vehicle [Δv y_lcs , Δv y_lts , Δv y_fts , Δy lcs , Δy lts , Δy fts , v x_s , xs -x o ], the state is updated according to the kinematic principle, the formula is:

[0108] v x_s (t+1)=v x_s (t)+a v (t)*sin a d (t)*ΔT

[0109] v y_s (t+1)=v y_s (t)+a v (t)*cos a d (t)*ΔT

[0110]

[0111]

[0112]

[0113] Among them, s, lc, lt, ft represent the autonomous driving vehicle, the front vehicle in the original lane, the front vehicle in the current lane, and the rear vehicle in the target lane, respectively, and ΔT is the measurement time step.

[0114] Among them, v represents the speed, y represents the longitudinal position, s represents the autonomous driving vehicle, l represents the front vehicle, f represents the rear vehicle, and ΔT is the measurement time step.

[0115] In an optional embodiment, the device further includes a control module, configured to:

[0116] The reward function is designed to guide the autonomous driving vehicle to complete the acceleration and deceleration movement within a given lateral range during the lane change process, and its lateral speed is less than the given range when the lane change is completed. The given lateral range is determined according to the center line of the lane, and the given final lateral speed is determined according to the straight-line driving requirements and the road speed limit. The reward function designs corresponding reward values ​​for the lateral movement position and lateral speed or lateral acceleration at different stages of the lane change process, thereby guiding the autonomous driving vehicle to learn how to complete the acceleration and deceleration process of lateral movement within a given range.

[0117] The determination of the reward value for each step in the lane change process is divided into four cases. First, during the lane change process, if the lateral position of the autonomous driving vehicle is less than the initial lateral position, a constant penalty value is given; when the lateral movement distance of the autonomous driving vehicle does not exceed half of the maximum lateral distance, the vehicle enters the acceleration process in the lateral direction, and a reward and penalty value r = f(v x_s); When the lateral movement distance of the autonomous driving vehicle exceeds half of the maximum lateral distance but is less than the maximum lateral distance, the vehicle enters a deceleration process in lateral motion, and a reward and penalty value r=f(a x_s ); If the lateral movement distance of the autonomous driving vehicle exceeds the maximum lateral distance, a corresponding penalty value is given according to the excess range.

[0118] At the same time, a collision avoidance strategy is introduced to ensure that the autonomous vehicle does not collide with surrounding vehicles during longitudinal movement. The collision avoidance strategy compares the longitudinal distance of the vehicle with the safety distance and assigns corresponding acceleration, deceleration and angle values ​​to the autonomous vehicle according to the situation. The following situations are divided into the following situations. The remaining situations are assigned according to the input of the model:

[0119] (1)(Δy lcs <D sf_lcs orΔy lts <D sf_lts )andΔy fts >D sf_fts , a v =-3m / s 2 , a d =0°;

[0120] (2)(Δy lcs >D sf_lcs andΔy lts >D sf_lts )andΔy fts <D sf_fts , a v =3m / s 2 , a d =0°;

[0121] (3)(Δy lcs <D sf_lcs orΔy lts <D sf_lts )andΔy fts <D sf_fts ,

[0122] min(Δy lcs / Δv y_lcs , Δy lts / Δv y_lts )<Δy fts / Δv y_fts , a v =-3m / s 2 , a d =0°

[0123] min(Δy lcs / Δv y_lcs , Δy lts / Δvy_lts )≥Δy fts / Δv y_fts , a v =3m / s 2 , a d =0°

[0124] In an optional embodiment, the device further comprises a training module, which is used to:

[0125] Inputting the driving information data of the lane-changing vehicle and surrounding vehicles into an initial deep reinforcement learning model for training, and updating various network parameters;

[0126] Based on iteratively updating the deep reinforcement learning model until the set reward function meets the convergence condition, a trained deep reinforcement learning model is obtained.

[0127] In an optional embodiment, the output module 53 is further used for:

[0128] Determining a current target state based on driving information data of the lane-changing vehicle and surrounding vehicles;

[0129] The trained deep reinforcement learning model is input based on the target state, and the acceleration value and direction angle of the lane-changing vehicle are output.

[0130] It should be noted that: the automatic driving lane change speed control device provided in the above embodiment only uses the division of the above program modules as an example to illustrate when implementing the automatic driving lane change speed control method. In actual applications, the above processing can be assigned to different program modules as needed, that is, the internal structure of the online car-hailing driver safety evaluation device is divided into different program modules to complete all or part of the processing described above. In addition, the automatic driving lane change speed control device provided in the above embodiment and the corresponding automatic driving lane change speed control embodiment belong to the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0131] An embodiment of the present invention provides a computer device, such as Figure 5 As shown, the computer device includes: a processor 110 and a memory 111 for storing a computer program that can be run on the processor 110; wherein, Figure 5 The processor 110 shown in the figure is not used to indicate that the number of the processor 110 is one, but is only used to indicate the positional relationship of the processor 110 relative to other devices. In actual applications, the number of the processor 110 may be one or more; similarly, Figure 5 The memory 111 shown in the figure has the same meaning, that is, it is only used to refer to the position relationship of the memory 111 relative to other devices. In actual applications, the number of memories 111 can be one or more.

[0132] When the processor 110 is used to run the computer program, the processor 110 performs the following steps:

[0133] Acquiring current driving information data of the lane-changing vehicle, wherein the driving information data includes position information and speed information, and the lane-changing vehicle is an automatic driving vehicle;

[0134] Determine driving information data of surrounding vehicles of the lane-changing vehicle based on the position information, wherein the surrounding vehicles include a front vehicle in the original lane of the lane-changing vehicle and front and rear vehicles in the target lane;

[0135] Based on the current driving information data of the lane-changing vehicle and the driving information data of the surrounding vehicles, a trained deep reinforcement learning model is input to output the acceleration value and direction angle of the lane-changing vehicle.

[0136] In an optional embodiment, the processor 110 is further configured to execute the following steps when running the computer program:

[0137] The vehicle driving information data, including position and speed information, is collected through cameras and road section detectors on the road sections in the target area, and the driving information data is transmitted to the lane-changing vehicle through a roadside communication unit.

[0138] In an optional embodiment, the processor 110 is further configured to execute the following steps when running the computer program:

[0139] determining lane position data of the lane-changing vehicle based on the position information of the lane-changing vehicle;

[0140] Determine an original lane and a target lane for lane change based on the lane position data and the lane change intention of the autonomous driving vehicle, and determine a starting point of the lane change process;

[0141] Obtain driving information data of the front vehicle in the current lane and the front and rear vehicles in the target lane at the starting time, and build a lane change simulation environment for autonomous driving vehicles.

[0142] In an optional embodiment, the processor 110 is further configured to execute the following steps when running the computer program:

[0143] Determine that the intelligent agent in the deep reinforcement learning model is an autonomous vehicle, and the action space is the acceleration value and direction angle of the autonomous vehicle [a v , a d ], the state space is the relative longitudinal speed and longitudinal distance between the lane-changing vehicle and the surrounding vehicles, the lateral speed of the lane-changing vehicle, and the lateral moving distance of the lane-changing vehicle [Δv y_lcs , Δv y_lts , Δv y_fts , Δylcs , Δy lts , Δy fts , v x_s , x s -x o ], the state is updated according to the kinematic principle, the formula is:

[0144] v x_s (t+1)=v x_s (t)+a v (t)*sin a d (t)*ΔT

[0145] v y_s (t+1)=v y_s (t)+a v (t)*cos a d (t)*ΔT

[0146]

[0147]

[0148]

[0149] Among them, s, lc, lt, ft represent the autonomous driving vehicle, the front vehicle in the original lane, the front vehicle in the current lane, and the rear vehicle in the target lane, respectively, and ΔT is the measurement time step.

[0150] In an optional embodiment, the processor 110 is further configured to execute the following steps when running the computer program:

[0151] The reward function is designed to guide the autonomous driving vehicle to complete the acceleration and deceleration movement within a given lateral range during the lane change process, and its lateral speed is less than the given range when the lane change is completed. The given lateral range is determined according to the center line of the lane, and the given final lateral speed is determined according to the straight-line driving requirements and the road speed limit. The reward function designs corresponding reward values ​​for the lateral movement position and lateral speed or lateral acceleration at different stages of the lane change process, thereby guiding the autonomous driving vehicle to learn how to complete the acceleration and deceleration process of lateral movement within a given range.

[0152] The determination of the reward value for each step in the lane change process is divided into four cases. First, during the lane change process, if the lateral position of the autonomous driving vehicle is less than the initial lateral position, a constant penalty value is given; when the lateral movement distance of the autonomous driving vehicle does not exceed half of the maximum lateral distance, the vehicle enters the acceleration process in the lateral direction, and a reward and penalty value r = f(v x_s); When the lateral movement distance of the autonomous driving vehicle exceeds half of the maximum lateral distance but is less than the maximum lateral distance, the vehicle enters a deceleration process in lateral motion, and a reward and penalty value r=f(a x_s ); If the lateral movement distance of the autonomous driving vehicle exceeds the maximum lateral distance, a corresponding penalty value is given according to the excess range.

[0153] In an optional embodiment, the processor 110 is further configured to execute the following steps when running the computer program:

[0154] The collision avoidance strategy is introduced to ensure that the autonomous vehicle does not collide with surrounding vehicles during longitudinal movement. The collision avoidance strategy compares the longitudinal distance of the vehicle with the safety distance and assigns corresponding acceleration, deceleration and angle values ​​to the autonomous vehicle according to the situation. It is divided into the following situations, and the rest of the situations are assigned according to the input of the model:

[0155] (1)(Δy lcs <D sf_lcs orΔy lts <D sf_lts )andΔy fts >D sf_fts , a v =-3m / s 2 , a d =0°;

[0156] (2)(Δy lcs >D sf_lcs andΔy lts >D sf_lts )andΔy fts <D sf_fts , a v =3m / s 2 , a d =0°;

[0157] (3)(Δy lcs <D sf_lcs orΔy lts <D sf_lts )andΔy fts <D sf_fts ,

[0158] min(Δy lcs / Δv y_lcs , Δy lts / Δv y_lts )<Δy fts / Δv y_fts , a v =-3m / s 2 , a d =0°

[0159] min(Δy lcs / Δv y_lcs , Δy lts / Δv y_lts )≥Δy fts / Δv y_fts , a v =3m / s 2 , a d =0°

[0160] In an optional embodiment, the processor 110 is further configured to execute the following steps when running the computer program:

[0161] Inputting the driving information data of the lane-changing vehicle and surrounding vehicles into an initial deep reinforcement learning model for training, and updating various network parameters;

[0162] The deep reinforcement learning model is iteratively updated until the set reward function meets the convergence condition, thereby obtaining a trained deep reinforcement learning model.

[0163] In an optional embodiment, the processor 110 is further configured to execute the following steps when running the computer program:

[0164] Determining a current target state based on driving information data of the lane-changing vehicle and surrounding vehicles;

[0165] The trained deep reinforcement learning model is input based on the target state, and the acceleration value and direction angle of the lane-changing vehicle are output.

[0166] The computer device also includes: at least one network interface 112. The various components in the device are coupled together via a bus system 113. It is understood that the bus system 113 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 113 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, Figure 3 Various buses are labeled as bus system 113 .

[0167] The memory 111 may be a volatile memory or a nonvolatile memory, or may include both volatile and nonvolatile memories. The nonvolatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a ferromagnetic random access memory (FRAM), a flash memory, a magnetic surface memory, an optical disk, or a compact disc read-only memory (CD-ROM); the magnetic surface memory may be a disk memory or a tape memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), synchronous static random access memory (SSRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM, SyncLink Dynamic Random Access Memory), and direct RAM bus random access memory (DRRAM, Direct Rambus Random Access Memory).The memory 111 described in the embodiments of the present invention is intended to include but is not limited to these and any other suitable types of memories.

[0168] The memory 111 in the embodiment of the present invention is used to store various types of data to support the operation of the device. Examples of these data include: any computer program used to operate on the device, such as an operating system and an application; contact data; phone book data; messages; pictures; videos, etc. Among them, the operating system includes various system programs, such as a framework layer, a core library layer, a driver layer, etc., which are used to implement various basic services and process hardware-based tasks. The application program may include various applications, such as a media player (Media Player), a browser (Browser), etc., which are used to implement various application services. Here, the program that implements the method of the embodiment of the present invention may be included in the application program.

[0169] This embodiment also includes a computer storage medium, in which a computer program is stored. The computer storage medium may be a ferromagnetic random access memory (FRAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a flash memory, a magnetic surface memory, an optical disk, or a compact disc read-only memory (CD-ROM); or it may be various devices including one or any combination of the above memories, such as a mobile phone, a computer, a tablet device, a personal digital assistant, etc. When the computer program stored in the computer storage medium is executed by the processor, the above vehicle identification method is implemented. For the specific steps implemented when the computer program is executed by the processor, please refer to Figure 1 The description of the illustrated embodiment will not be repeated here.

[0170] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0171] In this document, the terms "comprises," "comprising," or any other variations thereof, are intended to cover a non-exclusive inclusion of elements other than those listed and may also include additional elements not expressly listed.

[0172] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A method for controlling lane change speed in an automatic driving vehicle, characterized in that: The method comprises: Acquiring current driving information data of the lane-changing vehicle, wherein the driving information data includes position information and speed information, and the lane-changing vehicle is an automatic driving vehicle; Determine driving information data of surrounding vehicles of the lane-changing vehicle in the original lane and the target lane based on the position information, wherein the surrounding vehicles include the front vehicle in the original lane of the lane-changing vehicle and the front and rear vehicles in the target lane; Based on the current driving information data of the lane-changing vehicle and the driving information data of the surrounding vehicles, a trained deep reinforcement learning model is input to output the acceleration value and direction angle of the lane-changing vehicle; Determine the lane position data of the lane-changing vehicle based on the position information of the lane-changing vehicle, determine the original lane and the target lane of the lane change based on the lane position data and the lane-changing intention of the autonomous driving vehicle, determine the starting point of the lane-changing process, and obtain the driving information data of the front vehicle in the current lane and the front and rear vehicles in the target lane at the starting point, and build a lane-changing simulation environment for the autonomous driving vehicle; According to the lane change simulation environment of the autonomous driving vehicle, the intelligent agent in the deep reinforcement learning model is determined to be the autonomous driving vehicle, and the action space is the acceleration value a of the autonomous driving vehicle. v and direction angle a d , the state space is the relative longitudinal speed and longitudinal distance between the autonomous driving vehicle and the surrounding vehicles, the lateral speed of the autonomous driving vehicle, and the lateral movement distance of the autonomous driving vehicle [Δv y_lcs , Δv y_lts , Δv y_fts , Δy lc_s ,Δy lt_s , Δy ft_s ,v x_s , x s -x o ], the state is updated according to the kinematic principle, the formula is: v x_s (t+1)=v x_s (t)+a v (t)*sin a d (t)*ΔT v y_s (t+1)=v y_s (t)+a v (t)*cos a d (t)*ΔT Where s is the autonomous driving vehicle, lc is the front vehicle in the original lane, lt is the front vehicle in the target lane, ft is the rear vehicle in the target lane, ΔT is the measurement time step; Δv y represents the relative longitudinal velocity, Δy represents the relative longitudinal distance, v x represents the lateral velocity, x represents the lateral position; Δv y_lcs Indicates the relative longitudinal speed between the vehicle in front and the vehicle in the original lane, Δv y_lts Indicates the relative longitudinal speed between the vehicle in front of the target lane and the vehicle itself, Δv y_fts Indicates the relative longitudinal speed between the vehicle behind the target lane and the vehicle itself, Δy lc_s Indicates the relative longitudinal distance between the vehicle in front of the original lane and the vehicle itself, Δy lt_s Indicates the relative longitudinal distance between the vehicle in front of the target lane and the vehicle itself, Δy ft_s Indicates the relative longitudinal distance between the vehicle in front of the target lane and the vehicle itself, v x_s represents the lateral speed of the vehicle, x s -x o Indicates the lateral displacement of the vehicle; According to the above-mentioned states and corresponding actions, a reward function is designed to guide the autonomous driving vehicle to complete acceleration and deceleration movements within a given lateral range during the lane change process, and its lateral speed is less than the given range when the lane change is completed; wherein, the given lateral range is determined according to the center line of the lane, and the given final lateral speed is determined according to the straight-line driving requirements and the road speed limit, and the reward function designs corresponding reward values ​​for the lateral movement position and lateral speed or lateral acceleration at different stages of the lane change process.

2. The automatic driving lane change speed control method according to claim 1, characterized in that: Before inputting the current driving information data of the lane-changing vehicle and the driving information data of the surrounding vehicles into the trained deep reinforcement learning model, the method further includes: Inputting the driving information data of the lane-changing vehicle and surrounding vehicles into an initial deep reinforcement learning model for training, and updating various network parameters; The deep reinforcement learning model is iteratively updated until the set reward function meets the convergence condition, thereby obtaining a trained deep reinforcement learning model.

3. The automatic driving lane change speed control method according to claim 1, characterized in that: The method of inputting a trained deep reinforcement learning model based on the current driving information data of the lane-changing vehicle and the driving information data of surrounding vehicles to output the acceleration value and direction angle of the lane-changing vehicle includes: Determining a current target state based on driving information data of the lane-changing vehicle and surrounding vehicles; The trained deep reinforcement learning model is input based on the target state, and the acceleration value and direction angle of the lane-changing vehicle are output.

4. A computer device, characterized in that: include: a processor and a memory for storing a computer program capable of being executed on the processor; Wherein, when the processor is used to run the computer program, it implements the automatic driving lane change speed control method described in any one of claims 1 to 3.

5. A computer storage medium, characterized in that: The computer storage medium stores a computer program, wherein the computer program is executed by a processor to implement the automatic driving lane change speed control method according to any one of claims 1 to 3.

Images