Vehicle lane changing decision-making method and device, electronic equipment and storage medium

Abstract

The invention relates to a vehicle lane changing decision-making method and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining at least two types of environment data collected for the surrounding environment of a target vehicle, and obtaining the vehicle state data of the target vehicle; for each type of environment data, inputting the environment data into at least two corresponding weak learners to obtain an environment recognition result set corresponding to the type of environment data; inputting the vehicle state data into at least two corresponding weak learners to obtain a state recognition result set corresponding to the target vehicle; obtaining input data from each environment recognition result set and each state recognition result set, and inputting the input data into a reinforcement learning model to obtain a corresponding longitudinal acceleration and a corresponding steering angle change rate; and according to the longitudinal acceleration and the steering angle change rate, a lane changing decision instruction corresponding to the target vehicle is generated. Therefore, a better lane changing decision is made.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving technology, and in particular to a vehicle lane change decision method, device, electronic device and storage medium. Background Art

[0002] Autonomous vehicles (Self-driving automobile) are intelligent vehicles that are unmanned by computer systems. Autonomous vehicles rely on artificial intelligence, visual computing, radar, monitoring devices and global positioning systems to work together, allowing computers to automatically and safely operate motor vehicles without any active human operation.

[0003] Lane changing decision is one of the important functions in the autonomous driving system. An effective lane changing decision system can not only improve the driving safety of the vehicle, but also improve the smoothness and comfort of driving. At present, most lane changing decision methods make lane changing decisions through single modal input data (for example: camera images, lidar data, GPS information, etc.).

[0004] However, this method of making lane change decisions through single-modal input data is easily affected by factors such as weather and lighting, resulting in reduced system performance or misjudgment. Summary of the invention

[0005] In order to solve the above technical problems or at least partially solve the above technical problems, the present application provides a vehicle lane changing decision method, device, electronic device and storage medium.

[0006] In a first aspect, the present application provides a vehicle lane change decision method, comprising:

[0007] Acquire at least two types of environmental data collected for the surrounding environment of the target vehicle, and acquire vehicle status data of the target vehicle;

[0008] For each type of environmental data, input the environmental data into at least two corresponding weak learners to obtain a set of environmental recognition results corresponding to the type of environmental data;

[0009] Inputting the vehicle state data into at least two corresponding weak learners to obtain a state recognition result set corresponding to the target vehicle;

[0010] Acquire input data from each of the environment recognition result sets and the state recognition result sets, and input the input data into a reinforcement learning model to obtain corresponding longitudinal acceleration and steering angle change rate;

[0011] A lane change decision instruction corresponding to the target vehicle is generated according to the longitudinal acceleration and the steering angle change rate.

[0012] In a possible implementation manner, the acquiring input data from each of the environment recognition result sets and the state recognition result sets includes:

[0013] Determine at least two data collection moments;

[0014] For each environment recognition result set, obtain the environment recognition data collected at each data collection moment from the environment recognition result set, and combine at least two of the environment recognition data in a time-ordered order to obtain environment time series recognition data;

[0015] From the state recognition result set, obtain the state recognition data collected at each data collection moment, and combine at least two of the state recognition data in a time-ordered order to obtain state time series recognition data;

[0016] At least two of the environment timing identification data and the state timing identification data are used as the input data.

[0017] In a possible implementation, determining at least two data collection moments includes:

[0018] Obtain the target time, preset time step and vehicle lane change duration;

[0019] Determining a cut-off time according to the target time and the lane-changing duration of the vehicle;

[0020] Starting from the target moment, determining an intermediate moment at every preset time step until reaching the cutoff moment;

[0021] The target moment and all the intermediate moments are used as data collection moments.

[0022] In a possible implementation manner, generating a lane change decision instruction corresponding to the target vehicle according to the longitudinal acceleration and the steering angle change rate includes:

[0023] Obtaining the current vehicle speed and current steering angle corresponding to the target vehicle;

[0024] Determining a start time and an end time among at least two of the data collection times;

[0025] The time difference between the start time and the end time is determined as the target duration;

[0026] Performing a setting operation on the current vehicle speed, the longitudinal acceleration and the target duration to obtain a target vehicle speed;

[0027] Performing a setting operation on the current steering angle, the steering angle change rate and the target duration to obtain a target steering angle;

[0028] A lane change decision instruction corresponding to the target vehicle is generated according to the longitudinal acceleration, the target vehicle speed, the steering angle change rate and the target steering angle.

[0029] In a possible implementation manner, generating a lane change decision instruction corresponding to the target vehicle according to the longitudinal acceleration, the target vehicle speed, the steering angle change rate and the target steering angle includes:

[0030] Obtain preset correction conditions;

[0031] Correcting the longitudinal acceleration, the target vehicle speed, the steering angle change rate and the target steering angle according to the correction condition to obtain corresponding corrected longitudinal acceleration, corrected vehicle speed, corrected steering angle change rate and corrected steering angle;

[0032] A lane change decision instruction corresponding to the target vehicle is generated according to the corrected longitudinal acceleration, the corrected vehicle speed, the corrected steering angle change rate and the corrected steering angle.

[0033] In a possible implementation manner, the environmental data includes image data and point cloud data, and the acquiring of at least two types of environmental data collected for the surrounding environment of the target vehicle includes:

[0034] The image data is collected by a plurality of image collection devices arranged on the target vehicle, and the point cloud data is collected by a plurality of radars arranged on the target vehicle.

[0035] In a possible implementation, the method further includes:

[0036] In the process of training the reinforcement learning model, the lane change prediction data corresponding to the output result of the reinforcement learning model is obtained, wherein the lane change prediction data includes the comfort of the target vehicle after changing lanes according to the output result, the lane distance between the target vehicle and the center line of the lane line of the changed lane, the safety distance between the target vehicle and the preceding vehicle, and the vehicle collision condition of the target vehicle;

[0037] Determining a reward value for the lane change prediction data according to a reward function corresponding to the reinforcement learning model, wherein the reward value includes a comfort reward value corresponding to the comfort level, a lane distance reward value corresponding to the lane distance, a safety distance reward value corresponding to the safety distance, and a collision reward value corresponding to the vehicle collision situation;

[0038] The model parameters of the reinforcement learning model are adjusted according to the reward value until the reward value reaches the target reward value, and the model training ends.

[0039] In a second aspect, the present application provides a vehicle lane change decision device, comprising:

[0040] A data acquisition module, used to acquire at least two types of environmental data collected for the surrounding environment of the target vehicle, and to acquire vehicle status data of the target vehicle;

[0041] A first input module is used to input each type of environmental data into at least two corresponding weak learners to obtain an environmental recognition result set corresponding to the type of environmental data;

[0042] A second input module, used for inputting the vehicle state data into at least two corresponding weak learners to obtain a state recognition result set corresponding to the target vehicle;

[0043] A third input module, used for obtaining input data from each of the environment recognition result set and the state recognition result set, and inputting the input data into the reinforcement learning model to obtain corresponding longitudinal acceleration and steering angle change rate;

[0044] The instruction generation module is used to generate a lane change decision instruction corresponding to the target vehicle according to the longitudinal acceleration and the steering angle change rate.

[0045] In a possible implementation manner, the third input module is specifically used to:

[0046] Determine at least two data collection moments;

[0047] For each environment recognition result set, obtain the environment recognition data collected at each data collection moment from the environment recognition result set, and combine at least two of the environment recognition data in a time-ordered order to obtain environment time series recognition data;

[0048] From the state recognition result set, obtain the state recognition data collected at each data collection moment, and combine at least two of the state recognition data in a time-ordered order to obtain state time series recognition data;

[0049] At least two of the environment timing identification data and the state timing identification data are used as the input data.

[0050] In a possible implementation manner, the third input module is further used to:

[0051] Obtain the target time, preset time step and vehicle lane change duration;

[0052] Determining a cut-off time according to the target time and the lane-changing duration of the vehicle;

[0053] Starting from the target moment, determining an intermediate moment at every preset time step until reaching the cutoff moment;

[0054] The target moment and all the intermediate moments are used as data collection moments.

[0055] In a possible implementation, the instruction generation module is specifically used to:

[0056] Obtaining the current vehicle speed and current steering angle corresponding to the target vehicle;

[0057] Determining a start time and an end time among at least two of the data collection times;

[0058] The time difference between the start time and the end time is determined as the target duration;

[0059] Performing a setting operation on the current vehicle speed, the longitudinal acceleration and the target duration to obtain a target vehicle speed;

[0060] Performing a setting operation on the current steering angle, the steering angle change rate and the target duration to obtain a target steering angle;

[0061] A lane change decision instruction corresponding to the target vehicle is generated according to the longitudinal acceleration, the target vehicle speed, the steering angle change rate and the target steering angle.

[0062] In a possible implementation, the instruction generation module is further used to:

[0063] Obtain preset correction conditions;

[0064] Correcting the longitudinal acceleration, the target vehicle speed, the steering angle change rate and the target steering angle according to the correction condition to obtain corresponding corrected longitudinal acceleration, corrected vehicle speed, corrected steering angle change rate and corrected steering angle;

[0065] A lane change decision instruction corresponding to the target vehicle is generated according to the corrected longitudinal acceleration, the corrected vehicle speed, the corrected steering angle change rate and the corrected steering angle.

[0066] In a possible implementation, the environmental data includes image data and point cloud data, and the data acquisition module is specifically used to:

[0067] The image data is collected by a plurality of image collection devices arranged on the target vehicle, and the point cloud data is collected by a plurality of radars arranged on the target vehicle.

[0068] In a possible implementation, the device further includes a model training module, which is used to:

[0069] In the process of training the reinforcement learning model, the lane change prediction data corresponding to the output result of the reinforcement learning model is obtained, wherein the lane change prediction data includes the comfort of the target vehicle after changing lanes according to the output result, the lane distance between the target vehicle and the center line of the lane line of the changed lane, the safety distance between the target vehicle and the preceding vehicle, and the vehicle collision condition of the target vehicle;

[0070] Determining a reward value for the lane change prediction data according to a reward function corresponding to the reinforcement learning model, wherein the reward value includes a comfort reward value corresponding to the comfort level, a lane distance reward value corresponding to the lane distance, a safety distance reward value corresponding to the safety distance, and a collision reward value corresponding to the vehicle collision situation;

[0071] The model parameters of the reinforcement learning model are adjusted according to the reward value until the reward value reaches the target reward value, and the model training ends.

[0072] In a third aspect, an electronic device is provided, comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;

[0073] Memory, used to store computer programs;

[0074] The processor is used to implement any method step described in the first aspect when executing a program stored in the memory.

[0075] In a fourth aspect, a computer-readable storage medium is provided, characterized in that a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, any method step described in the first aspect is implemented.

[0076] The above technical solution provided by the embodiment of the present application has the following advantages compared with the prior art:

[0077] The vehicle lane changing decision method, device, electronic device and storage medium provided in the embodiments of the present application first obtain at least two types of environmental data collected for the surrounding environment of the target vehicle, and obtain the vehicle state data of the target vehicle, then, for each type of environmental data, input the environmental data into at least two corresponding weak learners to obtain an environmental recognition result set corresponding to the type of environmental data, and input the vehicle state data into at least two corresponding weak learners to obtain a state recognition result set corresponding to the target vehicle, next, obtain input data from each environmental recognition result set and state recognition result set, and input the input data into a reinforcement learning model to obtain the corresponding longitudinal acceleration and steering angle change rate, and finally, generate a lane changing decision instruction corresponding to the target vehicle according to the longitudinal acceleration and the steering angle change rate. In the present application, firstly, by using at least two types of environmental data to identify the vehicle's surrounding environment, the accuracy of environmental recognition can be improved. Secondly, by sending the vehicle status data and various types of environmental data to various weak learners, outputting the recognition results, and then giving the recognition results of each weak learner as input to the reinforcement learning model, and further extracting features from the input results, the vehicle condition and the vehicle's surrounding environment can be better identified, thereby making better lane changing decisions. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0079] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0080] Figure 1 A flow chart of a vehicle lane changing decision method provided in an embodiment of the present application;

[0081] Figure 2 A flowchart of another vehicle lane change decision method provided in an embodiment of the present application;

[0082] Figure 3 A flowchart of another vehicle lane change decision method provided in an embodiment of the present application;

[0083] Figure 4 A flowchart of another vehicle lane change decision method provided in an embodiment of the present application;

[0084] Figure 5 A basic framework diagram of reinforcement learning provided in an embodiment of the present application;

[0085] Figure 6 An overall flow chart of a vehicle lane change decision method provided in an embodiment of the present application;

[0086] Figure 7 A schematic diagram of the structure of a vehicle lane change decision device provided in an embodiment of the present application;

[0087] Figure 8 A structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0088] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0089] To facilitate understanding of the embodiments of the present application, further explanation will be given below with reference to specific embodiments in conjunction with the accompanying drawings. The embodiments do not constitute a limitation on the embodiments of the present application.

[0090] See also Figure 1 , is a flow chart of an embodiment of a vehicle lane change decision method provided in an embodiment of the present application. Figure 1 As shown, the process may include the following steps:

[0091] Step 101: Acquire at least two types of environmental data collected for the surrounding environment of a target vehicle, and acquire vehicle status data of the target vehicle.

[0092] The above-mentioned environmental data refers to data collected from the surrounding environment of the target vehicle, including image data of image type and point cloud data of point cloud type.

[0093] Specifically, obtaining at least two types of environmental data collected for the surrounding environment of the target vehicle may include the following steps: collecting the image data through multiple image acquisition devices arranged on the target vehicle, and collecting the point cloud data through multiple radars arranged on the target vehicle.

[0094] In the application, image acquisition devices (such as cameras, cameras, etc.) and radars are installed at various positions on the body of the target vehicle. The image acquisition devices can collect image data (such as pictures, videos, etc.) of the vehicle's surroundings, and the radars can collect radar signals of the vehicle's surroundings, which are finally presented as point cloud data.

[0095] Since image data lacks depth information and point cloud data is always sparse, using only one type of data as the input of the subsequent algorithm will have low accuracy. The fusion of image type data and point cloud type data in this scheme can offset the shortcomings of a single type of data, thereby improving the accuracy of subsequent algorithm recognition.

[0096] Vehicle status data refers to the numerical information of the current vehicle status of the target vehicle, such as the total CAN (Controller Area Network) information of the vehicle body, speed, wheel speed, angular velocity, distance to the vehicle in front, etc. In the application, the numerical information of the current vehicle status is collected by sensors installed at various positions of the vehicle body.

[0097] Step 102: For each type of environmental data, input the environmental data into at least two corresponding weak learners to obtain a set of environmental recognition results corresponding to the type of environmental data.

[0098] The weak learners corresponding to the above image data refer to image-based target recognition algorithms, such as CNN (Convolutional Neural Networks), SVM (Support Vector Machine), decision tree, K-nearest neighbor algorithm, etc. The weak learners corresponding to the above point cloud data refer to point cloud-based target recognition algorithms, such as VoxelNet, PointNet, PointRCNN, and octree.

[0099] In an embodiment of the present application, an integrated learning idea is adopted, and some weak learners are used to perform primary feature extraction on the collected environmental data. Specifically, the image data are respectively input into multiple picture-type target recognition algorithms, and each picture-type target recognition algorithm outputs a corresponding environment recognition result, thereby obtaining multiple environment recognition results to form an environment recognition result set. Similarly, the point cloud data are respectively input into multiple point cloud-type target recognition algorithms, and each point cloud-type target recognition algorithm outputs a corresponding environment recognition result, thereby obtaining multiple environment recognition results to form an environment recognition result set.

[0100] It should be noted that in the application, the algorithm corresponding to each type of environmental data has the following three requirements: 1) Different algorithms have the same or similar output labels; 2) The algorithm is applicable to the data to be labeled; 3) Each algorithm has a different network architecture or recognition difference. For example, an algorithm is more accurate in recognizing a certain category, but not so good in other categories or scenarios. Other algorithms should have the function of making up for such "weaknesses" of the algorithm to a certain extent, thereby meeting the requirement of "good but different" for each algorithm in ensemble learning.

[0101] Step 103: input the vehicle state data into at least two corresponding weak learners to obtain a state recognition result set corresponding to the target vehicle.

[0102] In the embodiment of the present application, the above-mentioned integrated learning idea is also adopted, and the vehicle state data is input into at least two corresponding weak learners, and each weak learner outputs a corresponding state recognition result, thereby obtaining multiple state recognition results to form a state recognition result set.

[0103] Step 104: Obtain input data from each of the environment recognition result sets and the state recognition result sets, and input the input data into a reinforcement learning model to obtain corresponding longitudinal acceleration and steering angle change rate.

[0104] Step 105: Generate a lane change decision instruction corresponding to the target vehicle according to the longitudinal acceleration and the steering angle change rate.

[0105] For ease of understanding, step 104 and step 105 are described uniformly below:

[0106] The above reinforcement learning model is a model obtained through reinforcement learning training. Reinforcement learning is a learning method that learns how to maximize by interacting in an environment. Reinforcement learning includes an agent (the core of which is a neural network), an environment, and a reward function. The agent performs actions in the environment and obtains rewards based on the reward function. The goal of the agent is to learn a strategy that maximizes the cumulative reward in the environment. The core idea of ​​reinforcement learning is to gradually learn how to maximize the reward in different states through exploration and utilization.

[0107] Input data refers to data obtained at the same time or in the same time period in each recognition result set.

[0108] Based on this, in the embodiment of the present application, by giving the results of each weak learner recognition as input to the reinforcement learning model, the reinforcement learning model further extracts features from the input data, so as to better identify the vehicle situation and environmental conditions, and output more accurate decision information (i.e., longitudinal acceleration and steering angle change rate) based on the recognized situation. Finally, according to the longitudinal acceleration and steering angle change rate, a lane change decision instruction corresponding to the target vehicle is generated, so that the target vehicle is controlled to perform the lane change operation through the lane change decision instruction. Specifically, the output result is converted into operation information and connected to the vehicle's operating system, so that the direction, throttle and brake of the vehicle can be directly controlled to operate the vehicle to obtain the effect of assisted driving.

[0109] The technical solution provided by the embodiment of the present application is as follows: first, at least two types of environmental data collected for the surrounding environment of the target vehicle are obtained, and vehicle state data of the target vehicle is obtained; then, for each type of environmental data, the environmental data is input into at least two corresponding weak learners to obtain an environmental recognition result set corresponding to the type of environmental data; and the vehicle state data is input into at least two corresponding weak learners to obtain a state recognition result set corresponding to the target vehicle; next, input data is obtained from each environmental recognition result set and state recognition result set, and the input data is input into a reinforcement learning model to obtain corresponding longitudinal acceleration and steering angle change rate; finally, a lane change decision instruction corresponding to the target vehicle is generated according to the longitudinal acceleration and the steering angle change rate. In the present application, firstly, by using at least two types of environmental data to identify the vehicle's surrounding environment, the accuracy of environmental recognition can be improved. Secondly, by sending the vehicle status data and various types of environmental data to various weak learners, outputting the recognition results, and then giving the recognition results of each weak learner as input to the reinforcement learning model, and further extracting features from the input results, the vehicle condition and the vehicle's surrounding environment can be better identified, thereby making better lane changing decisions.

[0110] See also Figure 2 , is a flow chart of another embodiment of a vehicle lane change decision method provided in an embodiment of the present application. Figure 2 The process shown in the above Figure 1 Based on the process shown, how to obtain input data from each of the environment recognition result sets and the state recognition result sets is described. Figure 2 As shown, the process may include the following steps:

[0111] Step 201: Determine at least two data collection moments.

[0112] The data collection time refers to the time when the environmental data and the vehicle status data are collected. In the application, the time and frequency of collecting corresponding data by the image acquisition device, radar and sensor are the same, that is, the corresponding data is collected at the same time every preset time interval.

[0113] In an embodiment of the present application, step 201 may specifically include the following steps: obtaining a target moment, a preset time step, and a vehicle lane changing duration; determining a cutoff moment according to the target moment and the vehicle lane changing duration; determining an intermediate moment at every preset time step starting from the target moment until the cutoff moment is reached; and using the target moment and all the intermediate moments as data collection moments.

[0114] Among them, the target time can be any time when data is collected; the preset time step refers to the time interval for the image acquisition device, radar and sensor to collect data; the vehicle lane change duration refers to the time it takes for the vehicle to cut into other lanes from the current lane (such as two to three seconds).

[0115] In this implementation, first, the cutoff time is obtained by subtracting the vehicle lane changing time from the target time, then an intermediate time is obtained by calculating the target time minus a preset time step, and another intermediate time is obtained by calculating the target time minus two preset time steps, and so on, until the cutoff time is reached, thereby obtaining multiple data collection times.

[0116] Step 202: for each environment recognition result set, obtain the environment recognition data collected at each data collection moment from the environment recognition result set, and combine at least two of the environment recognition data in order of time to obtain environment time series recognition data.

[0117] Step 203: Obtain the state identification data collected at each data collection moment from the state identification result set, and combine at least two of the state identification data in a time-ordered order to obtain state timing identification data.

[0118] Step 204: Use at least two of the environment timing identification data and the state timing identification data as the input data.

[0119] For ease of understanding, steps 202 to 204 are described in a unified manner as follows:

[0120] In an embodiment of the present application, the environment identification data collected at each data collection moment is obtained from each environment identification result set respectively, and is sorted in chronological order to obtain environment timing identification data, and the state identification data collected at each data collection moment is obtained from the state identification result set, and is sorted in chronological order to obtain state timing identification data.

[0121] Specifically, the neural network output tensor of the nth weak learner at time t (i.e., target time) is defined as C t n ,This value reflects the recognition result after the current learner extracts the features. At the same time, considering the temporal nature of the scene, the preset time step is defined as τ to add historical states.

[0122] Therefore, the output results of each learner at time t are:

[0123]

[0124] Assuming a 2τ historical state, that is, the data collection time includes t, t-τ (i.e., the collection time before time t), and t-2τ (i.e., the two collection times before time t), the state st at time t is defined as follows:

[0125] s t =[T t ,T t-τ ,T t-2τ ]

[0126] The current state st at time t in the above formula consists of two parts: is the neural network output tensor of the nth weak learner at time t, which is used to describe the recognition result of the learner; T t-τ ,T t-2τ It is a record of history, used to introduce time series variation characteristics and extract historical experience.

[0127] pass Figure 2 The process shown can take into account the timing of the lane change scenario by adding historical states, thereby helping the vehicle to better identify changes in the surrounding environment and improve the accuracy of decision-making.

[0128] See also Figure 3 , is a flow chart of another embodiment of a vehicle lane change decision method provided in an embodiment of the present application. Figure 3 The process shown in the above Figure 1 Based on the process shown in FIG. 1 , a description is given of how to generate a lane change decision instruction corresponding to the target vehicle according to the longitudinal acceleration and the steering angle change rate. Figure 3 As shown, the process may include the following steps:

[0129] Step 301, obtaining the current speed and current steering angle corresponding to the target vehicle;

[0130] Step 302: Determine a start time and an end time in at least two of the data collection times;

[0131] Step 303: Determine the time difference between the start time and the end time as the target duration;

[0132] Step 304, performing a setting operation on the current vehicle speed, the longitudinal acceleration and the target duration to obtain a target vehicle speed;

[0133] Step 305, performing a setting operation on the current steering angle, the steering angle change rate and the target duration to obtain a target steering angle;

[0134] Step 306: Generate a lane change decision instruction corresponding to the target vehicle according to the longitudinal acceleration, the target vehicle speed, the steering angle change rate and the target steering angle.

[0135] For ease of understanding, steps 301 to 306 are described in a unified manner as follows:

[0136] For controlling the vehicle in the lane-changing scenario, the reinforcement learning model mainly learns how to adjust the speed and direction angle to reach the destination. Therefore, the output of the action is a two-dimensional vehicle control signal, one of which is the longitudinal vehicle acceleration, and the other is the steering angle change rate. When changing lanes, the longitudinal acceleration is mainly for the control of the vehicle's throttle and brake, and the steering angle change rate is for the control of the steering angle of the vehicle's steering wheel. The range of the longitudinal vehicle acceleration av is [-3m / s2,3m / s2]. Specifically, the value in the interval [-3m / s2,0] is a deceleration operation, and the value in [0,3m / s2] is an acceleration operation. Similarly, the steering angle is mainly for the amplitude control of the vehicle's lane change, which is equivalent to the angle of the steering wheel. The range of the steering angle change rate ay is [-3° / s,3° / s]. Specifically, the value in the interval [-3° / s,0] is a left turn operation, and the value in [0,3° / s] is a right turn operation. The steering angle should be constrained by speed. The faster the speed, the smaller the steering angle for changing lanes should be. When the speed is slower, the steering angle can be larger.

[0137] In the embodiment of the present application, the action space of reinforcement learning is designed as a continuous action space, and the DDPG algorithm is selected, so the action spaces av and ay are defined as follows: av∈[-3m / s2,3m / s2], ay∈[-3° / s,3° / s];

[0138] Therefore, the speed conversion formula is: V t+τ =V t +a v *τ;

[0139] Among them, V t+τ is the target vehicle speed, V t is the current vehicle speed, a v is the longitudinal acceleration, τ is the target duration;

[0140] The transformation formula of the direction angle is: θ t+τ =θ t +a y *τ;

[0141] Among them, θ t+τ is the target steering angle, θ t is the current steering angle, a y is the steering angle change rate, and τ is the target duration.

[0142] That is, in the embodiment of the present application, first, the target duration is obtained by calculating the time difference between the start time and the end time, and then, the current vehicle speed, longitudinal acceleration and target duration are set and calculated through the speed conversion formula to obtain the target vehicle speed, and, the current steering angle, steering angle change rate and target duration are set and calculated through the direction angle conversion formula to obtain the target steering angle, and finally, according to the longitudinal acceleration, target vehicle speed, steering angle change rate and target steering angle, the lane change decision instruction corresponding to the target vehicle is generated to guide the target vehicle to adjust the current vehicle speed to the target vehicle speed according to the longitudinal acceleration, and adjust the current steering angle to the target steering angle according to the steering angle change rate.

[0143] In the application, directly generating a lane change decision instruction based on the output of the reinforcement learning model may easily lead to illegal actions and cause danger. For example, when the vehicle speed is fast, a relatively large steering angle is output, which may easily lead to the danger of the vehicle rolling over. Or, the output acceleration makes the vehicle speed too fast, faster than the speed of the front vehicle in the target lane after the lane change, which may easily lead to rear-end collision, etc. Such output actions are dangerous actions and are determined to be illegal actions. To this end, in another embodiment of the present application, the lane change decision instruction corresponding to the target vehicle is generated based on the longitudinal acceleration, the target vehicle speed, the steering angle change rate and the target steering angle, and the following steps may also be included:

[0144] Acquire preset correction conditions; correct the longitudinal acceleration, the target vehicle speed, the steering angle change rate and the target steering angle according to the correction conditions to obtain corresponding corrected longitudinal acceleration, corrected vehicle speed, corrected steering angle change rate and corrected steering angle; generate a lane change decision instruction corresponding to the target vehicle according to the corrected longitudinal acceleration, the corrected vehicle speed, the corrected steering angle change rate and the corrected steering angle.

[0145] The above correction conditions can be set by the user according to actual needs, for example, the steering angle range corresponding to the vehicle speed, the target vehicle speed is less than the speed of the front vehicle after changing lanes, etc.

[0146] From the above description, it can be seen that in this embodiment, the longitudinal acceleration, target vehicle speed, steering angle change rate and target steering angle are corrected according to the correction conditions, and the final lane change decision instruction is generated according to the corrected longitudinal acceleration, corrected vehicle speed, corrected steering angle change rate and corrected steering angle. This can avoid the final generation of illegal actions and improve the safety of automatic driving.

[0147] pass Figure 3The process shown in the figure can calculate the target speed and target steering angle of the target vehicle at the end of the lane changing action according to the current speed, current steering angle, longitudinal acceleration, steering angle change rate and lane changing time (i.e., target time) of the target vehicle for the continuous action space, thereby generating the lane changing decision instruction corresponding to the target vehicle according to the longitudinal acceleration, target speed, steering angle change rate and target steering angle.

[0148] See also Figure 4 , is a flow chart of another embodiment of a vehicle lane change decision method provided in an embodiment of the present application. Figure 4 As shown, the process may include the following steps:

[0149] Step 401: in the process of training the reinforcement learning model, obtaining lane change prediction data corresponding to the output result of the reinforcement learning model, wherein the lane change prediction data includes the comfort of the target vehicle after changing lanes according to the output result, the lane distance between the target vehicle and the center line of the lane line of the changed lane, the safety distance between the target vehicle and the preceding vehicle, and the vehicle collision condition of the target vehicle;

[0150] Step 402: determining a reward value of the lane change prediction data according to a reward function corresponding to the reinforcement learning model, wherein the reward value includes a comfort reward value corresponding to the comfort level, a lane distance reward value corresponding to the lane distance, a safety distance reward value corresponding to the safety distance, and a collision reward value corresponding to the vehicle collision situation;

[0151] Step 403: Adjust the model parameters of the reinforcement learning model according to the reward value until the reward value reaches the target reward value, and the model training ends.

[0152] For ease of understanding, steps 401 to 403 are described in a unified manner as follows:

[0153] In the present application embodiment, Figure 5As shown in the figure, the reinforcement learning model includes an agent, an environment and a reward function. The agent performs actions in the environment and obtains rewards according to the reward function. The goal of the agent is to learn a strategy to maximize the cumulative reward in the environment. The core idea of ​​reinforcement learning is to gradually learn how to maximize the reward in different states through exploration and utilization. Among them, the reward function is divided into the following four parts: the first is the lane change reward and penalty, that is, the situation of the vehicle changing lanes, which is expressed as the center of the target vehicle is as close to the center line of the target lane line (that is, the lane line after the lane change) as possible; the second is the distance reward and penalty, which is to maintain a safe distance from the vehicle in front throughout the process; the third is the comfort reward and penalty, which usually depends on the rate of change of the longitudinal acceleration and the steering angle change rate, and the lane change process should be kept as stable as possible; the fourth is the collision penalty, and the vehicle will be punished greatly if it collides.

[0154] 1) Lane change reward formula:

[0155] r l =-abs[L t -Y]

[0156] Where abs means taking the absolute value, L t It represents the horizontal coordinate position of the center point of the target vehicle (the diagonal center point of the entire vehicle body as a rectangle) at the current time t, and Y represents the horizontal coordinate of the center line of the target lane line. The farther the vehicle is from the center line of the target lane, the smaller the value and the more severe the penalty. The closer the vehicle is to the center line of the target lane, the larger the value.

[0157] 2) Distance reward and penalty formula:

[0158]

[0159] where r d is the distance reward, D is the distance to the front car, if it is greater than the safe distance, you will get a reward, if it is less than the safe distance, you will be punished. The safe distance is generally related to the speed of the front car, and the maximum safe distance is defined as:

[0160]

[0161] Among them, V t is the current vehicle speed, t' is the human reaction time, u is the speed of the vehicle ahead, d s The basic safety threshold is generally 5m. It can be seen that the safety distance is proportional to the speed and inversely proportional to the speed of the vehicle in front. The key point is that when the lane-changing vehicle passes through the center line of the lane line during the lane-changing process, the vehicle in front of the lane-changing vehicle will jump to the vehicle in front of the target lane line.

[0162] 3) Comfort rewards and penalties:

[0163] Comfort s, depends on the rate of change of longitudinal acceleration and steering angle change rate.

[0164] 4) Collision rewards and penalties:

[0165] Among them, r c is the collision penalty, whose value is a binary reward, for example: when no collision occurs, r c is 0, when a collision occurs r c is -100.

[0166] Therefore, the entire reward and punishment function formula is as follows:

[0167] r t =w l r l +w d r d +w s r s +w c r c

[0168] Among them, w l ,w d ,w s ,w c is the penalty coefficient, which is used to adjust the proportion of each reward and punishment in the reward function. c r c In the reward function, it is two or three orders of magnitude smaller than other reward and penalty items, so that safety can be ensured.

[0169] pass Figure 4 The process shown can train the reinforcement learning model based on the reward function, so that the reinforcement learning model gradually learns how to obtain the maximum reward under different states, thereby improving the processing capability of the reinforcement learning model and improving the accuracy of the output results.

[0170] See also Figure 6 , is an overall flow chart of a vehicle lane change decision method provided in an embodiment of the present application. Figure 6 As shown, the process may include the following steps:

[0171] When a lane change start instruction is received (the start instruction comes from the driver's request to change lanes, the navigation route's request to change lanes, or the automatic driving path planning system), the vehicle lane change decision method of the present application is started and the intelligent lane change action decision begins.

[0172] Specifically, first, the camera image data, radar point cloud data and signal value data of the vehicle's own state are obtained through the camera, radar and sensor on the vehicle body. Then, the camera image data, radar point cloud data and signal value data are sent to various weak learners respectively, and the corresponding target detection and recognition results are output. Then, the temporal features are extracted from the results of each learner as input to the reinforcement learning agent, so that the reinforcement learning agent can further extract features from the input results. The agent trains its own model by interacting with the environment, so as to better identify the environmental conditions and output the corresponding actions. Finally, the action output by the reinforcement learning agent is corrected through the correct operation of professional drivers, and the corrected action is output to obtain the final lane change decision. In addition, the corrected action output can also be input into the simulation environment and calculated through the reward function, so as to further optimize the reinforcement learning agent.

[0173] In the present application, firstly, by using at least two types of environmental data to identify the vehicle's surrounding environment, the accuracy of environmental recognition can be improved. Secondly, by sending the vehicle status data and various types of environmental data to various weak learners, outputting the recognition results, and then giving the recognition results of each weak learner as input to the reinforcement learning model, and further extracting features from the input results, the vehicle condition and the vehicle's surrounding environment can be better identified, thereby making better lane changing decisions.

[0174] Based on the same technical concept, the embodiment of the present application also provides a vehicle lane change decision device, such as Figure 7 As shown, the device comprises:

[0175] A data acquisition module 71 is used to acquire at least two types of environmental data collected for the surrounding environment of the target vehicle, and to acquire vehicle status data of the target vehicle;

[0176] A first input module 72 is used to input each type of environmental data into at least two corresponding weak learners to obtain a set of environmental recognition results corresponding to the type of environmental data;

[0177] A second input module 73, used to input the vehicle state data into at least two corresponding weak learners to obtain a state recognition result set corresponding to the target vehicle;

[0178] A third input module 74, configured to obtain input data from each of the environment recognition result set and the state recognition result set, and input the input data into the reinforcement learning model to obtain corresponding longitudinal acceleration and steering angle change rate;

[0179] The instruction generation module 75 is used to generate a lane change decision instruction corresponding to the target vehicle according to the longitudinal acceleration and the steering angle change rate.

[0180] In a possible implementation manner, the third input module is specifically used to:

[0181] Determine at least two data collection moments;

[0182] For each environment recognition result set, obtain the environment recognition data collected at each data collection moment from the environment recognition result set, and combine at least two of the environment recognition data in a time-ordered order to obtain environment time series recognition data;

[0183] From the state recognition result set, obtain the state recognition data collected at each data collection moment, and combine at least two of the state recognition data in a time-ordered order to obtain state time series recognition data;

[0184] At least two of the environment timing identification data and the state timing identification data are used as the input data.

[0185] In a possible implementation manner, the third input module is further used to:

[0186] Obtain the target time, preset time step and vehicle lane change duration;

[0187] Determining a cut-off time according to the target time and the lane-changing duration of the vehicle;

[0188] Starting from the target moment, determining an intermediate moment at every preset time step until reaching the cutoff moment;

[0189] The target moment and all the intermediate moments are used as data collection moments.

[0190] In a possible implementation, the instruction generation module is specifically used to:

[0191] Obtaining the current vehicle speed and current steering angle corresponding to the target vehicle;

[0192] Determining a start time and an end time among at least two of the data collection times;

[0193] The time difference between the start time and the end time is determined as the target duration;

[0194] Performing a setting operation on the current vehicle speed, the longitudinal acceleration and the target duration to obtain a target vehicle speed;

[0195] Performing a setting operation on the current steering angle, the steering angle change rate and the target duration to obtain a target steering angle;

[0196] A lane change decision instruction corresponding to the target vehicle is generated according to the longitudinal acceleration, the target vehicle speed, the steering angle change rate and the target steering angle.

[0197] In a possible implementation, the instruction generation module is further used to:

[0198] Obtain preset correction conditions;

[0199] Correcting the longitudinal acceleration, the target vehicle speed, the steering angle change rate and the target steering angle according to the correction condition to obtain corresponding corrected longitudinal acceleration, corrected vehicle speed, corrected steering angle change rate and corrected steering angle;

[0200] A lane change decision instruction corresponding to the target vehicle is generated according to the corrected longitudinal acceleration, the corrected vehicle speed, the corrected steering angle change rate and the corrected steering angle.

[0201] In a possible implementation, the environmental data includes image data and point cloud data, and the data acquisition module is specifically used to:

[0202] The image data is collected by a plurality of image collection devices arranged on the target vehicle, and the point cloud data is collected by a plurality of radars arranged on the target vehicle.

[0203] In a possible implementation, the device further includes a model training module, which is used to:

[0204] In the process of training the reinforcement learning model, the lane change prediction data corresponding to the output result of the reinforcement learning model is obtained, wherein the lane change prediction data includes the comfort of the target vehicle after changing lanes according to the output result, the lane distance between the target vehicle and the center line of the lane line of the changed lane, the safety distance between the target vehicle and the preceding vehicle, and the vehicle collision condition of the target vehicle;

[0205] Determining a reward value for the lane change prediction data according to a reward function corresponding to the reinforcement learning model, wherein the reward value includes a comfort reward value corresponding to the comfort level, a lane distance reward value corresponding to the lane distance, a safety distance reward value corresponding to the safety distance, and a collision reward value corresponding to the vehicle collision situation;

[0206] The model parameters of the reinforcement learning model are adjusted according to the reward value until the reward value reaches the target reward value, and the model training ends.

[0207] Based on the same technical concept, the embodiment of the present application also provides an electronic device, such as Figure 8 As shown, it includes a processor 111, a communication interface 112, a memory 113 and a communication bus 114, wherein the processor 111, the communication interface 112, and the memory 113 communicate with each other through the communication bus 114.

[0208] Memory 113, used for storing computer programs;

[0209] The processor 111 is used to execute the program stored in the memory 113 to implement the following steps:

[0210] Acquire at least two types of environmental data collected for the surrounding environment of the target vehicle, and acquire vehicle status data of the target vehicle;

[0211] For each type of environmental data, input the environmental data into at least two corresponding weak learners to obtain a set of environmental recognition results corresponding to the type of environmental data;

[0212] Inputting the vehicle state data into at least two corresponding weak learners to obtain a state recognition result set corresponding to the target vehicle;

[0213] Acquire input data from each of the environment recognition result sets and the state recognition result sets, and input the input data into a reinforcement learning model to obtain corresponding longitudinal acceleration and steering angle change rate;

[0214] A lane change decision instruction corresponding to the target vehicle is generated according to the longitudinal acceleration and the steering angle change rate.

[0215] The communication bus mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0216] The communication interface is used for communication between the above electronic device and other devices.

[0217] The memory may include a random access memory (RAM) or a non-volatile memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0218] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0219] In another embodiment provided in the present application, a computer-readable storage medium is also provided, in which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned vehicle lane change decision methods are implemented.

[0220] In another embodiment provided in the present application, a computer program product including instructions is also provided, which, when executed on a computer, enables the computer to execute any one of the vehicle lane change decision methods in the above embodiments.

[0221] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0222] The foregoing is merely a specific embodiment of the present invention, which enables those skilled in the art to understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A vehicle lane-changing decision method, characterized in that: The method comprises: Acquire at least two types of environmental data collected for the surrounding environment of the target vehicle, and acquire vehicle status data of the target vehicle; For each type of environmental data, input the environmental data into at least two corresponding weak learners to obtain a set of environmental recognition results corresponding to the type of environmental data; Inputting the vehicle state data into at least two corresponding weak learners to obtain a state recognition result set corresponding to the target vehicle; Acquire input data from each of the environment recognition result sets and the state recognition result sets, and input the input data into a reinforcement learning model to obtain corresponding longitudinal acceleration and steering angle change rate; A lane change decision instruction corresponding to the target vehicle is generated according to the longitudinal acceleration and the steering angle change rate.

2. The method according to claim 1, characterized in that The acquiring input data from each of the environment recognition result sets and the state recognition result sets comprises: Determine at least two data collection moments; For each environment recognition result set, obtain the environment recognition data collected at each data collection moment from the environment recognition result set, and combine at least two of the environment recognition data in a time-ordered order to obtain environment time series recognition data; From the state recognition result set, obtain the state recognition data collected at each data collection moment, and combine at least two of the state recognition data in a time-ordered order to obtain state time series recognition data; At least two of the environment timing identification data and the state timing identification data are used as the input data.

3. The method according to claim 2, characterized in that The determining of at least two data collection moments comprises: Obtain the target time, preset time step and vehicle lane change duration; Determining a cut-off time according to the target time and the lane-changing duration of the vehicle; Starting from the target moment, determining an intermediate moment at every preset time step until reaching the cutoff moment; The target moment and all the intermediate moments are used as data collection moments.

4. The method according to claim 2, characterized in that: The step of generating a lane change decision instruction corresponding to the target vehicle according to the longitudinal acceleration and the steering angle change rate includes: Obtaining the current vehicle speed and current steering angle corresponding to the target vehicle; Determining a start time and an end time among at least two of the data collection times; The time difference between the start time and the end time is determined as the target duration; Performing a setting operation on the current vehicle speed, the longitudinal acceleration and the target duration to obtain a target vehicle speed; Performing a setting operation on the current steering angle, the steering angle change rate and the target duration to obtain a target steering angle; A lane change decision instruction corresponding to the target vehicle is generated according to the longitudinal acceleration, the target vehicle speed, the steering angle change rate and the target steering angle.

5. The method according to claim 4, characterized in that The generating a lane change decision instruction corresponding to the target vehicle according to the longitudinal acceleration, the target vehicle speed, the steering angle change rate and the target steering angle comprises: Obtain preset correction conditions; Correcting the longitudinal acceleration, the target vehicle speed, the steering angle change rate and the target steering angle according to the correction condition to obtain corresponding corrected longitudinal acceleration, corrected vehicle speed, corrected steering angle change rate and corrected steering angle; A lane change decision instruction corresponding to the target vehicle is generated according to the corrected longitudinal acceleration, the corrected vehicle speed, the corrected steering angle change rate and the corrected steering angle.

6. The method according to claim 1, characterized in that The environmental data includes image data and point cloud data, and the acquiring of at least two types of environmental data collected for the surrounding environment of the target vehicle includes: The image data is collected by a plurality of image collection devices arranged on the target vehicle, and the point cloud data is collected by a plurality of radars arranged on the target vehicle.

7. The method according to claim 1, characterized in that The method further comprises: In the process of training the reinforcement learning model, the lane change prediction data corresponding to the output result of the reinforcement learning model is obtained, wherein the lane change prediction data includes the comfort of the target vehicle after changing lanes according to the output result, the lane distance between the target vehicle and the center line of the lane line of the changed lane, the safety distance between the target vehicle and the preceding vehicle, and the vehicle collision condition of the target vehicle; Determining a reward value for the lane change prediction data according to a reward function corresponding to the reinforcement learning model, wherein the reward value includes a comfort reward value corresponding to the comfort level, a lane distance reward value corresponding to the lane distance, a safety distance reward value corresponding to the safety distance, and a collision reward value corresponding to the vehicle collision situation; The model parameters of the reinforcement learning model are adjusted according to the reward value until the reward value reaches the target reward value, and the model training ends.

8. A vehicle lane-changing decision device, characterized in that: The device comprises: A data acquisition module, used to acquire at least two types of environmental data collected for the surrounding environment of the target vehicle, and to acquire vehicle status data of the target vehicle; A first input module is used to input each type of environmental data into at least two corresponding weak learners to obtain an environmental recognition result set corresponding to the type of environmental data; A second input module, used for inputting the vehicle state data into at least two corresponding weak learners to obtain a state recognition result set corresponding to the target vehicle; A third input module, used for obtaining input data from each of the environment recognition result set and the state recognition result set, and inputting the input data into the reinforcement learning model to obtain corresponding longitudinal acceleration and steering angle change rate; The instruction generation module is used to generate a lane change decision instruction corresponding to the target vehicle according to the longitudinal acceleration and the steering angle change rate.

9. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; Memory, used to store computer programs; The processor is used to implement the vehicle lane changing decision method described in any one of claims 1-7 when executing the program stored in the memory.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a vehicle lane-changing decision method program, and when the vehicle lane-changing decision method program is executed by a processor, the steps of the vehicle lane-changing decision method described in any one of claims 1 to 7 are implemented.

Images