Vehicle driving decision-making method, device, equipment and medium

Abstract

The invention discloses a vehicle driving decision-making method and device, equipment and a medium, and relates to the technical field of automatic driving, and the method comprises the steps: obtaining a first driving state of a vehicle at a current moment; based on the first driving state, predicting a second driving state at the next moment obtained after the vehicle executes different actions in the preset action set; the preset action set comprises a longitudinal / transverse action set; respectively constructing a corresponding target reward function based on the first driving state and each second driving state, and inputting the first driving state, the second driving states, a preset action set, the target reward function and the driving paths of the surrounding vehicles into a preset POMDP model, so as to solve and output a target action corresponding to the optimal reward function by using the POMDP model; and controlling the vehicle to run by using the target action. The POMDP is applied to automatic driving, so that the uncertainty in the driving process is modeled, and more accurate actions are output to control vehicle driving.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving technology, and in particular to a vehicle driving decision method, device, equipment and medium. Background Art

[0002] In MDP (Markov Decision Process), the agent interacts with a given environment. Once the agent takes an action, the state of the external environment will change, and the agent will receive a reward. In this process, the agent's goal is to take the next action to maximize the long-term cumulative reward. In MDP, the state of the environment is completely known and determined.

[0003] During the process of vehicle autonomous driving, the driving intention of human drivers is usually dominated by subjective emotions, driving habits, and objective road and surrounding vehicle conditions, which are usually difficult to model together. Therefore, there is inherent uncertainty in human driving behavior, and there is noise in the results of the perception system. POMDP (Partially Observable Markov Decision Process) is an extension of MDP. Its observation of the environment is incomplete and noisy. For POMDP, the true state of the environment is unknown, so it can choose actions based on historical information, or retain the current estimate of the external environment and calculate the corresponding actions based on the known confidence distribution.

[0004] In summary, how to apply POMDP to the decision-making process of autonomous driving to model the uncertainty in the driving process and output more accurate actions to control vehicle driving is a problem that needs to be solved. Summary of the invention

[0005] In view of this, the purpose of the present invention is to provide a vehicle driving decision method, device, equipment and medium, which can apply POMDP to the decision-making process of autonomous driving to achieve modeling of uncertainty in the driving process, so as to output more accurate actions to control vehicle driving. The specific scheme is as follows:

[0006] In a first aspect, the present application discloses a vehicle driving decision method, comprising:

[0007] Obtain the first driving state of the vehicle at the current moment;

[0008] Predicting, based on the first driving state, a second driving state at the next moment obtained by the vehicle after executing different actions in a preset action set; wherein the preset action set includes a longitudinal action set and a lateral action set;

[0009] Based on the first driving state and each of the second driving states, a corresponding target reward function is constructed respectively, and the first driving state, the second driving state, the preset action set, the target reward function and the driving path of the surrounding vehicles are input into a preset POMDP model, so as to solve and output the target action corresponding to the optimal reward function by using the POMDP model;

[0010] The target action is used to control the driving of the vehicle.

[0011] Optionally, the preset action set includes a longitudinal action set and a lateral action set; the longitudinal action set includes acceleration motion, uniform motion and deceleration motion, and the lateral action set includes left lane change, lane keeping and right lane change;

[0012] Correspondingly, the predicting, based on the first driving state, of the second driving state at the next moment obtained by the vehicle after executing different actions in the preset action set, includes:

[0013] Selecting any one longitudinal action and one lateral action from the longitudinal action set and the lateral action set to form an action combination, and determining an acceleration value corresponding to each action combination;

[0014] Predicting, based on the first driving state and the acceleration value, a second driving state of the vehicle at the next moment after executing different action combinations;

[0015] Accordingly, the use of the target action to control the vehicle to travel includes:

[0016] The vehicle is controlled to travel using a target action combination corresponding to an optimal reward function.

[0017] Optionally, the obtaining of the first driving state of the vehicle at the current moment includes:

[0018] Acquire a first driving state of the vehicle and current driving states of surrounding vehicles detected by the sensor at the current moment;

[0019] Correspondingly, the process of acquiring the driving paths of the surrounding vehicles includes:

[0020] Predicting the arrival position of the surrounding vehicles at the next moment based on the current driving state of the surrounding vehicles;

[0021] A target trajectory is generated between the current position of the surrounding vehicle and the arrival position using a B-spline curve, and the target trajectory is used as a driving path of the surrounding vehicle.

[0022] Optionally, predicting the arrival position of the surrounding vehicles at the next moment based on the current driving state of the surrounding vehicles includes:

[0023] Determine the center point of each lane in front of the surrounding vehicles according to the current road structure, and select a number of target positions near the center point using a Gaussian sampling method; wherein the distance between the center point and the self-vehicle satisfies a preset distance condition;

[0024] Inputting each of the target positions and the current driving state into an observation model constructed based on a normal distribution function to obtain a probability value of each of the target positions;

[0025] The target position corresponding to the maximum probability value is determined as the arrival position of the surrounding vehicles at the next moment.

[0026] Optionally, the target reward function includes a first reward function for characterizing speed gain, a second reward function for characterizing smoothness gain, a third reward function for characterizing collision gain, a fourth reward function for characterizing target point gain, and a fifth reward function for characterizing spacing gain; the first driving state includes the current position and current speed of the vehicle; the second driving state includes the predicted position and predicted speed of the vehicle at the next moment;

[0027] Accordingly, constructing a corresponding target reward function based on the first driving state and each of the second driving states respectively includes:

[0028] constructing the first reward function using the predicted speed and a current vehicle speed limit determined based on the predicted position;

[0029] constructing the second reward function based on the action corresponding to the second driving state;

[0030] Constructing the third reward function based on the actual distance between the vehicle and the surrounding vehicles and the target safety distance;

[0031] Constructing the fourth reward function based on the relationship between the predicted position and the navigation endpoint;

[0032] Constructing the fifth reward function based on the actual distance between the vehicle and the current nearest preceding vehicle and a preset optimal distance;

[0033] The first reward function, the second reward function, the third reward function, the fourth reward function and the fifth reward function are weighted by using a weight coefficient to obtain a target reward function.

[0034] Optionally, the process of determining the target safety distance includes:

[0035] determining a first distance based on the current vehicle speed limit and the vehicle maximum acceleration;

[0036] determining a second distance based on a vehicle width and a vehicle length of the own vehicle, and determining a third distance based on a vehicle width and a vehicle length of the surrounding vehicles;

[0037] The first distance, the second distance and the third distance are summed to obtain the target safety distance.

[0038] Optionally, the process of determining the target safety distance includes:

[0039] determining a first distance based on the current vehicle speed limit and the vehicle maximum acceleration;

[0040] determining a second distance based on a vehicle width and a vehicle length of the own vehicle, and determining a third distance based on a vehicle width and a vehicle length of the surrounding vehicles;

[0041] The first distance, the second distance and the third distance are summed to obtain the target safety distance.

[0042] In a second aspect, the present application discloses a vehicle driving decision-making device, comprising:

[0043] A driving state acquisition module, used to acquire a first driving state of the vehicle at the current moment;

[0044] A driving state prediction module, used for predicting, based on the first driving state, a second driving state obtained by the vehicle at the next moment after executing different actions in a preset action set; wherein the preset action set includes a longitudinal action set and a lateral action set;

[0045] an action determination module, configured to construct a corresponding target reward function based on the first driving state and each of the second driving states, and input the first driving state, the second driving state, the preset action set, the target reward function and the driving paths of surrounding vehicles into a preset POMDP model, so as to solve and output a target action corresponding to the optimal reward function using the POMDP model;

[0046] A control module is used to control the driving of the vehicle using the target action.

[0047] In a third aspect, the present application discloses an electronic device, comprising:

[0048] Memory, used to store computer programs;

[0049] The processor is used to execute the computer program to implement the steps of the vehicle driving decision-making method disclosed above.

[0050] In a fourth aspect, the present application discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, the steps of the vehicle driving decision-making method disclosed above are implemented.

[0051] It can be seen that the present application obtains the first driving state of the self-vehicle at the current moment; based on the first driving state, predicts the second driving state of the self-vehicle at the next moment after executing different actions in the preset action set; wherein the preset action set includes a longitudinal action set and a lateral action set; based on the first driving state and each of the second driving states, respectively constructs a corresponding target reward function, and inputs the first driving state, the second driving state, the preset action set, the target reward function and the driving paths of the surrounding vehicles into a preset POMDP model, so as to use the POMDP model to solve and output the target action corresponding to the optimal reward function; and uses the target action to control the driving of the self-vehicle.

[0052] Beneficial effects: The present application first needs to obtain the first driving state of the vehicle at the current moment, and the present application sets a preset action set, which specifically includes the actions that the vehicle may take during driving, and then predicts the second driving state of the vehicle at the next moment after executing different actions in the preset action set based on the first driving state, thereby constructing a corresponding target reward function based on the first driving state at the current moment and each second driving state predicted at the next moment; wherein the preset action set includes a longitudinal action set and a lateral action set. Further, the first driving state, the second driving state, the preset action set, the target reward function and the driving path of the surrounding vehicles are input into a preset POMDP model, so as to solve and output the target action corresponding to the optimal reward function using the POMDP model, and then use the target action to control the driving of the vehicle. That is, the present application applies POMDP to the decision-making process of autonomous driving, and also takes into account the driving paths of surrounding vehicles, so as to achieve modeling of uncertainty in the driving process, thereby outputting more accurate actions to control vehicle driving. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0054] Figure 1 A flow chart of a vehicle driving decision method disclosed in this application;

[0055] Figure 2 A schematic diagram of predicting the driving paths of surrounding vehicles disclosed in this application;

[0056] Figure 3 A flow chart of a specific vehicle driving decision method disclosed in this application;

[0057] Figure 4 A schematic diagram of a target safety distance disclosed in this application;

[0058] Figure 5 A schematic diagram of a vehicle driving decision process disclosed in this application;

[0059] Figure 6 This is a schematic diagram of the structure of a vehicle driving decision-making device disclosed in this application;

[0060] Figure 7 This is a structural diagram of an electronic device disclosed in this application. DETAILED DESCRIPTION

[0061] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0062] During the process of vehicle autonomous driving, the driving intention of human drivers is usually dominated by subjective emotions, driving habits, and objective road and surrounding vehicle conditions, which are usually difficult to model together. Therefore, there is inherent uncertainty in human driving behavior, and there is noise in the results of the perception system. POMDP (Partially Observable Markov Decision Process) is an extension of MDP. Its observation of the environment is incomplete and noisy. For POMDP, the true state of the environment is unknown, so it can select actions based on historical information, or retain the current estimate of the external environment and calculate the corresponding actions based on the known confidence distribution. To this end, the embodiments of the present application disclose a vehicle driving decision method, device, equipment and medium, which can apply POMDP to the decision-making process of autonomous driving to achieve modeling of uncertainty in the driving process, thereby outputting more accurate actions to control vehicle driving.

[0063] See also Figure 1 As shown, the embodiment of the present application discloses a vehicle driving decision method, the method comprising:

[0064] Step S11: Obtain the first driving state of the vehicle at the current moment.

[0065] In this embodiment, it is first necessary to obtain the first driving state of the vehicle at the current moment.

[0066] In a specific implementation, the above-mentioned obtaining the first driving state of the vehicle at the current moment includes: obtaining the first driving state of the vehicle and the current driving states of surrounding vehicles detected by the sensor at the current moment. It can be understood that sensors such as radars and cameras are installed on the vehicle, which can collect the first driving state of the vehicle and the current driving states of surrounding vehicles in real time.

[0067] Step S12: predicting, based on the first driving state, a second driving state of the vehicle at the next moment after executing different actions in a preset action set, wherein the preset action set includes a longitudinal action set and a lateral action set.

[0068] In this embodiment, a preset action set is set, which specifically includes some driving actions that the vehicle may take during driving. Then, based on the first driving state at the current moment, it is predicted that the second driving state of the vehicle at the next moment will be obtained after executing different actions in the preset action set.

[0069] Specifically, the preset action set includes a longitudinal action set and a lateral action set; the longitudinal action set includes acceleration, uniform motion and deceleration, and the lateral action set includes left lane change, lane keeping and right lane change. It can be understood that the vehicle has both lateral and longitudinal actions during driving, wherein the preset action set A is defined as:

[0070] ;

[0071] in, represents a set of vertical actions, It represents a set of lateral actions, that is, the first three items represent longitudinal actions, which respectively represent acceleration, uniform motion and deceleration; the last three items represent lateral actions, which respectively represent left lane change, right lane change and lane keeping.

[0072] Correspondingly, the prediction of the second driving state at the next moment obtained by the self-vehicle after executing different actions in the preset action set based on the first driving state includes: selecting a longitudinal action and a lateral action from the longitudinal action set and the lateral action set to form an action combination, and determining the acceleration value corresponding to each action combination; predicting the second driving state at the next moment obtained by the self-vehicle after executing different action combinations based on the first driving state and the acceleration value. It can be understood that the vehicle has both lateral and longitudinal actions during driving, so the present application selects a longitudinal action and a lateral action from the longitudinal action set and the lateral action set to form an action combination as the driving action that the vehicle may take, and it can be known from the above content that the longitudinal action includes three actions of acceleration, uniform motion and deceleration, and the lateral action includes three actions of left lane change, lane keeping and right lane change, so nine action combinations can be obtained. Further, the present embodiment determines the acceleration value corresponding to each action combination, and then predicts the second driving state at the next moment obtained by the self-vehicle after executing different action combinations based on the first driving state and the acceleration value. In this process, the kinematic model is used:

[0073] ;

[0074] in, is the first driving state, x and y represent the current lateral and longitudinal positions of the vehicle, v x and v y Indicates the current horizontal and vertical speed; is the second driving state, which is used to indicate the predicted position and predicted speed of the vehicle at the next moment; x and a y Indicates the acceleration value in the horizontal and vertical directions. Indicates the time difference between the current moment and the next moment.

[0075] In a specific implementation, a suitable acceleration can be set for each action combination according to the empirical value. In addition, the acceleration value can be set to a default value, such as 0.8. In another specific implementation, dynamic values ​​can also be used for planning. For example, when the speed given by the longitudinal planning is an acceleration value, the values ​​between 0.2, 0.4, 0.6~1.8, and 2.0 are taken for calculation respectively. When the speed given by the longitudinal planning is a deceleration value, the values ​​between -0.5, -1.0, and -1.5~-5.0 are respectively executed for calculation. That is, it is necessary to execute these 10 results separately to obtain different second driving states, and then input them into the POMDP model for decision making. Among them, the vehicle has a maximum acceleration a plus and a maximum deceleration a minus, and the gradient between them is a plus / 10 or a minus / 10. When these steps are completed, before the speed of each time step is sent to control the vehicle, the speed in each time needs to be fitted three times, and the speed at each point corresponding to the fitted curve is finally sent.

[0076] Step S13: Construct corresponding target reward functions based on the first driving state and each of the second driving states, and input the first driving state, the second driving state, the preset action set, the target reward function and the driving paths of surrounding vehicles into a preset POMDP model, so as to use the POMDP model to solve and output the target action corresponding to the optimal reward function.

[0077] In this embodiment, the corresponding target reward function is constructed based on the first driving state at the current moment and each second driving state predicted at the next moment, and then the first driving state, the second driving state, the preset action set, the target reward function and the driving path of the surrounding vehicles are input into the preset POMDP model, so as to use the POMDP model to solve and output the target action corresponding to the optimal reward function. That is, this application applies POMDP to the decision-making process of autonomous driving, and also takes into account the driving paths of surrounding vehicles, so as to achieve modeling of uncertainty in the driving process, thereby outputting more accurate actions to control vehicle driving.

[0078] The process of acquiring the driving path of the surrounding vehicles includes: predicting the arrival position of the surrounding vehicles at the next moment based on the current driving state of the surrounding vehicles; generating a target trajectory line between the current position of the surrounding vehicles and the arrival position using a B-spline curve, and using the target trajectory line as the driving path of the surrounding vehicles. It is understandable that the movement trend of the surrounding vehicles at the next moment in the environment cannot be directly measured by the vehicle sensor. Therefore, assuming that the driving intention of the surrounding vehicles remains unchanged during the current decision cycle, the movement trend of the next period of time can be predicted according to the driver's driving intention. Specifically, it is first necessary to predict the arrival position of the surrounding vehicles at the next moment based on the current driving state of the surrounding vehicles, and then generate a target trajectory line between the current position of the surrounding vehicles and the arrival position using a third-order B-spline curve, so as to use the target trajectory line as the driving path of the surrounding vehicles. The use of B-splines mainly considers the diversity of roads. B-splines can more flexibly represent the predicted trajectory while ensuring curvature continuity.

[0079] In a specific implementation, the above-mentioned prediction of the arrival position of the surrounding vehicles at the next moment based on the current driving state of the surrounding vehicles includes: determining the center point of each lane in front of the surrounding vehicles according to the current road structure, and selecting a number of target positions near the center point using a Gaussian sampling method; wherein the distance between the center point and the vehicle satisfies a preset distance condition; inputting each of the target positions and the current driving state into an observation model constructed based on a normal distribution function to obtain a probability value for each of the target positions; and determining the target position corresponding to the maximum probability value as the arrival position of the surrounding vehicles at the next moment. That is, when predicting the arrival position of the surrounding vehicles at the next moment, first determine the center point of each lane in front of the surrounding vehicles according to the current road structure, such as Figure 2 As shown, according to the road structure Figure 2 There are two lanes in Figure 2 P0 is the center point of the two lanes, and the longitudinal distance between the center point and the vehicle satisfies the preset distance condition, which can be set according to the specific scene requirements. Then, the Gaussian sampling method is used to select a number of target positions near the center point, that is, the small dots around P0 in the figure, and then the current driving status of each target position and the surrounding vehicles is input into the observation model constructed based on the normal distribution function to obtain the probability value of each target position. Among them, the observation model of the vehicle is:

[0080] ;

[0081] in, , N represents normal distribution, Indicates the state of the surrounding vehicles at the next moment, that is, the end position, Indicates the current driving status of surrounding vehicles.

[0082] Finally, the present application determines the target position corresponding to the maximum probability value as the arrival position of the surrounding vehicles at the next moment.

[0083] Step S14: Using the target action to control the vehicle to travel.

[0084] In this embodiment, the target action is used to control the driving of the vehicle.

[0085] It can be seen that the present application obtains the first driving state of the self-vehicle at the current moment; based on the first driving state, predicts the second driving state of the self-vehicle at the next moment after executing different actions in the preset action set; wherein the preset action set includes a longitudinal action set and a lateral action set; based on the first driving state and each of the second driving states, respectively constructs a corresponding target reward function, and inputs the first driving state, the second driving state, the preset action set, the target reward function and the driving paths of the surrounding vehicles into a preset POMDP model, so as to use the POMDP model to solve and output the target action corresponding to the optimal reward function; and uses the target action to control the driving of the self-vehicle.

[0086] Beneficial effects: The present application first needs to obtain the first driving state of the vehicle at the current moment, and the present application sets a preset action set, which specifically includes the actions that the vehicle may take during driving, and then predicts the second driving state of the vehicle at the next moment after executing different actions in the preset action set based on the first driving state, thereby constructing a corresponding target reward function based on the first driving state at the current moment and each second driving state predicted at the next moment; wherein the preset action set includes a longitudinal action set and a lateral action set. Further, the first driving state, the second driving state, the preset action set, the target reward function and the driving path of the surrounding vehicles are input into a preset POMDP model, so as to solve and output the target action corresponding to the optimal reward function using the POMDP model, and then use the target action to control the driving of the vehicle. That is, the present application applies POMDP to the decision-making process of autonomous driving, and also takes into account the driving paths of surrounding vehicles, so as to achieve modeling of uncertainty in the driving process, thereby outputting more accurate actions to control vehicle driving.

[0087] See also Figure 3 As shown, the embodiment of the present application discloses a specific vehicle driving decision method. Compared with the previous embodiment, this embodiment further explains and optimizes the technical solution. Specifically, it includes:

[0088] Step S21: Acquire a first driving state of the vehicle at the current moment; the first driving state includes the current position and current speed of the vehicle.

[0089] In this embodiment, the current position and current speed of the vehicle at the current moment are obtained. The current position includes the lateral and longitudinal positions, and the current speed also includes the lateral and longitudinal speeds.

[0090] Step S22: Based on the first driving state, predict the second driving state of the vehicle at the next moment after executing different actions in the preset action set; the second driving state includes the predicted position and predicted speed of the vehicle at the next moment.

[0091] In this embodiment, the current position and current speed of the vehicle at the current moment are used to obtain the predicted position and speed at the next moment.

[0092] Step S23: construct a first reward function using the predicted speed and the current vehicle speed limit determined based on the predicted position; construct a second reward function based on the action corresponding to the second driving state; construct a third reward function based on the actual distance between the vehicle and the surrounding vehicles and the target safety distance; construct a fourth reward function based on the relationship between the predicted position and the navigation end point; construct a fifth reward function based on the actual distance between the vehicle and the current nearest preceding vehicle and a preset optimal distance.

[0093] In this embodiment, the first reward function for characterizing speed gain is constructed using the predicted speed and the current vehicle speed limit determined based on the predicted position. It can be understood that in order to make the driving speed of the vehicle closer to the expected speed and to ensure that there is no frequent acceleration and deceleration near the expected speed, the smaller the difference between the vehicle speed and the expected speed per unit time, the smaller the penalty; wherein the expected speed is usually considered to be the current vehicle speed limit at the predicted position. The first reward function R v It is expressed as:

[0094]

[0095] in, To predict the speed, is the current vehicle speed limit, C exceed and C lower is a negative constant.

[0096] In this embodiment, a second reward function for representing the ride comfort benefit is constructed based on the action corresponding to the second driving state. Specifically, when the vehicle is accelerating, decelerating, or changing lanes, it will be penalized. a It is expressed as:

[0097] .

[0098] In this embodiment, a third reward function is constructed based on the actual distance between the vehicle and surrounding vehicles and the target safety distance to characterize the collision benefit. That is, when the vehicle collides with surrounding vehicles, it will also be punished. e It is expressed as:

[0099] ;

[0100] Where d is the actual distance between the vehicle and surrounding vehicles, d safe is the target safety distance between the vehicle and surrounding vehicles.

[0101] Specifically, the process of determining the target safety distance includes: determining a first distance based on the current vehicle speed limit and the maximum acceleration of the vehicle; determining a second distance based on the vehicle width and vehicle length of the own vehicle, and determining a third distance based on the vehicle width and vehicle length of the surrounding vehicles; and summing the first distance, the second distance and the third distance to obtain the target safety distance.

[0102] First, based on the current vehicle speed limit and the maximum vehicle acceleration a max Determine the first distance d threshold :

[0103] ;

[0104] It also needs to be based on the vehicle width W of the vehicle e and vehicle length L e Determine the second distance R e,boundary :

[0105] ;

[0106] The third distance R is determined based on the vehicle width and vehicle length of the surrounding vehicles. boundary :

[0107] ;

[0108] Then the first distance, the second distance and the third distance are summed to obtain the target safety distance d safe , the target safety distance is Figure 4 As shown in:

[0109] .

[0110] In this embodiment, a fourth reward function is constructed based on the relationship between the predicted position and the navigation end point to characterize the target point benefit, which means that when the vehicle completes the action and reaches the target position predetermined by the planning module. dIt is expressed as:

[0111] ;

[0112] Among them, S i Indicates the distance between the predicted position of the vehicle and the navigation starting point in the ST coordinate system, S end It represents the distance from the navigation end point to the navigation start point in the ST coordinate system, that is, the total navigation length. Here, ST refers to the origin of the vehicle, taking the front, for example, 200 meters, as the end point, and establishing an ST graph. s is the displacement, and t is the time. This formula mainly indicates that rewards are given when the planning result can reach the end point, and no rewards are given if it cannot reach the end point.

[0113] In this embodiment, a fifth reward function for characterizing the distance benefit is constructed based on the actual distance between the vehicle and the current nearest preceding vehicle and the preset optimal distance. d It is expressed as:

[0114] ;

[0115] Among them, c gap is a constant coefficient, Indicates the actual distance between the vehicle and the nearest preceding vehicle. Indicates the preset optimal distance between the vehicle and the current nearest preceding vehicle.

[0116] Specifically, the process of determining the preset optimal distance includes: determining the target speed when an emergency braking event occurs based on the current speed and the preset maximum braking acceleration; and determining the preset optimal distance based on the current speed, the target speed and the vehicle maximum acceleration.

[0117] That is, based on the current speed v and the preset maximum braking acceleration b max Determine target speed in the event of an emergency braking event :

[0118] ;

[0119] Then based on the current speed v, target speed and the maximum vehicle acceleration a max Determine the preset optimal distance :

[0120] .

[0121] Step S24: weighting the first reward function, the second reward function, the third reward function, the fourth reward function and the fifth reward function using a weight coefficient to obtain a target reward function.

[0122] In this embodiment, the weight coefficient is used to adjust the first reward function R v , the second reward function R a , the third reward function R e 、The fourth reward function R d And the fifth reward function R gap Weighted to get the target reward function R:

[0123] ;

[0124] Among them, w1 to w5 represent weight coefficients, which are obtained by the maximum entropy inverse reinforcement learning algorithm. This algorithm is based on the maximum entropy principle and infers the weight of the reward function from actual driving experimental data, thereby making driving more in line with human driving habits.

[0125] Step S25: Input the first driving state, the second driving state, the preset action set, the target reward function and the driving paths of surrounding vehicles into a preset POMDP model, so as to use the POMDP model to solve and output the target action corresponding to the optimal reward function, and use the target action to control the driving of the vehicle.

[0126] In this embodiment, the POMDP model specifically includes six parts {S, A, T, R, Z, O}, where S represents the state space, i.e., the first driving state and the second driving state mentioned above; A is the action that the vehicle may take during driving, i.e., the preset action set mentioned above; T is the state transition model of the vehicle, i.e., the kinematic model mentioned above; R is the target reward function mentioned above; Z is the observation data, i.e., the data detected by the sensors such as the radar and camera mentioned above; O is the observation function, i.e., the observation model mentioned above. Therefore, the above-obtained data is solved by the online POMCP solver to output the target action corresponding to the optimal reward function.

[0127] For more specific processing procedures of the above steps S21, S22 and S25, reference may be made to the corresponding contents disclosed in the aforementioned embodiments, which will not be described in detail here.

[0128] It can be seen that the target reward function in the embodiment of the present application includes a first reward function for characterizing speed benefits, a second reward function for characterizing smoothness benefits, a third reward function for characterizing collision benefits, a fourth reward function for characterizing target point benefits, and a fifth reward function for characterizing spacing benefits, and then the first reward function, the second reward function, the third reward function, the fourth reward function, and the fifth reward function are weighted by using a weight coefficient to obtain the target reward function. Finally, the present application is to select an action corresponding to the optimal reward function value to control the vehicle driving.

[0129] Below Figure 5 Take this as an example to explain the solution of this application in detail:

[0130] Assumptions Figure 5 In the figure, the dark blue box is the position of the vehicle at time t0, the light blue box (dashed edge line) is the position of the vehicle at time t1, the gray box is the position of the surrounding vehicles at time t0, and the light gray box (dashed edge line) is the position of the surrounding vehicles at time t1. The red point is the center point P0 of the two lanes (no other target position points are used here, only two P0 points are used as examples). The black line segment indicates the trajectory generated by the B-spline.

[0131] First, the sensor receives the surrounding environment and the vehicle data, and then calculates the driving path of the surrounding vehicles. Figure 5 There are two possible paths for the surrounding vehicles at time t: going straight and changing lanes right. The probability of going straight is 80%, and the probability of changing lanes right is 20%. That is, the straight path with a high probability is taken as the input of the model, and then the first driving state of the current vehicle S0=[x,y,v x , v y ], execute the actions in [ACC, KS, DEC, LCL, LCR, LK] in state S0 to obtain different action combinations. According to different action combinations, the second driving state of the vehicle at the next moment can be calculated. After the second driving state is calculated, the target reward function R can be calculated again, and then the above information is input into the POMDP model to output the highest reward R, and then execute the corresponding action combination, and the process ends at time t.

[0132] At the beginning of time t1, the surrounding environment and the vehicle data are received by the sensor, and then the driving paths of the surrounding vehicles are calculated. At this time, there are two possible paths for the surrounding vehicles at time t1, straight and right lane change, with a straight probability of 10% and a right lane change probability of 90%. That is, the right lane change with a high probability is taken as the input of the model. At this time, state S1=S0+t. Continue to execute the actions in [ACC, KS, DEC, LCL, LCR, LK] in state S1 to obtain different action combinations. From the above content, it can be seen that there are nine different action combinations here. Then, according to different action combinations, the second driving state of the vehicle at the next moment can be calculated. After the second driving state is calculated, the target reward function R can be calculated again, and the action with the highest reward is executed. The process ends at time t1.

[0133] According to the above description, the action combination taken at time t0 may be lane keeping (LK) and constant speed (KS), and the action A taken at time t1 may be lane keeping (LK) and deceleration (DEC). In this way, the position, speed and other information at time t1 can be predicted at time t0.

[0134] It should be noted that here we only use time t0 to calculate the position at time t1. Usually, the states at times t1, t2, t3...tn can be predicted at time t0, but only the observation data at time t0 is obtained directly through the sensor.

[0135] See also Figure 6 As shown, the embodiment of the present application discloses a vehicle driving decision device, which includes:

[0136] A driving state acquisition module 11 is used to acquire a first driving state of the vehicle at a current moment;

[0137] A driving state prediction module 12, configured to predict, based on the first driving state, a second driving state obtained by the vehicle at the next moment after executing different actions in a preset action set; wherein the preset action set includes a longitudinal action set and a lateral action set;

[0138] an action determination module 13, configured to construct a corresponding target reward function based on the first driving state and each of the second driving states, and input the first driving state, the second driving state, the preset action set, the target reward function and the driving path of the surrounding vehicles into a preset POMDP model, so as to solve and output a target action corresponding to the optimal reward function using the POMDP model;

[0139] The control module 14 is used to control the driving of the vehicle using the target action.

[0140] It can be seen that the present application obtains the first driving state of the self-vehicle at the current moment; based on the first driving state, predicts the second driving state of the self-vehicle at the next moment after executing different actions in the preset action set; wherein the preset action set includes a longitudinal action set and a lateral action set; based on the first driving state and each of the second driving states, respectively constructs a corresponding target reward function, and inputs the first driving state, the second driving state, the preset action set, the target reward function and the driving paths of the surrounding vehicles into a preset POMDP model, so as to use the POMDP model to solve and output the target action corresponding to the optimal reward function; and uses the target action to control the driving of the self-vehicle.

[0141] Beneficial effects: The present application first needs to obtain the first driving state of the vehicle at the current moment, and the present application sets a preset action set, which specifically includes the actions that the vehicle may take during driving, and then predicts the second driving state of the vehicle at the next moment after executing different actions in the preset action set based on the first driving state, thereby constructing a corresponding target reward function based on the first driving state at the current moment and each second driving state predicted at the next moment; wherein the preset action set includes a longitudinal action set and a lateral action set. Further, the first driving state, the second driving state, the preset action set, the target reward function and the driving path of the surrounding vehicles are input into a preset POMDP model, so as to solve and output the target action corresponding to the optimal reward function using the POMDP model, and then use the target action to control the driving of the vehicle. That is, the present application applies POMDP to the decision-making process of autonomous driving, and also takes into account the driving paths of surrounding vehicles, so as to achieve modeling of uncertainty in the driving process, thereby outputting more accurate actions to control vehicle driving.

[0142] Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Specifically, it may include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 is used to store a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the vehicle driving decision method executed by the electronic device disclosed in any of the aforementioned embodiments.

[0143] In this embodiment, the power supply 23 is used to provide working voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and the external device, and the communication protocol it follows is any communication protocol that can be applied to the technical solution of the present application, and is not specifically limited here; the input and output interface 25 is used to obtain external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs and is not specifically limited here.

[0144] Among them, the processor 21 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 21 can be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 21 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 21 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 21 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0145] In addition, the memory 22, as a carrier for storing resources, can be a read-only memory, a random access memory, a disk or an optical disk, etc. The resources stored thereon include an operating system 221, a computer program 222 and data 223, etc. The storage method can be temporary storage or permanent storage.

[0146] Among them, the operating system 221 is used to manage and control the hardware devices and computer programs 222 on the electronic device 20, so as to realize the operation and processing of the massive data 223 in the memory 22 by the processor 21, which can be Windows, Unix, Linux, etc. In addition to including a computer program that can be used to complete the vehicle driving decision method performed by the electronic device 20 disclosed in any of the aforementioned embodiments, the computer program 222 can further include a computer program that can be used to complete other specific tasks. In addition to data transmitted from an external device received by the electronic device, the data 223 can also include data collected by its own input and output interface 25, etc.

[0147] Furthermore, an embodiment of the present application also discloses a computer-readable storage medium, in which a computer program is stored. When the computer program is loaded and executed by a processor, the vehicle driving decision method steps disclosed in any of the aforementioned embodiments are implemented.

[0148] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0149] Those skilled in the art may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented with electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0150] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a compact disc read-only memory (CD-ROM), or any other form of storage medium known in the art.

[0151] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0152] The above is a detailed introduction to a vehicle driving decision method, device, equipment and storage medium provided by the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for a general technician in this field, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A vehicle driving decision method, characterized in that: include: Obtain the first driving state of the vehicle at the current moment; Predicting, based on the first driving state, a second driving state at the next moment obtained by the vehicle after executing different actions in a preset action set; wherein the preset action set includes a longitudinal action set and a lateral action set; Based on the first driving state and each of the second driving states, a corresponding target reward function is constructed respectively, and the first driving state, the second driving state, the preset action set, the target reward function and the driving path of the surrounding vehicles are input into a preset POMDP model, so as to solve and output the target action corresponding to the optimal reward function by using the POMDP model; The target action is used to control the driving of the vehicle.

2. The vehicle driving decision method according to claim 1, characterized in that: The longitudinal action set includes acceleration motion, uniform motion and deceleration motion, and the lateral action set includes left lane change, lane keeping and right lane change; Correspondingly, the predicting, based on the first driving state, of the second driving state at the next moment obtained by the vehicle after executing different actions in the preset action set, includes: Selecting any one longitudinal action and one lateral action from the longitudinal action set and the lateral action set to form an action combination, and determining an acceleration value corresponding to each action combination; Predicting, based on the first driving state and the acceleration value, a second driving state of the vehicle at the next moment after executing different action combinations; Accordingly, the use of the target action to control the vehicle to travel includes: The vehicle is controlled to travel using a target action combination corresponding to an optimal reward function.

3. The vehicle driving decision method according to claim 1, characterized in that: The obtaining of the first driving state of the vehicle at the current moment includes: Acquire a first driving state of the vehicle and current driving states of surrounding vehicles detected by the sensor at the current moment; Correspondingly, the process of acquiring the driving paths of the surrounding vehicles includes: Predicting the arrival position of the surrounding vehicles at the next moment based on the current driving state of the surrounding vehicles; A target trajectory is generated between the current position of the surrounding vehicle and the arrival position using a B-spline curve, and the target trajectory is used as a driving path of the surrounding vehicle.

4. The vehicle driving decision method according to claim 3, characterized in that: The predicting the arrival position of the surrounding vehicles at the next moment based on the current driving state of the surrounding vehicles includes: Determine the center point of each lane in front of the surrounding vehicles according to the current road structure, and select a number of target positions near the center point using a Gaussian sampling method; wherein the distance between the center point and the self-vehicle satisfies a preset distance condition; Inputting each of the target positions and the current driving state into an observation model constructed based on a normal distribution function to obtain a probability value of each of the target positions; The target position corresponding to the maximum probability value is determined as the arrival position of the surrounding vehicles at the next moment.

5. The vehicle driving decision method according to any one of claims 1 to 4, characterized in that: The target reward function includes a first reward function for representing speed gain, a second reward function for representing smoothness gain, a third reward function for representing collision gain, a fourth reward function for representing target point gain, and a fifth reward function for representing spacing gain; the first driving state includes the current position and current speed of the vehicle; the second driving state includes the predicted position and predicted speed of the vehicle at the next moment; Accordingly, constructing a corresponding target reward function based on the first driving state and each of the second driving states respectively includes: constructing the first reward function using the predicted speed and a current vehicle speed limit determined based on the predicted position; constructing the second reward function based on the action corresponding to the second driving state; Constructing the third reward function based on the actual distance between the vehicle and the surrounding vehicles and the target safety distance; Constructing the fourth reward function based on the relationship between the predicted position and the navigation endpoint; Constructing the fifth reward function based on the actual distance between the vehicle and the current nearest preceding vehicle and a preset optimal distance; The first reward function, the second reward function, the third reward function, the fourth reward function and the fifth reward function are weighted by using a weight coefficient to obtain a target reward function.

6. The vehicle driving decision method according to claim 5, characterized in that: The process of determining the target safety distance includes: determining a first distance based on the current vehicle speed limit and the vehicle maximum acceleration; determining a second distance based on a vehicle width and a vehicle length of the own vehicle, and determining a third distance based on a vehicle width and a vehicle length of the surrounding vehicles; The first distance, the second distance and the third distance are summed to obtain the target safety distance.

7. The vehicle driving decision method according to claim 5, characterized in that: The process of determining the preset optimal distance includes: determining a target speed when an emergency braking event occurs based on the current speed and a preset maximum braking acceleration; The preset optimal distance is determined based on the current speed, the target speed and a maximum acceleration of the vehicle.

8. A vehicle driving decision device, characterized in that: include: A driving state acquisition module, used to acquire a first driving state of the vehicle at the current moment; A driving state prediction module, used for predicting, based on the first driving state, a second driving state obtained by the vehicle at the next moment after executing different actions in a preset action set; wherein the preset action set includes a longitudinal action set and a lateral action set; an action determination module, configured to construct a corresponding target reward function based on the first driving state and each of the second driving states, and input the first driving state, the second driving state, the preset action set, the target reward function and the driving paths of surrounding vehicles into a preset POMDP model, so as to solve and output a target action corresponding to the optimal reward function using the POMDP model; A control module is used to control the driving of the vehicle using the target action.

9. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor is used to execute the computer program to implement the steps of the vehicle driving decision method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: Used to store computer programs; wherein, when the computer program is executed by a processor, the steps of the vehicle driving decision method as described in any one of claims 1 to 7 are implemented.

Images