Automatic driving method, device, equipment, medium and computer program product

Abstract

The present application discloses a method, device, equipment, medium and computer program product for autonomous driving. According to the embodiments provided in the present application, by obtaining the road information and driving status of the autonomous driving vehicle; according to the road information and driving status, the probability of the autonomous driving vehicle performing multiple first driving actions according to preset rules is determined by a pre-trained decision model, and the target driving action is determined based on the probability, and the preset rule is set based on the road information and driving status; the autonomous driving vehicle is driven according to the target driving action, so that the autonomous driving vehicle can select the target driving action through the preset rules, thereby satisfying the autonomous driving of high-degree-of-freedom scenarios.

Description

Technical Field

[0001] The present application relates to the field of autonomous driving, and in particular to a method, apparatus, device, medium and computer program product for autonomous driving. Background Art

[0002] With the rapid development of science and technology, self-driving vehicles are becoming more and more common in people's lives. Existing self-driving vehicles make decisions through scene and rule-based state machines. The state machine can make certain behavioral decisions based on the analysis of the current scene or based on rule-based judgments, thereby enabling the self-driving vehicle to complete automatic driving behavior.

[0003] However, the rules of the state machine are manually defined and cannot cover all possible situations. In scenarios with relatively high degrees of freedom, autonomous driving technology based only on general rules cannot meet people's needs. Summary of the invention

[0004] The embodiments of the present application provide a method, apparatus, device, medium and computer program product for autonomous driving, which can meet the needs of autonomous driving in high-degree-of-freedom scenarios and improve user experience.

[0005] In a first aspect, an embodiment of the present application provides a method for autonomous driving, the method comprising:

[0006] Obtain road information and driving status of autonomous vehicles;

[0007] According to the road information and the driving state, respectively determine the probability that the autonomous driving vehicle performs a plurality of first driving actions according to preset rules through a pre-trained decision model, and determine a target driving action based on the probabilities, wherein the preset rules are set based on the road information and the driving state;

[0008] The autonomous driving vehicle is driven according to the target driving action.

[0009] In a possible implementation, the road information includes road vehicle information, road lane line information, and road vehicle running track information. The method comprises: determining, based on the road information and the driving state, probabilities of the automatic driving vehicle performing a plurality of first driving actions according to preset rules through a pre-trained decision model, and determining a target driving action based on the probabilities. The preset rules are set based on the road information and the driving state. The method comprises:

[0010] Determine, by means of the pre-trained decision model, a first distance between the autonomous driving vehicle and the road vehicle, a second distance between the autonomous driving vehicle and the road lane line, and a predicted trajectory of the road vehicle according to the road vehicle information, the road lane line information, and the running trajectory information of the road vehicle;

[0011] Calculating, based on the first distance, the second distance, the predicted trajectory of the road vehicle, and the driving state, the probability that the autonomous driving vehicle performs the plurality of first driving actions respectively according to a preset rule;

[0012] A target driving action is determined according to the probability.

[0013] In one possible implementation, the method of calculating the probability that the autonomous driving vehicle respectively performs the plurality of first driving actions according to a preset rule based on the first distance, the second distance, the predicted trajectory of the road vehicle, and the driving state includes:

[0014] Calculating reward values ​​for collisions caused by the autonomous driving vehicle performing a plurality of first driving actions respectively according to the first distance, the second distance, the predicted trajectory of the road vehicle, and the driving state;

[0015] Calculating, based on the reward value, an estimated probability that the autonomous driving vehicle performs the first driving action;

[0016] The estimated probability is weighted according to a preset rule to obtain the probability that the autonomous driving vehicle performs the first driving action.

[0017] In a possible implementation, the method of determining a target driving action according to the probability includes:

[0018] According to the probability, the target driving action is determined by an ε-greedy algorithm.

[0019] In a possible implementation, before determining, based on the road information and the driving state, the probabilities of the autonomous driving vehicle performing a plurality of first driving actions according to preset rules by using a pre-trained decision model, and determining a target driving action based on the probabilities, and before the preset rules are set based on the road information and the driving state, the method further includes:

[0020] Acquire simulated road information and simulated driving status of a simulated autonomous driving vehicle;

[0021] The initial decision model is trained according to the simulated road information and the simulated driving state to obtain a trained decision model.

[0022] In a possible implementation, the initial decision model is trained according to the simulated road information and the simulated driving state to obtain a trained decision model, and the method includes:

[0023] Determine, according to the simulated road information and the simulated driving state, a simulated first distance between the simulated autonomous driving vehicle and a simulated road vehicle, a simulated second distance between the simulated autonomous driving vehicle and a lane line of a simulated road, and a simulated predicted trajectory of the simulated road vehicle, and train the initial decision model to obtain a trained decision model;

[0024] When the output result of the trained decision model meets the training stop condition, the trained decision model is determined to be the decision model.

[0025] In a second aspect, an embodiment of the present application provides an automatic driving device, the device comprising:

[0026] An acquisition module, used to obtain road information and driving status of the autonomous driving vehicle;

[0027] a determination module, configured to determine, according to the road information and the driving state, respectively the probability that the autonomous driving vehicle performs a plurality of first driving actions according to a preset rule by using a pre-trained decision model, and determine a target driving action based on the probabilities, wherein the preset rule is set based on the road information and the driving state;

[0028] An execution module is used to drive the automatic driving vehicle according to the target driving action.

[0029] In a possible implementation, the road information includes road vehicle information, road lane line information, and road vehicle running track information, and the determination module is specifically used to:

[0030] Determine, by means of the pre-trained decision model, a first distance between the autonomous driving vehicle and the road vehicle, a second distance between the autonomous driving vehicle and the road lane line, and a predicted trajectory of the road vehicle according to the road vehicle information, the road lane line information, and the running trajectory information of the road vehicle;

[0031] Calculating, based on the first distance, the second distance, the predicted trajectory of the road vehicle, and the driving state, the probability that the autonomous driving vehicle performs the plurality of first driving actions respectively according to a preset rule;

[0032] A target driving action is determined according to the probability.

[0033] In a possible implementation manner, the determination module may be used to:

[0034] Calculating reward values ​​for collisions caused by the autonomous driving vehicle performing a plurality of first driving actions respectively according to the first distance, the second distance, the predicted trajectory of the road vehicle, and the driving state;

[0035] Calculating, based on the reward value, an estimated probability that the autonomous driving vehicle performs the first driving action;

[0036] The estimated probability is weighted according to a preset rule to obtain the probability that the autonomous driving vehicle performs the first driving action.

[0037] In a possible implementation manner, the determination module may be used to:

[0038] According to the probability, the target driving action is determined by an ε-greedy algorithm.

[0039] In a possible implementation, the device may further include:

[0040] a first acquisition module, configured to determine, according to the road information and the driving state, respectively the probability that the autonomous driving vehicle performs a plurality of first driving actions according to a preset rule through a pre-trained decision model, determine a target driving action based on the probabilities, and obtain simulated road information and a simulated driving state of a simulated autonomous driving vehicle before the preset rule is set based on the road information and the driving state;

[0041] The training module is used to train the initial decision model according to the simulated road information and the simulated driving state to obtain a trained decision model.

[0042] In a possible implementation, the training module is specifically used for:

[0043] Determine, according to the simulated road information and the simulated driving state, a simulated first distance between the simulated autonomous driving vehicle and a simulated road vehicle, a simulated second distance between the simulated autonomous driving vehicle and a lane line of a simulated road, and a simulated predicted trajectory of the simulated road vehicle, and train the initial decision model to obtain a trained decision model;

[0044] When the output result of the trained decision model meets the training stop condition, the trained decision model is determined to be the decision model.

[0045] In a third aspect, an embodiment of the present application provides an autonomous driving device, the device comprising:

[0046] a processor and a memory storing computer program instructions;

[0047] When the processor executes the computer program instructions, it implements a method for autonomous driving as described in any one of the embodiments of the present application.

[0048] In a fourth aspect, an embodiment of the present application provides a computer storage medium, on which computer program instructions are stored. When the computer program instructions are executed by a processor, an automatic driving method such as any one of the embodiments of the present application is implemented.

[0049] In a fifth aspect, an embodiment of the present application provides a computer program product. When the instructions in the computer program product are executed by a processor of an electronic device, the electronic device performs the method of autonomous driving as described in any one of the embodiments of the present application.

[0050] The autonomous driving method, apparatus, device, medium and computer program product of the embodiments of the present application determine the target driving action in a pre-trained decision model by acquiring the road information and driving status of the autonomous driving vehicle. When determining the target driving action, it is determined based on the probability of the autonomous driving vehicle executing multiple first driving actions according to preset rules, and then the driving of the autonomous driving vehicle is completed according to the target driving action. In this way, the autonomous driving vehicle can select the target driving action according to preset rules, thereby satisfying autonomous driving in high-degree-of-freedom scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the technical solution of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0052] Figure 1 is a flowchart of an automatic driving method provided by an embodiment of the present application;

[0053] Figure 2 is a flowchart of an autonomous driving method provided by yet another embodiment of the present application;

[0054] Figure 3 is a schematic structural diagram of an automatic driving device provided by yet another embodiment of the present application;

[0055] Figure 4 This is a schematic diagram of the structure of an autonomous driving device provided by yet another embodiment of the present application. DETAILED DESCRIPTION

[0056] The features and exemplary embodiments of various aspects of the present application will be described in detail below. In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain the present application, rather than to limit the present application. For those skilled in the art, the present application can be implemented without the need for some of these specific details. The following description of the embodiments is only to provide a better understanding of the present application by illustrating the examples of the present application.

[0057] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "include..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0058] As mentioned in the background technology, existing autonomous driving vehicles make decisions through a state machine based on scenarios and rules. The state machine can make certain behavioral decisions based on the analysis of the current scenario or based on the judgment of rules, thereby enabling the autonomous driving vehicle to complete the automatic driving behavior. However, the rules of the state machine are all manually defined and cannot cover all possible situations. In scenarios with relatively high degrees of freedom, autonomous driving technology based only on general rules cannot meet people's needs.

[0059] In order to solve the problems of the prior art, the embodiments of the present application provide a method, apparatus, device and computer storage medium for autonomous driving.

[0060] In an embodiment of the present application, a target driving action is determined in a pre-trained decision model by acquiring road information and driving status of the autonomous driving vehicle. When determining the target driving action, it is determined based on the probability that the autonomous driving vehicle executes multiple first driving actions according to preset rules, and then the driving of the autonomous driving vehicle is completed according to the target driving action. In this way, the autonomous driving vehicle can select the target driving action according to preset rules, thereby meeting the autonomous driving requirements of high-degree-of-freedom scenarios.

[0061] An autonomous driving vehicle needs to determine the target driving action through a decision model, and the decision model must be trained before use. The following is a detailed description of the process of training the decision model in the autonomous driving method provided in an embodiment of the present application.

[0062] Figure 1 A flow chart of an autonomous driving method provided by an embodiment of the present application is shown.

[0063] like Figure 1 As shown, the method for autonomous driving provided by the embodiment of the present application may include the following steps:

[0064] Step 110, obtaining simulated road information and simulated driving status of the simulated autonomous driving vehicle.

[0065] Step 120 , training the initial decision model according to the simulated road information and the simulated driving state to obtain a trained decision model.

[0066] In some embodiments of the present application, the initial decision model is trained by acquiring simulated road information and simulated driving status of a simulated autonomous driving vehicle to obtain a trained decision model. This allows the autonomous driving vehicle to select and execute target autonomous driving actions through the trained decision model during driving, thereby completing autonomous driving.

[0067] In step 110, simulated road information and simulated driving status of the simulated autonomous driving vehicle are obtained.

[0068] The simulated road information and simulated driving status may be environmental information of the road on which the autonomous driving vehicle is traveling, simulated by simulation software, and the environmental information includes road information and driving status.

[0069] For example, the environmental information is information about the environment in which the autonomous driving vehicle is traveling on a highway, the road information may include the autonomous driving vehicle being in the leftmost lane of the highway, and the simulated driving state may be a simulated autonomous driving vehicle, such as traveling in the leftmost lane at a constant speed of 120 kilometers per hour.

[0070] The simulation of road information and driving status by simulation software can be set by the user by inputting parameters in the simulation software, or can be obtained by initialization of the simulation software, and the specific form is not limited here.

[0071] In one example, the user inputs the position of the autonomous driving vehicle, the road parameters of the road on which the autonomous driving vehicle is traveling, and the driving state parameters of the autonomous driving vehicle into the simulation software, and generates simulated road information and driving state through the simulation software. In the decision model training process, the simulated road information and simulated driving state generated by the simulation software are first obtained.

[0072] In step 120, the initial decision model is trained according to the simulated road information and the simulated driving state to obtain a trained decision model.

[0073] After obtaining the simulated road information and the simulated driving status, the initial decision model is trained according to the simulated road information and the simulated driving status.

[0074] When training the initial decision model, the initial decision model may be trained in simulation software.

[0075] In step 120, the above method may include the following steps:

[0076] According to the simulated road information and the simulated driving status, a simulated first distance between the simulated autonomous driving vehicle and the simulated road vehicle, a simulated second distance between the simulated autonomous driving vehicle and the simulated road lane line, and a simulated predicted trajectory of the simulated road vehicle are determined, and the initial decision model is trained to obtain a trained decision model.

[0077] When the output result of the trained decision model meets the training stop condition, the trained decision model is determined to be the decision model.

[0078] The simulated road information may include simulated road vehicles, simulated road lane lines, and simulated vehicle motion trajectories.

[0079] For example, the simulated road vehicles may be other vehicles traveling in the simulated traffic environment, the simulated road lane lines may be lane lines of the first lane and the second lane in the simulated traffic environment, and the motion trajectory of the simulated vehicle may be the motion trajectory of other vehicles in the simulated traffic environment.

[0080] The simulated first distance may be calculated based on the distance between the simulated autonomous driving vehicle and the simulated road vehicle, the simulated second distance may be calculated based on the distance between the simulated autonomous driving vehicle and the simulated lane line, and the simulated predicted trajectory of the simulated road vehicle may be the trajectory of the simulated road vehicle in the next driving.

[0081] Based on the position of the simulated autonomous driving vehicle and the position of the simulated lane line, the positions of the simulated autonomous driving vehicle and the simulated lane line are calculated and determined as the simulated second distance.

[0082] When predicting the next trajectory of the simulated road vehicle, the initial decision model may establish a prediction model for the simulated road vehicle in the simulated road information in advance, and then predict the trajectory of the simulated road vehicle through the prediction model.

[0083] For example, the next trajectory of surrounding vehicles can be predicted through a long short-term memory network (LSTM) to obtain a simulated predicted trajectory of a simulated road vehicle.

[0084] In one example, after the user obtains the simulated road information and driving status in the simulation software, the simulation software can be used to determine the simulated first distance, the simulated second distance, and the simulated predicted trajectory of the vehicle on the simulated road, and then the initial decision model is trained based on this information to obtain a trained decision model.

[0085] When inputting simulated vehicle information and simulated road information into the initial decision model, the input values ​​can be approximated and continuous values ​​can be discretized to control the accuracy within a certain range, so that the input state space is not too large and the training time is reduced.

[0086] The initial decision model may be pre-set according to a double deep learning network (Double Deep Q-Learning, DDQN), and the initial decision model may include a simulated estimated Q value network, a simulated target Q value network, and a simulated Q value regulator set according to preset rules.

[0087] The simulated estimated Q-value network is provided with a simulated reward function based only on whether the simulated autonomous driving vehicle can reach the destination without collision.

[0088] When the initial decision model is trained according to the simulated road information and the simulated driving state to obtain the trained decision model, it can be judged in the simulation software whether a collision will occur before the simulated autonomous driving vehicle performs the simulated first driving action and reaches the end point. According to the judgment result, the simulated reward for the simulated autonomous driving vehicle to perform the simulated first driving action is calculated in the simulation reward function. The simulation reward function is as follows:

[0089]

[0090] Among them, R represents the simulated reward, r_p represents the simulated reward constant, α represents the simulated amplification factor, r_n represents the simulated penalty constant, V final represents the speed at which the simulated autonomous vehicle reaches the end point, V design It represents the expected speed pre-set for simulating the autonomous driving vehicle. The first driving action may include five actions: simulating speed maintenance, simulating acceleration, simulating deceleration, simulating changing lanes to the left, and simulating changing lanes to the right.

[0091] When calculating the simulated estimated probability of the simulated autonomous driving vehicle performing the first driving action, the simulated estimated Q value of the simulated autonomous driving vehicle performing the first driving action may be calculated in a simulated estimated Q value function, and the simulated estimated Q value function is as follows:

[0092]

[0093] Among them, R t represents the reward value of the current state of the simulated autonomous driving vehicle, γ represents the parameter of the simulated estimated Q value function, Represents the next state sequence S t+1 The eigenvector of A t+1 represents the next state, ω t It represents the parameter set of the simulated Q network, maxQ represents the maximum simulated estimated Q value in the next state after the simulated autonomous driving vehicle performs the first driving action, and the simulated estimated Q value after the simulated autonomous driving vehicle performs the first driving action can be returned through the simulated estimated Q value network, and then the maximum simulated estimated Q value is selected.

[0094] In order to make the actions output by the initial decision model more reasonable, we also need to adjust the simulated estimated probability according to preset rules. The preset rules can be road traffic rules, physical restrictions on vehicle driving, etc. Specifically, a simulated Q value adjuster can be set to adjust the simulated estimated Q value and discard unreasonable driving actions.

[0095] For example, the maximum speed limit on a highway is 120 kilometers per hour, and the vehicle's maximum acceleration, maximum turning radius, etc. We can preset a simulated autonomous driving vehicle that is currently driving at a speed of 120 kilometers per hour in the leftmost lane of the highway. Due to physical limitations and the maximum speed limit on highways, it is unreasonable for the vehicle to change lanes to the left or accelerate at the current speed. The adjusted function is as follows:

[0096]

[0097] where y c represents the adjusted simulated estimated Q value, β represents the simulated influencing factor, and the simulated estimated Q value is adjusted by β. After the adjustment, the simulated target driving action is selected by the ε-greedy algorithm:

[0098]

[0099] Among them, A c represents the simulated target driving action, and a represents the first driving action.

[0100] After selecting the simulated target driving action, the current state S of the autonomous driving vehicle will be simulated. t , simulate the target driving action A c , income R t , next state S t+1The terminal state P is put into the experience pool as a sample, and the actual benefit of the current state of the simulated autonomous driving vehicle is calculated through the simulated target Q value network. The simulated target Q value function is as follows:

[0101]

[0102] Among them, y target represents the simulated target Q value, γ is the parameter of the simulated target Q value function, and then the mean square error loss function is used to update all parameters ω through the gradient back propagation of the neural network. The mean square error loss function is as follows:

[0103]

[0104] Where m represents the number of samples in the experience pool, and j represents the currently selected sample. When the update period T is a multiple of the set update period, the parameter ω of the simulated target Q value function is updated. - =ω.

[0105] Continue to train the initial decision model until the output result meets the training stop condition to obtain a trained decision model, wherein the training stop condition may be that the output result meets the user's preset requirements, which is not limited here.

[0106] The following is combined with Figure 2 The method for autonomous driving provided in the embodiments of the present application is described in detail.

[0107] Figure 2 A schematic diagram of a process flow of another automatic driving method provided by an embodiment of the present application is shown, such as Figure 2 As shown, the method for autonomous driving provided by the embodiment of the present application may include the following steps:

[0108] Step 210, obtaining the road information and driving status of the autonomous driving vehicle.

[0109] Step 220, based on the road information and driving status, determine the probability that the autonomous driving vehicle performs multiple first driving actions according to preset rules through a pre-trained decision model, determine the target driving action based on the probability, and the preset rules are set based on the road information and driving status.

[0110] Step 230: driving the autonomous driving vehicle according to the target driving action.

[0111] In some embodiments of the present application, a target driving action can be determined in a pre-trained decision model based on road information and driving status of the autonomous driving vehicle. When determining the target driving action, it is determined based on the probability that the autonomous driving vehicle performs multiple first driving actions according to preset rules, and then the autonomous driving vehicle drives according to the target driving action. In this way, the autonomous driving vehicle can select the target driving action according to preset rules, thereby satisfying autonomous driving in high-degree-of-freedom scenarios.

[0112] In step 210, the road information and driving status of the autonomous driving vehicle are obtained.

[0113] The road information may be information on the road on which the autonomous driving vehicle is currently traveling, and the driving state may be the state in which the autonomous driving vehicle is currently traveling.

[0114] For example, the road information may be information about an autonomous driving vehicle traveling on a highway. The road information may include an autonomous driving vehicle in the leftmost lane of the highway. The simulated driving state may be a simulated autonomous driving vehicle, such as traveling in the leftmost lane at a constant speed of 120 kilometers per hour.

[0115] When obtaining road information of an autonomous driving vehicle, a variety of sensors of the autonomous driving vehicle may be used. For example, a lidar may be used to obtain 3D point cloud information of the road, and a camera may be used to obtain an RGB image of the road. The specific form is not limited here.

[0116] When obtaining the driving status, the current driving speed of the autonomous driving vehicle can be obtained through the control system of the autonomous driving vehicle. For example, the driving status obtained through the control system of the autonomous driving vehicle can be a constant speed of 120 kilometers per hour.

[0117] In step 220, based on the road information and driving status, the probabilities of the autonomous driving vehicle executing multiple first driving actions according to preset rules are determined respectively through a pre-trained decision model, and the target driving action is determined based on the probability, and the preset rules are set based on the road information and driving status.

[0118] The pre-trained decision model can be Figure 1 The preset rules can be road traffic rules, physical restrictions on vehicle driving, etc. For example, the maximum speed limit on a highway is 120 kilometers per hour, the maximum acceleration and maximum turning radius of the vehicle, etc. The specific rules can be set according to the actual situation of the user and are not limited here.

[0119] The first driving action may be an action of the autonomous driving vehicle maintaining a constant speed, accelerating, decelerating, changing lanes to the left, or changing lanes to the right.

[0120] When determining the probability that the autonomous driving vehicle executes the first driving action according to the preset rule, the above Figure 1 The decision model trained in , the detailed method will not be repeated here.

[0121] In order to better determine the target driving action, in step 220, the road information includes road vehicle information, road lane line information and road vehicle running track information. The above method may include the following steps:

[0122] Through a pre-trained decision model, based on road vehicle information, road lane line information and road vehicle running trajectory information, the first distance between the autonomous driving vehicle and the road vehicle, the second distance between the autonomous driving vehicle and the road lane line and the predicted trajectory of the road vehicle are determined.

[0123] Based on the first distance, the second distance, the predicted trajectory and the driving state of the road vehicle, the probabilities of the autonomous driving vehicle executing multiple first driving actions according to preset rules are calculated.

[0124] Determine target driving actions based on probability.

[0125] The road vehicle information may be other vehicles around the autonomous driving vehicle, the road lane lines may be the lane lines of the road where the autonomous driving vehicle is located, and the road vehicle operation trajectory information may be the historical operation trajectory of the road vehicle.

[0126] The first distance may be the distance between the autonomous driving vehicle and the road vehicle, the second distance may be the distance between the autonomous driving vehicle and the lane line, and the predicted trajectory of the road vehicle may be the trajectory of the road vehicle in the next driving.

[0127] When determining the first distance, the second distance, and the predicted trajectory of the road vehicle, they can be calculated through a learning algorithm after obtaining the road information.

[0128] When calculating the probability of the autonomous driving vehicle performing the first driving action according to the preset rules, the probability can be calculated by the decision model on the first distance, the second distance, the predicted trajectory of the road vehicle and the driving state.

[0129] In some embodiments of the present application, in calculating the probability that the autonomous driving vehicle respectively performs a plurality of first driving actions according to a preset rule based on the first distance, the second distance, the predicted trajectory and the driving state of the road vehicle, the method may include the following steps:

[0130] According to the first distance, the second distance, the predicted trajectory and the driving state of the road vehicle, the reward values ​​of the autonomous driving vehicle when a collision occurs by performing multiple first driving actions respectively are calculated.

[0131] Based on the reward value, calculate the estimated probability that the autonomous driving vehicle will perform the first driving action.

[0132] The estimated probability is weighted according to a preset rule to obtain the probability that the autonomous driving vehicle performs the first driving action.

[0133] Based on the first distance, the second distance, the predicted trajectory and the driving state of the road vehicle, the reward values ​​for collisions caused by the autonomous driving vehicle performing multiple first driving actions are calculated, and then the estimated probability of the autonomous driving vehicle performing the first driving action can be calculated based on the reward values. The estimated probability is weighted according to preset rules to obtain the probability of the autonomous driving vehicle performing the first driving action. Since the calculation of the reward value is only based on whether the autonomous driving vehicle reaches the destination without collision, the amount of training for the reward function is greatly reduced.

[0134] After the first distance, the second distance, the predicted trajectory and the driving status of the road vehicle are input into the decision model, the reward value can be calculated based on whether a collision will occur when the autonomous driving vehicle executes multiple first driving actions and reaches the end point. The specific calculation method can refer to the above-mentioned model training process and will not be elaborated here.

[0135] After calculating the reward value, the decision model can calculate the estimated probability that the autonomous driving vehicle performs the first driving action. Specifically, the estimated Q value can be calculated by the estimated Q value function of the decision model, and then the estimated probability that the autonomous driving vehicle performs the first driving action can be obtained.

[0136] When weighting the estimated probability, the estimated probability can be weighted according to a preset rule to obtain a weighted probability. Specifically, the estimated Q value can be adjusted by a Q value adjuster of the decision model to obtain a weighted estimated Q value.

[0137] In one example, the first driving action that does not comply with traffic regulations and physical limitations of the vehicle can be discarded through preset rules. For example, the autonomous driving vehicle is currently driving at a constant speed of 120 kilometers per hour in the leftmost lane of the highway. Here we stipulate that the maximum speed limit of the leftmost lane of the highway is 120 kilometers per hour. At this time, if the autonomous driving vehicle chooses to accelerate, it does not comply with traffic regulations. If it chooses to change lanes to the left, it does not comply with physical limitations. The specific rules are not restricted here and can be selected by the user.

[0138] When determining the target driving action according to the probability, the first driving action with the highest probability may be selected as the target driving action.

[0139] For example, an autonomous vehicle is currently traveling at a constant speed of 120 kilometers per hour in the leftmost lane of a highway. Through the analysis of road information by the decision model, it can be known that there is a vehicle traveling at a constant speed of 110 kilometers per hour 200 meters in front of the autonomous vehicle. At this time, the decision model can determine that if the autonomous vehicle continues to travel at a constant speed, it may collide with the vehicle in front. The first driving action of changing lanes to the left and accelerating does not comply with the preset rules and is discarded. At this time, the decision model can choose between changing lanes to the right and slowing down. When the preset rules are more risk-prone for the decision model, the probability of the autonomous vehicle choosing to change lanes to the right may be greater than the probability of slowing down. When the preset rules are more safety-prone for the decision model, the probability of the autonomous vehicle choosing to slow down may be greater than the probability of changing lanes to the right.

[0140] In some embodiments of the present application, in determining the target driving action according to probability, the above method may include the following steps:

[0141] According to the probability, the target driving action is determined by the ε-greedy algorithm.

[0142] Among them, when determining the target driving according to probability, the ε-greedy algorithm can be used for selection, which makes the decision model not always select the first driving action with the highest probability, but in most cases it will still select the first driving action with the highest probability as the target driving action, so the ε-greedy algorithm can allow the decision model to have a certain exploration rate.

[0143] In one example, an autonomous vehicle is currently traveling at a constant speed of 120 kilometers per hour in the leftmost lane of a highway. The decision model calculates that there is a vehicle traveling at a constant speed of 110 kilometers per hour 200 meters ahead. The autonomous vehicle can choose to slow down or change lanes to the right. If the decision model tends to be more risk-taking, the autonomous vehicle can also choose to slow down. If the decision model tends to be more safety-oriented, the autonomous vehicle can also choose to change lanes to the right.

[0144] In step 230, the autonomous driving vehicle is driven according to the target driving action.

[0145] Among them, after selecting the target driving action, the control system of the autonomous driving vehicle can send an instruction of the target driving action to the autonomous driving vehicle, and the autonomous driving vehicle completes the driving action according to the target driving action.

[0146] For example, an autonomous vehicle is currently traveling at a constant speed of 120 kilometers per hour in the leftmost lane of a highway. The target driving action is to change lanes to the right. The autonomous vehicle can control the vehicle to deviate to the right lane until the vehicle enters the lane and then travel at a constant speed of 120 kilometers per hour in the second lane on the left side of the highway.

[0147] Based on the method of autonomous driving provided in the above embodiment, accordingly, the embodiment of the present application also provides a specific implementation method of an autonomous driving device, please refer to the following embodiment.

[0148] See Figure 3 , an embodiment of the present application provides an automatic driving device, the device comprising:

[0149] An acquisition module 310 is used to acquire road information and driving status of the autonomous driving vehicle;

[0150] A determination module 320, configured to determine, based on the road information and the driving state, respectively the probability that the autonomous driving vehicle performs a plurality of first driving actions according to a preset rule through a pre-trained decision model, and determine a target driving action based on the probability, wherein the preset rule is set based on the road information and the driving state;

[0151] The execution module 330 is used to drive the autonomous driving vehicle according to the target driving action.

[0152] In some embodiments of the present application, the road information and driving status of the autonomous driving vehicle are acquired by the acquisition module 310, and the determination module 320 determines the target driving action in a pre-trained decision model. When determining the target driving action, it is determined based on the probability that the autonomous driving vehicle executes multiple first driving actions according to preset rules. Then the execution module 330 completes the driving of the autonomous driving vehicle according to the target driving action. In this way, the autonomous driving vehicle can select the target driving action according to preset rules, thereby satisfying autonomous driving in high-degree-of-freedom scenarios.

[0153] In some embodiments of the present application, the road information includes road vehicle information, road lane line information, and road vehicle running track information, and the determination module 320 is specifically used to:

[0154] Through a pre-trained decision model, based on road vehicle information, road lane line information and road vehicle running trajectory information, the first distance between the autonomous driving vehicle and the road vehicle, the second distance between the autonomous driving vehicle and the road lane line and the predicted trajectory of the road vehicle are determined.

[0155] Based on the first distance, the second distance, the predicted trajectory and the driving state of the road vehicle, the probabilities of the autonomous driving vehicle executing multiple first driving actions according to preset rules are calculated.

[0156] Determine target driving actions based on probability.

[0157] In some embodiments of the present application, the determination module 320 may also be used to:

[0158] According to the first distance, the second distance, the predicted trajectory and the driving state of the road vehicle, the reward values ​​of the autonomous driving vehicle when a collision occurs by performing multiple first driving actions respectively are calculated.

[0159] Based on the reward value, calculate the estimated probability that the autonomous driving vehicle will perform the first driving action.

[0160] The estimated probability is weighted according to a preset rule to obtain the probability that the autonomous driving vehicle performs the first driving action.

[0161] In some embodiments of the present application, the determination module 320 may be used to:

[0162] According to the probability, the target driving action is determined by the ε-greedy algorithm.

[0163] In some embodiments of the present application, the above device may further include:

[0164] The first acquisition module is used to determine the probability of the autonomous driving vehicle executing multiple first driving actions according to preset rules through a pre-trained decision model based on road information and driving status, determine the target driving action based on the probability, and obtain simulated road information and simulated driving status of the simulated autonomous driving vehicle before the preset rules are set based on the road information and driving status.

[0165] The training module is used to train the initial decision model according to the simulated road information and the simulated driving state to obtain a trained decision model.

[0166] In some embodiments of the present application, the training module is specifically used to:

[0167] According to the simulated road information and the simulated driving status, a simulated first distance between the simulated autonomous driving vehicle and the simulated road vehicle, a simulated second distance between the simulated autonomous driving vehicle and the simulated road lane line, and a simulated predicted trajectory of the simulated road vehicle are determined, and the initial decision model is trained to obtain a trained decision model.

[0168] When the output result of the trained decision model meets the training stop condition, the trained decision model is determined to be the decision model.

[0169] Based on the method provided in the above embodiment, the embodiment of the present application also provides a specific implementation of the autonomous driving electronic device. Figure 4 A schematic diagram of the hardware structure of an electronic device for autonomous driving provided in an embodiment of the present application is shown.

[0170] Figure 4 A schematic diagram of the hardware structure of the autonomous driving provided in an embodiment of the present application is shown.

[0171] The device for autonomous driving may include a processor 401 and a memory 402 storing computer program instructions.

[0172] Specifically, the processor 401 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0173] Memory 402 may include a large capacity memory for data or instructions. By way of example and not limitation, memory 402 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive or a combination of two or more of these. In appropriate cases, memory 402 may include a removable or non-removable (or fixed) medium. In appropriate cases, memory 402 may be inside or outside of an integrated gateway disaster recovery device. In a specific embodiment, memory 402 is a non-volatile solid-state memory.

[0174] The memory may include read-only memory (ROM), random access memory (RAM), magnetic disk storage media devices, optical storage media devices, flash memory devices, electrical, optical or other physical / tangible memory storage devices. Thus, typically, the memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the method according to an aspect of the present disclosure.

[0175] The processor 401 implements any one of the autonomous driving methods in the above embodiments by reading and executing computer program instructions stored in the memory 402.

[0176] In one example, the autonomous driving device may further include a communication interface 403 and a bus 410. Figure 4 As shown, the processor 401, the memory 402, and the communication interface 403 are connected via a bus 410 and communicate with each other.

[0177] The communication interface 403 is mainly used to implement communication between various modules, devices, units and / or equipment in the embodiments of the present application.

[0178] Bus 410 includes hardware, software or both, and the parts of online data flow billing equipment are coupled to each other. For example, but not limitation, bus may include accelerated graphics port (AGP) or other graphics bus, enhanced industrial standard architecture (EISA) bus, front-side bus (FSB), hypertransport (HT) interconnection, industrial standard architecture (ISA) bus, infinite bandwidth interconnection, low pin count (LPC) bus, memory bus, micro channel architecture (MCA) bus, peripheral component interconnection (PCI) bus, PCI-Express (PCI-X) bus, serial advanced technology attachment (SATA) bus, video electronics standard association local (VLB) bus or other suitable bus or two or more of these combinations. In appropriate cases, bus 410 may include one or more buses. Although the present application embodiment describes and shows a specific bus, the present application considers any suitable bus or interconnection.

[0179] In addition, in combination with the autonomous driving method in the above embodiments, the present application embodiment may provide a computer storage medium for implementation. The computer storage medium stores computer program instructions; when the computer program instructions are executed by a processor, any one of the autonomous driving methods in the above embodiments is implemented.

[0180] It should be clear that the present application is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present application is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present application.

[0181] The functional blocks shown in the above-described block diagram can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present application are programs or code segments that are used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link by a data signal carried in a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, etc. The code segment can be downloaded via a computer network such as the Internet, an intranet, etc.

[0182] It should also be noted that the exemplary embodiments mentioned in this application describe some methods or systems based on a series of steps or devices. However, this application is not limited to the order of the above steps, that is, the steps can be performed in the order mentioned in the embodiment, or in a different order from the embodiment, or several steps can be performed simultaneously.

[0183] Aspects of the present disclosure are described above with reference to the flowchart and / or block diagram of the method, device (system) and computer program product according to the embodiment of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device to produce a machine so that these instructions executed by the processor of the computer or other programmable data processing device enable the implementation of the function / action specified in one or more boxes of the flowchart and / or block diagram. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field programmable logic circuit. It can also be understood that each box in the block diagram and / or flowchart and the combination of boxes in the block diagram and / or flowchart can also be implemented by dedicated hardware that performs a specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions.

[0184] The above is only a specific implementation of the present application. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, modules and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. It should be understood that the protection scope of the present application is not limited to this. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in this application, and these modifications or replacements should be included in the protection scope of this application.

Claims

1. A method for automatic driving, characterized in that: The method comprises: Obtain road information and driving status of autonomous vehicles; According to the road information and the driving state, respectively determine the probability that the autonomous driving vehicle performs a plurality of first driving actions according to preset rules through a pre-trained decision model, and determine a target driving action based on the probabilities, wherein the preset rules are set based on the road information and the driving state; driving the autonomous driving vehicle according to the target driving action; The road information includes road vehicle information, road lane line information, and road vehicle running track information. The method comprises: determining the probability of the automatic driving vehicle performing a plurality of first driving actions according to a preset rule based on the road information and the driving state through a pre-trained decision model, and determining a target driving action based on the probability. The preset rule is set based on the road information and the driving state. The method comprises: Determine, by means of the pre-trained decision model, a first distance between the autonomous driving vehicle and the road vehicle, a second distance between the autonomous driving vehicle and the road lane line, and a predicted trajectory of the road vehicle according to the road vehicle information, the road lane line information, and the running trajectory information of the road vehicle; Calculating, based on the first distance, the second distance, the predicted trajectory of the road vehicle, and the driving state, the probability that the autonomous driving vehicle performs the plurality of first driving actions respectively according to a preset rule; A target driving action is determined according to the probability.

2. The method according to claim 1, characterized in that The method of calculating the probability that the autonomous driving vehicle respectively performs the first driving actions according to a preset rule based on the first distance, the second distance, the predicted trajectory of the road vehicle and the driving state includes: Calculating reward values ​​for collisions caused by the autonomous driving vehicle performing a plurality of first driving actions respectively according to the first distance, the second distance, the predicted trajectory of the road vehicle, and the driving state; Calculating, based on the reward value, an estimated probability that the autonomous driving vehicle performs the first driving action; The estimated probability is weighted according to a preset rule to obtain the probability that the autonomous driving vehicle performs the first driving action.

3. The method according to claim 1 or 2, characterized in that: The method of determining a target driving action according to the probability comprises: According to the probability, the target driving action is determined by an ε-greedy algorithm.

4. The method according to claim 1, characterized in that: Before determining, according to the road information and the driving state, the probabilities of the automatic driving vehicle performing a plurality of first driving actions according to preset rules by using a pre-trained decision model, and determining a target driving action based on the probabilities, and before the preset rules are set based on the road information and the driving state, the method further includes: Acquire simulated road information and simulated driving status of a simulated autonomous driving vehicle; The initial decision model is trained according to the simulated road information and the simulated driving state to obtain a trained decision model.

5. The method according to claim 4, characterized in that The method includes training the initial decision model according to the simulated road information and the simulated driving state to obtain a trained decision model. Determine, according to the simulated road information and the simulated driving state, a simulated first distance between the simulated autonomous driving vehicle and a simulated road vehicle, a simulated second distance between the simulated autonomous driving vehicle and a lane line of a simulated road, and a simulated predicted trajectory of the simulated road vehicle, and train the initial decision model to obtain a trained decision model; When the output result of the trained decision model meets the training stop condition, the trained decision model is determined to be the decision model.

6. An automatic driving device, characterized in that: The device comprises: An acquisition module, used to obtain road information and driving status of the autonomous driving vehicle; a determination module, configured to determine, according to the road information and the driving state, respectively the probability that the autonomous driving vehicle performs a plurality of first driving actions according to a preset rule by using a pre-trained decision model, and determine a target driving action based on the probabilities, wherein the preset rule is set based on the road information and the driving state; An execution module, configured to drive the autonomous driving vehicle according to the target driving action; The road information includes road vehicle information, road lane line information, and road vehicle running track information, and the probabilities of the automatic driving vehicle performing a plurality of first driving actions according to preset rules are determined respectively according to the road information and the driving state through a pre-trained decision model, and a target driving action is determined based on the probabilities, and the preset rules are set based on the road information and the driving state; The method according to claim 1 comprises: Determine, by means of the pre-trained decision model, a first distance between the autonomous driving vehicle and the road vehicle, a second distance between the autonomous driving vehicle and the road lane line, and a predicted trajectory of the road vehicle according to the road vehicle information, the road lane line information, and the running trajectory information of the road vehicle; Calculating, based on the first distance, the second distance, the predicted trajectory of the road vehicle, and the driving state, the probability that the autonomous driving vehicle performs the plurality of first driving actions respectively according to a preset rule; A target driving action is determined according to the probability.

7. An automatic driving device, characterized in that: The device comprises: a processor and a memory storing computer program instructions; When the processor executes the computer program instructions, the method for autonomous driving as described in any one of claims 1 to 5 is implemented.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer program instructions, which, when executed by a processor, implement the method for autonomous driving as described in any one of claims 1 to 5.

9. A computer program product, characterized in that When the instructions in the computer program product are executed by a processor of an electronic device, the electronic device executes the method for autonomous driving as described in any one of claims 1 to 5.

Images