Vehicle safe driving method, device and equipment and storage medium

Abstract

The invention discloses a vehicle safe driving method, device and equipment and a storage medium, and relates to the technical field of man-machine interaction and reinforcement learning, and the method comprises the steps that according to obtained vehicle parameters, the driving intention of surrounding vehicles is determined, and the vehicle parameters comprise the vehicle parameters of the surrounding vehicles and the vehicle parameters of a control vehicle; according to the vehicle parameters and the driving intention, the trained neural network is applied to generate a driving decision for controlling the vehicle, and the vehicle is controlled to run according to the driving decision. According to the invention, when the driving decision is generated, the driving intention of the surrounding vehicles is considered, the problem that the human driving behavior cannot be predicted is improved, and the probability of occurrence of traffic accidents is reduced.

Description

Technical Field

[0001] The present application relates to the field of human-computer interaction and reinforcement learning technology, and in particular to a vehicle safe driving method, device, equipment and storage medium. Background Art

[0002] With the rapid development of science and technology, autonomous driving technology has developed rapidly, and vehicle behavior decision-making technology has become a key component of it.

[0003] Conventional automatic determination methods for driving decisions mainly rely on fixed rules and logic, but driving scenarios are complex and diverse, and the decision rules required for each scenario may be different. When faced with new scenarios, decision rules are prone to failure. In addition, human driving behavior is highly complex and diverse. In the same overtaking scenario, different drivers may make different choices based on their own intentions, risk preferences, road condition perception and other factors. For example, some drivers may choose to overtake without meeting the overtaking conditions in order to quickly overtake the vehicle in front, while other drivers may be more cautious and only overtake when it is safe. Therefore, conventional driving decision-making methods may have difficulty adapting to new scenarios or misunderstand the driver's intentions, leading to traffic accidents such as vehicle collisions.

[0004] The above contents are only used to assist in understanding the technical solution of the present application and do not constitute an admission that the above contents are prior art. Summary of the invention

[0005] The main purpose of this application is to provide a vehicle safe driving method, device, equipment and storage medium, aiming to reduce the probability of traffic accidents.

[0006] To achieve the above objectives, the present application proposes a vehicle safe driving method, the method comprising:

[0007] Determining the driving intention of surrounding vehicles according to the acquired vehicle parameters, wherein the vehicle parameters include vehicle parameters of the surrounding vehicles and vehicle parameters of the operating vehicle;

[0008] According to the vehicle parameters and the driving intention, the trained neural network is applied to generate a driving decision for controlling the vehicle, and the controlled vehicle is controlled to travel according to the driving decision.

[0009] In one embodiment, before the step of applying the trained neural network, the method further includes:

[0010] Construct a neural network based on the state parameters of the vehicles in the training samples;

[0011] The neural network is trained according to the state parameters of the vehicle and the reinforcement learning algorithm until a preset total number of training times is reached.

[0012] In one embodiment, the step of training the neural network according to the state parameters of the vehicle and the reinforcement learning algorithm comprises:

[0013] Obtaining a decision output according to the state parameters of the vehicle, the reinforcement learning algorithm and the training vehicle intention, wherein the training vehicle intention is obtained based on the state parameters of the vehicle and the Bayesian reasoning algorithm;

[0014] Determining a control amount of the manipulated vehicle according to the decision output, wherein the control amount is a change value of a driving parameter of the manipulated vehicle;

[0015] The control amount is applied to the controlled vehicle, a reward value of the neural network is determined according to the adjusted vehicle state parameters, the neural network is updated, and the steps of obtaining a decision output according to the vehicle state parameters, the training vehicle intention and the reinforcement learning algorithm are repeated until a preset training stop condition is met.

[0016] In one embodiment, the step of determining the control amount of manipulating the vehicle according to the decision output includes:

[0017] In a case where the decision output is a mean and a variance of the driving parameter, determining a normal distribution of the driving parameter based on the decision output;

[0018] According to the normal distribution, a change value of the driving parameter is determined.

[0019] In one embodiment, after the step of training the neural network, the method further includes:

[0020] Generate a reward value curve according to the training rounds and the reward value;

[0021] When the reward value curve tends to be stable, determining the neural network with the highest reward value as the trained neural network;

[0022] When the reward value curve shows an upward trend, resetting the total number of training times;

[0023] When the reward value curve oscillates, the reinforcement learning algorithm is replaced and the neural network is retrained.

[0024] In one embodiment, before the step of training the neural network, the method further includes:

[0025] The training stop condition is determined according to the current scene, wherein, when the current scene is an overtaking scene, the training stop condition includes that the horizontal interval between the operating vehicle and the surrounding vehicles is less than a preset center axis deviation distance, and the front and rear interval between the operating vehicle and the surrounding vehicles is not less than a preset vehicle distance.

[0026] In one embodiment, before the step of training the neural network, the method further includes:

[0027] The total number of training times of the neural network is determined based on the historical training data and the vehicle parameters.

[0028] In addition, to achieve the above-mentioned purpose, the present application also proposes a vehicle safety driving device, the vehicle safety driving device comprising:

[0029] A vehicle intention determination module, used to determine the vehicle driving intention of the surrounding vehicles according to the acquired vehicle parameters of the operating vehicle and the surrounding vehicles;

[0030] A driving decision generation module is used to apply a trained neural network according to the vehicle parameters and the vehicle driving intention to generate a driving decision for the controlled vehicle, and control the controlled vehicle to drive according to the driving decision.

[0031] In addition, to achieve the above-mentioned purpose, the present application also proposes a vehicle safety driving device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, and the computer program is configured to implement the steps of the vehicle safety driving method described above.

[0032] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the vehicle safety driving method described above are implemented.

[0033] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, it implements the steps of the vehicle safety driving method described above.

[0034] One or more technical solutions proposed in this application have at least the following technical effects: by obtaining the vehicle parameters of the controlled vehicle and the vehicle parameters of the surrounding vehicles, the vehicle driving intention of the surrounding vehicles is determined, the prediction accuracy of the surrounding vehicle behavior is improved, and the controlled vehicle can perceive and understand the potential intention of the surrounding vehicles in advance, thereby effectively avoiding traffic accidents caused by misunderstanding or delayed judgment. Further, based on the vehicle parameters and the determined vehicle driving intention, the trained neural network is applied to generate the driving decision of the controlled vehicle, the current traffic environment can be determined based on the vehicle parameters of the surrounding vehicles obtained, and the execution capability of the current controlled vehicle can be determined based on the vehicle parameters of the controlled vehicle. Then, the current traffic environment, the execution capability of the controlled vehicle and the driving intention of the surrounding vehicles are comprehensively considered through the neural network to formulate a driving strategy suitable for the current scene, so that the controlled vehicle can make a more rapid and accurate response when facing a complex and changeable traffic environment; further control the controlled vehicle to drive according to the driving strategy formulated above, to achieve safe and efficient driving, reduce traffic accidents caused by human misjudgment or improper operation, and reduce the probability of safety accidents. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0036] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0037] Figure 1 A schematic diagram of a process flow provided for the first embodiment of the vehicle safe driving method of the present application;

[0038] Figure 2 A schematic diagram of the initial environment provided in Example 1 of the present application;

[0039] Figure 3 A schematic diagram of the actual driving process of a vehicle provided in Example 1 of the present application;

[0040] Figure 4 A curve diagram showing the change in reward value during training provided in Example 3 of the present application;

[0041] Figure 5 A schematic diagram of a neural network training process provided in Example 3 of the present application;

[0042] Figure 6 This is a schematic diagram of the module structure of the vehicle safety driving device according to an embodiment of the present application;

[0043] Figure 7 Schematic diagram of the device structure of the hardware operating environment involved in the vehicle safety driving method in the embodiment of the present application. DETAILED DESCRIPTION

[0044] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0045] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0046] In this embodiment, for the convenience of description, the following description is made with the vehicle terminal as the execution subject.

[0047] Since conventional automatic existing driving decision-making systems mainly rely on fixed rules and logic written by humans, these rules need to be manually written by experienced technicians based on driving experience, traffic rules and road safety standards. Due to the complexity and diversity of driving scenarios, the decision rules required for each scenario may be different. Therefore, it is time-consuming and labor-intensive to develop a decision maker that can cover all possible scenarios. When faced with new scenarios, the decision maker is prone to failure and the migration cost is also high. In addition, human driving behavior is highly diverse and has individual intention differences. Traditional decision makers lack understanding and modeling of driver intentions, and only rely on rule judgments and historical data. They cannot accurately predict the intentions of human driving behavior, which leads to the disconnection between decision results and actual driving behavior, increasing the possibility of traffic accidents.

[0048] The present application provides a solution, which firstly obtains the vehicle parameters of the controlled vehicle and the vehicle parameters of the surrounding vehicles, combines with the Bayesian reasoning algorithm, predicts the driving intentions of the surrounding vehicles, captures the dynamic changes of the surrounding vehicles, and provides reliable data support for the determination of subsequent driving decisions; then, based on the vehicle parameters and vehicle intentions, applies the trained neural network to generate driving decisions for the controlled vehicle, and the neural network can learn the decision logic and reactions of human drivers when facing different scenarios through training with a large amount of sample data; at the same time, during the training process of the neural network, the reinforcement learning algorithm is applied to continuously update the relevant parameters and weights of the neural network, and the best performing round is selected as the trained neural network, which is applied to the actual autonomous driving decision, thereby improving the accuracy and rationality of the driving decision; finally, the controlled vehicle drives according to the generated driving decision, which can effectively avoid potential conflicts with surrounding vehicles and reduce the probability of traffic accidents.

[0049] It should be noted that the execution subject of this embodiment may be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device capable of realizing the above functions, etc. The following takes the vehicle terminal as an example of the execution subject to illustrate this embodiment and the following embodiments.

[0050] Based on this, the embodiment of the present application provides a vehicle safe driving method, referring to Figure 1 , Figure 1 This is a flow chart of the first embodiment of the vehicle safe driving method of the present application.

[0051] In this embodiment, the vehicle safe driving method includes steps S10 to S20:

[0052] Step S10, determining the driving intention of surrounding vehicles according to the acquired vehicle parameters, wherein the vehicle parameters include vehicle parameters of surrounding vehicles and vehicle parameters of the operating vehicle;

[0053] In one feasible embodiment, parameters such as speed, position, acceleration, etc. of surrounding vehicles are collected, and a Bayesian reasoning algorithm is used to predict the next possible actions or intentions of these vehicles.

[0054] It should be noted that vehicle parameters refer to various physical states and behavioral characteristics of the vehicle during driving, such as speed, position, acceleration, etc.; the acquired vehicle parameters include parameters of the currently driven vehicle and surrounding vehicles. Regarding the selection of surrounding vehicles, it can be all vehicles within a preset range, or it can be a preset number of vehicles closest to the currently driven vehicle. This embodiment does not make specific limitations on this.

[0055] In addition, it should be noted that driving intention refers to the actions that the vehicle driver or the autonomous driving system plans to take, such as changing lanes, accelerating, stopping, etc., which can be inferred based on vehicle parameters through intention reasoning methods.

[0056] For example, the Bayesian inference algorithm can be used to infer the driving intention of the surrounding vehicles. When the preset range of the plane road only includes the operating vehicle and one surrounding vehicle, the vehicle intention includes the tendency to drive to the left or right, and whether the attention is focused. It can be determined based on the horizontal and vertical coordinates, speed and acceleration of the operating vehicle and the surrounding vehicles. The joint probability distribution of the two intentions at time t+1 is b t+1 =[θ M ,M|x t+1 ], that is, given the state x at time t+1 t+1 In the state of attention θ M The joint probability distribution of the vehicle's driving direction M is given by the inference formula based on Bayes' law as follows:

[0057]

[0058] in, represents the vehicle parameters at time t, the superscript H represents the relevant parameters of the surrounding vehicles, and the superscript R represents the relevant parameters of the controlled vehicle; I t =[x t ,I t-1 ], which represents the cumulative list of environmental states at time t; Indicates the relevant adjustment amount of the vehicle at time t, such as vehicle speed, deviation angle, etc.; It indicates whether the attention is focused when the driving tendency intention M is given at time t+1. M The posterior probability of and the current state I t Next, we observe x t+1 The probability of M in I t The prior probability under t+1 ) represents the posterior probability of the driving tendency intention M at time t+1, and also considers the given control amount and the current state I t Next, we observe x t+1 The probability that M is in I t The prior probability under ; represents the temporary update of the joint probability distribution at time t+1, and its value is and M - The product of the posterior probabilities of t (·) is a probability distribution calculation function, which integrates and adjusts the temporarily updated probability distribution generated above, including smoothing and normalizing the probability distribution; From (θ M ,M) to The transition model describes the probability of state transition; b t+1 Represents the final joint probability distribution at time t+1, and calculates the function g through the probability distribution t (·) Update, which involves updating all possible and 's points.

[0059] For example, in addition to the vehicle intentions of surrounding vehicles, traffic signals and pedestrian behavior can also be combined to comprehensively judge traffic conditions through Bayesian reasoning algorithms to optimize driving decisions. Traffic signals, pedestrian behavior, and vehicle driving data are collected in real time through sensors, cameras, and other equipment, and the prior probability of each node in the Bayesian network is updated based on the collected data, such as the probability that the traffic light is red in a certain period of time, the probability that pedestrians run red lights, etc. The posterior probability of each node is calculated based on the real-time data and the updated prior probability using Bayesian theorem, that is, the various possibilities under the current traffic conditions.

[0060] It can be understood that by predicting the driving intentions of surrounding vehicles, responses can be made in advance, safer decisions can be made, and the occurrence of traffic accidents can be reduced; at the same time, by accurately identifying the intentions of surrounding vehicles, smoother driving actions can be produced, improving passenger comfort.

[0061] Step S20, applying the trained neural network according to the vehicle parameters and driving intention, generating a driving decision for controlling the vehicle, and controlling the vehicle to drive according to the driving decision.

[0062] For example, the number of surrounding vehicles in the current environment can be determined based on the vehicle parameters obtained within the preset range, the input dimension of the neural network can be determined based on the number of surrounding vehicles, and the corresponding neural network can be selected. For example, referring to Figure 2 The initialization environment includes two lanes and two cars A and B on the road. Car A is a surrounding vehicle and car B is the current control vehicle. Figure 2 In the example, car A is heading to the right, car B is behind car A, and eight-dimensional variables, including the horizontal coordinate, vertical coordinate, speed, and acceleration of both cars, are selected as vehicle parameters. The probability distribution of human intentions determined based on the Bayesian inference algorithm is added. Human intentions include the tendency to drive to the left or right, and whether attention is focused, for a total of 12 dimensions. Therefore, the input of the trained neural network should be [x H ,y h ,v H ,a H ,x R ,y R ,v R ,a R ,θ,1-θ,m l ,m r ], where θ is the probability of focused attention, 1-θ is the probability of distracted attention, and m l is the probability of driving left, m r is the probability of driving on the right. During driving, the intention of car A is perceived to be left-leaning, attention is focused [θ,1-θ,m l ,m r]=[1,0,1,0], change to rightward tendency, distraction [θ,1-θ,m l ,m r ]=[0,1,0,1], it is predicted that car A may change lanes to the right, please refer to Figure 3 The actual driving situation of the car, Figure 3 In the box, car B takes human intention into consideration and predicts car A's lane change to the right. It makes the driving decision to evade to the left and overtake in the left lane. Figure 2 and Figure 3 , it can be seen that car A drove from the left lane to the right lane. Car B observed car A's intention to change lanes to the right and changed lanes to the left in advance to avoid a vehicle collision. Figure 2 The left rear of the original car A has reached Figure 3 In front of the right side of car A after changing lanes.

[0063] This embodiment provides a vehicle safe driving method, which can generate driving decisions by collecting surrounding environment, and has the generalization ability to face different driving environments and traffic conditions; by generating driving decisions through a trained neural network, it can more accurately judge the vehicle's state parameters and vehicle intentions, and generate smooth driving decisions accordingly, while ensuring the riding experience, avoiding or reducing the occurrence of traffic accidents.

[0064] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above-mentioned embodiment 1 can be referred to the above introduction, and will not be repeated later. On this basis, before step S20, it also includes:

[0065] Step A10, constructing a neural network according to the state parameters of the vehicles in the training samples;

[0066] In a feasible embodiment, the structure of the neural network is designed according to the vehicle parameters (vehicle state parameters) of the surrounding cars and the controlled car in the training samples, and the vehicle intentions of the surrounding cars determined based on the Bayesian inference algorithm, including the number of layers, the number of neurons, the activation function, the input dimension of the hidden layer, etc.

[0067] Step A20, training the neural network according to the state parameters of the vehicle and the reinforcement learning algorithm until a preset total number of training times is reached.

[0068] It should be noted that the reinforcement learning algorithm is a machine learning algorithm that allows an intelligent agent (such as a neural network) to take actions in the environment and optimizes its behavior strategy based on the rewards or penalties brought about by the actions, such as the PPO (Proximal Policy Optimization) algorithm.

[0069] For example, the agent predicts that the car has two possible actions C and D in a certain state. Initially, it predicts that the probability of taking action C in this state is 0.7, and the probability of taking action D is 0.3; after observing the execution of action C, the probability distribution is updated based on this new experience, and the probability of action C may be temporarily adjusted to 0.8 and action D to 0.2; the agent is repeatedly trained and the relevant parameters are adjusted until the preset number of training times is reached.

[0070] In this embodiment, through the training of the reinforcement learning algorithm, it is possible to gradually learn to make optimal decisions in complex environments and adapt to different environments and task requirements.

[0071] In a feasible implementation manner, step A20 includes:

[0072] Step A21, obtaining a decision output according to the state parameters of the vehicle, the reinforcement learning algorithm and the training vehicle intention, wherein the training vehicle intention is obtained based on the state parameters of the vehicle and the Bayesian reasoning algorithm;

[0073] It should be noted that decision output refers to specific driving decisions generated based on input information, such as acceleration, changing direction, etc. Since the reinforcement learning algorithm is used, the decision output is expressed in the form of probability distribution.

[0074] For example, the horizontal and vertical coordinates, speed and acceleration of the controlled vehicle and surrounding vehicles are used as the state parameters of the vehicle, and the concentration and the tendency to drive left or right are determined as the vehicle intention. The specific calculation process based on the Bayesian algorithm reasoning can refer to the above-mentioned embodiment 1, so it can be determined that the input dimension of the neural network is twelve dimensions. The PPO algorithm is applied, and a four-dimensional output can be obtained after the neural network calculation. in, is the mean value of speed adjustment, v var is the speed adjustment variance, is the mean value of the angle adjustment, is the angle adjustment variance.

[0075] It can be understood that by combining the vehicle's state parameters, training vehicle intentions and reinforcement learning algorithms, accurate decision outputs can be generated, thereby enhancing adaptability to complex environments.

[0076] Step A22, determining a control amount for manipulating the vehicle according to the decision output, wherein the control amount is a change value of a driving parameter for manipulating the vehicle;

[0077] It should be noted that driving parameters refer to variables used to control vehicle behavior, such as speed, direction, acceleration, etc.

[0078] For example, it is detected that the vehicle ahead is decelerating, and the probability of deceleration in the decision output is 0.8, and the probability of overtaking is 0.2. Combined with the status of surrounding vehicles, the angle control amount is determined to be 0 and the speed control amount is -30 kilometers per hour.

[0079] It can be understood that by determining the control amount according to the decision output, precise control of the vehicle can be achieved, thereby improving driving safety and stability.

[0080] Step A23, applying the control amount to control the vehicle, determining the reward value of the neural network according to the adjusted vehicle state parameters, updating the neural network, and repeating the steps of obtaining the decision output according to the vehicle state parameters, training vehicle intentions and reinforcement learning algorithms until the preset training stop conditions are met.

[0081] In a feasible embodiment, while the surrounding vehicles are driving, the manipulating vehicle is also changing the driving parameters according to the control amount. According to the changed state parameters of the surrounding vehicles and the manipulating vehicle, such as whether a collision occurs, the distance difference between the surrounding vehicles and the manipulating vehicle, etc., a reward value is assigned to the current neural network, and the relevant weights of the neural network are updated.

[0082] It should be noted that the reward value is an important feedback signal for training neural networks. In reinforcement learning, it is used to evaluate the quality of a certain behavior. For example, if the vehicle successfully avoids a collision, it will receive a positive reward value; if a collision occurs, it will receive a negative reward value.

[0083] For example, when it is observed that the vehicle in front slows down because of the red light and the current speed of the controlled vehicle is 30 kilometers per hour, a corresponding decision output is generated, and a speed control amount of -30 kilometers per hour is applied to the controlled vehicle to smoothly reduce the speed of the controlled vehicle to 0; after the controlled vehicle stops, check whether the distance to the vehicle in front is within a safe range (preset stop condition), give the current neural network a positive reward value, and dynamically adjust the relevant parameters of the neural network, such as learning rate, batch size, etc., to end the current training; adjust the training sample to the initial state, and apply the adjusted neural network to start a new round of training until the preset number of training times is reached.

[0084] For example, in addition to the reward value, the weight update of the neural network can also be adjusted according to user feedback. During driving, natural language processing technology can be combined to understand the user's language instructions and adjust the relevant decision output according to the instructions.

[0085] In this implementation, by continuously learning and optimizing driving strategies, environmental information around the vehicle, such as road signs, pedestrians and other vehicles, can be more accurately identified, thereby reducing the occurrence of traffic accidents such as collisions.

[0086] In a feasible implementation manner, step A22 includes:

[0087] Step A221, when the decision output is the mean and variance of the driving parameter, determining the normal distribution of the driving parameter according to the decision output;

[0088] For example, when it is predicted that the vehicle needs to decelerate at a red light, the mean and variance of the speed are calculated based on the decision output to determine the speed distribution during the deceleration process.

[0089] Step A222, determining the change value of the driving parameter according to the normal distribution.

[0090] In a feasible embodiment, according to the probability of occurrence of each possible value in the normal distribution, a value is randomly selected from the normal distribution and determined as the change value of the driving parameter.

[0091] For example, 4 (preset number) speed change values ​​can be randomly selected according to the normal distribution, and combined with the vehicle's fuel consumption data, the change pattern of fuel consumption at different speeds can be analyzed, and the one with the lowest fuel consumption can be selected as the change value of the driving parameter, thereby further improving the vehicle's energy efficiency.

[0092] In this implementation, by randomly extracting the normal distribution, it can be ensured that the speed change value is smooth and reasonable in most cases, avoiding extreme or unrealistic speed changes.

[0093] Based on the first embodiment and / or the second embodiment of the present application, in the third embodiment of the present application, the same or similar contents as those of the above-mentioned first and second embodiments can be referred to the above introduction, and will not be repeated later. On this basis, after step A20, it also includes:

[0094] Step A30, generating a reward value curve according to the training rounds and the reward value;

[0095] It should be noted that the reward value curve is used to visualize and analyze the changing trend of the reward value during the training process, so as to evaluate the performance and convergence of the current neural network. By analyzing the reward value curve, trends in the training process, such as overfitting or convergence, can be identified.

[0096] Step A31, when the reward value curve tends to be stable, determining the neural network with the highest reward value as the trained neural network;

[0097] It can be understood that a smooth reward value curve indicates that the neural network has learned to make stable and reliable decisions on a specific task. Selecting the neural network with the highest reward value can ensure that the decision-making process is based on the best strategy learned by the network during training.

[0098] For example, please refer to Figure 4 , Figure 4 A reward value change curve during training is provided. Figure 4 The horizontal axis of the curve graph represents the number of training times, and the vertical axis represents the corresponding reward value. Figure 4 It is not difficult to see that the reward value changes eventually stabilize at around -1. Figure 4 The reward value at the middle dotted line is the highest, so the neural network of the corresponding training round is used as the neural network finally applied in the actual driving process.

[0099] Step A32, when the reward value curve shows an upward trend, reset the total number of training times;

[0100] It is understandable that after the training is completed, the reward value curve still shows an upward trend, indicating that the reward value has not yet reached the fitting, the performance of the neural network has not yet reached the best, the reinforcement learning algorithm is learning and gradually adjusting the parameters of the neural network to obtain a higher reward value. The number of training times can be further increased so that the neural network can learn the best decision-making strategy.

[0101] Step A33, when the reward value curve oscillates, replace the reinforcement learning algorithm and retrain the neural network.

[0102] It is understandable that the reward value curve fluctuates, that is, the reward value curve remains high and low, indicating that the current reinforcement learning algorithm may not be able to stably optimize the neural network, or there are problems such as overfitting. Replacing other reinforcement learning algorithms may be more helpful for the neural network to learn driving strategies and thus show better performance.

[0103] In this embodiment, by analyzing the changes in the reward value curve, the current training status of the neural network is determined, and then a decision is made whether to end the training or adjust the relevant configuration of the neural network to obtain higher performance.

[0104] In a feasible implementation manner, before step A20, the method further includes:

[0105] Step B01, determining the training stop conditions according to the current scene, wherein, when the current scene is an overtaking scene, the training stop conditions include that the horizontal interval between the operating vehicle and the surrounding vehicles is less than a preset center axis deviation distance, and the front and rear interval between the operating vehicle and the surrounding vehicles is not less than a preset vehicle distance.

[0106] It should be noted that the center axis deviation distance refers to the distance deviation of the center line of the vehicle in the horizontal direction. If the horizontal distance between the two vehicles is less than the center axis deviation distance, it means that the two vehicles are currently traveling in the center of the road and have not deviated to the outside of the lane; at the same time, the front and rear distance between the two vehicles is not less than the preset distance, which indicates that the current distance is safe. Therefore, it can be determined that the controlled vehicle has safely completed the overtaking task.

[0107] For example, in Figure 2 In the road situation, car B tries to overtake car A. We can establish a plane lane coordinate system with the right as the positive direction of the x-axis, and use the x and y distributions to represent the horizontal and vertical coordinates of the vehicle in the lane coordinate system. The training stop condition can refer to the following formula:

[0108] x B -x A ≥d x

[0109] |y B -y A | <d y

[0110] Among them, d x is the preset minimum front and rear safe distance, d y is the preset deviation distance from the central axis. Once the above conditions are met, it is considered that car B has completed the overtaking task. According to the principle of reinforcement learning reward setting, a high score is given to this situation and this round of training ends.

[0111] In this embodiment, by ensuring that the controlled vehicle maintains a safe distance from surrounding vehicles when overtaking, the probability of traffic accidents is reduced; by setting different training stop conditions according to different scenarios, the adaptability to different scenarios is enhanced.

[0112] In a feasible implementation manner, before step A20, the method further includes:

[0113] Step E01, determining the total number of training times of the neural network based on historical training data and state parameters of the vehicle.

[0114] Exemplarily, the total number of training rounds for the current training sample is determined based on the total number of training rounds required to achieve optimal performance under the historical training data, i.e., the same number of historical state parameters. For example, there are two vehicles on the current road, and under the same circumstances in the historical training data, the total number of training rounds required to achieve optimal performance is about 1000. Therefore, the total number of training rounds for the neural network can be set to 1200.

[0115] Exemplarily, the total number of training times of the neural network may be further adjusted according to the depth of the neural network, the number of hidden layers, etc.

[0116] In this embodiment, by rationally utilizing historical training data and vehicle parameters, the training cost can be reduced, the training process of the neural network can be optimized, the training efficiency of the neural network can be improved, and over-training or under-training can be avoided.

[0117] For example, to help understand the intention-based reasoning autonomous driving vehicle reinforcement learning training process obtained after the above embodiment, please refer to Figure 5 , Figure 5 A neural network training process diagram is provided, specifically:

[0118] Step S101, reading the state parameters of the vehicle, reading the state parameters of the currently controlled vehicle and surrounding vehicles, including but not limited to the horizontal and vertical coordinates of the vehicle based on the world coordinate system, the speed and acceleration of the vehicle, etc.;

[0119] Step S102, determining the vehicle intention of the surrounding vehicles, and inferring the driving intention of the vehicle based on the Bayesian reasoning algorithm and the state parameters of the vehicle;

[0120] Step S103, determining the control amount of the controlled vehicle, using a reinforcement learning algorithm, based on the vehicle intention and state parameters, determining the decision output of the current controlled vehicle, and then determining the change value of the driving parameter of the controlled vehicle, i.e., the control amount, according to the decision output;

[0121] Step S104, when the above determined control amount is applied, the current state parameters of the controlled vehicle and surrounding vehicles are updated;

[0122] Step S105, determine whether the distance between the controlled vehicle and the surrounding vehicles is in a safe state (reaching the preset stop condition) under the current state. If the preset stop condition is not reached, repeat the above steps S101 to S105 until the preset stop condition is reached, and enter step S106 to end this round of training.

[0123] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the vehicle safety driving method of the present application. More simple transformations based on this technical concept are all within the scope of protection of the present application.

[0124] The present application also provides a vehicle safety driving device, please refer to Figure 6 , the vehicle safety driving device includes:

[0125] The vehicle intention determination module 10 is used to determine the driving intention of the surrounding vehicles according to the acquired vehicle parameters, wherein the vehicle parameters include vehicle parameters of the surrounding vehicles and vehicle parameters of the operating vehicle;

[0126] The driving decision generation module 20 is used to generate a driving decision for controlling the vehicle based on vehicle parameters and driving intentions by applying the trained neural network, and control the vehicle to drive according to the driving decision.

[0127] Optionally, the vehicle safety driving device further includes a neural network training module, and the neural network training module is used to:

[0128] Construct a neural network based on the state parameters of the vehicles in the training samples;

[0129] According to the vehicle's state parameters and reinforcement learning algorithm, the neural network is trained until the preset total number of training times is reached.

[0130] Optionally, the neural network training module is further used to:

[0131] Obtaining a decision output according to the state parameters of the vehicle, the reinforcement learning algorithm and the training vehicle intention, wherein the training vehicle intention is obtained based on the state parameters of the vehicle and the Bayesian reasoning algorithm;

[0132] Determining a control amount for manipulating the vehicle according to the decision output, wherein the control amount is a change value of a driving parameter for manipulating the vehicle;

[0133] The control amount is applied to control the vehicle, the reward value of the neural network is determined according to the adjusted vehicle state parameters, the neural network is updated, and the steps of obtaining the decision output according to the vehicle state parameters, training vehicle intentions and reinforcement learning algorithms are repeated until the preset training stop conditions are met.

[0134] Optionally, the neural network training module is further used to:

[0135] In a case where the decision output is the mean and variance of the driving parameter, determining a normal distribution of the driving parameter based on the decision output;

[0136] According to the normal distribution, the change value of the driving parameter is determined.

[0137] Optionally, the neural network training module is further used to:

[0138] Generate a reward value curve based on the training rounds and reward values;

[0139] When the reward value curve tends to be stable, the neural network with the highest reward value is determined as the trained neural network;

[0140] When the reward value curve shows an upward trend, reset the total number of training times;

[0141] When the reward value curve oscillates, replace the reinforcement learning algorithm and retrain the neural network.

[0142] Optionally, the neural network training module is further used to:

[0143] The training stop conditions are determined according to the current scene. When the current scene is an overtaking scene, the training stop conditions include that the horizontal interval between the operating vehicle and the surrounding vehicles is less than a preset center axis deviation distance, and the front and rear interval between the operating vehicle and the surrounding vehicles is not less than a preset vehicle distance.

[0144] Optionally, the neural network training module is further used to:

[0145] The total number of neural network training times is determined based on historical training data and vehicle parameters.

[0146] The vehicle safety driving device provided in the embodiment of the present application adopts the vehicle safety driving method in the above embodiment to reduce the probability of traffic accidents. Compared with the prior art, the beneficial effects of the vehicle safety driving device provided in the present application are the same as the beneficial effects of the vehicle safety driving method provided in the above embodiment, and other technical features in the vehicle safety driving device are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0147] An embodiment of the present application provides a vehicle safety driving device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the vehicle safety driving method in the above-mentioned embodiment one.

[0148] Reference below Figure 7 , which shows a schematic diagram of the structure of a vehicle safety driving device suitable for implementing an embodiment of the present application. The vehicle safety driving device in the embodiment of the present application may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 7 The vehicle safety driving device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0149] like Figure 7As shown, the vehicle safety driving device may include a processing device 1001 (e.g., a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the vehicle safety driving device are also stored. The processing device 1001, ROM1002, and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the vehicle safety driving device to communicate wirelessly or wired with other devices to exchange data. Although the vehicle safety driving device with various systems is shown in the figure, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or provided alternatively.

[0150] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0151] The vehicle safety driving device provided in the embodiment of the present application adopts the vehicle safety driving method in the above embodiment to reduce the probability of traffic accidents. Compared with the prior art, the beneficial effects of the vehicle safety driving device provided in the present application are the same as the beneficial effects of the vehicle safety driving method provided in the above embodiment, and the other technical features in the vehicle safety driving device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0152] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0153] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0154] An embodiment of the present application provides a computer-readable storage medium having computer-readable program instructions (ie, a computer program) stored thereon, and the computer-readable program instructions are used to execute the vehicle safe driving method in the above-mentioned embodiment.

[0155] The computer-readable storage medium provided in the embodiment of the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM: Random Access Memory), a read-only memory (ROM: Read Only Memory), an erasable programmable read-only memory (EPROM: Erasable Programmable Read Only Memory or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM: CD-Read Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency: Radio Frequency), etc., or any suitable combination of the above.

[0156] The computer-readable storage medium may be included in the vehicle safety driving device; or may exist independently without being assembled into the vehicle safety driving device.

[0157] The above-mentioned computer-readable storage medium carries one or more programs. When the above-mentioned one or more programs are executed by the vehicle safety driving device, the vehicle safety driving device: determines the vehicle driving intention of the surrounding vehicles based on the acquired vehicle parameters, wherein the vehicle parameters include the vehicle parameters of the controlled vehicle and the vehicle parameters of the surrounding vehicles; based on the vehicle parameters and the vehicle driving intention, applies the trained neural network to generate a driving decision for the controlled vehicle, and controls the controlled vehicle to drive according to the driving decision.

[0158] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0159] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0160] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0161] The readable storage medium provided in the embodiment of the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned vehicle safe driving method, and can reduce the probability of traffic accidents. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in the present application are the same as the beneficial effects of the vehicle safe driving method provided in the above-mentioned embodiment, and will not be repeated here.

[0162] An embodiment of the present application also provides a computer program product, including a computer program, which implements the steps of the above-mentioned vehicle safety driving method when executed by a processor.

[0163] The computer program product provided in the embodiment of the present application can reduce the probability of traffic accidents. Compared with the prior art, the beneficial effects of the computer program product provided in the present application are the same as the beneficial effects of the vehicle safe driving method provided in the above embodiment, which will not be repeated here.

[0164] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A vehicle safe driving method, characterized in that: The vehicle safe driving method comprises: Determining the driving intention of surrounding vehicles according to the acquired vehicle parameters, wherein the vehicle parameters include vehicle parameters of the surrounding vehicles and vehicle parameters of the operating vehicle; According to the vehicle parameters and the driving intention, the trained neural network is applied to generate a driving decision for the controlled vehicle, and the controlled vehicle is controlled to travel according to the driving decision.

2. The vehicle safe driving method according to claim 1, characterized in that: Before the step of applying the trained neural network, the method further comprises: Construct a neural network based on the state parameters of the vehicles in the training samples; The neural network is trained according to the state parameters of the vehicle and the reinforcement learning algorithm until a preset total number of training times is reached.

3. The vehicle safe driving method according to claim 2, characterized in that: The step of training the neural network according to the state parameters of the vehicle and the reinforcement learning algorithm comprises: Obtaining a decision output according to the state parameters of the vehicle, the reinforcement learning algorithm and the training vehicle intention, wherein the training vehicle intention is obtained based on the state parameters of the vehicle and the Bayesian reasoning algorithm; Determining a control amount of the manipulated vehicle according to the decision output, wherein the control amount is a change value of a driving parameter of the manipulated vehicle; The control amount is applied to the controlled vehicle, a reward value of the neural network is determined according to the adjusted state parameters of the vehicle, the neural network is updated, and the steps of obtaining a decision output according to the state parameters of the vehicle, the training vehicle intention and the reinforcement learning algorithm are repeated until a preset training stop condition is met.

4. The vehicle safe driving method according to claim 3, characterized in that: The step of determining the control amount of manipulating the vehicle according to the decision output comprises: In a case where the decision output is a mean and a variance of the driving parameter, determining a normal distribution of the driving parameter based on the decision output; According to the normal distribution, a change value of the driving parameter is determined.

5. The vehicle safe driving method according to claim 3, characterized in that: After the step of training the neural network, the method further comprises: Generate a reward value curve according to the training rounds and the reward value; When the reward value curve tends to be stable, determining the neural network with the highest reward value as the trained neural network; When the reward value curve shows an upward trend, resetting the total number of training times; When the reward value curve oscillates, the reinforcement learning algorithm is replaced and the neural network is retrained.

6. The vehicle safe driving method according to claim 3, characterized in that: Before the step of training the neural network, the method further includes: The training stop condition is determined according to the current scene, wherein, when the current scene is an overtaking scene, the training stop condition includes that the horizontal interval between the operating vehicle and the surrounding vehicles is less than a preset center axis deviation distance, and the front and rear interval between the operating vehicle and the surrounding vehicles is not less than a preset vehicle distance.

7. The vehicle safe driving method according to claim 2, characterized in that: Before the step of training the neural network, the method further includes: The total number of training times of the neural network is determined based on the historical training data and the vehicle parameters.

8. A vehicle safety driving device, characterized in that: The vehicle safety driving device comprises: A vehicle intention determination module, used to determine the driving intention of surrounding vehicles according to the acquired vehicle parameters, wherein the vehicle parameters include vehicle parameters of the surrounding vehicles and vehicle parameters of the operating vehicle; A driving decision generation module is used to generate a driving decision for the controlled vehicle based on the vehicle parameters and the driving intention by applying a trained neural network, and control the controlled vehicle to travel according to the driving decision.

9. A vehicle safety driving device, characterized in that: The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the vehicle safe driving method according to any one of claims 1 to 7.

10. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the vehicle safety driving method according to any one of claims 1 to 7 are implemented.

Images