Driving behavior decision planning method, system and device and storage medium

Abstract

The invention provides a driving behavior decision planning method, system and device, and a storage medium, and the method comprises the steps: constructing a state space according to the driving state of a current vehicle, and constructing an observation space according to a plurality of state spaces and environmental factors; constructing an action space according to the current stress conditions of front and rear wheels of the vehicle, and constructing a reward function according to behavior prediction, emergency handling and riding comfort in the historical driving data; the state space and the observation space are changed by executing the action space, and a reward value is given to the change of the state space and the observation space through the reward function so as to train each action space; and in response to completion of training, determining a maximum reward value according to the current state space and the current observation space, and executing an action space corresponding to the maximum reward value to realize decision planning of the driving behavior. The method can make a safe and reasonable decision about the shortest distance to the target location, is high in capability of coping with emergencies, and improves the riding comfort of passengers.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving, and more specifically, to a method, system, device and storage medium for driving behavior decision planning. Background Art

[0002] An autonomous driving system refers to an autonomous system that enables safe driving of a vehicle with partial or complete separation from the driver. It mainly includes major technical modules such as environmental perception, path planning, behavioral decision-making, and navigation control. The main obstacle to the large-scale application of autonomous driving technology at this stage is the uncertainty of the driving environment, including weather, road conditions, and the complex interaction between vehicles and other traffic participants. Specifically, in real traffic conditions, the interactive behaviors between different entities are intricate, and the prediction of the behavior and trajectory of vehicle-road intelligent entities is the focus and difficulty of research; in addition, the vehicle-road collaborative method needs to deal with sudden road conditions, and considering the riding comfort of passengers, it is necessary to ensure that the acceleration curve of decision planning is always smooth. Summary of the invention

[0003] In view of this, the purpose of the embodiments of the present invention is to propose a method, system, electronic device and computer-readable storage medium for driving behavior decision planning. The present invention designs reward functions related to behavior prediction, emergency handling, and ride comfort, which can effectively realize the behavior prediction of other intelligent agents in the vehicle-road cooperative system and judge the current road conditions and traffic environment. It can make the shortest distance, safe and reasonable decision to reach the target location, has a strong ability to deal with emergencies, and improves the ride comfort of passengers.

[0004] Based on the above purpose, one aspect of an embodiment of the present invention provides a method for driving behavior decision planning, comprising the following steps: constructing a state space according to the current driving state of a vehicle, and constructing an observation space according to multiple state spaces and environmental factors; constructing an action space according to the force conditions of the front and rear wheels of the current vehicle, and constructing a reward function according to behavior prediction, emergency handling and ride comfort in historical driving data; changing the state space and the observation space by executing the action space, and assigning a reward value to the change of the state space and the observation space through the reward function to train each action space; and in response to the completion of the training, determining the maximum reward value according to the current state space and the current observation space, and executing the action space corresponding to the maximum reward value to realize decision planning for driving behavior.

[0005] In some embodiments, the step of constructing a state space based on the driving state of the current vehicle includes: constructing a plane coordinate system based on longitude and latitude, and constructing a body coordinate system based on the original running direction of the current vehicle; obtaining the coordinates, heading angle and lane information of the current vehicle from the plane coordinate system, and obtaining the speed, acceleration and angular velocity of the current vehicle from the body coordinate system; and constructing a state space based on the coordinates, heading angle, lane information, speed, acceleration and angular velocity.

[0006] In some embodiments, the step of constructing a body coordinate system according to the current original running direction of the vehicle includes: setting the current original running direction of the vehicle as a first direction of the body coordinate system, and setting a direction perpendicular to the first direction as a second direction of the body coordinate system.

[0007] In some embodiments, the step of acquiring the speed, acceleration and angular velocity of the current vehicle from the body coordinate system includes: decomposing the current running speed of the current vehicle into the first direction and the second direction respectively, and determining the acceleration in the first direction and the second direction respectively.

[0008] In some embodiments, the step of constructing an observation space based on multiple state spaces and environmental factors includes: obtaining the state space of the current vehicle at the current moment and multiple consecutive moments before the current moment; respectively obtaining the distance between the current vehicle and the vehicle in front and the vehicle behind, respectively obtaining the congestion coefficient of the current vehicle in the current lane with the vehicle in front and the vehicle behind, and obtaining the map information at the current moment; and constructing the observation space based on the multiple state spaces, the distances, the congestion coefficients and the map information.

[0009] In some implementations, the step of acquiring the map information at the current moment includes: acquiring lane lines, positions of vehicles adjacent to the current vehicle on the left and right, and traffic light status.

[0010] In some embodiments, the step of constructing an action space based on the current force conditions of the front and rear wheels of the vehicle includes: obtaining a first force of the front wheel in the current direction of movement and a second force of the front wheel in a direction perpendicular to the current direction of movement, and obtaining a third force of the rear wheel in the original direction of movement and a fourth force of the rear wheel in a direction perpendicular to the original direction of movement; and constructing an action space based on the first force, the second force, the third force and the fourth force.

[0011] In some embodiments, the step of assigning reward values ​​to changes in the state space and the observation space through the reward function to train each action space includes: in response to the changes in the state space and the observation space resulting in violation of traffic regulations or collision with other people or objects, assigning a first negative reward value to the corresponding action space; in response to any congestion coefficient corresponding to the vehicle exceeding a congestion coefficient threshold or any acceleration exceeding an acceleration threshold, assigning a second negative reward value to the corresponding action space; and in response to the vehicle arriving at the destination and without the above situation, assigning a positive reward value to the corresponding action space, wherein the absolute value of the first negative reward value is greater than the absolute value of the second negative reward value.

[0012] In some embodiments, the step of determining the maximum reward value based on the current state space and the current observation space, and executing the action space corresponding to the maximum reward value to achieve decision planning for driving behavior includes: determining the action space with the maximum reward value at the current moment based on the state space and observation space at the current moment, and continuously executing the action space with the maximum reward value at each moment.

[0013] Another aspect of an embodiment of the present invention provides a system for driving behavior decision planning, including: a construction module, used to construct a state space according to the current driving state of a vehicle, and to construct an observation space according to multiple state spaces and environmental factors; a reward module, used to construct an action space according to the force conditions of the front and rear wheels of the current vehicle, and to construct a reward function according to behavior prediction, emergency handling and ride comfort in historical driving data; an execution module, used to change the state space and the observation space by executing the action space, and to assign a reward value to the change of the state space and the observation space through the reward function to train each action space; and a decision module, used to determine the maximum reward value according to the current state space and the current observation space in response to the completion of training, and to execute the action space corresponding to the maximum reward value to realize decision planning for driving behavior.

[0014] According to another aspect of an embodiment of the present invention, there is provided an electronic device, comprising: at least one processor; and a memory, wherein the memory stores computer instructions executable on the processor, and the instructions implement the steps of the above method when executed by the processor.

[0015] According to another aspect of the embodiments of the present invention, a computer-readable storage medium is provided, which stores a computer program that implements the above method steps when executed by a processor.

[0016] The present invention has the following beneficial technical effects: a reward function related to behavior prediction, emergency handling, and ride comfort is designed, which can effectively realize the behavior prediction of other intelligent entities in the vehicle-road cooperative system and judge the current road conditions and traffic environment, and can make the shortest distance, safe and reasonable decision to reach the target location, has a strong ability to deal with emergencies, and improves passenger ride comfort. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other embodiments can be obtained based on these drawings without paying creative work.

[0018] Figure 1 A schematic diagram of an embodiment of a method for driving behavior decision planning provided by the present invention;

[0019] Figure 2 The architecture diagram of the driving behavior decision planning provided by the present invention;

[0020] Figure 3 A schematic diagram of a vehicle dynamics model provided by the present invention;

[0021] Figure 4 A schematic diagram of an embodiment of a system for driving behavior decision planning provided by the present invention;

[0022] Figure 5 A schematic diagram of the hardware structure of an embodiment of an electronic device for driving behavior decision planning provided by the present invention;

[0023] Figure 6 A schematic diagram of an embodiment of a computer storage medium for driving behavior decision planning provided by the present invention. DETAILED DESCRIPTION

[0024] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the embodiments of the present invention are further described in detail below in combination with specific embodiments and with reference to the accompanying drawings.

[0025] It should be noted that all expressions using "first" and "second" in the embodiments of the present invention are for distinguishing two non-identical entities with the same name or non-identical parameters. It can be seen that "first" and "second" are only for the convenience of expression and should not be understood as limitations on the embodiments of the present invention. The subsequent embodiments will not explain this one by one.

[0026] A first aspect of the embodiments of the present invention provides an embodiment of a method for driving behavior decision planning. Figure 1FIG. 2 is a schematic diagram of an embodiment of a method for driving behavior decision planning provided by the present invention. Figure 1 As shown, the embodiment of the present invention includes the following steps:

[0027] S1. Construct a state space according to the current driving state of the vehicle, and construct an observation space according to multiple state spaces and environmental factors;

[0028] S2: Construct an action space based on the force conditions of the front and rear wheels of the current vehicle, and construct a reward function based on behavior prediction, emergency handling, and ride comfort in historical driving data;

[0029] S3, changing the state space and the observation space by executing the action space, and assigning a reward value to the change of the state space and the observation space through the reward function to train each action space; and

[0030] S4. In response to the completion of the training, a maximum reward value is determined according to the current state space and the current observation space, and an action space corresponding to the maximum reward value is executed to realize decision planning for the driving behavior.

[0031] Figure 2 The architecture diagram of the driving behavior decision planning provided by the present invention is based on Figure 2 The embodiments of the present invention are described.

[0032] In the embodiment of the invention, the self-driving car in the driving scene is equipped with a laser radar to scan and perceive the surrounding environment. In addition, each self-driving car is equipped with a sensor for V2V communication, and the vehicle-road cooperative driving scene parameters can be transmitted to the self-driving car through the V2V communication module. Furthermore, the autonomous vehicle participating in the decision-making can obtain the position, speed, acceleration and other information of other self-driving cars in the vehicle-road cooperative system.

[0033] In some embodiments, the step of constructing a state space based on the driving state of the current vehicle includes: constructing a plane coordinate system based on longitude and latitude, and constructing a body coordinate system based on the original running direction of the current vehicle; obtaining the coordinates, heading angle and lane information of the current vehicle from the plane coordinate system, and obtaining the speed, acceleration and angular velocity of the current vehicle from the body coordinate system; and constructing a state space based on the coordinates, heading angle, lane information, speed, acceleration and angular velocity.

[0034] In some embodiments, the step of constructing a body coordinate system according to the current original running direction of the vehicle includes: setting the current original running direction of the vehicle as a first direction of the body coordinate system, and setting a direction perpendicular to the first direction as a second direction of the body coordinate system. Figure 3 A schematic diagram of a vehicle dynamics model provided by the present invention, such as Figure 3 As shown, X B-CG-Y B and X I -OY I Represent the body coordinate system and plane coordinate system of the current vehicle. In the body coordinate system, X B Indicates the first direction, Y B Indicates the second direction.

[0035] In some embodiments, the step of obtaining the speed, acceleration and angular velocity of the current vehicle from the body coordinate system includes: decomposing the current running speed of the current vehicle into the first direction and the second direction respectively, and determining the acceleration in the first direction and the second direction respectively. The horizontal coordinate of the vehicle at the current moment is (X t , Y t ), the heading angle is X B -CG direction speed is CG-Y B The direction speed is X B -CG direction acceleration is CG-Y B The direction acceleration is Angular velocity is W t .

[0036] The current vehicle status S t S t =[X t , Y t , W t , i], where i∈{l, c, r} represents the lane information, l represents the left lane, c represents the current lane, and r represents the right lane.

[0037] Let the road congestion coefficient of the i-th lane be C r，i and C p，i , where C r，i Indicates the congestion coefficient in front of the current vehicle, C p，i Indicates the congestion coefficient behind the current vehicle.

[0038] The distance between the front and rear vehicles and the time difference between the two vehicles passing a certain point are important bases for reflecting the road capacity. Therefore, the road congestion coefficient of lane i is as follows:

[0039]

[0040]

[0041] Among them, D p and D rare defined as the longitudinal position difference between the current vehicle and the vehicle in front and the vehicle behind, respectively. B The velocities in the -CG direction are expressed as and Assume that the length of all vehicles on the road is D l Similarly, the distance threshold between vehicles is expressed as D th (usually set to 50-100 meters), the embodiment of the present invention considers the road congestion situation within the range of less than 200 meters between vehicles. In the embodiment of the present invention, the road congestion coefficient reflects the size of the space occupied by vehicles around the current vehicle. Specifically, the closer the longitudinal distance between vehicles, the higher the road congestion coefficient value of the lane, and the smaller the driving space.

[0042] In some embodiments, the step of constructing an observation space based on multiple state spaces and environmental factors includes: obtaining the state space of the current vehicle at the current moment and multiple consecutive moments before the current moment; respectively obtaining the distance between the current vehicle and the vehicle in front and the vehicle behind, respectively obtaining the congestion coefficient of the current vehicle in the current lane with the vehicle in front and the vehicle behind, and obtaining the map information at the current moment; and constructing the observation space based on the multiple state spaces, the distances, the congestion coefficients and the map information.

[0043] In some implementations, the step of acquiring the map information at the current moment includes: acquiring lane lines, positions of vehicles adjacent to the current vehicle on the left and right, and traffic light status.

[0044] Observation Space

[0045] Among them, D r Indicates the distance between the current vehicle and the vehicle behind, D p Indicates the distance between the current vehicle and the vehicle ahead. Indicates that the rear vehicle is moving along X B -The speed in the CG direction, M is the map information at the current moment (including lane lines, adjacent vehicle positions, traffic light status, etc.).

[0046] In some embodiments, the step of constructing an action space based on the current force conditions of the front and rear wheels of the vehicle includes: obtaining a first force of the front wheel in the current direction of movement and a second force of the front wheel in a direction perpendicular to the current direction of movement, and obtaining a third force of the rear wheel in the original direction of movement and a fourth force of the rear wheel in a direction perpendicular to the original direction of movement; and constructing an action space based on the first force, the second force, the third force and the fourth force.

[0047] The lateral acceleration at the current moment Longitudinal acceleration and angular acceleration It can be expressed as:

[0048]

[0049]

[0050]

[0051] in, Represents the force on the current direction of motion of the front wheel, Represents the force of the front wheel perpendicular to the current direction of motion, Represents the force in the current direction of motion of the rear wheel, represents the force of the rear wheel perpendicular to the current direction of motion, δ is the steering angle of the front wheel, m is the mass of the current vehicle, and I z is the moment of inertia about the longitudinal axis.

[0052] In some embodiments, the step of assigning reward values ​​to changes in the state space and the observation space through the reward function to train each action space includes: in response to the changes in the state space and the observation space resulting in violation of traffic regulations or collision with other people or objects, assigning a first negative reward value to the corresponding action space; in response to any congestion coefficient corresponding to the vehicle exceeding a congestion coefficient threshold or any acceleration exceeding an acceleration threshold, assigning a second negative reward value to the corresponding action space; and in response to the vehicle arriving at the destination and without the above situation, assigning a positive reward value to the corresponding action space, wherein the absolute value of the first negative reward value is greater than the absolute value of the second negative reward value.

[0053] The embodiment of the present invention uses a high-definition map containing semantic information of the vehicle-road cooperative driving scene, and rasterizes the map to form a multi-channel tensor, where each channel represents a different map element, including roads, intersections, lanes, lane boundaries, traffic lights, etc. Furthermore, by identifying the location of the lane, the type of lane boundary (solid or dashed), the location of the stop sign, and the status of the traffic light, the autonomous driving vehicle can make driving decisions that comply with traffic rules through map recognition and behavior learning (such as stopping at a red light, driving along the lane, and changing lanes only when allowed), and respond to emergencies such as other vehicles changing lanes and sudden obstacles. In the embodiment of the present invention, k is used to count the number of violations of traffic rules during the vehicle-road cooperative driving process.

[0054] In addition, in order to ensure the comfort of passengers during driving, the acceleration threshold is set to ensure the stability of the vehicle during driving:

[0055]

[0056] in, Indicates the acceleration in the current direction of motion, Indicates the acceleration perpendicular to the current direction of motion.

[0057] In an embodiment of the present invention, the goal is to minimize the total driving distance to the target point, and constraints can be set: (a) the number of traffic rule violations in the vehicle-road cooperative driving decision-making planning method and system is 0; (b) avoid collisions with vehicles in front and behind that suddenly appear in the current lane; (c) whether the congestion coefficient of the current lane meets the requirements, and if it does not meet the requirements, determine whether lane change is possible; (d) ensure passenger comfort.

[0058] Specifically it can be defined as:

[0059]

[0060] st(a)k=0,

[0061] (b)D r ≥D collision ,D p ≥D collision ,

[0062] (c)C r,i ≤C r,max ,C p,i ≤C p,max ,

[0063] (d)

[0064] Among them, D collision Indicates the collision distance, C r，max Indicates the maximum congestion coefficient with the preceding vehicle, C p，max Indicates the maximum congestion coefficient with the following vehicle.

[0065] In some embodiments, the step of determining the maximum reward value based on the current state space and the current observation space, and executing the action space corresponding to the maximum reward value to achieve decision planning for driving behavior includes: determining the action space with the maximum reward value at the current moment based on the state space and observation space at the current moment, and continuously executing the action space with the maximum reward value at each moment.

[0066] The embodiment of the present invention uses DDPG to solve the decision-making problem of the vehicle-road cooperative driving system. DDPG can be replaced by deep reinforcement learning methods in continuous action space such as the PPO algorithm. DDPG can explore the environment, try multiple actions in different states, and finally learn the best strategy through experience. The DDPG model applied to the vehicle-road cooperative driving system consists of state space, observation space, action space, and reward function:

[0067] State Space

[0068] Observation Space

[0069] Action Space

[0070] Reward function: After taking action a(t), from state S t To state S t+1 The transformation generates a reward r(t). r(t) encourages the autonomous vehicle to learn general policy behaviors and navigate automatically under specific conditions:

[0071]

[0072] Among them, R1 is the positive reward value of the vehicle reaching the destination, which is inversely proportional to the total driving distance. R2 is the negative reward value obtained by task failure, indicating that the vehicle violates traffic regulations or is about to collide in the current single training.

[0073] When the reward function converges, the planning decision is completed, and the output is the control of the vehicle (the action space is the output each time).

[0074] The embodiment of the present invention can directly output the control signal of the vehicle according to the current and past vehicle status and road information, complete the control of the vehicle motion state such as acceleration, deceleration, left turn, right turn, and maintenance, and further complete the behavior layer strategies such as parking, lane change, following, overtaking, and passing intersections. The vehicle-road cooperative driving decision-making scheme proposed by the present invention plans the shortest path to the destination in real time, while considering the optimization conditions such as avoiding collisions, maintaining safe driving space, and passenger comfort, improving the decision-making efficiency, optimizing the path to the destination, and providing a safer and more comfortable solution for autonomous driving vehicles.

[0075] It should be particularly pointed out that the various steps in the various embodiments of the above-mentioned driving behavior decision planning method can be cross-linked, replaced, added, and deleted. Therefore, these reasonable permutations, combinations and transformations of the driving behavior decision planning method should also fall within the scope of protection of the present invention, and the scope of protection of the present invention should not be limited to the embodiments.

[0076] Based on the above purpose, the second aspect of the embodiment of the present invention provides a system for driving behavior decision planning. Figure 4As shown, the system 200 includes the following modules: a construction module, which is used to construct a state space according to the current driving state of the vehicle, and to construct an observation space according to multiple state spaces and environmental factors; a reward module, which is used to construct an action space according to the force conditions of the front and rear wheels of the current vehicle, and to construct a reward function according to behavior prediction, emergency handling and ride comfort in historical driving data; an execution module, which is used to change the state space and the observation space by executing the action space, and to assign reward values ​​to the changes in the state space and the observation space through the reward function to train each action space; and a decision module, which is used to determine the maximum reward value according to the current state space and the current observation space in response to the completion of training, and to execute the action space corresponding to the maximum reward value to realize decision planning for driving behavior.

[0077] In some embodiments, the construction module is also used to: construct a plane coordinate system based on longitude and latitude, and construct a body coordinate system based on the original running direction of the current vehicle; obtain the coordinates, heading angle and lane information of the current vehicle from the plane coordinate system, and obtain the speed, acceleration and angular velocity of the current vehicle from the body coordinate system; and construct a state space based on the coordinates, heading angle, lane information, speed, acceleration and angular velocity.

[0078] In some embodiments, the construction module is further used to: set the current original running direction of the vehicle as the first direction of the body coordinate system, and set the direction perpendicular to the first direction as the second direction of the body coordinate system.

[0079] In some embodiments, the construction module is further used to: decompose the current running speed of the current vehicle into the first direction and the second direction respectively, and determine the acceleration in the first direction and the second direction respectively.

[0080] In some embodiments, the construction module is also used to: obtain the state space of the current vehicle at the current moment and multiple consecutive moments before the current moment; obtain the distance between the current vehicle and the vehicle in front and the vehicle behind, respectively, obtain the congestion coefficient of the current vehicle in the current lane with the vehicle in front and the vehicle behind, respectively, and obtain the map information at the current moment; and construct an observation space based on multiple state spaces, distances, congestion coefficients and map information.

[0081] In some implementations, the construction module is further used to obtain lane lines, positions of left and right adjacent vehicles of the current vehicle, and traffic light status.

[0082] In some embodiments, the reward module is also used to: obtain a first force of the front wheel in the current direction of movement and a second force of the front wheel in the direction perpendicular to the current direction of movement, and obtain a third force of the rear wheel in the original direction of movement and a fourth force of the rear wheel in the direction perpendicular to the original direction of movement; and construct an action space based on the first force, the second force, the third force and the fourth force.

[0083] In some embodiments, the execution module is also used to: in response to changes in the state space and the observation space resulting in a violation of traffic regulations or a collision with other people or objects, assign a first negative reward value to the corresponding action space; in response to any congestion coefficient corresponding to the vehicle exceeding a congestion coefficient threshold or any acceleration exceeding an acceleration threshold, assign a second negative reward value to the corresponding action space; and in response to the vehicle arriving at the destination and without the above situation, assigning a positive reward value to the corresponding action space, wherein the absolute value of the first negative reward value is greater than the absolute value of the second negative reward value.

[0084] In some embodiments, the decision module is further used to: determine the action space with the maximum reward value at the current moment according to the state space and observation space at the current moment, and continuously execute the action space with the maximum reward value at each moment.

[0085] Based on the above purpose, the third aspect of an embodiment of the present invention proposes an electronic device, comprising: at least one processor; and a memory, the memory storing computer instructions that can be run on the processor, the instructions being executed by the processor to implement the following steps: S1, constructing a state space according to the current driving state of the vehicle, and constructing an observation space according to multiple state spaces and environmental factors; S2, constructing an action space according to the force conditions of the front and rear wheels of the current vehicle, and constructing a reward function according to behavior prediction, emergency handling and ride comfort in historical driving data; S3, changing the state space and observation space by executing the action space, and assigning reward values ​​to the changes in the state space and observation space through the reward function to train each action space; and S4, in response to the completion of the training, determining the maximum reward value according to the current state space and the current observation space, and executing the action space corresponding to the maximum reward value to achieve decision planning for driving behavior.

[0086] In some embodiments, the step of constructing a state space based on the driving state of the current vehicle includes: constructing a plane coordinate system based on longitude and latitude, and constructing a body coordinate system based on the original running direction of the current vehicle; obtaining the coordinates, heading angle and lane information of the current vehicle from the plane coordinate system, and obtaining the speed, acceleration and angular velocity of the current vehicle from the body coordinate system; and constructing a state space based on the coordinates, heading angle, lane information, speed, acceleration and angular velocity.

[0087] In some embodiments, the step of constructing a body coordinate system according to the current original running direction of the vehicle includes: setting the current original running direction of the vehicle as a first direction of the body coordinate system, and setting a direction perpendicular to the first direction as a second direction of the body coordinate system.

[0088] In some embodiments, the step of acquiring the speed, acceleration and angular velocity of the current vehicle from the body coordinate system includes: decomposing the current running speed of the current vehicle into the first direction and the second direction respectively, and determining the acceleration in the first direction and the second direction respectively.

[0089] In some embodiments, the step of constructing an observation space based on multiple state spaces and environmental factors includes: obtaining the state space of the current vehicle at the current moment and multiple consecutive moments before the current moment; respectively obtaining the distance between the current vehicle and the vehicle in front and the vehicle behind, respectively obtaining the congestion coefficient of the current vehicle in the current lane with the vehicle in front and the vehicle behind, and obtaining the map information at the current moment; and constructing the observation space based on the multiple state spaces, the distances, the congestion coefficients and the map information.

[0090] In some implementations, the step of acquiring the map information at the current moment includes: acquiring lane lines, positions of vehicles adjacent to the current vehicle on the left and right, and traffic light status.

[0091] In some embodiments, the step of constructing an action space based on the current force conditions of the front and rear wheels of the vehicle includes: obtaining a first force of the front wheel in the current direction of movement and a second force of the front wheel in a direction perpendicular to the current direction of movement, and obtaining a third force of the rear wheel in the original direction of movement and a fourth force of the rear wheel in a direction perpendicular to the original direction of movement; and constructing an action space based on the first force, the second force, the third force and the fourth force.

[0092] In some embodiments, the step of assigning reward values ​​to changes in the state space and the observation space through the reward function to train each action space includes: in response to the changes in the state space and the observation space resulting in violation of traffic regulations or collision with other people or objects, assigning a first negative reward value to the corresponding action space; in response to any congestion coefficient corresponding to the vehicle exceeding a congestion coefficient threshold or any acceleration exceeding an acceleration threshold, assigning a second negative reward value to the corresponding action space; and in response to the vehicle arriving at the destination and without the above situation, assigning a positive reward value to the corresponding action space, wherein the absolute value of the first negative reward value is greater than the absolute value of the second negative reward value.

[0093] In some embodiments, the step of determining the maximum reward value based on the current state space and the current observation space, and executing the action space corresponding to the maximum reward value to achieve decision planning for driving behavior includes: determining the action space with the maximum reward value at the current moment based on the state space and observation space at the current moment, and continuously executing the action space with the maximum reward value at each moment.

[0094] like Figure 5 , which is a schematic diagram of the hardware structure of an embodiment of the electronic device for driving behavior decision planning provided by the present invention.

[0095] As Figure 5 Taking the device shown as an example, the device includes a processor 301 and a memory 302.

[0096] The processor 301 and the memory 302 may be connected via a bus or other means. Figure 5 The example of connecting through bus is taken in the following.

[0097] The memory 302 is a non-volatile computer-readable storage medium that can be used to store non-volatile software programs, non-volatile computer executable programs and modules, such as program instructions / modules corresponding to the method for driving behavior decision planning in the embodiment of the present application. The processor 301 executes various functional applications and data processing of the server by running the non-volatile software programs, instructions and modules stored in the memory 302, that is, implementing the method for driving behavior decision planning.

[0098] The memory 302 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required by at least one function; the data storage area may store data created according to the use of the method for driving behavior decision planning, etc. In addition, the memory 302 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 302 may optionally include a memory remotely arranged relative to the processor 301, and these remote memories may be connected to the local module via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0099] Computer instructions 303 corresponding to one or more driving behavior decision planning methods are stored in the memory 302. When executed by the processor 301, the driving behavior decision planning method in any of the above method embodiments is executed.

[0100] Any embodiment of the electronic device that executes the above-mentioned driving behavior decision planning method can achieve the same or similar effect as any corresponding embodiment of the above-mentioned method.

[0101] The present invention also provides a computer-readable storage medium, which stores a computer program for executing the method for driving behavior decision planning when executed by a processor.

[0102] like Figure 6 As shown, it is a schematic diagram of an embodiment of the computer storage medium of the driving behavior decision planning provided by the present invention. Figure 6 Taking the computer storage medium shown as an example, the computer readable storage medium 401 stores a computer program 402 that performs the above method when executed by a processor.

[0103] Finally, it should be noted that a person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program of the driving behavior decision-making planning method can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the storage medium of the program can be a disk, an optical disk, a read-only storage memory (ROM) or a random access memory (RAM), etc. The above-mentioned computer program embodiment can achieve the same or similar effect as any of the above-mentioned method embodiments corresponding thereto.

[0104] The above are exemplary embodiments disclosed in the present invention, but it should be noted that various changes and modifications may be made without departing from the scope disclosed in the embodiments of the present invention as defined in the claims. The functions, steps and / or actions of the method claims according to the disclosed embodiments described herein do not need to be performed in any particular order. In addition, although the elements disclosed in the embodiments of the present invention may be described or required in individual form, they may also be understood as multiple unless explicitly limited to the singular.

[0105] It should be understood that, as used herein, the singular forms "a", "an" are intended to include the plural forms as well, unless the context clearly supports an exception. It should also be understood that, as used herein, "and / or" refers to any and all possible combinations including one or more of the associated listed items.

[0106] The serial numbers of the embodiments disclosed in the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0107] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0108] A person skilled in the art should understand that the discussion of any of the above embodiments is only exemplary and is not intended to imply that the scope of the disclosure of the embodiments of the present invention (including the claims) is limited to these examples; under the concept of the embodiments of the present invention, the technical features in the above embodiments or different embodiments can also be combined, and there are many other changes in different aspects of the embodiments of the present invention as above, which are not provided in detail for the sake of simplicity. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present invention should be included in the protection scope of the embodiments of the present invention.

Claims

1. A method for driving behavior decision planning, characterized in that: The steps include: Construct a state space according to the current driving state of the vehicle, and construct an observation space according to multiple state spaces and environmental factors; The action space is constructed based on the current force conditions of the front and rear wheels of the vehicle, and the reward function is constructed based on the behavior prediction, emergency handling and ride comfort in the historical driving data; The state space and the observation space are changed by executing the action space, and a reward value is assigned to the change of the state space and the observation space by the reward function to train each action space; as well as In response to the training being completed, a maximum reward value is determined according to a current state space and a current observation space, and an action space corresponding to the maximum reward value is executed to implement decision planning for driving behavior.

2. The method for driving behavior decision planning according to claim 1, characterized in that: The step of constructing the state space according to the current driving state of the vehicle comprises: Construct a plane coordinate system based on longitude and latitude, and construct a volume coordinate system based on the original running direction of the current vehicle; Acquire the coordinates, heading angle and lane information of the current vehicle from the plane coordinate system, and acquire the speed, acceleration and angular velocity of the current vehicle from the body coordinate system; and A state space is constructed according to the coordinates, heading angle, lane information, speed, acceleration and angular velocity.

3. The method for driving behavior decision planning according to claim 2, characterized in that: The step of constructing a body coordinate system according to the current original running direction of the vehicle comprises: The current original running direction of the vehicle is set as the first direction of the body coordinate system, and the direction perpendicular to the first direction is set as the second direction of the body coordinate system.

4. The method for driving behavior decision planning according to claim 3, characterized in that: The step of acquiring the current speed, acceleration and angular velocity of the vehicle from the body coordinate system comprises: The current running speed of the current vehicle is decomposed into the first direction and the second direction respectively, and the acceleration in the first direction and the second direction are determined respectively.

5. The method for driving behavior decision planning according to claim 3, characterized in that: The step of constructing an observation space according to a plurality of state spaces and environmental factors comprises: Obtain the state space of the current vehicle at the current moment and multiple consecutive moments before the current moment; respectively obtaining the distances between the current vehicle and the vehicle in front and the vehicle behind, respectively obtaining the congestion coefficients between the current vehicle and the vehicle in front and the vehicle behind in the current lane, and obtaining the map information at the current moment; and An observation space is constructed according to the plurality of state spaces, the distances, the congestion coefficients and the map information.

6. The method for driving behavior decision planning according to claim 5, characterized in that: The step of obtaining the map information at the current moment includes: Get the lane lines, the positions of the vehicles adjacent to the current vehicle on the left and right, and the status of traffic lights.

7. The method for driving behavior decision planning according to claim 1, characterized in that: The step of constructing the action space according to the current force conditions of the front and rear wheels of the vehicle comprises: Obtaining a first force of the front wheel in the current moving direction and a second force of the front wheel in the direction perpendicular to the current moving direction, and obtaining a third force of the rear wheel in the original moving direction and a fourth force of the rear wheel in the direction perpendicular to the original moving direction; and An action space is constructed according to the first force, the second force, the third force and the fourth force.

8. The method for driving behavior decision planning according to claim 1, characterized in that: The step of assigning a reward value to the change of the state space and the observation space by the reward function to train each action space comprises: In response to changes in the state space and the observation space resulting in a violation of traffic regulations or a collision with other persons or objects, assigning a first negative reward value to the corresponding action space; In response to any congestion coefficient corresponding to the vehicle exceeding a congestion coefficient threshold or any acceleration exceeding an acceleration threshold, assigning a second negative reward value to the corresponding action space; and In response to the vehicle arriving at the destination without the above situation, a positive reward value is assigned to the corresponding action space, wherein the absolute value of the first negative reward value is greater than the absolute value of the second negative reward value.

9. The method for driving behavior decision planning according to claim 1, characterized in that: The step of determining the maximum reward value according to the current state space and the current observation space, and executing the action space corresponding to the maximum reward value to realize the decision planning of the driving behavior includes: Determine the action space with the maximum reward value at the current moment based on the state space and observation space at the current moment, and continue to execute the action space with the maximum reward value at each moment.

10. A system for driving behavior decision planning, characterized in that: include: A construction module, used to construct a state space according to the current driving state of the vehicle, and to construct an observation space according to multiple state spaces and environmental factors; The reward module is used to construct the action space based on the current force conditions of the front and rear wheels of the vehicle, and to construct the reward function based on the behavior prediction, emergency handling and ride comfort in the historical driving data; An execution module, configured to change the state space and the observation space by executing the action space, and assign a reward value to the change of the state space and the observation space through the reward function to train each action space; as well as The decision module is used to determine the maximum reward value according to the current state space and the current observation space in response to the completion of the training, and execute the action space corresponding to the maximum reward value to realize the decision planning of the driving behavior.

11. An electronic device, characterized in that: include: at least one processor; as well as A memory storing computer instructions executable on the processor, wherein the instructions, when executed by the processor, implement the steps of the method according to any one of claims 1 to 9.

12. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 9 are implemented.

Images