Intelligent vehicle optimal decision-making control model construction and solution method, device and storage medium

Abstract

The present disclosure provides a method, device and storage medium for constructing and solving the optimal decision-making control model of an intelligent vehicle, including: constructing an optimal decision-making control model of an intelligent vehicle, minimizing the path tracking performance of the vehicle and the sum of the predicted tracking error as the objective function of the optimal decision-making control model, using the vehicle dynamics model, the distance between the vehicle and surrounding traffic participants, the distance between the vehicle and the road, and the distance between the vehicle and the traffic light as the motion constraints and safety constraints of the optimal decision-making control model, and using the evaluation function and the strategy function as the output of the optimal decision-making control model; establishing a learning penalty function method to handle safety constraints, monitoring unsafe driving conditions during the exploration process and imposing penalties on strategies that violate the constraints, so that the strategies iterate in the direction of satisfying the constraints, and finally outputting the optimal strategy function and the optimal evaluation function of the optimal decision-making control model. The present disclosure has high security and strong intelligence, breaking through the bottleneck of existing decision-making control solvers that are independently optimized and rely on artificial rules.

Description

Technical Field

[0001] The present invention belongs to the field of autonomous decision-making and control of intelligent vehicles, and in particular relates to a method and device for constructing and solving an optimal decision-making and control model of an intelligent vehicle. Background Art

[0002] Intelligence is one of the directions of automotive technology change, and it has great potential in reducing traffic accidents, reducing fuel consumption, and improving traffic efficiency. The decision-making ability in complex urban scenes such as intersections is the core element to measure the level of driving intelligence, and it is necessary to consider the dynamic and random nature of traffic scenes. At present, the traditional hierarchical solution divides the decision-making problem into a series of sub-optimization problems and uses online optimization methods to solve them separately, resulting in low generalization ability and difficulty in meeting the real-time calculation of control instructions in complex scenes. With the rapid development of artificial intelligence technology, learning-based solution methods with self-learning ability and strong generalization ability are expected to break through the decision-making problem faced by high-level autonomous driving. For cars, their optimization solution needs to consider the traffic status of roads, pedestrians, bicycles and surrounding motor vehicles to ensure the safety of vehicle driving. Secondly, it needs to consider the limitations of physical conditions such as traffic rules, traffic lights, and diversified road structures, and has high intelligence.

[0003] The existing learning-based solution to the decision-making control problem is based on a process of trial and error from scratch, and generates better driving strategies through continuous iteration of interactive data. On the one hand, it does not take into account effective prior knowledge such as the center line of the road and vehicle dynamics, and is prone to converge to a local optimal solution, making it difficult to obtain an optimal strategy with strong intelligence. On the other hand, the learning process is based on the vehicle's trial and error process in the driving environment, and the driving strategy is adjusted by the reward and punishment signals obtained. The safety of the algorithm cannot be fully guaranteed, and it is difficult to actually apply it on a real vehicle. Summary of the invention

[0004] The present disclosure aims to solve one of the above-mentioned problems.

[0005] To this end, the first aspect of the present disclosure provides a method for constructing and solving an optimal decision-making control model for an intelligent vehicle with high security and strong intelligence, including:

[0006] An optimal decision-making control model for an intelligent vehicle is constructed, wherein the objective function of the optimal decision-making control model is to minimize the sum of the path tracking performance and the predicted tracking error of the ego vehicle, the ego vehicle dynamics model is used as the motion constraint of the optimal decision-making control model, the distance between the ego vehicle and surrounding traffic participants, the distance between the ego vehicle and the road, and the distance between the ego vehicle and the traffic light are used as the safety constraints of the optimal decision-making control model, and the evaluation function for selecting the optimal path from the candidate path set and the strategy function for determining the optimal ego vehicle control command corresponding to the optimal path are used as the output of the optimal decision-making control model; wherein the path tracking performance characterizes the difference between the ego vehicle and the candidate path in the future predicted time domain and the stability and energy consumption of the ego vehicle during the tracking process, and the predicted tracking error is used to evaluate the pros and cons of tracking different paths;

[0007] A learning penalty function method is established to process the safety constraints. During the exploration process, unsafe driving conditions are monitored and penalties are imposed on strategies that violate the constraints, so that the strategies are iterated in the direction of satisfying the constraints, and finally the optimal strategy function and optimal evaluation function of the optimal decision-making model are output.

[0008] The method for constructing and solving the optimal decision-making control model for an intelligent vehicle provided by the first embodiment of the present disclosure has the following characteristics and beneficial effects:

[0009] The model construction and solution method proposed in the present disclosure comprehensively considers nonlinear vehicle dynamics and safety constraints of different types of traffic participants, and continuously punishes states that violate constraints until an optimal strategy that strictly satisfies the constraints is obtained. Therefore, this solution method can effectively ensure the safety of driving strategies.

[0010] The solution method proposed in the present invention optimizes the evaluation function and the strategy function simultaneously, which are used to select the optimal path and generate control instructions respectively. It breaks through the bottleneck of existing decision-making and control solvers that independently optimize and rely on artificial rules, and ensures the intelligence of the decision-making and control process.

[0011] In some embodiments, the mathematical expression of the optimal decision-making model of the smart car is as follows:

[0012]

[0013]

[0014]

[0015] x self (τ+1)=F self (x self (τ),u τ (θ)) (2)

[0016] x surr(τ+1)=F surr (x surr (τ),x self (τ),u τ (θ)) (3)

[0017] x road (τ) = F road (x self (τ)) (4)

[0018] g surr (x self (τ),x surr (τ))≥D surr (5)

[0019] g road (x self (τ),x road )≥D road (6)

[0020] g light (x self (τ),x light (τ)=R)≥D light (7)

[0021] in:

[0022] Formula (1) is the objective function of the optimal decision-making model, J track (θ) represents the performance index of the strategy function, that is, the path tracking performance, J score (w) represents the performance index of the evaluation function, that is, the prediction tracking error, θ and w represent the strategy function parameters to be optimized and the evaluation function parameters respectively; t is the initial time, τ is the virtual time of the prediction time domain, and T is the length of the prediction time domain; s τ represents the driving state at time τ, s τ =[x ref (τ),x self (τ),x surr (τ),x road (τ),x light (τ)] T , x ref (τ) represents the candidate path state at time τ; x self (τ), x surr (τ), x road (τ) represent the state of the vehicle, the state of surrounding traffic participants and the road state at time τ respectively; x light (τ) represents the state of the traffic light at time τ; Represents different initial driving states s tThe expected performance under the distribution, wherein the different initial driving state distributions are the joint distributions of different candidate paths, vehicle states, surrounding traffic participant states, road states, and signal light states; u τ (θ) is the control command of the ego vehicle, satisfying u τ (θ)=π θ (s τ ), π θ (s τ ) is the driving state s from time τ τ Mapping to the optimal ego-car action; V w (s t ) is the driving state s from time t t to the optimal path tracking performance; l(·) represents the tracking cost at each step;

[0023] Formula (2) is the dynamic model of the vehicle, F self (·) is the vehicle dynamics equation;

[0024] Formula (3) is the motion model of surrounding traffic participants, F surr (·) Used to calculate the motion state of traffic participants such as pedestrians, bicycles and motor vehicles at the next moment;

[0025] Formula (4) is the road state related to the vehicle driving, F road (·) is used to calculate the road information corresponding to the current state of the vehicle;

[0026] Formula (5) is the safety constraint of surrounding traffic participants, g surr (x self (τ),x surr (τ)) represents the distance between the vehicle and surrounding traffic participants. The distance between the vehicle and surrounding traffic participants is greater than the safety threshold D surr When , it indicates that no collision accident occurred;

[0027] Formula (6) is the road safety constraint, g road (x self (τ),x road (τ)) represents the distance between the vehicle and the driving lane. The distance between the vehicle and the driving lane is greater than the safety threshold D road When the vehicle is in the lane, it indicates that the vehicle has not left or deviated from the lane.

[0028] Formula (7) is the safety constraint of the red light, g light (x self (τ),x light (τ) = R) represents the distance between the vehicle and the stop line when the traffic light is red. This distance is greater than the safety threshold D light , it means that the vehicle did not run a red light.

[0029] In some embodiments, the optimal strategy function and the optimal evaluation function of the optimal decision-making model are obtained by the following steps:

[0030] 2-1) Initialize the evaluation function parameter w and the strategy function parameter θ, and set the evaluation function learning rate to β w , the learning rate of the policy function is β θ The starting point of the iteration is the initial time t, the initial penalty factor is ρ = 1, and the increase factor is c>1; the safety constraint performance J corresponding to the tracking constraint is introduced safe (θ);

[0031] 2-2) Construct the initial driving state represents the state of the i-th candidate path at the initial time t;

[0032] 2-3) Taking the initial driving state as the starting point, let J track (θ) = J safe (θ)=0, calculate the expected cumulative tracking performance J in the prediction time domain τ∈[t,t+T] track and expected cumulative safety constraint performance J safe ;

[0033] 2-4) Update the evaluation function V w (s t ) and the corresponding evaluation function parameter w: Update the strategy function parameter θ, and optimize the expected cumulative tracking performance J by the penalty factor ρ track and expected cumulative safety constraint performance J safe ;

[0034] 2-5) Increase the penalty factor, that is, set ρ = cρ, and set t = t+1;

[0035] 2-6) Continue to execute steps 2-2) to 2-5) until the optimal evaluation function parameter w is obtained. * and policy function parameters θ * , and finally output the optimal evaluation function With the optimal policy function

[0036] In some embodiments, step 2-3) specifically includes:

[0037] 2-3-1) Let J track (θ) = J safe (θ) = 0; according to the current driving state s τ Get the driving state s at the next moment τ+1 ;

[0038] 2-3-2) Calculate the tracking cost l(τ) and safety constraint performance g corresponding to time τ surr (xself (τ),x surr (τ))- D surr , g road (x self (τ),x road (τ))-D road ,g light (x self (τ),x light (τ) = R) - D light ; Set the penalty function corresponding to each safety constraint;

[0039] 2-3-3) Calculate the cumulative tracking performance and the cumulative safety constraint performance:

[0040] J track (θ) = J track (θ)+l(τ)

[0041]

[0042] 2-3-4) Let τ = τ + 1, repeat 2-3-1) to 2-3-3) until τ = t + T, and get the initial driving state s t Expected cumulative tracking performance J in the T-step prediction domain track and expected cumulative safety constraint performance J safe .

[0043] In some embodiments, the penalty functions corresponding to the security constraints are set respectively according to the following formulas:

[0044]

[0045]

[0046]

[0047] in, is the penalty function for the safety constraints of surrounding traffic participants, is the penalty function of road safety constraints, is the penalty function for the safety constraint of the red light.

[0048] In some embodiments, the evaluation function V is calculated according to the following formula: w (s τ ) and the corresponding evaluation function parameter w are updated:

[0049]

[0050] Among them, V w (s τ ) is the driving state s from time τ τMapping to optimal path following performance.

[0051] In some embodiments, the policy function parameter θ is updated according to the following formula to optimize the expected cumulative tracking performance J track and expected cumulative safety constraint performance J safe :

[0052] J π =J track +ρJ safe

[0053]

[0054] Among them, J π Used to evaluate ego-vehicle control strategies and characterize the balance between path tracking performance and safety.

[0055] The second aspect of the present disclosure provides an optimal decision-making and control model construction and solution device for an intelligent vehicle, including:

[0056] An optimal decision-making control model construction module is used to minimize the path tracking performance and predicted tracking error of the ego vehicle as the objective function of the optimal decision-making control model, use vehicle dynamics as the motion constraint of the optimal decision-making control model, use the distance between the ego vehicle and surrounding traffic participants, the distance between the ego vehicle and the road, and the distance between the ego vehicle and the traffic light as the safety constraint of the optimal decision-making control model, and use an evaluation function for selecting an optimal path from a set of candidate paths and a strategy function for determining an optimal ego vehicle control instruction corresponding to the optimal path as outputs of the optimal decision-making control model; wherein the path tracking performance characterizes the difference between the ego vehicle and the candidate path in the future prediction time domain and the stability and energy consumption of the ego vehicle during the tracking process, and the predicted tracking error is used to evaluate the pros and cons of tracking different paths;

[0057] The optimal decision-making control model solving module is used to establish a learning penalty function method to process the safety constraints, monitor unsafe driving conditions during the exploration process and impose penalties on strategies that violate the constraints, so that the strategies are iterated in the direction of satisfying the constraints, and finally output the optimal strategy function and optimal evaluation function of the optimal decision-making control model.

[0058] The computer-readable storage medium provided in the third aspect of the present disclosure stores computer instructions, and the computer instructions are used to enable the computer to execute the method for constructing and solving the optimal decision-making and control model of an intelligent vehicle. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 It is an overall flow chart of the construction and solution method of the optimal decision-making and control model for an intelligent vehicle provided in the embodiment of the first aspect of the present disclosure.

[0060] Figure 2 It is a structural schematic diagram of a device for constructing and solving an optimal decision-making and control model for an intelligent vehicle provided in an embodiment of the second aspect of the present disclosure.

[0061] Figure 3 It is a structural schematic diagram of an electronic device provided in the third aspect of the present disclosure. DETAILED DESCRIPTION

[0062] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0063] On the contrary, the present application covers any substitution, modification, equivalent method and scheme made on the essence and scope of the present application as defined by the claims. Further, in order to make the public have a better understanding of the present application, some specific details are described in detail in the detailed description of the present invention below. Those skilled in the art can fully understand the present application without the description of these details.

[0064] See also Figure 1 The first aspect of the present disclosure provides an optimal decision-making control model construction and solution method for a smart car, including:

[0065] The optimal decision-making control model of the intelligent vehicle is constructed, with minimizing the sum of the path tracking performance and the prediction tracking error of the ego vehicle as the objective function of the optimal decision-making control model, with the vehicle dynamics model as the motion constraint of the optimal decision-making control model, with the distance between the ego vehicle and the surrounding traffic participants, the distance between the ego vehicle and the road, and the distance between the ego vehicle and the traffic light as the safety constraint of the optimal decision-making control model, and with the evaluation function for selecting the optimal path from the candidate path set and the strategy function for determining the optimal ego vehicle control command corresponding to the optimal path as the output of the optimal decision-making control model; wherein the path tracking performance characterizes the difference between the ego vehicle and the candidate path in the future prediction time domain and the stability and energy consumption of the ego vehicle during the tracking process, and the prediction tracking error is used to evaluate the pros and cons of tracking different paths;

[0066] A learning penalty function method is established to deal with safety constraints. During the exploration process, unsafe driving conditions are monitored and penalties are imposed on strategies that violate the constraints, so that the strategies are iterated in the direction of satisfying the constraints, and finally the optimal policy function and optimal evaluation function of the optimal decision-making model are output.

[0067] In some embodiments, the mathematical expression of the optimal decision-making model of the intelligent vehicle is constructed as follows:

[0068]

[0069]

[0070]

[0071] x self (τ+1)=F self (x self (τ),u τ (θ)) (2)

[0072] x surr (τ+1)=F surr (x surr (τ),x self (τ),u τ (θ)) (3)

[0073] x road (τ) = F road (x self (τ)) (4)

[0074] g surr (x self (τ),x surr (τ))≥D surr (5)

[0075] g road (x self (τ),x road )≥D road (6)

[0076] g light (x self (τ),x light (τ)=R)≥D light (7)

[0077] in:

[0078] Formula (1) is the objective function of the optimal decision-making model, J track (θ) represents the performance index of the policy function, i.e., path tracking performance, J score (w) represents the performance index of the evaluation function, namely the prediction tracking error, which is an evaluation of the tracking effect of a certain path and represents the quality of tracking the path. θ and w represent the strategy function parameters to be optimized and the evaluation function parameters respectively; t is the initial time, τ is the virtual time of the prediction time domain, and T is the length of the prediction time domain; s τ represents the driving state at time τ, which can be specifically expressed as: s τ =[x ref (τ),x self (τ),x surr (τ),x road (τ),xlight (τ)] T , x ref (τ) represents the candidate path state at time τ; x self (τ), x surr (τ), x road (τ) represent the state of the vehicle, the state of surrounding traffic participants and the road state at time τ respectively; x light (τ) represents the state of the traffic light at time τ, x light (τ) = R means that the traffic light in the lane of the vehicle is red at time τ, x light (τ) = Y means that the traffic light in the lane of the vehicle is yellow at time τ, x light (τ) = G means that the traffic light in the lane of the vehicle is green at time τ. In driving scenarios without traffic lights, such as ramps or highways, set x light =R. Represents different initial driving states s t The expected performance under the distribution, the different initial driving state distribution is the joint distribution of different candidate paths, vehicle states, surrounding traffic participants states, road states and signal light states. Specifically, For different initial driving states s t The average performance of {·} is obtained by sampling a certain number of initial driving states s from the initial driving state distribution. t , calculate the corresponding {·} respectively, and finally calculate the average; u τ (θ) is the control command of the ego vehicle, satisfying u τ (θ)=π θ (s τ ), π θ (s τ ) is the driving state s from time τ τ Mapping to the optimal ego-car action; V w (s t ) is the driving state s from time t t to the optimal path tracking performance; l(·) represents the tracking cost of each step, which may include the path tracking error of the ego vehicle, the speed tracking error, the stability penalty and the energy consumption of the vehicle control. The tracking cost at time τ is recorded as l(τ).

[0079] Formula (2) is the dynamic model of the vehicle, F self (·) is the ego vehicle dynamics equation, which can be calculated based on the ego vehicle state x at time τ. self (τ) and the ego vehicle control command u τ (θ) Calculate the vehicle state x at the next time τ+1 self (τ+1);

[0080] Formula (3) is the motion model of surrounding traffic participants, where F surr (·) Used to calculate the motion state of traffic participants such as pedestrians, bicycles and motor vehicles at the next moment;

[0081] Formula (4) is the road state related to the vehicle driving, where F road (·) Used to calculate the road information corresponding to the current state of the vehicle, such as lane number, lane edge position, etc.;

[0082] Formula (5) is the safety constraint of surrounding traffic participants, where g surr (x self (τ),x surr (τ)) represents the distance between the vehicle and surrounding traffic participants. The distance between the vehicle and surrounding traffic participants is greater than the safety threshold D surr When , it indicates that no collision accident occurred;

[0083] Formula (6) is the road safety constraint, where g road (x self (τ),x road (τ)) represents the distance between the vehicle and the driving lane. The distance between the vehicle and the driving lane is greater than the safety threshold D road When the vehicle is in the lane, it indicates that the vehicle has not left or deviated from the lane.

[0084] Formula (7) is the safety constraint of the red light, where g light (x self (τ),x light (τ) = R) represents the distance between the vehicle and the stop line when the traffic light is red. This distance is greater than the safety threshold D light , it means that the vehicle did not run a red light.

[0085] In some embodiments, solving the optimal decision-making control model of the smart car specifically includes the following steps:

[0086] 2-1) Initialize the evaluation function parameter w and the strategy function parameter θ, and set the evaluation function learning rate to β w , the learning rate of the policy function is β θ The starting point of the iteration is the initial time t, the initial penalty factor is ρ = 1, and the increase factor is c>1; the safety constraint performance J corresponding to the tracking constraint is introduced safe (θ);

[0087] 2-2) Sample the vehicle state x at time t from the driving environment self (t), the state of surrounding traffic participants x surr (t) and the signal light state x light (t), the center lines of all drivable lanes are taken as the corresponding candidate paths τ i, and thus construct a candidate path set, i∈{1,N lane},N lane is the number of candidate paths, and the road state x at the initial time t is calculated based on the vehicle position at the initial time t road (t) and the status of each candidate path Status of each candidate path Including the reference position, reference speed and heading angle of the vehicle in the i-th candidate path, the initial driving state s is constructed based on the above information t ;

[0088] 2-3) Taking the initial driving state as the starting point, let J track (θ) = J safe (θ) = 0, calculate the cumulative tracking performance J in the prediction time domain τ∈[t,t+T] track and safety constraint performance J safe , the specific steps are as follows:

[0089] 2-3-1) Let J track (θ) = J safe (θ)=0; according to the current strategy function π θ (s τ ) generates the vehicle control command u self (τ)=π θ (s τ ), according to the dynamic model of the vehicle, the vehicle state x at the next moment is obtained self (τ+1), and at the same time, the state x of the surrounding traffic participants at the next moment is obtained according to the motion model of the surrounding traffic participants surr (τ+1); according to the current candidate path τ i Calculate the candidate path state at the next moment based on the vehicle position And road information x road (τ+1), construct the driving state s at the next moment τ+1 .

[0090] 2-3-2) Calculate the tracking cost l(τ) and safety constraint performance g corresponding to time τ surr (x self (τ),x surr (τ))- D surr , g road (x self (τ),x road (τ))-D road ,g light (x self (τ),x light (τ) = R) - D light ; Set the penalty function corresponding to each safety constraint as:

[0091]

[0092] in, is the penalty function for the safety constraints of surrounding traffic participants, is the penalty function of road safety constraints, is the penalty function for the safety constraint of the red light;

[0093] 2-3-3) Calculate the cumulative tracking performance and the cumulative safety constraint performance:

[0094]

[0095] 2-3-4) Let τ = τ + 1, repeat 2-3-1) to 2-3-3) until τ = t + T, and get the initial driving state s t Expected cumulative tracking performance J in the T-step prediction domain track and expected cumulative safety constraint performance J safe .

[0096] 2-4) Update the evaluation function V w (s τ ) and the corresponding evaluation function parameter w to fit the current initial driving state (s t ) to make the tracking performance output by the evaluation function as close as possible to the actual tracking performance. The update method is:

[0097]

[0098] Update the policy function π θ (s τ ) and its corresponding policy function parameter θ, optimizing the expected cumulative tracking performance J track and expected cumulative safety constraint performance J safe , the performance index after weighting by the penalty factor ρ is:

[0099] J π =J track +ρJ saff (11)

[0100] Among them, J π It is used to evaluate the control strategy of the ego vehicle and characterize the balance between tracking performance and safety. When the penalty factor ρ is small, such as 1≤ρ≤10, the strategy takes minimizing the expected cumulative tracking performance as the main goal; when the penalty factor ρ is large, such as ρ>10, the main goal is to ensure safety, that is, to minimize the expected cumulative safety constraint performance.

[0101] The update method of the policy function is:

[0102]

[0103] 2-5) Increase the penalty factor, that is, set ρ = cρ, and set t = t+1.

[0104] 2-6) Continue to execute steps 2-2) to 2-5) until the optimal evaluation function parameter w is obtained. * and policy function parameters θ * , and finally output the optimal evaluation function With the optimal policy function Used for subsequent path optimization and vehicle control.

[0105] In order to better understand the present application, the following describes in detail the application examples of the problem of the optimal decision-making control model for urban intersection scenarios and the solution method.

[0106] This embodiment is applied to a two-way six-lane intersection with traffic lights. The driving goal of the smart car (hereinafter referred to as the self-car) is to pass through the intersection safely and efficiently. This is a typical traffic scene with mixed pedestrians and vehicles. Pedestrians, bicycles and surrounding vehicles (hereinafter referred to as surrounding vehicles) all drive according to the right of way specified by the traffic lights. Taking left turn as an example, according to the characteristics of the road topology, the candidate path set for the self-car to drive includes 3 different paths, corresponding to the center lines of different target lanes. The self-car can perceive the information of the road and surrounding traffic participants through its own sensors. With the road, traffic lights, surrounding vehicles, pedestrians, bicycles, etc. as constraints, a constrained optimal tracking problem with quadratic performance indicators is constructed, and its tracking cost function is:

[0107]

[0108] Among them, the state of the vehicle at time τ is x self (τ)= [x self (τ),y self (τ),v self (τ),u self (τ),φ self (τ),ω self (τ),L self ,W self ] T , x self (τ), y self (τ), v self (τ), u self (τ), φ self (τ),ω self (τ), L self , W selfThey represent the lateral position, longitudinal position, lateral velocity, longitudinal velocity, heading angle, yaw rate, vehicle length, and vehicle width of the vehicle at time τ respectively; the i-th candidate path state at time τ They respectively represent the lateral position, longitudinal position, expected speed and expected heading angle of the candidate path point on the i-th candidate path; Q and R are positive definite diagonal weighted matrices.

[0109] This embodiment adopts a safety constraint construction method based on the distance between the center of the circle. Each vehicle is represented by two circles distributed one after the other. The center position of the circle in front of the vehicle at time τ is selected as The center position of the back circle is selected as Similarly, the state of the cycle at time τ can be expressed as x veh (τ), y veh (τ), v veh (τ),u veh (τ), φ veh (τ),ω veh (τ), L veh , W veh They are the lateral position, longitudinal position, lateral velocity, longitudinal velocity, heading angle, yaw rate, vehicle length, and vehicle width of the vehicle at time τ; the center position of the front circle of the vehicle at time τ is selected as The center position of the back circle is selected as The two centers of the ego vehicle and the two centers of the surrounding vehicles form four vehicle-to-vehicle distance constraints at each moment. If all constraints are greater than the vehicle-to-vehicle safety distance threshold, It means that there is no collision with the surrounding vehicle at this time, and the expression is as follows:

[0110]

[0111] Among them, λ(·,·) represents the distance measure between two points, represents the center position of the vehicle, represents the center of the front circle, represents the center of the back circle; Indicates the center position of the pericycle, #=F represents the center of the front circle, #=B represents the center of the back circle, The range that can be selected is 2 to 100 m. Similarly, the bicycle state at time τ is x bike (τ), y bike (τ), v bike (τ),u bike (τ), φ bike (τ),ω bike (τ), Lbike , W bike are the lateral position, longitudinal position, lateral velocity, longitudinal velocity, heading angle, yaw angular velocity, vehicle length, and vehicle width of the bicycle at time τ; the center of the front circle of the bicycle at time τ is The center of the back circle is The bicycle and the ego vehicle form four distance constraints at each moment, namely:

[0112]

[0113] in, Indicates the center position of the bicycle, #=F represents the center of the front circle, and #=B represents the center of the rear circle. is the safety distance threshold between bicycles and the vehicle itself, with a typical value range of 2.5 to 100 m.

[0114] In addition, the pedestrian is represented by a single circle, whose center is the position O of the pedestrian at time τ man (τ) = [x man (τ),y man (τ)] T , x man (τ),y man (τ) are the lateral and longitudinal positions of the pedestrian at time τ, respectively. Each pedestrian and the vehicle form two distance constraints, and the corresponding safety distance threshold is Considering the vulnerability of pedestrians, the typical value range is 3.5 to 100 m, that is:

[0115]

[0116] Among them, O man Indicates the location of pedestrians.

[0117] The road edge constraint is the position of the vehicle and the road edge O road (τ) = [x road (τ),y road (τ)] T should be less than a certain distance, x road (τ),y road (τ) are the lateral and longitudinal positions of the road edge, respectively, to ensure that the vehicle will not drive out of the intersection or enter the opposite lane. The safety threshold δ Road The typical value range is 0.1-10m:

[0118]

[0119] Among them, O road Indicates the location of the road edge.

[0120] The traffic light is constrained to the center point O of the stop line stop(τ) = [x stop (τ),y stop (τ)] T The distance from the two centers of the ego vehicle should be greater than the distance threshold, x stop (τ),y stop (τ) are the horizontal and vertical positions of the center point of the stop line, respectively, to ensure that the vehicle will not run a red light, that is:

[0121]

[0122] Among them, O stop Indicates the center position of the stop line of the lane where the vehicle is traveling.

[0123] For the constrained optimal decision-making problem constructed for the intersection scenario above, the approximate solution method proposed in this embodiment can be used to solve the optimal evaluation function and strategy function offline, which specifically includes the following steps:

[0124] 2-1) Initialize the evaluation function parameters w and strategy function parameters θ, and set the evaluation function learning rate to β w , the learning rate of the policy function is β θ The starting point of the iteration is the initial time t, the initial penalty factor is ρ = 1, and the increase factor is c>1; the safety constraint performance J corresponding to the tracking constraint is introduced safe (θ);

[0125] 2-2) Sample the initial driving state s at time t from the driving environment t ;

[0126] 2-3) Taking the initial driving state as the starting point, let J track (θ) = J safe (θ) = 0, calculate the cumulative tracking performance J in the prediction time domain τ∈[t,t+T] track (θ) and safety constraint performance J safe (θ), the specific steps are as follows:

[0127] 2-3-1) Let J track (θ) = J safe (θ)=0; according to the current strategy function π θ (s τ ) generates the vehicle control command u self (τ)=π θ (s τ ), according to the dynamic model of the vehicle, the vehicle state x at the next moment is obtained self (τ+1), and at the same time, the state x of the surrounding traffic participants at the next moment is obtained according to the motion model of the surrounding traffic participants surr (τ+1); according to the current candidate path τ iCalculate the candidate path state at the next moment based on the vehicle position And road information x road (τ+1), construct the driving state s at the next moment τ+1 ;

[0128] 2-3-2) Calculate the tracking performance l(τ) corresponding to time τ, construct constraints according to equations 14) to 18) and calculate the penalty function corresponding to the safety constraint according to equation 8) in,

[0129]

[0130] 2-3-3) Update the accumulated tracking performance J according to formula 9) track (θ) and the cumulative safety constraint performance J safe (θ);

[0131] 2-3-4) Let τ = v + 1, repeat 2-3-1) to 2-3-3) until τ = t + T, and get the initial driving state s t The expected cumulative tracking performance J track and expected cumulative safety constraint performance J safe ;

[0132] 2-4) Update the evaluation function V according to formula 10) w (s t ) and the corresponding evaluation function parameter w, and update the strategy function π according to formula (12) θ (s τ ) and its corresponding policy function parameter θ;

[0133] 2-5) Update the penalty factor ρ, set ρ = cρ, and set t = t+1.

[0134] Continue to execute steps 2-2) to 2-5) until the optimal evaluation function parameter w is obtained. * and policy function parameters θ * , and finally output the optimal evaluation function and the optimal policy function parameters Used for subsequent path optimization and vehicle control.

[0135] The second aspect of the present disclosure provides a device for constructing and solving an optimal decision-making control model for an intelligent vehicle, and its structure is shown in FIG. Figure 2 ,include:

[0136] An optimal decision-making control model construction module is used to minimize the path tracking performance and prediction tracking error of the ego vehicle as the objective function of the optimal decision-making control model, use the vehicle dynamics model as the motion constraint of the optimal decision-making control model, use the distance between the ego vehicle and surrounding traffic participants, the distance between the ego vehicle and the road, and the distance between the ego vehicle and the traffic light as the safety constraint of the optimal decision-making control model, and use the evaluation function for selecting the optimal path from the candidate path set and the strategy function for determining the optimal ego vehicle control command corresponding to the optimal path as the output of the optimal decision-making control model; wherein the path tracking performance characterizes the difference between the ego vehicle and the candidate path in the future prediction time domain and the stability and energy consumption of the ego vehicle during the tracking process, and the prediction tracking error is used to evaluate the pros and cons of tracking different paths;

[0137] The optimal decision-making control model solving module is used to establish a learning penalty function method to deal with safety constraints. During the exploration process, it monitors unsafe driving conditions and imposes penalties on strategies that violate constraints, so that the strategies iterate in the direction of satisfying the constraints. Finally, it outputs the optimal policy function and optimal evaluation function of the optimal decision-making control model.

[0138] In order to implement the above-mentioned embodiments, the embodiments of the present disclosure also propose a computer-readable storage medium on which a computer program is stored. The program is executed by a processor to execute the static path planning method for an autonomous driving vehicle of the above-mentioned embodiments.

[0139] Reference below Figure 3 , which shows a schematic diagram of the structure of an electronic device 100 suitable for implementing the embodiment of the present disclosure. It should be noted that the electronic device 100 includes a decision-making control system for an autonomous driving vehicle, wherein the electronic device in the embodiment of the present disclosure may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, servers, etc. Figure 3 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0140] like Figure 3As shown, the electronic device 100 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 101, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 102 or a program loaded from a storage device 108 to a random access memory (RAM) 103. In the RAM 103, various programs and data required for the operation of the electronic device 100 are also stored. The processing device 101, the ROM 102, and the RAM 103 are connected to each other via a bus 104. An input / output (I / O) interface 105 is also connected to the bus 104.

[0141] Typically, the following devices may be connected to the I / O interface 105: an input device 106 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, etc.; an output device 107 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 108 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 109. The communication device 109 may allow the electronic device 100 to communicate with other devices wirelessly or by wire to exchange data. Although FIG. 3 shows an electronic device 100 with various devices, it should be understood that it is not required to implement or have all the devices shown. More or fewer devices may be implemented or have alternatively.

[0142] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the present embodiment includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 109, or installed from the storage device 108, or installed from the ROM 102. When the computer program is executed by the processing device 101, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.

[0143] It should be noted that the computer-readable medium mentioned above in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, - but not limited to - an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0144] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0145] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device: constructs an optimal decision-making control model for an intelligent vehicle, minimizes the sum of the path tracking performance and the predicted tracking error of the vehicle as the objective function of the optimal decision-making control model, uses the vehicle dynamics model as the motion constraint of the optimal decision-making control model, uses the distance between the vehicle and surrounding traffic participants, the distance between the vehicle and the road, and the distance between the vehicle and the traffic light as the safety constraint of the optimal decision-making control model, and uses an evaluation function for selecting an optimal path from a set of candidate paths and a strategy function for determining the optimal vehicle control command corresponding to the optimal path as the output of the optimal decision-making control model; wherein the path tracking performance characterizes the difference between the vehicle and the candidate path in the future prediction time domain and the stability and energy consumption of the vehicle during the tracking process, and the predicted tracking error is used to evaluate the pros and cons of tracking different paths; establishes a learning penalty function method to handle safety constraints, monitors unsafe driving conditions during the exploration process and imposes penalties on strategies that violate constraints, so that the strategies iterate in the direction of satisfying the constraints, and finally outputs the optimal strategy function and optimal evaluation function of the optimal decision-making control model.

[0146] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, python, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0147] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0148] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0149] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0150] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute the instructions), or in conjunction with these instruction execution systems, devices or apparatuses. For the purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by or in conjunction with an instruction execution system, device or apparatus. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic devices), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or otherwise processing in a suitable manner if necessary, and then stored in a computer memory.

[0151] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0152] A person skilled in the art may understand that all or part of the steps of the above-mentioned embodiment method may be implemented by instructing the relevant hardware through a program, and the developed program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0153] In addition, each functional unit in each embodiment of the present application may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0154] The storage medium mentioned above may be a read-only memory, a disk or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limiting the present application. A person of ordinary skill in the art may change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A method for constructing and solving an optimal decision-making control model for an intelligent vehicle, characterized in that: include: An optimal decision-making control model for an intelligent vehicle is constructed, wherein the objective function of the optimal decision-making control model is to minimize the sum of the path tracking performance and the predicted tracking error of the ego vehicle, the ego vehicle dynamics model is used as the motion constraint of the optimal decision-making control model, the distance between the ego vehicle and surrounding traffic participants, the distance between the ego vehicle and the road, and the distance between the ego vehicle and the traffic light are used as the safety constraints of the optimal decision-making control model, and the evaluation function for selecting the optimal path from the candidate path set and the strategy function for determining the optimal ego vehicle control command corresponding to the optimal path are used as the output of the optimal decision-making control model; wherein the path tracking performance characterizes the difference between the ego vehicle and the candidate path in the future predicted time domain and the stability and energy consumption of the ego vehicle during the tracking process, and the predicted tracking error is used to evaluate the pros and cons of tracking different paths; Establishing a learning penalty function method to process the safety constraints, monitoring unsafe driving conditions during the exploration process and imposing penalties on strategies that violate the constraints, so that the strategies are iterated in the direction of satisfying the constraints, and finally outputting the optimal strategy function and optimal evaluation function of the optimal decision-making model; The mathematical expression of the optimal decision-making model of the intelligent vehicle is as follows: x self (τ+1)=F self (x self (t),u τ (i)) (2) x surr (τ+1)=F surr (x surr (t),x self (t),u τ (i)) (3) x road (τ)=F road (x self (t)) (4) g surr (x self (t),x surr (τ))≥D surr (5) g road (x self (t),x road )≥D road (6) g light (x self (t),x light (τ)=R)≥D light (7) in: Formula (1) is the objective function of the optimal decision-making model, J track (θ) represents the performance index of the strategy function, that is, the path tracking performance, J score (w) represents the performance index of the evaluation function, that is, the prediction tracking error, θ and w represent the strategy function parameters to be optimized and the evaluation function parameters respectively; t is the initial time, τ is the virtual time of the prediction time domain, and T is the length of the prediction time domain; s τ represents the driving state at time τ, s τ =[x ref (τ), x self (τ), x surr (τ), x road (τ), x light (τ)] T , x ref (τ) represents the candidate path state at time τ; x self (τ), x surr (τ), x road (τ) represents the state of the vehicle, the state of surrounding traffic participants and the road state at time τ respectively; x light (τ) represents the state of the traffic light at time τ; Represents different initial driving states s t distribution, where the different initial driving state distributions are the joint distributions of different candidate paths, vehicle states, surrounding traffic participant states, road states, and signal light states; uτ(θ) is the vehicle control command, satisfying u τ (θ)=π θ (s τ ), π θ (s τ ) is the driving state s from time τ τ Mapping to the optimal ego-car action; V w (s t ) is the driving state s from time t t to the optimal path tracking performance; l(·) represents the tracking cost at each step; Formula (2) is the dynamic model of the vehicle, F self (·) is the vehicle dynamics equation; Formula (3) is the motion model of surrounding traffic participants, F surr (·) Used to calculate the motion state of traffic participants such as pedestrians, bicycles and motor vehicles at the next moment; Formula (4) is the road state related to the vehicle driving, F road (·) is used to calculate the road information corresponding to the current state of the vehicle; Formula (5) is the safety constraint of surrounding traffic participants, g surr (x self (τ), x surr (τ)) represents the distance between the vehicle and surrounding traffic participants. The distance between the vehicle and surrounding traffic participants is greater than the safety threshold D surr When , it indicates that no collision accident occurred; Formula (6) is the road safety constraint, g road (x self (τ), x road (τ)) represents the distance between the vehicle and the driving lane. The distance between the vehicle and the driving lane is greater than the safety threshold D road When the vehicle is in the lane, it indicates that the vehicle has not left or deviated from the lane. Formula (7) is the safety constraint of the red light, g light (x self (τ), x light (τ) = R) represents the distance between the vehicle and the stop line when the traffic light is red. This distance is greater than the safety threshold D light , it means that the vehicle did not run a red light.

2. The method for constructing and solving the optimal decision-making model of an intelligent vehicle according to claim 1, characterized in that: The optimal strategy function and the optimal evaluation function of the optimal decision-making model are obtained by the following steps: 2-1) Initialize the evaluation function parameter w and the strategy function parameter θ, and set the evaluation function learning rate to β w , the learning rate of the policy function is β θ The starting point of the iteration is the initial time t, the initial penalty factor is ρ = 1, and the increase factor is c>1; the safety constraint performance J corresponding to the tracking constraint is introduced safe (θ); 2-2) Construct the initial driving state represents the state of the i-th candidate path at the initial time t; 2-3) Taking the initial driving state as the starting point, let J track (θ) = J safe (θ) = 0, calculate the expected cumulative tracking performance J in the prediction time domain τ∈[t, t+T] track and expected cumulative safety constraint performance J safe ; 2-4) Update the evaluation function V w (s t ) and the corresponding evaluation function parameter w: Update the strategy function parameter θ, and optimize the expected cumulative tracking performance J by the penalty factor ρ track and expected cumulative safety constraint performance J safe ; 2-5) Increase the penalty factor, that is, set ρ = cρ, and set t = t+1; 2-6) Continue to execute steps 2-2) to 2-5) until the optimal evaluation function parameter w* and strategy function parameter θ* are obtained, and finally the optimal evaluation function is output. With the optimal policy function 3. The method for constructing and solving the optimal decision-making model for an intelligent vehicle according to claim 2, characterized in that: Steps 2-3) specifically include: 2-3-1) Let J track (θ) = J safe (θ) = 0; according to the current driving state s τ Get the driving state s at the next moment τ+1 ; 2-3-2) Calculate the tracking cost l(τ) and safety constraint performance g corresponding to time τ surr (x self (τ), x surr (τ))-D surr , g road (x self (τ), x road (τ))-D road , g light (x self (τ), x light (τ) = R) - D light ; Set the penalty function corresponding to each safety constraint; 2-3-3) Calculate the cumulative tracking performance and the cumulative safety constraint performance: J track (θ)=J track (θ)+l(τ) 2-3-4) Let τ = τ + 1, repeat 2-3-1) to 2-3-3) until τ = t + T, and get the initial driving state s t Expected cumulative tracking performance J in the T-step prediction domain track and expected cumulative safety constraint performance J safe .

4. The method for constructing and solving the optimal decision-making model for an intelligent vehicle according to claim 3 is characterized in that: The penalty functions corresponding to the security constraints are set according to the following formula: in, is the penalty function for the safety constraints of surrounding traffic participants, is the penalty function of road safety constraints, is the penalty function for the safety constraint of the red light.

5. The method for constructing and solving the optimal decision-making model for an intelligent vehicle according to claim 2, characterized in that: The evaluation function V is expressed as follows: w (s τ ) and the corresponding evaluation function parameter w are updated: Among them, V w (s τ ) is the driving state s from time τ τ Mapping to optimal path following performance.

6. The method for constructing and solving the optimal decision-making model for an intelligent vehicle according to claim 2, characterized in that: Update the policy function parameter θ according to the following formula to optimize the expected cumulative tracking performance J track and expected cumulative safety constraint performance J safe : J π =J track +ρJ safe Among them, J π Used to evaluate ego-vehicle control strategies and characterize the balance between path tracking performance and safety.

7. A device for constructing and solving an optimal decision-making control model for an intelligent vehicle based on the method described in any one of claims 1 to 6, characterized in that: include: An optimal decision-making control model construction module is used to minimize the path tracking performance and predicted tracking error of the ego vehicle as the objective function of the optimal decision-making control model, use vehicle dynamics as the motion constraint of the optimal decision-making control model, use the distance between the ego vehicle and surrounding traffic participants, the distance between the ego vehicle and the road, and the distance between the ego vehicle and the traffic light as the safety constraint of the optimal decision-making control model, and use an evaluation function for selecting an optimal path from a set of candidate paths and a strategy function for determining an optimal ego vehicle control instruction corresponding to the optimal path as outputs of the optimal decision-making control model; wherein the path tracking performance characterizes the difference between the ego vehicle and the candidate path in the future prediction time domain and the stability and energy consumption of the ego vehicle during the tracking process, and the predicted tracking error is used to evaluate the pros and cons of tracking different paths; The optimal decision-making control model solving module is used to establish a learning penalty function method to deal with the safety constraints, monitor unsafe driving conditions during the exploration process and impose penalties on strategies that violate the constraints, so that the strategies are iterated in the direction of satisfying the constraints, and finally output the optimal strategy function and optimal evaluation function of the optimal decision-making control model.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the intelligent vehicle optimal decision-making and control model construction and solution method according to any one of claims 1 to 6.

Images