Intelligent networked PHEV multi-driving-mode autonomous decision control method based on deep learning

Abstract

The invention relates to the technical field of intelligent traffic, in particular to an intelligent networked PHEV multi-driving-mode autonomous decision control method based on deep learning. The method comprises the steps of designing a state space and an action space based on a deterministic strategy gradient algorithm by constructing three driving modes of a cruise scene, a car-following scene and a passing scene; a multi-target award function is constructed for driving safety, riding comfort and energy consumption economy, a Markov decision process is utilized to construct a multi-target driving mode autonomous decision control model, and it is ensured that when the intelligent network connection PHEV is driven in a complex environment, the optimal target vehicle speed can be planned by autonomously deciding different driving modes. The comprehensive optimization of safety, trafficability and energy consumption economy of the intelligent networked PHEV in the driving process is realized, and the decision-making ability of the intelligent vehicle in a complex traffic environment is further improved.

Description

Technical Field

[0001] The present invention relates to the field of intelligent transportation technology, and in particular to an autonomous decision-making control method for multiple driving modes of an intelligent networked PHEV based on deep learning. Background Art

[0002] With the continuous development of intelligent transportation system ITS and artificial intelligence technology, traditional cars and intelligent network technology are gradually integrated and regarded as an important direction for the future development of automobiles. Vigorously developing intelligent networked cars has become a consensus of the international community. On the other hand, intelligent networked cars combine the advantages of automatic driving and network interconnection. It is of great significance to study the control method of vehicle driving mode to improve the safety, comfort and energy economy of intelligent networked cars.

[0003] At present, with the increasing complexity of traffic scenarios and the rapid development of intelligent connected vehicles (ICV) and intelligent transportation systems (ITS), traditional vehicles usually rely on driver experience and traditional assisted driving systems to control vehicle driving, and do not deeply integrate intelligent network information. Faced with changeable and complex road conditions, the safety, comfort and energy economy of vehicles during driving are difficult to guarantee.

[0004] The methods used to solve the optimal control problem of intelligent connected vehicles mainly include rule-based, optimization and learning algorithms. The control strategy based on rule logic has low adaptability and flexibility and cannot meet the needs in complex traffic environments. The control method based on optimization requires real-time online processing of a large amount of data, which places high demands on the on-board processor and adapts to complex traffic environments. Most of the research in the field of vehicle autonomous driving based on deep reinforcement learning theory is about vehicle following driving decisions. Further research is needed to comprehensively consider the safety, comfort and energy economy of vehicles in complex traffic environments. Summary of the invention

[0005] The purpose of the present invention is to provide a multi-driving mode autonomous decision-making control method for an intelligent connected PHEV based on deep learning, so as to solve the problem that most of the research in the field of vehicle autonomous driving based on deep reinforcement learning theory in the prior art is about vehicle following driving decisions, and further in-depth research is still needed on comprehensively considering the safety, comfort and energy economy of the vehicle in a complex traffic environment.

[0006] 2. To achieve the above object, the present invention provides a deep learning-based intelligent networked PHEV multi-driving mode autonomous decision-making control method, the deep learning-based intelligent networked PHEV multi-driving mode autonomous decision-making control method comprising the following steps:

[0007] S1: Construct three traffic scenarios in an intelligent connected environment, namely, cruising scenario, following scenario and passing scenario;

[0008] S2: Using ITS, GPS and V2X technologies, the basic information of road traffic and the vehicle ahead is obtained, and the cruising intelligent connected PHEV driving model, the following intelligent connected PHEV driving model and the passing intelligent connected PHEV driving model are constructed. The MPC algorithm considering multiple objective functions is used to solve the optimal driving speed.

[0009] S3: Build a multi-objective driving mode autonomous decision-making control model based on the DDPG algorithm;

[0010] S4: Use the public data of the next generation traffic simulation project as the deep reinforcement learning network training data set, and extract the cruise following data set as the training set for algorithm training;

[0011] S5: The agent continuously adjusts its strategy based on the reward signal it receives, and after multiple training optimizations, it becomes stable, allowing it to explore a relatively stable behavior strategy that can obtain stable rewards in the environment. The training is terminated when the maximum number of training times is reached.

[0012] S6: Use the trained optimized strategy and the rule-based ACC control strategy to conduct simulation tests in the same typical operating conditions and actual operating conditions, and evaluate the effectiveness and adaptability of the driving mode autonomous decision-making control strategy.

[0013] Among them, in step S2, the specific method of constructing the cruising intelligent networked PHEV driving model, the following intelligent networked PHEV driving model and the passing intelligent networked PHEV driving model is:

[0014] Design of cruise intelligent network PHEV driving model: When the vehicle is in cruise mode, the control target only needs to consider speed and acceleration. Therefore, a PID algorithm with good real-time performance and fast response is used to adjust the vehicle speed. Safety, comfort and energy economy are used as constraints, and the desired speed is tracked based on PID control.

[0015] Design of the driving model of the smart connected PHEV: Based on the real-time traffic road environment information and the status information of the vehicle ahead, combined with the vehicle safety spacing strategy and the longitudinal dynamics model, the speed of the following vehicle is optimized based on the model predictive control strategy with the goal of comprehensively optimizing the safety, comfort and energy economy of the following vehicle;

[0016] Design of intelligent connected PHEV driving model: Relying on the intelligent connected environment, the target speed is planned according to the acquired traffic light information and the status information of the vehicle in front, and a prediction model based on RBF neural network is established to predict the speed of the vehicle in front. The objective function is designed according to the planned target speed and the safe distance maintained with the vehicle in front. At the same time, according to the limitations of the vehicle itself and the road, the driving torque, braking force and speed are constrained, and the multi-objective optimization problem is solved based on nonlinear MPC.

[0017] Among them, in step S3, the DDPG algorithm adopts a deterministic policy gradient method combined with an Actor-Critic algorithm, and the multi-objective driving mode autonomous decision-making control model includes a state space, an action space and a reward function.

[0018] Among them, in step S3, the design of the state space, action space and reward function is specifically as follows:

[0019] Design of state space and action space: When the target vehicle is driving in a complex traffic environment, the autonomous decision-making control model of the expected driving mode can autonomously decide which driving mode to use based on the surrounding information obtained;

[0020] When designing the reward function, the three optimization objectives of driving safety, economy and comfort need to be considered.

[0021] Among them, in step S4, the specific steps of algorithm training are: building a model in Matlab / simulink for simulation experiments, using the public data of the next generation of traffic simulation projects as the deep reinforcement learning network training data set to improve the training quality.

[0022] Among them, in step S6, the strategy after training optimization and the rule-based ACC control strategy are used to perform simulation tests in the same typical working condition and actual working condition respectively, specifically including typical working condition simulation result analysis and actual working condition simulation result analysis.

[0023] Among them, in step S6, the analysis of the simulation results of typical working conditions is specifically: using the ACC control strategy based on rule logic to perform simulation analysis and comparison under the same working conditions.

[0024] Among them, in step S6, the analysis of the actual working condition simulation results is specifically: generalization ability in real traffic scenarios.

[0025] The present invention discloses a multi-driving mode autonomous decision-making control method for intelligent connected PHEV based on deep learning. By constructing three driving modes of cruising scenario, following scenario and passing scenario, a state space and action space based on a deterministic policy gradient algorithm are designed, and a multi-objective reward function is constructed for driving safety, riding comfort and energy economy. A multi-objective driving mode autonomous decision-making control model is built using a Markov decision process to ensure that when the intelligent connected PHEV is driving in a complex environment, the optimal target vehicle speed can be planned by autonomously deciding on different driving modes, thereby achieving the comprehensive optimization of safety, passability and energy economy of the intelligent connected PHEV during driving, and further improving the decision-making ability of intelligent vehicles in complex traffic environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0027] Figure 1 It is a schematic diagram of three traffic scenarios provided by the present invention.

[0028] Figure 2 This is a signal light phase and timing information diagram provided by the present invention.

[0029] Figure 3 It is a framework structure diagram of the cruise model provided by the present invention.

[0030] Figure 4 It is a structural framework diagram of the traffic model provided by the present invention.

[0031] Figure 5 It is a DDPG multi-objective driving mode autonomous decision-making model diagram provided by the present invention.

[0032] Figure 6 It is a framework diagram of the DDPG driving mode autonomous decision-making model provided by the present invention.

[0033] Figure 7 It is a graph of reward function and Q value changes during the training iteration process provided by the present invention.

[0034] Figure 8 This is a typical working condition simulation result analysis diagram provided by the present invention.

[0035] Fig. 9 It is an analysis diagram of the actual working condition simulation results provided by the present invention. DETAILED DESCRIPTION

[0036] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.

[0037] See also Figures 1 to 9 The present invention provides a deep learning-based intelligent networked PHEV multi-driving mode autonomous decision-making control method, the deep learning-based intelligent networked PHEV multi-driving mode autonomous decision-making control method comprising the following steps:

[0038] S1: Construct three traffic scenarios in an intelligent connected environment, namely, cruising scenario, following scenario and passing scenario;

[0039] S2: Using ITS, GPS and V2X technologies, the basic information of road traffic and the vehicle ahead is obtained, and the cruising intelligent connected PHEV driving model, the following intelligent connected PHEV driving model and the passing intelligent connected PHEV driving model are constructed. The MPC algorithm considering multiple objective functions is used to solve the optimal driving speed.

[0040] S3: Build a multi-objective driving mode autonomous decision-making control model based on the DDPG algorithm;

[0041] S4: Use the public data of the next generation traffic simulation project as the deep reinforcement learning network training data set, and extract the cruise following data set as the training set for algorithm training;

[0042] S5: The agent continuously adjusts its strategy based on the reward signal it receives, and after multiple training optimizations, it becomes stable, allowing it to explore a relatively stable behavior strategy that can obtain stable rewards in the environment. The training is terminated when the maximum number of training times is reached.

[0043] S6: Use the trained optimized strategy and the rule-based ACC control strategy to conduct simulation tests in the same typical operating conditions and actual operating conditions, and evaluate the effectiveness and adaptability of the driving mode autonomous decision-making control strategy.

[0044] In this embodiment, by constructing three driving modes, namely cruising scenario, following scenario and passing scenario, the state space and action space based on the deterministic policy gradient algorithm are designed, and a multi-objective reward function is constructed for driving safety, riding comfort and energy economy. The Markov decision process is used to build a multi-objective driving mode autonomous decision-making control model to ensure that when the smart connected PHEV is driving in a complex environment, it can plan the optimal target vehicle speed by autonomously deciding on different driving modes, thereby achieving the comprehensive optimization of safety, passability and energy economy of the smart connected PHEV during driving, and further improving the decision-making ability of smart vehicles in complex traffic environments.

[0045] Among them, in step S1, the vehicle obtains the accurate location information of other vehicles through the global positioning system GPS (Global Positioning System) and intelligent sensing on-board sensors, and obtains real-time and accurate road traffic information by using vehicle-to-vehicle communication V2V and vehicle-to-infrastructure communication V2I. The target vehicle is selected as an intelligent networked plug-in hybrid vehicle, equipped with sensors such as lidar, camera, radar, etc., which can sense the position, speed, acceleration, driving trajectory and other information of surrounding vehicles, and can also interact intelligently with the traffic system to obtain current traffic light information. In order to simplify the subsequent solution process, it is set to only consider the longitudinal movement of the target vehicle and the influence caused by the state of the preceding vehicle, without overtaking and lane changing. The intelligent networked environment information is shown in Table 1:

[0046] Table 1 Intelligent network environment information

[0047]

[0048]

[0049] Furthermore, in step S2, the specific method of constructing the cruising intelligent networked PHEV driving model, the following intelligent networked PHEV driving model and the passing intelligent networked PHEV driving model is:

[0050] Design of cruise intelligent network PHEV driving model: When the vehicle is in cruise mode, the control target only needs to consider speed and acceleration. Therefore, a PID algorithm with good real-time performance and fast response is used to adjust the vehicle speed. Safety, comfort and energy economy are used as constraints, and the desired speed is tracked based on PID control, such as Figure 3As shown in the figure, in the same lane, if the on-board sensor does not identify the following target within the detection range, the vehicle will work in cruise mode. The target vehicle uses the on-board communication unit (OBU) to communicate with the road test unit (RSU) to obtain the maximum speed limit of the road. When the road regulations are met, the energy management system allocates the engine and motor torque to make the car quickly accelerate / decelerate to the cruising speed set by the driver for stable driving. The cruise model expression is:

[0051]

[0052] Among them, a des is the expected acceleration, v put is the output vehicle speed, v des is the set cruising speed, k p is the proportional coefficient, which is used to adjust the output of the system. When the error is large, the effect of the proportional term is obvious and the system will respond more quickly. l is the integral coefficient, which helps to solve the problem of steady-state error in the system and ensures that the system can gradually eliminate the persistent error. D is the differential coefficient, which helps to suppress the oscillation and overshoot of the system and improve the dynamic response of the system. To ensure the riding comfort, the expected acceleration of the cruise model is limited.

[0053] Design of the driving model of the smart connected PHEV with the following car: Through the real-time traffic road environment information and the status information of the vehicle in front, combined with the vehicle safety distance strategy and the longitudinal dynamics model, the safety, comfort and energy economy of the following vehicle are optimized as the comprehensive optimization. The speed of the following vehicle is optimized based on the model predictive control strategy. In the design of the algorithm for solving the optimal speed, important constraints such as safe vehicle distance, speed, acceleration and vehicle physical performance are considered to ensure that the vehicle can maintain a safe distance and comfort while taking into account energy economy in actual driving. The safety distance strategy selects the constant headway CTH strategy, takes the target vehicle speed v as the state variable, and the acceleration a as the control variable. The multi-objective objective function and constraints are converted into a standard quadratic programming form within the MPC framework. And the interior point method is used to solve it. By continuously iterating the quadratic programming problem, the optimal acceleration a is obtained. *Control sequence, take the first control input control system, and repeat this process at the next moment. In this way, the optimal control sequence can be determined by solving the quadratic programming problem at each time step, and finally the optimal vehicle speed sequence in the entire mileage can be obtained. The steps are as follows: convert the safety, comfort and energy economy requirements into corresponding objective functions, add relevant constraints under the MPC algorithm framework, and finally convert the multi-objective optimization problem of the following model into a constrained quadratic programming problem for optimization and solution. The objective function consists of the following three parts:

[0054] 1. J 1 Following safety: Vehicle safety is the primary concern during driving. During the entire driving process, it is necessary to ensure that the distance between the vehicle and the vehicle in front is always greater than the safe vehicle distance S. 0 To ensure good following performance when following a vehicle, the actual vehicle spacing S a For the expected vehicle spacing d des The following error approaches 0. l (t),S f (t) is the position of the leading vehicle and the following vehicle, v f (t) is the speed of the following vehicle, D is the minimum safe distance 10m, where J 1 =S a =D+τv f (t)-[S l (t)-S f (t)];

[0055] 2. J 2 Following comfort: The comfort of following involves multiple factors, among which the rate of change of acceleration is a very important consideration. A smaller rate of change of acceleration usually brings a more comfortable riding experience and helps reduce the discomfort of passengers during the driving process. By constraining the rate of change of acceleration accordingly, the comfort of the vehicle during driving can be guaranteed. 2 ={jerk min ≤jerk(t)≤jerk max}, and the acceleration change rate jerk range in this technical solution is (-2.5m / s 3 , 2.5m / s 3 ).

[0056] 3. J 3 Energy consumption economy: The energy consumption economy of hybrid vehicles can be expressed by fuel consumption rate. In order to optimize the energy consumption economy of the following model, the average power consumption per unit distance of the vehicle is defined as the fuel consumption model and expressed by equivalent energy consumption. Finally, the fuel consumption model of PHEV is established:

[0057]

[0058] Among them, S is the total mileage of the following vehicle at time 0-k, P req (t) is the current vehicle driving power, η eff Indicates the driving efficiency of the vehicle. LHV Represents the lower limit of the calorific value of gasoline. Based on the model predictive control theory, the overall objective function is specifically composed of the following expression:

[0059]

[0060] Where α 1 , α 2 , α 3 It is a weight factor used to adjust the proportion of safety performance index, comfort index and fuel economy index in the cost function. In most cases, J 1 , J 2 and J 3 The value is usually around 10 2 and 10 3 magnitude, so α 3 The value of is selected as 1000 to ensure that it is at the same level to balance the dynamic performance index and fuel economy index. The physical constraint formula that the vehicle needs to meet during driving is as follows:

[0061]

[0062] Among them, T eng,min and T eng,max is the minimum and maximum torque of the engine; ω eng,min ,ω eng,max is the minimum and maximum engine speed. em,min and T em,max are the minimum and maximum constraints of the motor torque; ω em,min ,ω em,max are the minimum and maximum speeds of the motor.

[0063] Here, the target vehicle speed v is used as the state variable and the acceleration a is used as the control variable. The multi-objective objective function and constraints are converted into a standard quadratic programming form within the MPC framework. The interior point method is used to solve the quadratic programming problem. By continuously iterating and solving the quadratic programming problem, the optimal acceleration a is obtained. * Control sequence, take the first control input control system, and repeat this process at the next moment. In this way, the optimal control sequence can be determined by solving the quadratic programming problem at each time step, and finally the optimal vehicle speed sequence in the entire mileage can be obtained.

[0064] Design of driving model for intelligent connected PHEV: Figure 4As shown in the figure, the design of the traffic mode mainly includes the following steps:

[0065] 1. Relying on the intelligent network environment, the target speed is planned according to the obtained signal light information and the status information of the preceding vehicle. Here, when the target vehicle is driving on a road with multiple signal lights, its speed will be greatly affected by the driving status of the preceding vehicle and the current traffic light status. V2V technology is used to obtain the driving status information of the preceding vehicle, including the current position and speed. V2I technology is used to obtain the status of the preceding signal light, as well as the remaining time of the signal light and the position of the target vehicle to the signal light. At the same time, the speed limit of the current road must be considered. The target reference speed is defined according to the obtained SPAT information. That is, when the target vehicle is driving within the target reference speed range, the number of starts and stops at the signal light intersection can be reduced, optimizing the driving efficiency of the entire journey. Its expression is as follows:

[0066]

[0067] Among them, v target_h (t) and v target_l (t) represent the upper and lower limits of the target reference speed respectively; K w is the number of cycles of the signal light; t r ,t g ,t c Respectively represent the duration of a single red light, the duration of a single green light, and the single cycle time of the signal light; d i (t) represents the distance between the i-th vehicle and the traffic light at time t (i = 1, 2, 1 is the front vehicle, 2 is the target vehicle); v max Speed ​​limit for the road.

[0068] 2. A prediction model based on RBF neural network is established to predict the speed of the preceding vehicle. In the process of speed prediction, the historical vehicle speed sequence in the short time domain is used as input, and the future vehicle speed sequence is used as output, as shown in the following formula:

[0069] V his =[v k-h+1 , v k-h ,…,v k ]

[0070] V ful =[v k+1 , v k+2 ,…,v k+p ]

[0071] The relationship between the historical speed sequence and the future speed sequence is as follows:

[0072] [v k+1 , v k+2 ,…,v k+p ] = fNN (v k-h+1 , v k-h ,…,v k )

[0073] Where V his ,V ful Represent the historical and future vehicle speed sequences, respectively, and f NN Represents the nonlinear mapping function predicted by the RBF neural network. The parameter settings for RBF training are shown in Table 2:

[0074] Table 2 RBF neural network parameter settings

[0075] Parameter name Parameter Value History step length h 5 Prediction step length p 5，10，15 The number of nodes in the hidden layer 10 Iterations 300 Learning Rate 0.01 Learning objectives 0.001

[0076] 3. Design the objective function based on the planned target speed and the safe distance to be maintained with the vehicle in front. At the same time, constrain the driving torque, braking force and speed according to the limitations of the vehicle itself and the road, and solve the multi-objective optimization problem based on nonlinear MPC. First, use the SPAT information to solve the optimal target reference speed, which can ensure that the target vehicle is in the green light zone as much as possible when passing the traffic light intersection. Secondly, establish a speed prediction model through the RBF neural network, and establish a multi-objective function based on the MPC algorithm. While ensuring the safety, comfort and energy economy of the target vehicle, the number of vehicle starts and stops is also optimized. Finally, the quadratic programming algorithm is used to optimize and solve the multi-objective problem. The established multi-objective function is shown in the following formula:

[0077]

[0078] J 4 (t) = [v(t) - v target (t)] 2

[0079] J 1 (t), J 2 (t), J 3 (t) are the objective functions of safety, comfort and energy economy respectively; J 4 (t represents the degree of deviation between the vehicle speed and the target reference speed; v target The target reference speed to be solved.

[0080] The weight coefficients are as follows:

[0081]

[0082] Where: s l (t)-s f (t) represents the distance between the preceding vehicle and the target vehicle at time t; v l (t)-v f(t represents the speed difference between the front vehicle and the target vehicle at time t; τ is the time step.

[0083] At the same time, other constraints should also satisfy the limitations of the vehicle physical model.

[0084] In step S3, the DDPG algorithm combines a deterministic policy gradient method with an Actor-Critic algorithm, and the multi-objective driving mode autonomous decision-making control model includes a state space, an action space, and a reward function.

[0085] Among them, the design of state space and action space: When the target vehicle is driving in a complex traffic environment, the expected driving mode autonomous decision control model can autonomously decide which driving mode to use based on the surrounding information obtained. The current driving state of the target vehicle and its relationship with the surrounding environment directly determine the next "action" of the target vehicle. According to the definition of deep reinforcement, the state space is set to S = [ν f ,ν l ,d rel ,ν rel ,t f ,ν lim After determining the driving mode, the system decides the corresponding actions, such as acceleration, deceleration, etc., which are finally converted into the control of vehicle acceleration. The acceleration parameter is used as the output action of the decision model to establish a continuous action space A = [a f ]. The meaning of each variable is shown in Table 3:

[0086] Table 3. Design of state space and action space

[0087]

[0088] In the autonomous decision-making and control process, the intelligent agent considers the interaction between the speed control algorithm and the traffic environment. The expression is as follows:

[0089]

[0090] Where, ν f represents the vehicle speed; ν l represents the speed of the preceding vehicle; a f represents the vehicle acceleration; d rel Represents the distance between two vehicles; T s represents the time interval, t f is the total travel time, t green With t red They are respectively the green light phase duration and the starting point of the passable time.

[0091] Among them, when designing the reward function, it is necessary to consider the three optimization goals of driving safety, economy and comfort;

[0092] 1. Safety reward function: Safety is the first thing to consider when driving. The distance between the target vehicle and the vehicle in front is controlled to ensure that there is no rear-end collision. However, the distance between the two vehicles cannot be too large, which will affect traffic efficiency. Therefore, the design of the safety reward function needs to consider the relationship between the actual vehicle distance and the expected vehicle distance, and give penalties or rewards according to the degree of deviation from the expected distance. The safety reward function is designed as follows:

[0093]

[0094] Where, d des is the expected vehicle distance; d is the actual vehicle distance; and D is the minimum safe distance.

[0095] 2. Comfort reward function: During vehicle driving, due to the need to decide on different driving modes, it is easy for the vehicle to accelerate or decelerate suddenly. It is generally believed that comfort is related to acceleration and acceleration rate of change. Therefore, acceleration and acceleration rate of change during driving are used as parameters to measure comfort, and they are considered for reward when designing the reward function. The comfort reward function is designed as follows:

[0096]

[0097] In the formula, ego a The maximum and minimum values ​​allowed for the acceleration and deceleration of the vehicle are (3, -3) m / s 2 .

[0098] 4. Economic reward function: Usually, researchers use the fuel consumption rate per unit time to express the fuel economy of the vehicle. In the previous article, the fuel consumption model of PHEV was established through the equivalent fuel consumption method. However, in real life, when driving from the starting point A to the destination B, the time cost needs to be considered. Therefore, the travel time parameter is added to the economic function. The vehicle needs to complete the specified mileage within a certain time, saving time cost and reducing energy consumption. The economic reward function is designed as follows:

[0099]

[0100] Where, d a is the actual driving distance; D s is the standard distance for training conditions; t a is the actual driving time; T s It is the standard time for training conditions.

[0101] Furthermore, in step S4, the specific steps of algorithm training are: building a model in Matlab / simulink for simulation experiments, using the public data of the next generation of traffic simulation projects as a deep reinforcement learning network training data set to improve the training quality.

[0102] In this embodiment, if Figure 6 As shown in the figure, the driving mode autonomous decision-making control model built in Simulink, the parameter settings of the deep neural network training phase of the DDPG algorithm are shown in Table 4:

[0103] Table 4. Parameters of deep neural network

[0104]

[0105]

[0106] Depend on Figure 6 As shown in the figure, the Q value change trend of the DDPG algorithm is very stable, indicating that the agent has found a relatively ideal strategy in the learning process and has a relatively accurate understanding of the environment. When the vehicle has a better performance in terms of safety, comfort, and energy economy after making decisions, the corresponding reward function value is higher, which is represented by an increase in value in the figure. When the vehicle adopts a mode that does not meet the preset expectations, it will be punished and the reward function will show a downward trend. Among them, the training effect of the agent is poor in the first 25 trainings. This is because at the beginning, the agent will explore multiple strategies and try different control strategies with random strategies, and the reward is low at this time. After 25 times, the reward function tends to be stable, but in order to avoid falling into the local optimum, the agent may use strategy exploration many times, which will cause the reward value to fluctuate. In the 25th to 80th training, the agent continuously adjusts the strategy according to the reward signal obtained, and finally gradually stabilizes after 90 trainings. This shows that the agent has explored a relatively stable behavior strategy in the learning process, and the strategy can obtain stable rewards in the environment. The training ends when the maximum number of training times is finally reached.

[0107] Further, in step S6, the strategy after training optimization and the rule-based ACC control strategy are used to perform simulation tests in the same typical working condition and actual working condition, respectively, including analysis of simulation results of typical working condition and analysis of simulation results of actual working condition:

[0108] 1. Analysis of simulation results of typical working conditions: The simulation analysis and comparison are carried out under the same working conditions using the rule-based logic ACC control strategy, and the typical working condition (WLTC-SPAT) is used as the front vehicle speed for simulation. The WLTC-SPAT working condition uses the WLTC working condition as the original data, in which a road with a total length of 2550 meters is inserted between the low-speed section and the medium-speed section. A traffic light will appear at every interval. The distribution distance of the four traffic lights and the traffic light timing change information are consistent with the network data established by the traffic mode in this article. While passing through the traffic light intersection, the front vehicle maintains a constant speed while complying with the speed limits and traffic regulations of the corresponding speed sections. It is determined that the entire mileage of the working condition is 25.80km long and lasts 2110s. The simulation results are as follows: Figure 8 It can be seen from (b) that in the entire driving condition, starting from 0s, both the rule-based ACC control strategy and the autonomous decision-making control strategy adopt the following mode. At 610s, the autonomous decision-making control strategy changes from the following mode to the passing mode, while the rule-based ACC control strategy continues to maintain the following state. During the traffic light intersection, the autonomous decision-making control strategy significantly reduces the number of stops, while the rule-based ACC control strategy maintains the same behavior mode as the front vehicle, stopping at a red light and driving at the maximum allowed speed at a green light. Therefore, it stops multiple times when passing through the traffic light intersection. After 920s, after passing the traffic light intersection, the autonomous decision-making driving mode changes to the following mode. When the speed of the front vehicle suddenly changes, the speed of the target vehicle can also change accordingly to ensure followability. At the same time, the speed change trend is softer than that of the rule-based ACC control strategy. At the same time, the driving mode changes to the cruise model at 1180s, 1490s and 1850s. This is because of the road speed limit, such as Figure 8 As can be seen from (c), during the entire driving process, the acceleration variation range of the autonomous decision-making control strategy is obvious. Since the speed is relatively stable during the following process, the randomness and delay of the acceleration are small, so the acceleration variation in the autonomous decision-making control strategy is significantly reduced. When the speed changes suddenly under the following process, the rule-based ACC control strategy follows the established control law due to its single control mode, and there is a certain lag, but it still needs to follow the vehicle in front. At this time, it is necessary to provide a larger acceleration and deceleration, which is reflected in the absolute value of the acceleration in the figure. Figure 8 As shown in (d), when encountering a multi-signal intersection, both the autonomous decision-making control strategy and the ACC control strategy can safely pass the signal light intersection, but the autonomous decision-making control strategy reduces the number of start-stop times of the target vehicle, and the time cost is the same as the ACC control strategy, indicating that its traffic efficiency is higher and can better optimize the energy consumption of the target vehicle. In summary, the driving mode autonomous decision-making control strategy based on the DDPG algorithm has more advantages than the ACC control strategy based on rule logic, and can better complete the mode decision in complex working conditions.

[0109] Analysis of simulation results of actual working conditions: In order to verify the generalization ability of this algorithm in real traffic scenarios, this section selects Chongqing's real driving data working condition CQR2 as a test sample for simulation. The entire mileage of the CQR2 working condition is 14.01km, the duration is 1407s, and there are traffic lights at 190s, 680s, and 860s respectively. The speed of the front vehicle adopts the CQR2 working condition, and the rear vehicle adopts the autonomous decision-making control strategy and the rule-based ACC control strategy for solution. Fig. 9 (b) It can be seen that the autonomous decision control strategy can effectively make driving mode decisions for the target vehicle. The traffic model is used at the three traffic light intersections to plan the vehicle speed and reduce the number of stops, while the rule-based ACC strategy is consistent with the behavior of the vehicle in front. When the speed reaches 50km / h, both strategies use the cruise mode. When the distance to the vehicle in front is reduced to a safe following distance, the cruise mode is changed to the following mode. Fig. 9 (c) It can be seen that when the vehicle changes from acceleration to deceleration or from deceleration to acceleration, the autonomous decision-making control strategy optimizes the acceleration to make its change smoother, thereby improving the comfort during driving.

[0110] What is disclosed above is only a preferred embodiment of the present invention, and it certainly cannot be used to limit the scope of rights of the present invention. Ordinary technicians in this field can understand that all or part of the processes of the above embodiment and equivalent changes made according to the claims of the present invention still fall within the scope of the invention.

Claims

1. A multi-driving mode autonomous decision-making control method for intelligent networked PHEV based on deep learning, characterized in that: The steps include: S1: Construct three traffic scenarios in an intelligent connected environment, namely, cruising scenario, following scenario and passing scenario; S2: Using ITS, GPS and V2X technologies, the basic information of road traffic and the vehicle ahead is obtained, and the cruising intelligent connected PHEV driving model, the following intelligent connected PHEV driving model and the passing intelligent connected PHEV driving model are constructed. The MPC algorithm considering multiple objective functions is used to solve the optimal driving speed. S3: Build a multi-objective driving mode autonomous decision-making control model based on the DDPG algorithm; S4: Use the public data of the next generation traffic simulation project as the deep reinforcement learning network training data set, and extract the cruise following data set as the training set for algorithm training; S5: The agent continuously adjusts its strategy based on the reward signal it receives, and after multiple training optimizations, it becomes stable, allowing it to explore a relatively stable behavior strategy that can obtain stable rewards in the environment. The training is terminated when the maximum number of training times is reached. S6: Use the trained optimized strategy and the rule-based ACC control strategy to conduct simulation tests in the same typical operating conditions and actual operating conditions, and evaluate the effectiveness and adaptability of the driving mode autonomous decision-making control strategy.

2. The deep learning-based intelligent networked PHEV multi-driving mode autonomous decision-making control method according to claim 1, characterized in that: In step S2, the specific method of constructing the cruising intelligent networked PHEV driving model, the following intelligent networked PHEV driving model and the passing intelligent networked PHEV driving model is: Design of cruise intelligent network PHEV driving model: When the vehicle is in cruise mode, the control target only needs to consider speed and acceleration. Therefore, a PID algorithm with good real-time performance and fast response is used to adjust the vehicle speed. Safety, comfort and energy economy are used as constraints, and the desired speed is tracked based on PID control. Design of the driving model of the smart connected PHEV: Based on the real-time traffic road environment information and the status information of the vehicle ahead, combined with the vehicle safety spacing strategy and the longitudinal dynamics model, the speed of the following vehicle is optimized based on the model predictive control strategy with the goal of comprehensively optimizing the safety, comfort and energy economy of the following vehicle; Design of intelligent connected PHEV driving model: Relying on the intelligent connected environment, the target speed is planned according to the acquired traffic light information and the status information of the vehicle in front, and a prediction model based on RBF neural network is established to predict the speed of the vehicle in front. The objective function is designed according to the planned target speed and the safe distance maintained with the vehicle in front. At the same time, according to the limitations of the vehicle itself and the road, the driving torque, braking force and speed are constrained, and the multi-objective optimization problem is solved based on nonlinear MPC.

3. The deep learning-based intelligent networked PHEV multi-driving mode autonomous decision-making control method according to claim 2, characterized in that: In step S3, the DDPG algorithm combines a deterministic policy gradient method with an Actor-Critic algorithm, and the multi-objective driving mode autonomous decision-making control model includes a state space, an action space, and a reward function.

4. The deep learning-based intelligent networked PHEV multi-driving mode autonomous decision-making control method according to claim 3, characterized in that: In step S3, the design of the state space, action space and reward function is specifically as follows: Design of state space and action space: When the target vehicle is driving in a complex traffic environment, the autonomous decision-making control model of the expected driving mode can autonomously decide which driving mode to use based on the surrounding information obtained; When designing the reward function, the three optimization objectives of driving safety, economy and comfort need to be considered.

5. The deep learning-based intelligent networked PHEV multi-driving mode autonomous decision-making control method according to claim 4, characterized in that: In step S4, the specific steps of algorithm training are: building a model in Matlab / simulink for simulation experiments, using the public data of the next generation of traffic simulation projects as the deep reinforcement learning network training data set to improve the training quality.

6. The deep learning-based intelligent networked PHEV multi-driving mode autonomous decision-making control method according to claim 5, characterized in that: In step S6, simulation tests are performed using the trained optimized strategy and the rule-based ACC control strategy in the same typical operating condition and actual operating condition, respectively, including analysis of simulation results of typical operating condition and analysis of simulation results of actual operating condition.

7. The deep learning-based intelligent networked PHEV multi-driving mode autonomous decision-making control method according to claim 6, characterized in that: In step S6, the analysis of the simulation results of typical working conditions is specifically as follows: using the rule-based logic ACC control strategy to perform simulation analysis and comparison under the same working conditions.

8. The deep learning-based intelligent networked PHEV multi-driving mode autonomous decision-making control method according to claim 7, characterized in that: In step S6, the analysis of the actual working condition simulation results is specifically: generalization ability in real traffic scenarios.

Images