A Deep Learning-Based Autonomous Decision-Making Control Method for Multi-Objective Driving Modes in Intelligent Connected PHEVs
By constructing a multi-objective autonomous decision-making control model for driving modes and combining deep learning and reinforcement learning algorithms, intelligent connected PHEVs achieve comprehensive optimization of safety, comfort, and energy economy in complex traffic environments, solving the problem of insufficient adaptability and flexibility of traditional methods in complex environments.
Patent Information
- Application Number
- CN202411518852.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-29
- Publication Date
- 2026-11-10
- Estimated Expiration
- 2044-10-29
AI Technical Summary
Existing technologies struggle to comprehensively consider vehicle safety, comfort, and energy efficiency in complex traffic environments for intelligent connected vehicles. Traditional methods lack adaptability and flexibility, and research based on deep reinforcement learning has not yet been sufficiently in-depth.
Three traffic scenarios—cruising, following, and passing—are constructed. Information is acquired by combining ITS, GPS, and V2X technologies. A multi-objective autonomous decision-making and control model for driving modes is built using the MPC and DDPG algorithms with multi-objective functions. The state space, action space, and reward function are designed, and the optimization strategy is achieved through deep learning to realize comprehensive optimization of safety, comfort, and energy economy.
In complex traffic environments, intelligent connected PHEVs can autonomously decide on different driving modes, plan the optimal vehicle speed, improve safety, comfort and energy economy, and enhance decision-making capabilities in complex traffic environments.
Smart Images

Figure CN119527341B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of intelligent transportation technology, and in particular to an autonomous decision-making and control method for multi-objective driving modes of intelligent connected PHEVs based on deep learning. Background Technology
[0002] With the continuous development of Intelligent Transportation Systems (ITS) and artificial intelligence technologies, traditional automobiles and intelligent connected technologies are gradually becoming more integrated, which is regarded as an important direction for the future development of automobiles. Vigorously developing intelligent connected vehicles has become an international consensus. On the other hand, intelligent connected vehicles combine the advantages of autonomous driving and network interconnection. Researching control methods for vehicle driving modes to improve the safety, comfort, and energy economy of intelligent connected vehicles is of great significance.
[0003] Currently, with the increasing complexity of traffic scenarios and the rapid development of intelligent connected vehicles (ICV) and intelligent transportation systems (ITS), traditional vehicles typically rely on driver experience and traditional driver assistance systems to control the vehicle, without deeply integrating intelligent connected information. Faced with changing and complex road conditions, the safety, comfort, and energy economy of the vehicle during driving are difficult to guarantee.
[0004] Methods for solving the optimal control problem of intelligent connected vehicles mainly include rule-based, optimization, and learning algorithms. Rule-based control strategies have low adaptability and flexibility, failing to meet the demands of complex traffic environments. Optimization-based control methods require real-time online processing of large amounts of data, placing high demands on onboard processors and hindering their adaptability to complex traffic conditions. Research on deep reinforcement learning in the field of autonomous driving largely focuses on vehicle following decisions; further in-depth research is needed to comprehensively consider vehicle safety, comfort, and energy efficiency in complex traffic environments. Summary of the Invention
[0005] The purpose of this invention is to provide an autonomous decision-making control method for multi-objective driving modes of intelligent connected PHEVs based on deep learning. This addresses the issue that most existing research on autonomous driving based on deep reinforcement learning theory focuses on vehicle following decisions, while further research is needed to comprehensively consider vehicle safety, comfort, and energy economy in complex traffic environments.
[0006] 1. To achieve the above objectives, this invention provides a deep learning-based autonomous decision-making control method for multi-objective driving modes of intelligent connected PHEVs, comprising the following steps: S1: Construct three traffic scenarios in an intelligent connected environment: cruise scenario, following scenario, and through scenario; S2: Utilize ITS, GPS and V2X technologies to obtain road traffic information and basic information about vehicles ahead, and construct cruise intelligent connected PHEV driving models, follow-the-car intelligent connected PHEV driving models and traffic intelligent connected PHEV driving models. The optimal driving speed is solved by using the MPC algorithm that considers multiple objective functions. S3: Building an autonomous decision-making and control model for multi-objective driving modes based on the DDPG algorithm; S4: Use the publicly available data from the next-generation traffic simulation project as the training dataset for the deep reinforcement learning network, and extract the car-following cruise dataset as the training set for algorithm training. S5: The agent continuously adjusts its strategy based on the reward signals it receives. After multiple training and optimizations, it tends to stabilize, allowing it to explore a relatively stable behavioral strategy that can obtain stable rewards in the environment. Training terminates when the maximum number of training iterations is reached. S6: Simulation tests were conducted using the trained and optimized strategy and the rule-based ACC control strategy under the same typical and actual operating conditions, respectively, to evaluate the effectiveness and adaptability of the multi-objective driving mode autonomous decision control strategy.
[0007] Specifically, in step S2, the methods for constructing the cruise intelligent connected PHEV driving model, the following intelligent connected PHEV driving model, and the through intelligent connected PHEV driving model are as follows: Cruise Intelligent Connected PHEV Driving Model Design: When the vehicle is in cruise mode, the control target only needs to consider speed and acceleration. Therefore, the PID algorithm with good real-time performance and fast response is used to adjust the vehicle speed. Safety, comfort and energy economy are used as constraints. The desired speed is tracked based on PID control. The design of the driving model for the intelligent connected PHEV is as follows: By obtaining real-time traffic and road environment information and the status information of vehicles ahead, combined with vehicle safety distance strategy and longitudinal dynamics model, the speed of the following vehicle is optimized based on the model predictive control strategy with the goal of comprehensively optimizing the safety, comfort and energy economy of the following vehicle. Design of a driving model for a connected intelligent PHEV: Based on the intelligent connected environment, the target speed is planned according to the acquired traffic light information and the status information of the vehicle in front. A prediction model based on RBF neural network is established to predict the speed of the vehicle in front. The objective function is designed based on the planned target speed and the safe distance to the vehicle in front. At the same time, the driving torque, braking force and speed are constrained according to the limitations of the vehicle itself and the road. The multi-objective optimization problem is solved based on nonlinear MPC.
[0008] In step S3, the DDPG algorithm combines the deterministic policy gradient method with the Actor-Critic algorithm, and the multi-objective driving mode autonomous decision control model includes a state space, an action space, and a reward function.
[0009] Specifically, in step S3, the design of the state space, action space, and reward function is as follows: Design of state space and action space: When the target vehicle is driving in a complex traffic environment, it is expected that the multi-objective driving mode autonomous decision control model can make autonomous decisions on which driving mode to use based on the acquired surrounding information. When designing the reward function, three optimization objectives—driving safety, economy, and comfort—must be considered.
[0010] In step S4, the specific steps for algorithm training are as follows: build a model in Matlab / simulink to conduct simulation experiments, and use the public data of the next-generation traffic simulation project as the training dataset for the deep reinforcement learning network to improve the training quality.
[0011] In step S6, the trained and optimized strategy and the rule-based ACC control strategy are used to conduct simulation tests under the same typical working condition and the actual working condition, respectively. This includes the analysis of simulation results under the typical working condition and the analysis of simulation results under the actual working condition.
[0012] In step S6, the simulation results analysis of typical working conditions specifically involves: using a rule-based ACC control strategy to conduct simulation analysis and comparison under the same working conditions.
[0013] In step S6, the analysis of the actual working condition simulation results specifically refers to the generalization ability in real traffic scenarios.
[0014] This invention discloses a deep learning-based autonomous decision-making control method for multi-objective driving modes in intelligent connected PHEVs. It constructs three driving modes: cruising, following, and navigating. A state space and action space based on a deterministic policy gradient algorithm are designed. A multi-objective reward function is built to consider driving safety, passenger comfort, and energy economy. A Markov decision process is used to construct an autonomous decision-making control model for multi-objective driving modes. This ensures that the intelligent connected PHEV can autonomously decide on different driving modes to plan the optimal target vehicle speed when driving in complex environments. This achieves optimal overall safety, navigability, and energy economy during driving, further improving the decision-making ability of intelligent vehicles in complex traffic environments. Attached Figure Description
[0015] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0016] Figure 1 These are schematic diagrams of three traffic scenarios provided by this invention.
[0017] Figure 2 This is a traffic light phase and timing information diagram provided by the present invention.
[0018] Figure 3 This is a structural diagram of the cruise model framework provided by the present invention.
[0019] Figure 4 This is a structural framework diagram of the common model provided by the present invention.
[0020] Figure 5 This is a diagram of the DDPG multi-objective driving mode autonomous decision-making model provided by the present invention.
[0021] Figure 6 This is a framework diagram of the DDPG autonomous decision-making model for driving modes provided by this invention.
[0022] Figure 7 This is a graph showing the changes in the reward function and Q-value during the training iteration process provided by this invention.
[0023] Figure 8 This is a simulation result analysis diagram of typical working conditions provided by the present invention.
[0024] Figure 9 This is an analysis diagram of the actual working condition simulation results provided by the present invention. Detailed Implementation
[0025] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.
[0026] Please see Figures 1 to 9 This invention provides a deep learning-based autonomous decision-making control method for multi-objective driving modes of intelligent connected PHEVs. The deep learning-based autonomous decision-making control method for multi-objective driving modes of intelligent connected PHEVs includes the following steps: S1: Construct three traffic scenarios in an intelligent connected environment: cruise scenario, following scenario, and through scenario; S2: Utilize ITS, GPS and V2X technologies to obtain road traffic information and basic information about vehicles ahead, and construct cruise intelligent connected PHEV driving models, follow-the-car intelligent connected PHEV driving models and traffic intelligent connected PHEV driving models. The optimal driving speed is solved by using the MPC algorithm that considers multiple objective functions. S3: Building an autonomous decision-making and control model for multi-objective driving modes based on the DDPG algorithm; S4: Use the publicly available data from the next-generation traffic simulation project as the training dataset for the deep reinforcement learning network, and extract the car-following cruise dataset as the training set for algorithm training. S5: The agent continuously adjusts its strategy based on the reward signals it receives. After multiple training and optimizations, it tends to stabilize, allowing it to explore a relatively stable behavioral strategy that can obtain stable rewards in the environment. Training terminates when the maximum number of training iterations is reached. S6: Simulation tests were conducted using the trained and optimized strategy and the rule-based ACC control strategy under the same typical and actual operating conditions, respectively, to evaluate the effectiveness and adaptability of the multi-objective driving mode autonomous decision control strategy.
[0027] In this embodiment, three driving modes—cruising, following, and passing—are constructed. A state space and action space based on a deterministic policy gradient algorithm are designed, and a multi-objective reward function is constructed for driving safety, ride comfort, and energy economy. A multi-objective autonomous decision-making control model for driving modes is built using Markov decision processes. This ensures that when the intelligent connected PHEV is driving in complex environments, it can autonomously decide on different driving modes to plan the optimal target vehicle speed. This achieves the comprehensive optimization of safety, passability, and energy economy of the intelligent connected PHEV during driving, further improving the decision-making ability of intelligent vehicles in complex traffic environments.
[0028] In step S1, the vehicle obtains accurate location information of other vehicles through the Global Positioning System (GPS) and intelligent sensing onboard sensors, and acquires real-time and accurate road traffic information using vehicle-to-vehicle (V2V) and vehicle-to-infrastructure (V2I) communication. The target vehicle is selected as an intelligent connected plug-in hybrid electric vehicle (PHEV), equipped with sensors such as lidar, cameras, and radar, capable of sensing the position, speed, acceleration, and trajectory of surrounding vehicles. It can also intelligently interact with the traffic system to obtain current traffic light information. To simplify the subsequent solution process, only the longitudinal movement of the target vehicle and the influence of the preceding vehicle's state are considered, excluding overtaking and lane-changing behaviors. The intelligent connected environment information is shown in Table 1. Table 1 Intelligent Connected Environment Information Furthermore, in step S2, the specific methods for constructing the cruise intelligent connected PHEV driving model, the following intelligent connected PHEV driving model, and the through intelligent connected PHEV driving model are as follows: Cruise-enabled intelligent connected PHEV driving model design: When the vehicle is in cruise mode, the control objective only needs to consider speed and acceleration. Therefore, a PID algorithm with good real-time performance and fast response is used to adjust the vehicle speed. Safety, comfort, and energy economy are used as constraints. Based on PID control, the desired speed is tracked. Figure 3 As shown, if no following target is detected within the working range of the onboard sensors in the same lane, the vehicle operates in cruise mode. The target vehicle communicates with the Road Side Unit (RSU) via the Onboard Unit (OBU) to obtain the maximum speed limit of the road. While meeting road regulations, the energy management system allocates engine and motor torque to rapidly accelerate / decelerate the vehicle to the cruise speed set by the driver for stable driving. The cruise model expression is: in, For the desired acceleration, To output vehicle speed, The set cruising speed, The proportional term is used to adjust the system output. When the error is large, the effect of the proportional term is obvious, and the system will react more quickly. As integral coefficients, they help to address the problem of steady-state errors in the system, ensuring that the system can gradually eliminate persistent errors. These are differential coefficients, which help suppress system oscillations and overshoot, improving the system's dynamic response. The desired acceleration of the cruise model is limited to ensure ride comfort.
[0029] Following the design of the intelligent connected PHEV driving model: By acquiring real-time traffic environment information and the status information of vehicles ahead, combined with vehicle safety distance strategy and longitudinal dynamics model, the goal is to achieve the optimal balance of safety, comfort, and energy economy for the following vehicle. Based on model predictive control strategy, the speed of the following vehicle is optimized. In the design of the algorithm for solving the optimal speed problem, important constraints such as safe distance, speed, acceleration, and vehicle physical performance are considered to ensure that the vehicle can maintain a safe distance and comfort while also considering energy economy in actual driving. The safety distance strategy adopts the constant headway (CTH) strategy, which sets the target speed... As a state variable, acceleration As control variables, the multi-objective objective function and constraints are transformed into a standard quadratic programming form within the MPC framework, and solved using the interior-point method. By iteratively solving the quadratic programming problem, the optimal acceleration is obtained. The control sequence is determined by taking the first control input to the control system and repeating this process at the next time step. In this way, the optimal control sequence can be determined by solving a quadratic programming problem at each time step, ultimately yielding the optimal vehicle speed sequence over the entire driving distance. The steps are as follows: Safety, comfort, and energy economy requirements are transformed into corresponding objective functions. Relevant constraints are added within the MPC algorithm framework, and finally, the multi-objective optimization problem of the car-following model is transformed into a constrained quadratic programming problem for optimization. The objective function consists of the following three parts: 1. Following-the-car safety: Vehicle safety is the primary concern during driving. Throughout the entire driving process, it is essential to ensure that the distance between your vehicle and the vehicle in front is always greater than the safe following distance. To ensure good following distance while in a following vehicle, the actual distance between vehicles should be kept within a certain range. For the expected vehicle spacing The following error approaches 0. Positions of the vehicle in front and the following vehicles. The speed of the vehicle behind. The minimum safe distance is 10m, where, ; 2. Car-following comfort: Comfort during car-following involves multiple factors, among which the rate of change of acceleration is a crucial consideration. A smaller rate of change of acceleration generally results in a more comfortable ride, helping to reduce passenger discomfort during vehicle movement. By appropriately constraining the rate of change of acceleration, vehicle comfort during driving can be ensured. Furthermore, the acceleration change rate is used in this technical solution. The range is 。; 3. Energy consumption economy: The energy consumption economy of hybrid electric vehicles (PHEVs) can be represented by the fuel consumption rate. To optimize the energy consumption economy of the car-following model, the average power consumption per unit distance is defined as the fuel consumption model and represented by equivalent energy consumption, ultimately establishing the fuel consumption model for PHEVs: in, This represents the total distance traveled by the vehicle during time intervals 0-k. This represents the current vehicle's driving power. This indicates the overall driving efficiency of the vehicle. This represents the lower limit of the calorific value of gasoline. Based on model predictive control theory, the overall objective function is specifically composed of the following expression: In the formula These are weighting factors used to adjust the proportions of safety performance, comfort, and fuel economy indicators in the cost function. In most cases, and The numerical value is generally in and The magnitude, therefore The value of is chosen to be 1000 to ensure they are within the same order of magnitude, thus balancing dynamic performance and fuel economy indicators. The physical constraints that the vehicle must satisfy during operation are as follows: in, and These are the engine's minimum and maximum torques; , These are the minimum and maximum speeds of the engine. and These are the minimum and maximum constraints for the motor torque; , These are the minimum and maximum speeds of the motor.
[0030] Here, the target vehicle speed will be... As a state variable, acceleration As control variables, the multi-objective objective function and constraints are transformed into a standard quadratic programming form within the MPC framework, and solved using the interior-point method. By iteratively solving the quadratic programming problem, the optimal acceleration is obtained. The control sequence is determined by taking the first control input to the control system and repeating this process at the next time step. In this way, the optimal control sequence can be determined by solving a quadratic programming problem at each time step, and finally the optimal vehicle speed sequence over the entire driving distance can be obtained.
[0031] Design of a driving model for a connected intelligent PHEV: such as Figure 4 As shown, the design of the access pattern mainly includes the following steps: 1. Relying on the intelligent connected environment, target speed planning is performed based on acquired traffic light information and preceding vehicle status information. Here, when the target vehicle is traveling on a road with multiple traffic lights, its speed is significantly affected by the preceding vehicle's driving status and the current traffic light status. V2V technology is used to obtain the preceding vehicle's driving status information, including its current position and speed. V2I technology is used to obtain the preceding traffic light status, as well as the remaining time of the traffic light and the target vehicle's position at the traffic light. Simultaneously, the current road speed limit is considered. Based on the obtained SPAT information, a target reference speed is defined. When the target vehicle travels within the target reference speed range, the number of stops and starts at traffic light intersections can be reduced, optimizing the overall travel efficiency. The expression is as follows: in, These represent the upper and lower limits of the target reference vehicle speed, respectively. This represents the number of cycles of the traffic light. These represent the duration of a single red light, the duration of a single green light, and the time of a single traffic light cycle, respectively. Indicates in At this moment, the first The distance between a vehicle and a traffic light ( (1 is the vehicle in front, 2 is the target vehicle). Speed limits are imposed on roads.
[0032] 2. A prediction model based on RBF neural network was established to predict the speed of the vehicle in front. In the process of speed prediction, the historical vehicle speed sequence in the short time domain is used as input, and the future vehicle speed sequence is used as output, as shown in the following formula: The relationship between historical vehicle speed sequences and future vehicle speed sequences is as follows: In the formula , The nonlinear mapping function representing the prediction of the RBF neural network, and the parameter settings for training the RBF are shown in Table 2: Table 2 RBF Neural Network Parameter Settings
[0033] 3. Based on the planned target speed and the safe distance to the vehicle in front, an objective function is designed. Simultaneously, constraints are imposed on driving torque, braking force, and speed according to vehicle and road limitations. A multi-objective optimization problem is solved using nonlinear MPC. First, SPAT information is used to solve for the optimal target reference speed, ensuring the target vehicle passes through the traffic light intersection within the green light zone as much as possible. Second, an RBF neural network is used to establish a vehicle speed prediction model, and a multi-objective function is established based on the MPC algorithm to ensure the safety of the target vehicle. Comfort While improving energy efficiency, the number of vehicle start-stop cycles was also optimized. Finally, a quadratic programming algorithm was used to optimize and solve the multi-objective problem, and the established multi-objective function is shown in the following formula: Security Comfort The objective function for energy efficiency; Indicates the degree of deviation between the vehicle speed and the target reference speed; The target reference speed is to be solved.
[0034] The weighting coefficients are as follows: in: This represents the distance between the vehicle in front and the target vehicle at time t; This represents the speed difference between the vehicle in front and the target vehicle at time t; For time step.
[0035] At the same time, other constraints should also meet the limitations of the vehicle's physical model.
[0036] In step S3, the DDPG algorithm combines the deterministic policy gradient method with the Actor-Critic algorithm, and the multi-objective driving mode autonomous decision control model includes a state space, an action space, and a reward function.
[0037] The design of the state space and action space is as follows: When the target vehicle is driving in a complex traffic environment, the multi-objective driving mode autonomous decision-making control model is expected to autonomously decide which driving mode to use based on the acquired surrounding information. The current driving state of the target vehicle and its relationship with the surrounding environment directly determine the next "action" of the target vehicle. According to the definition of deep reinforcement, the state space is set as follows: After determining the driving mode, the system decides on corresponding actions, such as acceleration and deceleration, ultimately translating into control of vehicle acceleration. Acceleration parameters are used as the output actions of the decision model to establish a continuous action space. The meanings of each variable are shown in Table 3: Table 3. Design of State Space and Action Space In the autonomous decision-making and control process, the intelligent agent considers the interaction between the speed control algorithm and the traffic environment. Its expression is as follows: In the formula, Represents the vehicle's speed; Represents the speed of the vehicle in front; Represents the vehicle's acceleration; Represents the distance between the two vehicles; Represents a time interval. Total travel time This refers to the duration of the green light phase and the start point of the permitted passage time.
[0038] When designing the reward function, three optimization objectives—driving safety, economy, and comfort—must be considered. 1. Safety Reward Function: Safety is the primary consideration during vehicle operation. Controlling the distance between the target vehicle and the vehicle in front helps prevent rear-end collisions. However, the distance cannot be too large, as this would affect traffic efficiency. Therefore, the safety reward function needs to consider the relationship between the actual and desired vehicle distance, assigning penalties or rewards based on the degree of deviation from the desired distance. The safety reward function is designed as follows: In the formula, The desired vehicle distance; This is the actual distance between vehicles; This is the minimum safe distance.
[0039] 2. Comfort Reward Function: During vehicle operation, decisions about different driving modes are required, which can easily lead to rapid acceleration or deceleration. Comfort is generally considered to be related to acceleration and the rate of change of acceleration. Therefore, acceleration and the rate of change of acceleration during driving are used as parameters to measure comfort, and these are considered as rewards when designing the reward function. The comfort reward function is designed as follows: In the formula, The maximum and minimum allowable acceleration and deceleration values for the vehicle are (3, -3). .
[0040] 1. Economic Reward Function: Researchers typically use fuel consumption rate per unit time to represent a vehicle's fuel economy. In the previous section, an equivalent fuel consumption method was used to establish a fuel consumption model for PHEVs. However, in real-world driving from point A to destination B, time costs must also be considered. Therefore, a travel time parameter is added to the economic reward function. The vehicle needs to complete a specified mileage within a certain time, reducing energy consumption while saving time. The economic reward function is designed as follows: In the formula, This represents the actual distance traveled. Standard distance for training conditions; This refers to the actual travel time. This is the standard time for training conditions.
[0041] Furthermore, in step S4, the specific steps for algorithm training are as follows: build a model in Matlab / simulink to conduct simulation experiments, and use the public data of the next-generation traffic simulation project as the training dataset for the deep reinforcement learning network to improve the training quality.
[0042] In this embodiment, such as Figure 6 As shown in Table 4, the parameter settings for the deep neural network training phase of the DDPG algorithm in the multi-objective driving mode autonomous decision control model built in Simulink are as follows: Table 4 Parameters of Deep Neural Networks Depend on Figure 6 As shown, the Q-value of the DDPG algorithm exhibits a very stable trend, indicating that the agent has found a relatively ideal strategy during the learning process and has a fairly accurate understanding of the environment. When the vehicle performs well in terms of safety, comfort, and energy economy after making a decision, the corresponding reward function value is higher, represented by an increase in value in the figure. When the vehicle adopts a driving mode that does not meet the preset expectations, it will be penalized, and the reward function will show a decreasing trend. In particular, the agent's training effect is poor in the first 25 training iterations. This is because at the beginning, the agent will explore multiple strategies and try different control strategies randomly, resulting in low rewards. After 25 iterations, the reward function tends to stabilize, but to avoid getting trapped in local optima, the agent may repeatedly explore strategies, which will lead to fluctuations in the reward value. In the 25 to 80 training iterations, the agent continuously adjusts its strategy based on the obtained reward signals, and finally gradually stabilizes after 90 training iterations. This indicates that the agent has explored a relatively stable behavioral strategy during the learning process, and this strategy can obtain stable rewards in the environment. Finally, the training terminates when the maximum number of training iterations is reached.
[0043] Furthermore, in step S6, simulation tests are conducted using the trained and optimized strategy and the rule-based ACC control strategy under the same typical operating condition and actual operating condition, respectively. This specifically includes analysis of the simulation results under the typical operating condition and analysis of the simulation results under the actual operating condition. 1. Typical Working Condition Simulation Results Analysis: A rule-based ACC control strategy was used to conduct simulation analysis and comparison under the same working conditions. The typical working condition (WLTC-SPAT) was used as the preceding vehicle speed for simulation. The WLTC-SPAT working condition used the WLTC working condition as the original data, with a 2550-meter-long road section inserted between the low-speed and medium-speed sections. A traffic light appeared at intervals between these sections. The distribution distance and timing information of the four traffic lights were consistent with the network data established for the traffic pattern in this paper. During the passage through the traffic light intersection, the preceding vehicle maintained a constant speed while complying with the corresponding speed limits and traffic regulations for each speed segment. The total travel distance for the working condition was determined to be 25.80 km, with a duration of 2110 seconds. The simulation results are as follows: Figure 8 As shown in (b), throughout the entire driving cycle, both the rule-based ACC control strategy and the autonomous decision-based control strategy adopted a following mode from 0s. At 610s, the autonomous decision-based control strategy changed from following mode to a passing mode, while the rule-based ACC control strategy continued to maintain the following state. During intersections with traffic lights, the autonomous decision-based control strategy significantly reduced the number of stops, while the rule-based ACC control strategy maintained the same behavior as the vehicle in front, stopping at red lights and traveling at the maximum permitted speed at green lights, thus resulting in multiple stops at intersections with traffic lights. After 920s, after passing through the intersection with traffic lights, the autonomous decision-based driving mode changed back to following mode. When the speed of the vehicle in front suddenly changed, the target vehicle's speed also changed accordingly, ensuring following performance. Furthermore, the speed change trend was smoother than that of the rule-based ACC control strategy. At 1180s, 1490s, and 1850s, the driving mode switched to cruise mode due to road speed limits. Figure 8 As shown in (c), during the entire driving process, the acceleration variation range of the autonomous decision-making control strategy is significantly reduced because the speed is relatively stable while following the vehicle, and the randomness and delay of acceleration are small. However, when the speed suddenly changes during following, the rule-based ACC control strategy, due to its single control mode, follows a predetermined control law and exhibits a certain degree of lag. Since it still needs to follow the vehicle in front, it requires providing greater acceleration and deceleration, which is reflected in the figure as a larger absolute value of acceleration. Figure 8As shown in (d), both the autonomous decision-making control strategy and the ACC control strategy can safely pass through intersections with multiple traffic lights. However, the autonomous decision-making control strategy, while reducing the number of stops and starts for the target vehicle, incurs the same time cost as the ACC control strategy, indicating that it has higher traffic efficiency and can better optimize the energy consumption of the target vehicle. In summary, the multi-target driving mode autonomous decision-making control strategy based on the DDPG algorithm has more advantages than the rule-based ACC control strategy and can better complete mode decision-making in complex working conditions.
[0044] Analysis of Simulation Results under Actual Driving Conditions: To verify the generalization ability of this algorithm in real traffic scenarios, this section selects the Chongqing City real-world driving data scenario CQR2 as the test sample for simulation. The total driving distance in the CQR2 scenario is 14.01km, lasting 1407s, with traffic lights at 190s, 680s, and 860s. The speed of the preceding vehicle is calculated using the CQR2 scenario, while the speed of the following vehicle is calculated using both autonomous decision-making control strategy and rule-based ACC control strategy. Figure 9 (b) It can be seen that the autonomous decision-making control strategy can effectively make driving mode decisions for the target vehicle, using a passage model at all three traffic light intersections to plan vehicle speed and reduce the number of stops, while the rule-based ACC strategy maintains consistency with the behavior of the vehicle in front. When the vehicle speed reaches 50km / h, both strategies adopt cruise mode. When the distance to the vehicle in front decreases to a safe following distance, the cruise mode is switched to following mode, such as... Figure 9 (c) It can be seen that when the vehicle changes from acceleration to deceleration or from deceleration to acceleration, the autonomous decision control strategy optimizes the acceleration to make the change smoother and improves the comfort during driving.
[0045] The above description discloses only one preferred embodiment of the present invention, and should not be construed as limiting the scope of the present invention. Those skilled in the art will understand that all or part of the processes of the above embodiments can be implemented, and equivalent changes made in accordance with the claims of the present invention are still within the scope of the invention.
Claims
1. A deep learning-based autonomous decision-making and control method for multi-objective driving modes in intelligent connected PHEVs, characterized in that, Includes the following steps: S1: Construct three traffic scenarios in an intelligent connected environment: cruise scenario, following scenario, and through scenario; S2: Utilize ITS, GPS, and V2X technologies to acquire road traffic information and basic information about vehicles ahead, and construct a cruise intelligent connected PHEV driving model, a following intelligent connected PHEV driving model, and a through intelligent connected PHEV driving model. The cruise intelligent connected PHEV driving model uses a PID algorithm to adjust the vehicle speed, while the following intelligent connected PHEV driving model and the through intelligent connected PHEV driving model use an MPC algorithm that considers multiple objective functions to solve for the optimal driving speed. S3: A multi-objective driving mode autonomous decision-making control model is built based on the DDPG algorithm. The DDPG algorithm combines the deterministic policy gradient method with the Actor-Critic algorithm. The multi-objective driving mode autonomous decision-making control model includes a state space, an action space, and a reward function. The state space and action space are designed as follows: when the target vehicle is driving in a complex traffic environment, the multi-objective driving mode autonomous decision-making control model is expected to autonomously decide to use any one of the following driving modes: cruise intelligent connected PHEV driving mode, follow intelligent connected PHEV driving mode, and pass intelligent connected PHEV driving mode, based on the acquired surrounding information. The reward function considers three optimization objectives: driving safety, economy, and comfort. S4: The publicly available data from the Next Generation Traffic Simulation Project is used as the training dataset for the deep reinforcement learning network, and the car-following cruise dataset is extracted as the training set for training the DDPG algorithm. S5: The agent of the DDPG algorithm continuously adjusts its strategy based on the reward signal it receives. After multiple training optimizations, it tends to stabilize, allowing it to explore a relatively stable behavioral strategy. This behavioral strategy can obtain stable rewards in the environment, and training terminates when the maximum number of training iterations is reached. S6: Simulation tests were conducted using the trained and optimized behavior strategy and the rule-based ACC control strategy under the same typical and actual working conditions, respectively, and the effectiveness and adaptability of the multi-objective driving mode autonomous decision control model were evaluated. In step S2, the basic information of the vehicle ahead obtained specifically includes its position, speed, acceleration, and driving trajectory information.
2. The autonomous decision-making and control method for multi-objective driving modes of intelligent connected PHEVs based on deep learning as described in claim 1, characterized in that, In step S2, the specific methods for constructing the cruise intelligent connected PHEV driving model, the following intelligent connected PHEV driving model, and the through intelligent connected PHEV driving model are as follows: Cruise Intelligent Connected PHEV Driving Model Design: When the vehicle is in cruise mode, the control target needs to consider speed and acceleration. Therefore, the PID algorithm is used to adjust the vehicle speed, with safety, comfort and energy economy as constraints. Based on PID control, the desired speed is tracked. The design of the driving model for the intelligent connected PHEV is as follows: By obtaining real-time traffic and road environment information and the status information of vehicles ahead, combined with vehicle safety distance strategy and longitudinal dynamics model, the speed of the following vehicle is optimized based on the model predictive control strategy with the goal of comprehensively optimizing the safety, comfort and energy economy of the following vehicle. Design of a driving model for a connected intelligent PHEV: Based on the intelligent connected environment, the target speed is planned according to the acquired traffic light information and the status information of the vehicle in front. A prediction model based on RBF neural network is established to predict the speed of the vehicle in front. The objective function is designed based on the planned target speed and the safe distance to the vehicle in front. At the same time, the driving torque, braking force and speed are constrained according to the limitations of the vehicle itself and the road. The multi-objective optimization problem is solved based on nonlinear MPC.
3. The autonomous decision-making and control method for multi-objective driving modes of intelligent connected PHEVs based on deep learning as described in claim 2, characterized in that, In step S4, the specific steps for algorithm training are as follows: build a model in Matlab / simulink to conduct simulation experiments, and use the public data of the next-generation traffic simulation project as the training dataset for the deep reinforcement learning network to improve the training quality.
4. The autonomous decision-making and control method for multi-objective driving modes of intelligent connected PHEVs based on deep learning as described in claim 3, characterized in that, In step S6, simulation tests are conducted using the trained and optimized strategy and the rule-based ACC control strategy under the same typical operating condition and actual operating condition, respectively. Specifically, this includes analysis of the simulation results under the typical operating condition and analysis of the simulation results under the actual operating condition.
5. The autonomous decision-making and control method for multi-objective driving modes of intelligent connected PHEVs based on deep learning as described in claim 4, characterized in that, In step S6, the simulation results analysis of typical working conditions specifically involves: using a rule-based ACC control strategy to conduct simulation analysis and comparison under the same working conditions.
6. The autonomous decision-making and control method for multi-objective driving modes of intelligent connected PHEVs based on deep learning as described in claim 5, characterized in that, In step S6, the analysis of the actual working condition simulation results specifically refers to the generalization ability in real traffic scenarios.
Citation Information
Patent Citations
Anthropomorphic automatic driving car-following model based on deep reinforcement learning
CN109733415A
Electric vehicle economical self-adaptive cruise control method and system based on reinforcement learning
CN114771520A
