A hybrid electric vehicle adaptive energy management system and method

Abstract

This invention discloses an adaptive energy management system and method for hybrid electric vehicles. The system comprises a cloud-based driving behavior data acquisition and preprocessing module, a vehicle usage style recognition module, a driving style recognition module, and a control parameter adaptive adjustment module, as well as an on-vehicle communication unit, a central domain controller, a gateway, and an energy management controller on the vehicle side. The system collects driver driving behavior and vehicle usage behavior data in real time. The cloud-based control platform identifies the driver's vehicle usage style and driving style. The control parameter adaptive adjustment module performs cloud-based adaptive calculation of the hybrid electric vehicle energy management strategy control parameters. Finally, using OTA (Over-the-Air) remote download technology, the energy management strategy parameters are transmitted from the cloud server to the vehicle's energy management controller, enabling personalized customization of the energy management strategy and improving the intelligence and adaptability of the hybrid electric vehicle's energy management strategy across a wide range of operating conditions.

Description

Technical Field

[0001] This invention belongs to the technical field of vehicle energy conservation and emission reduction, specifically relating to an adaptive energy management system and method for hybrid electric vehicles. Background Technology

[0002] With economic development, energy and environmental issues are becoming increasingly prominent. Faced with this severe situation, countries around the world are vigorously developing new energy vehicles. Due to the limitations of battery technology development in electric vehicles, hybrid electric vehicles are becoming a research hotspot for major automakers. Energy management strategies for hybrid electric vehicles are key to achieving energy conservation and emission reduction. In recent years, research on energy management strategies for hybrid electric vehicles has mainly fallen into two categories: rule-based and optimization-based. Rule-based control strategies are easy to implement and computationally fast, but have poor adaptability. Optimization-based control strategies include global optimization and real-time optimization, such as dynamic programming and equivalent fuel consumption strategies. While global optimization has significant optimization effects, it involves large computational loads and requires prediction of the entire driving condition. Real-time optimization algorithms can perform online optimization, addressing the shortcomings of global optimization algorithms that cannot be implemented online. However, real-time optimization energy management strategies can ensure local optima but not global optima; therefore, their fuel economy is inferior to global optimization strategies.

[0003] Fuel consumption in hybrid electric vehicles (PHEVs) is significantly influenced by driver behavior and style. Frequent charging increases the use of electrical energy, effectively reducing engine fuel consumption, while less frequent charging increases reliance on engine fuel consumption. Different driving styles, such as aggressive or conservative, also lead to varying overall vehicle fuel economy. However, current mass-produced PHEV energy management strategies exhibit high uniformity, failing to meet the demands of efficient energy management under diverse driving and usage behaviors. With the development of vehicle electrification, intelligence, and connectivity, and the increasing prevalence of Intelligent Transportation Systems (ITS) and vehicle-to-everything (V2X) technologies, data-driven PHEV energy management strategies are an inevitable future trend. By collecting and statistically analyzing large amounts of driver driving data, combined with user driving characteristics, charging habits, and traffic conditions, personalized energy management strategies can be developed. Remote upgrades can then be used to adaptively adjust the energy management control strategy, further unlocking the energy-saving potential of PHEVs. Summary of the Invention

[0004] The technical problem this invention aims to solve is the challenge of achieving efficient energy management for mass-produced hybrid electric vehicles (HEVs) by meeting the diverse driving and usage behaviors of different drivers, thereby improving the adaptability of HEV energy management strategies to different drivers. To address this problem, this invention proposes an adaptive system and method for HEV energy management strategies based on remote upgrades. This system transmits data representing driver behavior and usage behavior to a cloud control platform in real time. The cloud control platform identifies different driver styles and usage patterns, and adaptively adjusts the energy management strategy control parameters accordingly. The energy management control strategy is updated and upgraded online via remote download, thereby rationally planning battery SOC trajectories and allocating power between the engine and motor based on different drivers, ultimately improving the overall fuel economy of the HEV.

[0005] The technical solution adopted by this invention to solve this technical problem is:

[0006] An adaptive energy management system for hybrid electric vehicles, characterized in that it includes a cloud control platform with data connectivity and a vehicle-side terminal; wherein,

[0007] The cloud control platform includes a driving behavior data acquisition and preprocessing module, a vehicle usage style recognition module, a driving style recognition module, and a control parameter adaptive adjustment module.

[0008] The vehicle-mounted unit includes an onboard communication unit, a central domain controller, a gateway, and an energy management controller.

[0009] The gateway connects to the vehicle-side sensors and transmits the driver's vehicle use behavior data and driver driving behavior data collected by the vehicle-side sensors to the driving behavior data acquisition and preprocessing module through the vehicle communication unit.

[0010] The driving behavior data acquisition and preprocessing module preprocesses the collected driver vehicle use behavior data and driver driving behavior data, and transmits the preprocessed corresponding data to the vehicle use style recognition module and the driving style recognition module, respectively.

[0011] The vehicle usage style recognition module determines the driver's vehicle usage style based on the preprocessed driver vehicle usage behavior data, and the driving style recognition module determines the driver's driving style based on the preprocessed driver driving behavior data.

[0012] The adaptive adjustment module of control parameters adjusts the final SOC and trajectory factors and deep reinforcement learning network parameters in the battery state of charge trajectory planning strategy based on the different driving styles and driving styles of different drivers determined by the vehicle usage style recognition module and the driving style recognition module, and generates a control strategy upgrade package.

[0013] The vehicle communication unit remotely pushes the control policy upgrade package to the vehicle and downloads it to the flash memory of the central domain controller;

[0014] The central domain controller is connected to the energy management controller via the vehicle communication unit and gateway to perform self-tests on the energy management controller, determine whether the current operating environment is suitable for updating, and perform corresponding update or no update operations.

[0015] The energy management controller is connected to the cloud control platform via a gateway and an on-board communication unit to feed back updated driver driving behavior data and driver vehicle usage data to the cloud control platform.

[0016] As a further preferred embodiment of the present invention, the driver's vehicle behavior data collected in the driving behavior data acquisition and preprocessing module includes at least the SOC of the hybrid vehicle during charging, the driving range during charging, and the average daily driving mileage; the driver's driving behavior data collected includes at least the vehicle speed, accelerator pedal opening and its rate of change, and brake pedal opening and its rate of change.

[0017] As a further preferred embodiment of the present invention, the preprocessing method used in the driving behavior data acquisition and preprocessing module includes at least abnormal and duplicate data removal, missing data reconstruction, and data smoothing filtering.

[0018] As a further preferred embodiment of the present invention, the vehicle usage style recognition module uses K-means clustering to cluster and analyze vehicle usage behavior data to obtain the number of driver vehicle usage style types. The vehicle usage style recognition method can employ BP neural network and support vector machine. The vehicle usage styles include anxious, normal, and calm types.

[0019] As a further preferred embodiment of the present invention, in the driving style recognition module, principal component analysis is first used to reduce the dimensionality of the feature parameters to remove redundant information between the original variables; then, K-means clustering is used to cluster the driving styles; finally, BP neural network and support vector machine methods are used to recognize the driving styles; the driving styles include conservative, normal and aggressive.

[0020] As a further preferred embodiment of the present invention, the energy management controller adopts a hierarchical architecture, with the upper layer being a battery SOC trajectory planning module and the lower layer being a deep reinforcement learning module, which uses a deep reinforcement learning algorithm to solve the hybrid transmission gear positions and the torque distribution between the motor and the engine.

[0021] As a further preferred embodiment of the present invention, the upper-level battery SOC trajectory planning module adopts a mileage-related trajectory planning method, wherein the battery SOC trajectory planning method is as follows:

[0022]

[0023] In the formula,

[0024] soc ref The reference SOC obtained by the battery SOC trajectory planning module;

[0025] λ is the battery SOC trajectory factor;

[0026] soc ini and soc fin These are the initial SOC and final SOC of the battery, respectively.

[0027] L rem Represents the remaining mileage;

[0028] L tot Represents the total mileage;

[0029] As a further preferred embodiment of the present invention, the state space S of the deep reinforcement learning algorithm used in the lower-level deep reinforcement learning module is:

[0030] S = {v, acc, T} req ,soc} (7)

[0031] In the formula:

[0032] v represents the vehicle speed;

[0033] acc is the longitudinal acceleration;

[0034] T req This is the torque required by the entire vehicle;

[0035] soc represents the current state of battery charge.

[0036] For hybrid electric vehicles with discrete gearboxes, a deep reinforcement learning algorithm based on PDQN is used to solve the hybrid gearbox gears and the torque distribution between the motor and the engine.

[0037] At this point, the output action space A of the deep reinforcement learning algorithm is:

[0038] A = {T e i g o e} (8)

[0039] In the formula:

[0040] i g and i e These are the gear positions for the electric motor and the engine, respectively.

[0041] T e This refers to engine torque;

[0042] For hybrid electric vehicles that only have continuous torque output from the engine and motor and no discrete gearbox, a deep reinforcement learning algorithm based on DDPG is used to solve the torque distribution between the engine and motor.

[0043] At this point, the output action space A of the deep reinforcement learning algorithm is:

[0044] A = {T e n e T M} (9)

[0045] In the formula:

[0046] n e Engine speed;

[0047] T M This is the torque of the drive motor.

[0048] As a further preferred embodiment of the present invention, the reward function of the deep reinforcement learning is:

[0049]

[0050] In the formula:

[0051] fugl refers to engine fuel consumption;

[0052] α and β are the engine fuel consumption weighting coefficient and the battery SOC weighting coefficient, respectively.

[0053] t represents time;

[0054] SOC stands for State of Charge.

[0055] Furthermore, the present invention also provides a management method for an adaptive energy management system for hybrid electric vehicles, characterized by comprising the following steps:

[0056] 1) The vehicle-side sensors collect driver vehicle behavior data and driver driving behavior data in real time, and transmit the real-time collected driver vehicle behavior data and driver driving behavior data to the driving behavior data collection and preprocessing module of the cloud control platform through the gateway and vehicle communication unit;

[0057] 2) The driving behavior data acquisition and preprocessing module preprocesses the acquired driver vehicle use behavior data and driver driving behavior data, and transmits the preprocessed corresponding data to the vehicle use style recognition module and the driving style recognition module respectively.

[0058] 3) The vehicle usage style recognition module determines the driver's vehicle usage style based on the preprocessed driver vehicle usage behavior data, and the driving style recognition module determines the driver's driving style based on the preprocessed driver driving behavior data.

[0059] 4) The adaptive adjustment module of control parameters adaptively adjusts the final SOC and trajectory factors and deep reinforcement learning network parameters in the battery state of charge trajectory planning strategy based on the different driving styles and driving styles of different drivers obtained by the vehicle usage style recognition module and the driving style recognition module, and generates a control strategy upgrade package.

[0060] 5) The vehicle communication unit remotely pushes the control strategy upgrade package to the vehicle and downloads it to the flash memory of the central domain controller;

[0061] 6) The user is prompted to update via the human-machine interface. After receiving confirmation from the user that they agree to the update, the central domain controller performs a self-test on the vehicle energy management controller to determine whether the current operating environment is suitable for the update. After the self-test is completed, the energy management domain controller enters boot mode and waits for the program to be flashed.

[0062] 7) After the program is successfully flashed, the energy management controller parameters are updated, and the updated driver driving behavior data and driver vehicle use behavior data are fed back to the cloud control platform.

[0063] Compared with existing technologies, the technical solution provided by this invention has the following advantages:

[0064] 1) This invention provides an adaptive energy management system and method for hybrid electric vehicles. Based on cloud computing and V2X network communication, it adopts remote upgrade technology to realize the download of energy management strategies from the cloud server to the vehicle controller. It realizes the personalized customization and online update of energy management strategies for different drivers' driving habits and driving styles, improves the intelligence and adaptability of hybrid electric vehicle energy management strategies, and helps to improve the energy saving and emission reduction benefits of mass-produced hybrid electric vehicles.

[0065] 2) This invention provides an adaptive energy management system and method for hybrid electric vehicles. It utilizes deep reinforcement learning to train energy management strategies. The training of energy management strategies based on deep reinforcement learning is no longer limited to specific operating conditions, but is conducted using data uploaded to the cloud in real time from the actual vehicle, which helps improve the operating condition adaptability of the deep reinforcement learning strategy. Furthermore, this solution transfers the computationally intensive deep reinforcement learning training process to the cloud control platform, which helps reduce the hardware requirements of the vehicle-side controller, thereby reducing controller costs. Attached Figure Description

[0066] Figure 1 This is a logical structure diagram of the energy management system provided by the present invention;

[0067] Figure 2 This is a schematic diagram illustrating the working principle of the vehicle style recognition module of this invention.

[0068] Figure 3 This is a schematic diagram illustrating the working principle of the driving style recognition module of this invention.

[0069] Figure 4 This is a schematic diagram of the battery SOC trajectory planning method provided by the present invention.

[0070] Figure 5 This is a logical block diagram of the deep reinforcement learning algorithm provided by the present invention.

[0071] Figure 6 This is an upgrade flowchart of the control strategy upgrade package provided by the present invention. Detailed Implementation

[0072] The technical solution of the present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0073] [First Embodiment]

[0074] Figure 1 The first embodiment of this invention provides an adaptive energy management system for hybrid electric vehicles, comprising a cloud control platform with data connectivity and a vehicle-side terminal; wherein,

[0075] The cloud control platform includes a driving behavior data acquisition and preprocessing module, a vehicle usage style recognition module, a driving style recognition module, and a control parameter adaptive adjustment module.

[0076] The vehicle-mounted unit includes an onboard communication unit, a central domain controller, a gateway, and an energy management controller.

[0077] The gateway connects to the vehicle-side sensors and transmits the driver's vehicle use behavior data and driver driving behavior data collected by the vehicle-side sensors to the driving behavior data acquisition and preprocessing module through the vehicle communication unit.

[0078] The driving behavior data acquisition and preprocessing module preprocesses the collected driver vehicle use behavior data and driver driving behavior data, and transmits the preprocessed corresponding data to the vehicle use style recognition module and the driving style recognition module, respectively.

[0079] The vehicle usage style recognition module determines the driver's vehicle usage style based on the preprocessed driver vehicle usage behavior data, and the driving style recognition module determines the driver's driving style based on the preprocessed driver driving behavior data.

[0080] The adaptive adjustment module of control parameters adjusts the final SOC and trajectory factors and deep reinforcement learning network parameters in the battery state of charge trajectory planning strategy based on the different driving styles and driving styles of different drivers determined by the vehicle usage style recognition module and the driving style recognition module, and generates a control strategy upgrade package.

[0081] The vehicle communication unit remotely pushes the control policy upgrade package to the vehicle and downloads it to the flash memory of the central domain controller;

[0082] The central domain controller is connected to the energy management controller via the vehicle communication unit and gateway to perform self-tests on the energy management controller, determine whether the current operating environment is suitable for updating, and perform corresponding update or no update operations.

[0083] The energy management controller is connected to the cloud control platform via a gateway and an on-board communication unit to feed back updated driver driving behavior data and driver vehicle usage data to the cloud control platform.

[0084] Taking a mass-produced passenger vehicle as an example, firstly, the vehicle-side sensors record the driver's driving style data from the vehicle's CAN bus, including vehicle status information operated by the driver, vehicle speed, accelerator pedal opening and its rate of change, brake pedal opening and its rate of change, acceleration, etc., as well as user travel data, i.e., data related to vehicle usage style, including SOC during charging, SOC at the end of charging, electricity consumed per trip, charging frequency, charging range and average daily driving mileage, etc., and then package and upload the data to the database of the cloud control platform.

[0085] Then, the driver behavior data acquisition and preprocessing module receives driver vehicle use behavior data and driver driving behavior data for preprocessing. The preprocessing methods used include at least abnormal and duplicate data removal, missing data reconstruction, and data smoothing filtering to ensure the effectiveness, continuity and smoothness of the data.

[0086] Subsequently, the driving style recognition module uses K-means clustering to cluster and analyze driving behavior data to obtain the number of driver driving style types. Driving style recognition methods can employ BP neural networks and support vector machines. For example, based on driver driving habits and the urgency of their charging needs, drivers can be categorized into anxious, normal, and calm types.

[0087] The driving style recognition module uses principal component analysis to reduce the dimensionality of feature parameters, removing redundant information between original variables; then, K-means clustering is used to cluster driving styles; finally, a backpropagation neural network and support vector machine method are used for driving style recognition. For example, based on selected feature parameters, driving styles can be divided into conservative, normal, and aggressive types according to the degree of aggressiveness of vehicle driving.

[0088] In this embodiment,

[0089] The algorithm steps of Support Vector Machine include:

[0090] 1) Data preparation:

[0091] Collect and prepare datasets for training and testing, and then preprocess the data, including normalization, standardization, and feature engineering.

[0092] 2) Select the SVM type:

[0093] For classification tasks, choose either linear SVM or nonlinear SVM (by using kernel functions to handle nonlinear problems).

[0094] 3) Select the kernel function:

[0095] If a nonlinear SVM is chosen, an appropriate kernel function needs to be selected, such as a linear kernel, polynomial kernel, or Gaussian kernel. The choice of kernel function depends on the nature of the data and the requirements of the problem.

[0096] 4) Parameter tuning:

[0097] Adjust the hyperparameters of the SVM, such as the regularization parameter C and the parameters of the kernel function (e.g., the degree of the polynomial kernel, the bandwidth of the Gaussian kernel, etc.). Then, use methods such as cross-validation to find the optimal combination of hyperparameters.

[0098] 5) Model training:

[0099] Train the SVM model using the training set. Ensure the model has learned appropriately based on the chosen kernel function and parameters.

[0100] 6) Model Evaluation:

[0101] Use a test set to evaluate the model's performance, calculating metrics such as accuracy, precision, and recall. Analyze the model's performance as needed and consider whether parameter adjustments are necessary.

[0102] 7) Decision function and support vectors:

[0103] Once the trained model is obtained, its decision function can be used to classify or regress new samples. Support vectors are sample points that play a crucial role in the model training process.

[0104] 8) Model Interpretation:

[0105] Analyze the support vectors and hyperplane to understand the model's decision boundary in the feature space. Visualization techniques, such as plotting the decision boundary, can aid in understanding the model.

[0106] 9) Model Application:

[0107] The trained SVM model is applied to new, unseen data for prediction or regression.

[0108] These steps constitute the training and recognition process of a support vector machine. In practical applications, parameter tuning and optimization may be required depending on the specific circumstances to obtain better model performance.

[0109] A PHEV vehicle dynamics model is established on a cloud server to build the environment for training deep reinforcement learning agents. The vehicle dynamics model can include a driver model, engine model, motor model, battery model, vehicle controller, etc.

[0110] Finally, communication between the real vehicle and the cloud control platform is established. Training is conducted using real vehicle driving conditions, with driving style as one of the state variables in the reinforcement learning process. The reinforcement learning agent, i.e., the energy management strategy, is trained online. By identifying the current driving style, corresponding energy management strategies are pushed online.

[0111] Figure 3 This is a driving style recognition module that integrates connected vehicle information. Assuming a PHEV model has 10,000 drivers, driver behavior data and vehicle driving data are synchronously recorded in the vehicle's backend to establish a corresponding database. Vehicle speed, accelerator pedal opening, brake pedal opening, following distance, and relative speed are collected via the vehicle's CAN bus. Vehicle location information, traffic light phase times, average speed of the road segment, road slope, and remaining mileage are obtained through roadside RSUs and map apps. The onboard terminal uploads the data recorded in the backend to the cloud server in real time. The cloud control platform preprocesses the collected data, selecting the maximum, minimum, average, and standard deviation of vehicle speed, accelerator pedal opening, brake pedal opening, average speed of the road segment, road slope, and remaining mileage as feature parameters. Then, PCA principal component analysis is performed to reduce the dimensionality of the data and remove redundant information.

[0112] Preferably, K-Means clustering is used to classify driver driving styles into nine categories: Conservative (smooth traffic), Normal (smooth traffic), Aggressive (smooth traffic), Conservative (mild congestion), Normal (mild congestion), Aggressive (mild congestion), Conservative (heavy congestion), Normal (heavy congestion), and Aggressive (heavy congestion). The principle of K-Means clustering is to divide the sample set into k classes, assigning each sample to the k classes that minimizes the distance from each sample to the center of each class. The K-Means clustering algorithm is an iterative process. First, the cluster center of each class is determined, and each sample is assigned to the class closest to its center. Then, the mean of each class is updated as the new cluster center. This process is repeated until convergence. The specific algorithm flow is as follows:

[0113] Step 1: Determine the final number of clusters. Based on the actual situation and existing research, subjectively determine the number of clusters, and continuously adjust it according to the clustering results to seek the optimal number of clusters.

[0114] Step 2: Determine the initial cluster centers. Set the initial cluster centers yourself based on the characteristics of the existing data.

[0115] Step 3: Calculate the distance from each sample to each cluster center. Based on the characteristics of the K-means algorithm, assign each sample to the cluster closest to the cluster center.

[0116] Step 4: Recalculate the distance from the sample to the cluster center and determine the new cluster center.

[0117] Step 5: Repeat Step 4 until the cluster centers are completely consistent in both steps. Otherwise, continue to calculate Step 4.

[0118] Preferably, an SVM (Support Vector Machine) is used to train the above driving style recognition. The input of the neural network is vehicle speed, accelerator pedal opening, brake pedal opening, following distance, relative vehicle speed, average vehicle speed of the road segment, road slope and remaining driving distance. The output is the driving style category under the current working conditions.

[0119] In this embodiment, the energy management controller adopts a hierarchical architecture. The upper layer is a battery SOC trajectory planning module, and the lower layer is a deep reinforcement learning module, which uses deep reinforcement learning algorithms to solve for the hybrid transmission gear positions and the torque distribution between the motor and engine.

[0120] The upper-layer battery SOC trajectory planning module adopts a mileage-related trajectory planning method, which is as follows:

[0121]

[0122] In the formula,

[0123] soc ref The reference SOC obtained by the battery SOC trajectory planning module;

[0124] λ is the battery SOC trajectory factor;

[0125] soc ini and soc fin These are the initial SOC and final SOC of the battery, respectively.

[0126] L rem Represents the remaining mileage;

[0127] L tot Represents the total mileage;

[0128] The state space S of the deep reinforcement learning algorithm used in the lower-level deep reinforcement learning module is:

[0129] S = {v, acc, T} req ,soc} (12)

[0130] In the formula:

[0131] v represents the vehicle speed;

[0132] acc is the longitudinal acceleration;

[0133] T req This is the torque required by the entire vehicle;

[0134] soc represents the current state of battery charge.

[0135] For hybrid electric vehicles with discrete gearboxes, a deep reinforcement learning algorithm based on PDQN is used to solve the hybrid gearbox gears and the torque distribution between the motor and the engine.

[0136] At this point, the output action space A of the deep reinforcement learning algorithm is:

[0137] A = {T e i g i e} (13)

[0138] In the formula:

[0139] i g and i e These are the gear positions for the electric motor and the engine, respectively.

[0140] T e This refers to engine torque;

[0141] For hybrid electric vehicles that only have continuous torque output from the engine and motor and no discrete gearbox, a deep reinforcement learning algorithm based on DDPG is used to solve the torque distribution between the engine and motor.

[0142] At this point, the output action space A of the deep reinforcement learning algorithm is:

[0143] A = {T e n e T M} (14)

[0144] In the formula:

[0145] n e Engine speed;

[0146] T M This is the torque of the drive motor;

[0147] Simultaneously outputting discrete gears and continuous torque can further optimize energy consumption.

[0148] The DDPG algorithm flow is as follows:

[0149] Randomly initialize the Critic network Q(s, a|θ) Q ) and actor network μ(s|θ μ )

[0150] 1) Initialize the target network θ Q′ ←θ Q θ μ′ ←θ μ ;

[0151] 2) Initialize the experience buffer R;

[0152] 3) Repeat step 1 for each round, i.e., process 4-process 16;

[0153] 4) Within M rounds;

[0154] 5) Explore actions during random initialization;

[0155] 6) Receive initialization status s1;

[0156] 7) Repeat step 2 for each round, i.e., process 8) - process 16);

[0157] 8) Within the number of iterations T:

[0158] 9) Select action a based on the current strategy and the noise exploration. t =μ(s) t |θ μ )+N t ;

[0159] 10) Perform action a t Observational report r t Observe the new state s t+1 ;

[0160] 11) Store the interactive data in the experience buffer R;

[0161] 12) Randomly sample N mini-batch data (s) from the experience buffer R. t a t r t s t+1 (data);

[0162] 13) Set the critic network target y i =r i +γQ′(s i+1 ,μ′(si+1 |θ μ′ )|θ Q′ );

[0163] 14) By minimizing the loss function Update the critic network;

[0164] 15) Use sampling gradients to update the policy network

[0165]

[0166] 16) Update the target network;

[0167] θ Q′ ←τθ Q +(1-τ)θ Q′

[0168] θ μ ←τθ μ +(1-τ)θ μ′

[0169] 17) End loop 2;

[0170] 18) End loop 1;

[0171] The reward function for the deep reinforcement learning is:

[0172]

[0173] In the formula:

[0174] fuel refers to the engine's fuel consumption;

[0175] α and β are the engine fuel consumption weighting coefficient and the battery SOC weighting coefficient, respectively.

[0176] t represents time;

[0177] SOC stands for State of Charge.

[0178] Here, the reward function is a weighted average of instantaneous fuel consumption and SOC offset. By using a training network and fully utilizing techniques such as experience replay, priority replay, and small-scale approximation, the stability and computational efficiency of the algorithm are enhanced through a target value network method. The training process is complete when the reward function of reinforcement learning converges after multiple training iterations.

[0179] PDQN combines DQN and DDPG to handle discrete and continuous action spaces, respectively. The PDQN algorithm is suitable for hybrid action spaces. Hybrid electric vehicles' power systems include an engine and an electric motor, which output power through power coupling. While the torque and speed changes of both the engine and motor are continuous, the gear positions of the motor and engine are discrete. Furthermore, there is a correlation between discrete and continuous actions. Therefore, energy management strategies based on the PDQN algorithm are suitable for hybrid systems. PDQN uses two neural networks, also inputting the state and action together into the second network. However, PDQN does not have an Actor-Critic structure; instead, it consists of a continuous action Q-network and a discrete action Q-network.

[0180] The specific algorithm flow of PDQN is as follows:

[0181] 1) Initialization: Action-Value Network Q(s, x) k |θ μ ), Action parameter network μ(s|θ) μ ), and the target action value network Q′ and the target action parameter network μ′, and change their weights to θ Q′ ←θ Q θ μ′ ←θ μ Initialize the experience buffer R;

[0182] 2) Perform cycle 1 for each round, i.e., process 3-process 19;

[0183] 3) Observe the current state S t ;

[0184] 4) Initialize the random process for motion parameter exploration;

[0185] 5) Perform loop 2 at each step of this environment, i.e., process 6) - process 18);

[0186] 6) Calculate motion parameters x k ←μ(S t |θ μ )+N k ;

[0187] 7) Calculate the action value Q k ←Q(S t x k |θ Q );

[0188] 8) Select an action based on the greedy strategy ε:

[0189] 9) Perform action a t ;

[0190] 10) Observe the next state S t+1 and reward r t =R(S) t a t );

[0191] 11) Store (S) in the return visit buffer experience pool t a t S t+1 a t+1 );

[0192] 12) Update the current state S t ←S t+1 ;

[0193] 13) Sample small batches of experience e from the revisit buffer experience pool i =(s i a i s i+1 r i );

[0194] 14) Action a i Decompose into k i and Where k i Represents discrete actions, and corresponding continuous actions;

[0195] 15) Training Movement Value Network Q:

[0196] Calculate y i =r i +γ max k i ∈|K|Q′(s i+1 1, μ′(s i+1 |θ μ′ )|θ Q′ );

[0197] Calculate loss

[0198] Perform a stochastic gradient descent step on the loss L;

[0199] 16) Training the motion parameter network μ:

[0200] Calculate loss

[0201] Perform a stochastic gradient descent step on the loss L;

[0202] 17) Update the parameters of the target action value network and the target action parameter network:

[0203] θ Q′ ←τθQ +(1-τ)θ Q′ ;

[0204] θ μ′ ←τθ μ +(1-τ)θ μ′ Where τ represents the average rate

[0205] 18) End loop 2;

[0206] 19) End loop 1.

[0207] In the energy management optimization of hybrid power systems based on the PDQN algorithm, the optimization objective is to minimize the equivalent energy consumption of the hybrid power system while meeting the same power demand of the driver. The constraints are the allowable variation range of the power battery charge, the maximum output power of the engine, and the maximum output power of the motor. Action variables and state variables are selected to construct the action space and state space. The optimal reward function to achieve the optimization function is set in combination with the optimization objective.

[0208] The optimization goals for new energy vehicles go beyond just energy consumption; they must also consider traffic efficiency, driving comfort, operating and time costs, and power performance. Constraints no longer solely stem from the vehicle's powertrain but also include factors such as traffic signals, other road users, and traffic signs. This necessitates establishing a multi-objective optimization evaluation system that considers these multi-dimensional constraints. The PDQN-based energy management strategy for new energy vehicles leverages multi-source perception information and vehicle status information acquired by the onboard controller. It constructs a multi-source input energy management optimization framework based on deep reinforcement learning to solve for the optimal energy allocation between the engine and electric motor, further achieving multi-objective parameter optimization.

[0209] like Figure 6 As shown, the trained control strategy upgrade package generates a C code upgrade package, which is remotely pushed to the vehicle communication terminal via OTA through the cloud control platform's 5G-LTE communication technology, and downloaded to the flash memory of the central domain controller. The user is then prompted to update through the human-machine interface.

[0210] After receiving confirmation of the user's consent to the update, the central domain controller performs a self-check on the vehicle's energy pipeline system to determine if the current operating environment is suitable for the update. After the self-check is complete, the energy management domain controller enters boot mode, waiting for the program to be flashed. Once the program is successfully flashed, the OTA update is complete, and the latest system status is fed back to the cloud control platform via the vehicle communication unit.

[0211] The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above embodiments. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.

Claims

1. An adaptive energy management system for hybrid electric vehicles, characterized in that: This includes a cloud control platform and vehicle-mounted terminals for data connectivity; among which, The cloud control platform includes a driving behavior data acquisition and preprocessing module, a vehicle usage style recognition module, a driving style recognition module, and a control parameter adaptive adjustment module. The vehicle-mounted unit includes an onboard communication unit, a central domain controller, a gateway, and an energy management controller. The gateway connects to the vehicle-side sensors and transmits the driver's vehicle use behavior data and driver driving behavior data collected by the vehicle-side sensors to the driving behavior data acquisition and preprocessing module through the vehicle communication unit. The driving behavior data acquisition and preprocessing module preprocesses the collected driver vehicle use behavior data and driver driving behavior data, and transmits the preprocessed corresponding data to the vehicle use style recognition module and the driving style recognition module, respectively. The vehicle usage style recognition module determines the driver's vehicle usage style based on the preprocessed driver vehicle usage behavior data, and the driving style recognition module determines the driver's driving style based on the preprocessed driver driving behavior data. The adaptive adjustment module of control parameters adjusts the final SOC and trajectory factors and deep reinforcement learning network parameters in the battery state of charge trajectory planning strategy based on the different driving styles and driving styles of different drivers determined by the vehicle usage style recognition module and the driving style recognition module, and generates a control strategy upgrade package. The vehicle communication unit remotely pushes the control policy upgrade package to the vehicle and downloads it to the flash memory of the central domain controller; The central domain controller is connected to the energy management controller via the vehicle communication unit and gateway to perform self-tests on the energy management controller, determine whether the current operating environment is suitable for updating, and perform corresponding update or no update operations. The energy management controller is connected to the cloud control platform via a gateway and an on-board communication unit to feed back the updated driver driving behavior data and driver vehicle usage data to the cloud control platform.

2. The adaptive energy management system for hybrid electric vehicles according to claim 1, characterized in that, The driving behavior data collection and preprocessing module collects driver vehicle behavior data including at least the State of Charge (SOC) of the hybrid vehicle during charging, the driving range during charging, and the average daily driving mileage; and collects driver driving behavior data including at least vehicle speed, accelerator pedal opening and its rate of change, and brake pedal opening and its rate of change.

3. The adaptive energy management system for hybrid electric vehicles according to claim 1, characterized in that: The preprocessing methods used in the driving behavior data acquisition and preprocessing module include at least the removal of abnormal and duplicate data, reconstruction of missing data, and data smoothing filtering.

4. The adaptive energy management system for hybrid electric vehicles according to claim 1, characterized in that: In the vehicle usage style recognition module, K-means clustering is used to cluster and analyze vehicle usage behavior data to obtain the number of driver vehicle usage style types. The vehicle usage style recognition method adopts BP neural network and support vector machine. The vehicle usage styles include anxious, normal and calm types.

5. The adaptive energy management system for hybrid electric vehicles according to claim 1, characterized in that, In the driving style recognition module, principal component analysis is first used to reduce the dimensionality of the feature parameters to remove redundant information between the original variables; then, K-means clustering is used to cluster the driving styles; finally, BP neural network and support vector machine methods are used to recognize the driving styles; the driving styles include conservative, normal and aggressive.

6. The adaptive energy management system for hybrid electric vehicles according to claim 1, characterized in that: The energy management controller adopts a hierarchical architecture, with the upper layer being the battery SOC trajectory planning module and the lower layer being the deep reinforcement learning module, which uses deep reinforcement learning algorithms to solve the hybrid transmission gear positions and the torque distribution between the motor and the engine.

7. The adaptive energy management system for a hybrid electric vehicle according to claim 6, characterized in that: The upper-layer battery SOC trajectory planning module adopts a mileage-related trajectory planning method, which is as follows: （1）； In the formula, soc ref The reference SOC obtained by the battery SOC trajectory planning module; λ This refers to the battery SOC trajectory factor. soc ini and soc fin These are the initial SOC and final SOC of the battery, respectively. L rem Represents the remaining mileage; L tot This represents the total mileage traveled.

8. The adaptive energy management system for a hybrid electric vehicle according to claim 6, characterized in that: The state space of the deep reinforcement learning algorithm used in the lower-level deep reinforcement learning module. S for: （2）； In the formula: v The vehicle speed; acc It is longitudinal acceleration; T req This is the torque required by the entire vehicle; soc This indicates the current state of battery charge. For hybrid electric vehicles with discrete gearboxes, a deep reinforcement learning algorithm based on PDQN is used to solve the hybrid gearbox gears and the torque distribution between the motor and the engine. At this point, the deep reinforcement learning algorithm outputs the action space. A for: （3）； In the formula: i g and i e These are the gear positions for the electric motor and the engine, respectively. T e This refers to engine torque; For hybrid electric vehicles that only have continuous torque output from the engine and motor and no discrete gearbox, a deep reinforcement learning algorithm based on DDPG is used to solve the torque distribution between the engine and motor. At this point, the deep reinforcement learning algorithm outputs the action space. A for: （4）； In the formula: n e Engine speed; T M This is the torque of the drive motor.

9. The adaptive energy management system for a hybrid electric vehicle according to claim 8, characterized in that: The reward function for the deep reinforcement learning is: （5）； In the formula: fuel For engine fuel consumption; α and β are the engine fuel consumption weighting coefficient and the battery SOC weighting coefficient, respectively. t Represents time; soc This represents the battery's state of charge.

10. The management method of the adaptive energy management system for a hybrid electric vehicle according to claim 1, characterized in that: Includes the following steps: 1) Vehicle-side sensors collect driver vehicle behavior data and driver driving behavior data in real time, and transmit the collected driver vehicle behavior data and driver driving behavior data to the driving behavior data collection and preprocessing module of the cloud control platform through the gateway and vehicle communication unit; 2) The driving behavior data acquisition and preprocessing module preprocesses the acquired driver vehicle use behavior data and driver driving behavior data, and transmits the preprocessed corresponding data to the vehicle use style recognition module and the driving style recognition module respectively. 3) The vehicle usage style recognition module determines the driver's vehicle usage style based on the preprocessed driver vehicle usage behavior data, and the driving style recognition module determines the driver's driving style based on the preprocessed driver driving behavior data. 4) The adaptive adjustment module of control parameters adaptively adjusts the final SOC and trajectory factors and deep reinforcement learning network parameters in the battery state of charge trajectory planning strategy based on the different driving styles and driving styles of different drivers obtained by the vehicle usage style recognition module and the driving style recognition module, and generates a control strategy upgrade package. 5) The vehicle communication unit remotely pushes the control strategy upgrade package to the vehicle and downloads it to the flash memory of the central domain controller; 6) The central domain controller prompts the user whether to update through the human-machine interface. After receiving the user's confirmation of consent to update, the central domain controller performs a self-test on the vehicle energy management controller to determine whether the current operating environment is suitable for updating. After the self-test is completed, the energy management domain controller enters boot mode and waits for the program to be flashed. 7) After the program is successfully flashed, the energy management controller parameters are updated, and the updated driver driving behavior data and driver vehicle use behavior data are fed back to the cloud control platform.

Images