Hybrid electric vehicle energy management method considering driving condition change and system thereof

Abstract

The invention discloses a hybrid electric vehicle energy management method considering driving condition change and a system thereof, which can avoid damage to the health state of a battery caused by excessive charging and discharging or violent fluctuation of electric quantity, and can effectively improve the economical efficiency of a hybrid electric vehicle. The method comprises the steps of inputting vehicle speed information into a trained driving condition recognition model, and outputting a driving condition recognition result which is an urban driving condition or a suburban driving condition; the driving condition recognition model is established based on a sequence network, parameters are initialized by adopting glorot normal distribution, an Adam optimization algorithm is used as an optimizer, and a cross entropy loss function is used for model training to obtain the driving condition recognition model; the sequence network comprises a recurrent neural network (RNN), a long short-term memory (LSTM) network or a neural network based on an attention mechanism Transform; and selecting an energy optimization controller according to the driving condition recognition result, inputting the driving condition recognition result into the selected energy optimization controller, and outputting an instantaneous energy optimization control result.

Description

Technical Field

[0001] The embodiments of the present application relate to the technical field of hybrid vehicle energy management, and in particular, to a hybrid vehicle energy management method and system that takes into account changes in driving conditions. Background Art

[0002] As a solution for the transition from traditional fuel vehicles to pure electric vehicles, hybrid vehicles mostly use internal combustion engines to support long-distance driving, while using battery packs to improve fuel economy. The energy management strategy distributes the power output of multiple power sources during vehicle operation, which plays a key role in the economy and stability of the vehicle.

[0003] With the rapid development of various sensor technologies, adaptive energy management strategies can further improve vehicle economy by analyzing external vehicle data to adjust the underlying strategy. In recent years, adaptive energy management strategies have attracted much attention. However, problems such as the accuracy of variable parameter recognition and the performance of the underlying control strategy have greatly limited the performance of the vehicle. How to combine road condition information with energy management strategies to maximize energy utilization has become a major difficulty.

[0004] Vehicle driving condition recognition often uses fuzzy rule-based or clustering methods to extract multidimensional features from data as input, and the recognition accuracy is not high. The underlying control strategy development methods can be divided into three categories: rule-based, optimization-based, and learning-based. Among them, learning-based strategies have gradually emerged in literature research. However, in the development of adaptive energy management strategies, rule-based or optimization-based methods are still used to build underlying strategies. In summary, building a road recognition model based on a sequence network architecture and developing underlying strategies based on deep reinforcement learning can better combine road condition information with energy management strategies, thereby avoiding damage to the battery health status caused by excessive charging and discharging or drastic fluctuations in power, and effectively improving the economy of hybrid vehicles. Summary of the invention

[0005] In order to overcome the above technical defects, the present application provides a deep learning hybrid electric vehicle energy management method and system taking into account driving condition changes, and its technical solutions are mainly as follows:

[0006] S1. Inputting vehicle speed information into a trained driving condition recognition model and outputting a driving condition recognition result, where the driving condition recognition result is an urban driving condition or a suburban driving condition;

[0007] The driving condition recognition model is built based on a sequence network, and the parameters are initialized using the glorot normal distribution. The Adam optimization algorithm is used as the optimizer, and the cross entropy loss function is used for model training. The sequence network includes a recurrent neural network RNN, a long short-term memory network LSTM, or a neural network based on the attention mechanism Transformer.

[0008] S2. Selecting an energy optimization controller according to the driving condition recognition result, inputting the driving condition recognition result into the selected energy optimization controller, and outputting an instantaneous energy optimization control result;

[0009] The energy optimization controller achieves instantaneous energy optimization control through the energy management strategy of the hybrid vehicle under different driving conditions. The energy management strategy is obtained by training the driving cycle data using a deep reinforcement learning algorithm to adjust the weights of the multi-objective reward function. The multi-objective reward function is established based on the fuel consumption and battery SOC changes.

[0010] In addition, a corresponding hybrid electric vehicle energy management system is also provided in an embodiment of the present application, which is used to execute the above-mentioned deep learning hybrid electric vehicle energy management method that takes into account changes in driving conditions.

[0011] The above technical solution can achieve the following technical effects in the embodiments of the present application:

[0012] The accuracy of driving condition recognition is improved by extracting long-term dependency information from vehicle speed information through a sequence network. The underlying control strategy suitable for different driving conditions is obtained by adjusting the reward function weights in deep reinforcement learning and training driving cycle data. This application combines changes in driving conditions with energy management strategies, which can reduce the unreasonable energy distribution caused by switching driving conditions, further optimize the energy planning effect of hybrid vehicles, and improve vehicle fuel economy and battery health. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The accompanying drawings exemplarily illustrate the embodiments and constitute a part of the specification, and together with the text description of the specification, are used to explain the exemplary implementation of the embodiments. The embodiments shown are for illustrative purposes only and do not limit the scope of the claims. In all drawings, the same reference numerals refer to similar but not necessarily identical elements.

[0014] Figure 1 A flow chart of a hybrid vehicle energy management method considering driving condition changes in this application;

[0015] Figure 2 It is an overall framework diagram of the hybrid vehicle energy management method considering driving condition changes in this application;

[0016] Figure 3 It is a comparison chart of the results of holdout tests on different standard driving cycles between the driving condition recognition model built based on LSTM provided by the present application and the recognition model developed by other developers;

[0017] Figure 4 It is a comparison chart of control results of multiple reinforcement learning agents trained under different driving conditions using different multi-objective reward function weights provided by the present application;

[0018] Figure 5 It is a comparison chart of the control results of the controller Agent-S for suburban conditions and the controller Agent-U for urban conditions under different standard driving cycles obtained by training provided by the present application;

[0019] Figure 6 This is a comparison chart of the control results of the driving condition adaptive deep reinforcement learning energy control strategy provided by this application, the traditional deep learning algorithm and the rule-based control strategy in a dynamic and complex environment obtained by combining multiple standard driving cycles. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.

[0021] It should be noted that the descriptions involving "first", "second", etc. in the embodiments of the present application are only for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in the field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by this application.

[0022] In the description of the present application, it should be understood that the numerical labels before the steps do not indicate the order in which the steps are executed, but are only used to facilitate the description of the present application and to distinguish each step, and therefore should not be understood as a limitation on the present application.

[0023] First, the following terms are explained:

[0024] Glorot normal distribution: Glorot normal distribution, also known as Xavier initialization, is a weight initialization method that is commonly used in the training of deep neural networks. This method helps the network train more efficiently by initializing the network weights with a normal distribution and avoiding the problem of gradient disappearance or explosion. In neural networks, Glorot initialization selects a weight distribution with a mean of 0 and a standard deviation of The values ​​are drawn from a normal distribution, where n in and n out are the number of input nodes and output nodes of the current layer respectively.

[0025] Adam optimization algorithm: Adam (Adaptive Moment Estimation) is a widely used gradient optimization algorithm that combines the advantages of Momentum and RMSprop to accelerate the training process of deep learning models. The Adam algorithm adjusts the learning rate of each parameter by calculating the first-order moment (the mean of the gradient) and the second-order moment (the mean of the square of the gradient). It adjusts the learning rate adaptively and can effectively handle sparse gradients, noisy data, and non-steady-state objective functions.

[0026] DQN algorithm: DQN (Deep Q-Network) is a method in deep reinforcement learning. It solves the computational difficulties of traditional Q learning algorithms in high-dimensional state space by using deep neural networks to approximate the Q value function. DQN estimates the action value (Q value) by training a neural network, that is, the expected return of taking an action in a specific state. DQN combines technologies such as Experience Replay and Target Network, enabling the algorithm to converge stably and optimize strategies in complex environments.

[0027] DDPG algorithm: DDPG (Deep Deterministic Policy Gradient) is a deep reinforcement learning algorithm used to solve decision-making problems in continuous action space. It combines deep learning with the policy gradient method and uses a deterministic strategy instead of a random strategy. DDPG is an algorithm based on the Actor-Critic framework, where the Actor is responsible for generating action strategies and the Critic is responsible for evaluating the value of the current strategy. It further stabilizes the learning process by using a target network and experience replay mechanism.

[0028] TD3 algorithm: TD3 (Twin Delayed Deep Deterministic Policy Gradient) is an improved version of the DDPG algorithm, which aims to solve the problem of over-estimation of Q values ​​in the DDPG training process. Based on DDPG, 1) two Q networks are used to reduce the over-estimation of Q values; 2) the update frequency of the Critic network is reduced to reduce the variance in the training process; 3) noise is added to the target Q value to increase the robustness of the training.

[0029] HWFET: HWFET is a standardized driving cycle used to test the fuel economy of hybrid vehicles on highways. The HWFET test is a fuel efficiency test method based on the US EPA (Environmental Protection Agency) standard, simulating the situation of a vehicle driving at a certain speed on a highway. This test is used to evaluate the vehicle's fuel consumption on highways and the performance of the hybrid system.

[0030] UDDS: UDDS is another standardized driving cycle specifically designed to test the fuel economy of a vehicle in an urban environment, simulating driving conditions on city streets. The UDDS test includes low-speed driving, frequent acceleration and braking, reflecting common conditions in city driving. This test is used to evaluate the fuel consumption and emissions of a vehicle in urban traffic. It is often used in conjunction with HWFET to provide comprehensive performance data on hybrid vehicles in different driving environments.

[0031] Secondly, in order to facilitate those skilled in the art to understand the technical solutions provided in the embodiments of the present application, the relevant technologies are described below:

[0032] Embodiment 1

[0033] See also Figure 1 and Figure 2 In the embodiment of the present application, a deep learning hybrid vehicle energy management method considering driving condition changes includes:

[0034] S1. Input vehicle speed information into a trained driving condition recognition model, and output a driving condition recognition result, where the driving condition recognition result is an urban driving condition or a suburban driving condition.

[0035] The driving condition recognition model is built based on a sequence network, and the parameters are initialized using the glorot normal distribution, the Adam optimization algorithm is used as the optimizer, and the cross entropy loss function is used for model training. The sequence network can be a recurrent neural network RNN, a long short-term memory network LSTM, or a neural network based on the attention mechanism Transformer.

[0036] S2. Select an energy optimization control model according to the driving condition recognition result, input the driving condition recognition result into the selected energy optimization control model, and output the instantaneous energy optimization control result.

[0037] Among them, the energy optimization control model realizes instantaneous energy optimization control through the energy management strategy of hybrid vehicles under different driving conditions. The energy management strategy is obtained by training the driving cycle data using deep reinforcement learning to adjust the weights of the multi-objective reward function. The multi-objective reward function is established based on the fuel consumption and battery SOC changes.

[0038] The multi-objective reward function plays a vital role in the training process of the underlying energy management controller. It not only affects the learning process of the intelligent agent (i.e., the energy management controller), but also determines how the final control strategy balances multiple optimization objectives. In deep reinforcement learning, the reward function gives feedback based on the performance of the intelligent agent at each step in the environment. For the energy management problem of hybrid vehicles, there are usually multiple objectives that need to be optimized at the same time. For example: 1) Fuel consumption: It is hoped to minimize fuel consumption to improve the fuel economy of the vehicle. 2) SOC change: SOC is the state of charge of the battery. Excessive consumption of the battery or frequent charging will affect the battery life. Therefore, it is also an important goal to reasonably control the SOC change and avoid excessive charging and discharging of the battery.

[0039] A simple multi-objective reward function can be expressed as follows: R = ω fuel ·R fuel +ω SOC ·R SOC , where ω fuel and ω SOC is the weight of the objective, which is used to control the contribution of each objective to the overall reward function. fuel and R SOC are the reward items related to fuel consumption and SOC change respectively. During the training process, weight adjustment is a very important mechanism in the multi-objective reward function, which determines the relative importance of different objectives in the final strategy. The multi-objective reward function guides the deep reinforcement learning agent to make optimal decisions during the training process by combining feedback from multiple objectives (such as fuel consumption and SOC change). By designing a suitable reward function and adjusting the training data, it is possible to ensure that the energy management controller performs stably in complex and changing driving environments and optimize the vehicle's energy efficiency and battery management.

[0040] The above-mentioned deep reinforcement learning algorithm can be DQN, DDPG, TD3 and other algorithms.

[0041] In the technical solution of the present application, the driving condition recognition model constructed by the sequence network can extract the long-term dependent information in the vehicle speed and improve the accuracy of driving condition recognition. Since the energy optimization control model realizes instantaneous energy optimization control through the energy management strategy suitable for different driving conditions and the energy management strategy is obtained by training the driving cycle data using deep reinforcement learning to adjust the weights of the multi-objective reward function, the changes in driving conditions are combined with the energy management strategy to output the instantaneous energy optimization control results in real time, which can reduce the unreasonable energy distribution caused by the switching of driving conditions, further optimize the energy planning effect of hybrid vehicles, improve vehicle fuel economy, and extend battery life.

[0042] Embodiment 2

[0043] The driving condition recognition model described above requires model training in advance. The following will take the long short-term memory network LSTM as an example to illustrate the technical solution for building and training the driving condition recognition model.

[0044] Specifically, the model input of the driving condition recognition model is a fixed-length speed data sequence, and its model output is the probability that the current driving condition belongs to a certain category, such as the probability of suburban or urban driving conditions.

[0045] The construction of the driving condition recognition model based on the long short-term memory network LSTM specifically includes the following steps S1.1-S1.3, as follows:

[0046] S1.1. Data processing: Divide the standard driving cycle into data segments according to the vehicle start and stop points.

[0047] The number of standard driving cycles is at least two, and the data segment is of a fixed length.

[0048] Specifically, urban and suburban driving conditions are used as identification targets, and the start and stop points of the selected cycle are divided into multiple data segments, and data cleaning is performed according to their changing characteristics. "0" and "1" are used to represent suburban and urban driving conditions respectively. The segmented driving cycle data x can be expressed as: x = [v 1+nt , v 2+nt , ..., v m+nt ], where m is the segment length and t is the segment interval. n is a non-negative integer and (m+nt) is less than or equal to the total length of the driving cycle.

[0049] Specifically, in a specific implementation, the number of standard driving cycles may be 12, i.e., 12 standard driving cycles; the data segments are divided as follows: a time window of a fixed length is used to segment the cleaned driving cycle, and the segment size is generally 60s to 120s.

[0050] The data comes from 12 standard driving cycles. Since the vehicle speed level is also an important basis for judging driving conditions, and the vehicle speed is often within a certain range, it can be normalized without processing. The standard driving cycle often contains more than just the simulation of a single driving condition. Therefore, after first classifying it as a whole, the driving cycle is divided into smaller driving segments based on the vehicle start and stop points, and the data segments obtained after the division are further cleaned. Then, a fixed-length time window of 90s is used and a step size of 5s is used to segment the cleaned driving cycle. The obtained data is sampled without replacement to obtain training data.

[0051] S1.2. Model framework construction: Build the model framework of the driving condition recognition model based on the long short-term memory network LSTM.

[0052] The constructed model framework contains multiple memory units, each of which uses the following three gating mechanisms to process information in the sequence:

[0053] Where f, i, o represent the opening of the forget gate, input gate, and output gate respectively, and W and b are trainable matrix parameters and biases.

[0054] The expression of the implicit vector h is as follows: t =σ(W o ·[h t-1 , x t ]+b o )·tanh(C t ), where h is the latent vector and C is the cell state containing long-term information.

[0055] Because it introduces the implicit vector h and the cell state C, LSTM can well capture the long-term dependency information contained in the sequence data, thereby achieving better results in problems that require processing longer sequence data.

[0056] Further, the driving condition recognition model is constructed as follows: First, an input layer is used to receive the vehicle speed data, and the output is passed to an LSTM layer, which is activated by the ReLU function to obtain the complete output sequence, and the overfitting is reduced by the dropout layer. The output sequence is then fed back to another LSTM layer, which is also activated by the ReLU function and another dropout layer to further reduce overfitting. This dropout layer only returns the output of the last time step. The result is then passed through a fully connected layer and activated with the sigmoid function, and the final model output is This is the predicted probability. If the value is close to 1, it means that the driving condition is more likely to belong to a certain category, otherwise it means it belongs to another category.

[0057] S1.3. Model training: During the training process, the glorot normal distribution is used to initialize the parameters, the Adam optimization algorithm is used as the optimizer, and the cross entropy loss function is used for training.

[0058] It is understandable that using glorot normal distribution to initialize parameters can enhance model convergence, avoid gradient vanishing or exploding, and help accelerate the training process and improve stability. Using the Adam optimization algorithm as an optimizer usually performs better than the traditional gradient descent method, and can adaptively adjust the learning rate of each parameter to speed up convergence, which is especially suitable for dealing with large-scale data sets and complex problems with high dimensions. Using the cross entropy loss function to calculate the loss can effectively measure the gap between the model output and the true label. .

[0059] During the model training process, set the maximum number of training iterations, for example 250, and use the following loss function to calculate the loss value:

[0060] Among them, N is the number of samples, y is the true label value of the sample, and y i is the true label value of the sample in each iteration, The predicted value output by the model in each iteration.

[0061] For example, the driving condition recognition model constructed and trained based on the long short-term memory network LSTM in this application is compared with the current mainstream driving condition recognition model developed based on fuzzy rules, clustering and supervised learning. The recognition accuracy of various methods is shown in Table 1.

[0062] Table 1

[0063]

[0064] It can be seen from Table 1 that the recognition accuracy of the driving condition recognition model constructed and trained based on the long short-term memory network LSTM in this application is higher than that of the recognition models constructed in the three ways.

[0065] Furthermore, the above four models are used for holdout tests on WVUSUB, WVUINTER, and WVUCITY, and the test results are as follows: Figure 3 As shown, it should be noted that Figure 3 The first row is the speed curves of the three working conditions of WVUSUB, WVUINTER and WVUCITY. The following are the recognition results of fuzzy rules, clustering, supervised learning and LSTM model. Figure 3 The leftmost part of the image is marked. The WVUSUB recognition result is taken as an example for analysis. This driving cycle includes high-speed driving that lasts shorter than traditional suburban driving conditions, and also covers a low-speed range with drastic changes. It belongs to a comprehensive working condition with characteristics of urban and suburban driving conditions. From the recognition results, the LSTM-based model recognizes the high-speed driving segments of approximately 150 seconds to 200 seconds and 800 seconds to 1450 seconds as suburban driving conditions, and recognizes the low-speed fluctuation segments of approximately 250 seconds to 800 seconds and 1450 seconds to 1650 seconds as urban driving conditions, showing high accuracy. At the transition points between the high-speed and low-speed driving segments (approximately 250 seconds and 800 seconds), the model is able to recognize the transition of driving conditions in a timely manner. In the short-term speed fluctuation segment (approximately 400 seconds to 600 seconds) that maintains the characteristics of suburban driving, the model maintains the stability of recognition by considering the long-term dependencies in the speed feature sequence, thereby avoiding the switching interference of recognition results caused by short-term speed changes. For the recognition models based on fuzzy rules and clustering, both of them recognized most of the operating conditions as urban driving conditions, which is inconsistent with the actual state. For the model based on supervised learning, the recognition results showed drastic fluctuations in multiple intervals, such as between 150 seconds and 250 seconds and between 1200 seconds and 1600 seconds, showing low stability. The same conclusion can be drawn from the results of WVINTER and WVUCITY. In general, from Figure 3 It can be seen that the LSTM-based recognition model has higher recognition accuracy and better stability.

[0066] Embodiment 3

[0067] The energy optimization control model described above is a mathematical modeling of the vehicle energy management problem to obtain a deep reinforcement learning optimization controller under different driving conditions. The following uses the DDPG algorithm as an example to illustrate the modeling scheme of the energy optimization control model.

[0068] Specifically, by setting a multi-objective reward function, the balance between multiple optimization objectives is adjusted based on the characteristics of different driving conditions, and finally the multi-objective reward function parameters and the driving cycle data used in the training process are modified to obtain multiple deep reinforcement learning energy optimization controllers.

[0069] The construction of the energy optimization control model includes the following steps S2.1-S2.3, as follows:

[0070] S2.1. Modeling of hybrid vehicle simulation model: Modeling of vehicle longitudinal dynamics, engine, motor and power battery.

[0071] Build a hybrid vehicle simulation model: The hybrid vehicle simulation model can belong to a variety of architectures. Taking the input power split type as an example, its powertrain architecture includes an engine and two motors that act as motors and generators respectively. This application does not consider vehicle steering and road slope, and the longitudinal dynamics of the whole vehicle can be modeled as:

[0072]

[0073] Among them C D is the air resistance coefficient, A f is the cross-sectional area of ​​the vehicle, v is the vehicle speed, m is the vehicle mass, g represents the acceleration due to gravity, and f represents the coefficient of rolling resistance.

[0074] Build a model of the key power components of the vehicle: Ignoring the specific implementation details and the impact of temperature, the engine is modeled as a function of efficiency r with respect to torque T and speed ω:

[0075] r f =f e (ω eng T eng ).

[0076] The power P and efficiency η of the two motors are calculated and modeled as follows:

[0077]

[0078] The battery is modeled based on equivalent circuit theory as:

[0079]

[0080] Where P bat is the battery output power, V represents the open-loop voltage, I represents the current, R is the battery internal resistance, SOC t is the remaining power at time t, and C bat is the total battery capacity, and the required power P during vehicle operation is a function of t:

[0081]

[0082] In summary, during vehicle operation, the functions of fuel consumption and SOC change with respect to t are as follows:

[0083]

[0084] Improving vehicle economy requires reasonable power distribution to comprehensively improve the output efficiency of the engine and battery.

[0085] S2.2. Based on the DDPG algorithm, the vehicle energy management problem is modeled to obtain the corresponding state vector, action vector and real-time reward function.

[0086] Specifically, using the DDPG algorithm as an example, it uses two deep networks to simulate the action function and the evaluation function respectively, which can better handle complex control problems in the continuous action space. In DDPG, the action network μ and the evaluation network Q are updated at each time step t as follows:

[0087]

[0088] Where J represents the optimization objective function, N is the number of samples, and θ μ and θ Q are the parameters of the action network and the evaluation network respectively.

[0089] Specifically, the DDPG algorithm is divided into three parts: state space, action space, and reward function. In this application, the state space S = {v, acc, SOC} includes the current speed v, the current acceleration acc, and the remaining SOC, where the range of SOC is [0,1]. The engine output power is selected as the action space A = {P eng}, where P eng The output range is defined between 0kW and 56kW. The fuel consumption and battery SOC changes are used to establish the reward function. In summary, the state vector s, action vector a, and real-time reward function r are set as follows:

[0090]

[0091] Where α and β are constant factors for the change in reward value caused by fuel consumption and battery power changes, respectively. Take the state vector s as the input of the action network, and add gradually weakening noise to its output action a to enhance the strategy exploration ability. Execute the action and record the next state s` and the immediate reward value r returned by the environment. Store (s, a, s`, r) as a sample data in the experience replay buffer pool. When there is enough data in the buffer pool, randomly sample a batch size of data from it and calculate the target Q value:

[0092] y t =r t +γQ'(s t+1 ,μ′(s t+1 |θ μ′ )|θ Q );

[0093] Where y represents the target Q value, which is usually calculated as the current reward plus the discounted value of the next state Q value. Update the evaluation network by minimizing the error between the Q value and the target Q value:

[0094] L(θ Q )=E[(Q(s,a|θ Q)-y) 2 ];

[0095] Use the gradient ascent algorithm to optimize the action network, and finally update the parameters of the target network through the soft update method:

[0096] where τ is the soft update parameter.

[0097] S2.3. Classify vehicle driving conditions according to the characteristics of vehicle speed and required power changes, and adjust the multi-objective reward function weight selection and training data in the deep reinforcement learning training process according to the characteristics of different categories.

[0098] In order to obtain controllers suitable for suburban driving conditions and urban driving conditions respectively, different driving cycles are used as training data, and the relative importance of different optimization objectives in the final strategy is adjusted by modifying the multi-objective reward function. Models (such as energy management controllers) are trained using deep reinforcement learning algorithms based on vehicle simulation models.

[0099] Specifically, S2.1 builds a vehicle simulation model, S2.2 is the specific training process and training settings (action space, state space, etc.) of a single controller, S2.3 adjusts the training data and reward function weights based on the set driving conditions, and uses deep reinforcement learning to obtain two controllers for suburban driving conditions and urban driving conditions respectively. During the training process, the vehicle model built in S2.1 is used as the external environment. The specific training process of a single controller is as shown in S2.2.

[0100] (1) Multi-objective reward function weight setting: By training multiple controllers with different reward function weights, these controllers are used in suburban and urban conditions respectively, and the control results are compared to select more appropriate multi-objective reward function weights for different driving conditions.

[0101] (2) Training driving cycle setting: Use driving cycle data that can represent the corresponding driving conditions as training data for the deep reinforcement learning energy optimization controller. The above driving conditions include suburban and urban driving conditions. Take HWFET and UDDS as examples. These two standard driving cycles are often used to simulate suburban conditions and urban conditions, respectively. For the instantaneous energy optimization controller used for suburban driving conditions, HWFET is set as the training driving cycle; similarly, for urban driving conditions, training is based on UDDS.

[0102] In this scheme, the controller is trained using different multi-objective reward function weights, and the obtained controller results are used for suburban and urban driving conditions respectively. The control results are shown in Figure 4As shown in (a) and (b), based on this, 1×200 are selected to train the suburban condition controller Agent-S and 1×150 are selected to train the urban condition controller Agent-U.

[0103] Figure 5 This is a comparison chart of the control results of the controller Agent-S for suburban conditions and the controller Agent-U for urban conditions under different standard driving cycles obtained by training provided in this application.

[0104] Combine multiple standard driving cycles as dynamic complex working conditions, and apply the hybrid vehicle energy management system that considers driving condition changes proposed in this application, the traditional deep enhancement energy management strategy that does not use driving condition recognition, and the rule-based strategy to this working condition respectively. The control results are as follows: Figure 6 And as shown in Table 2.

[0105] Table 2: Comparison of results of different control strategies under complex working conditions

[0106]

[0107] Combination Figure 6 As can be seen from Table 2, the present application can stabilize the battery charge and discharge state trajectory, avoiding damage to the battery health state caused by excessive charge and discharge or drastic fluctuations in power. At the same time, the highest MPGe (equivalent miles per gallon) value is achieved in multiple strategies, which can effectively improve fuel economy. Obviously, those skilled in the art should understand that the modules or steps of the above-mentioned embodiments of the present application can be implemented with general-purpose computer devices, which can be concentrated on a single computer device, or distributed on a network composed of multiple computer devices. Optionally, they can be implemented with executable program codes of computer devices, so that they can be stored in a storage device and executed by a computer device, and in some cases, the steps shown or described can be performed in a different order from that here, or they can be made into individual integrated circuit modules respectively, or multiple modules or steps therein can be made into a single integrated circuit module to implement. In this way, the embodiments of the present application are not limited to any specific combination of hardware and software.

[0108] It should be noted that the above are only preferred embodiments of the present application, and the patent protection scope of the present application is not limited thereto. Any equivalent structure or equivalent process transformation made using the contents of the specification and drawings of the present application, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A hybrid vehicle energy management method considering driving condition changes, characterized in that: include: S1. Inputting vehicle speed information into a trained driving condition recognition model and outputting a driving condition recognition result, where the driving condition recognition result is an urban driving condition or a suburban driving condition; The driving condition recognition model is built based on a sequence network, and the parameters are initialized using the glorot normal distribution. The Adam optimization algorithm is used as the optimizer, and the cross entropy loss function is used for model training. The sequence network includes a recurrent neural network RNN, a long short-term memory network LSTM, or a neural network based on the attention mechanism Transformer. S2. Selecting an energy optimization controller according to the driving condition recognition result, inputting the driving condition recognition result into the selected energy optimization controller, and outputting an instantaneous energy optimization control result; The energy optimization controller achieves instantaneous energy optimization control through the energy management strategy of the hybrid vehicle under different driving conditions. The energy management strategy is obtained by training the driving cycle data using a deep reinforcement learning algorithm to adjust the weights of the multi-objective reward function. The multi-objective reward function is established based on the fuel consumption and battery SOC changes.

2. The hybrid vehicle energy management method according to claim 1, characterized in that: When the sequence network is a long short-term memory network LSTM, the construction of the driving condition recognition model based on the long short-term memory network LSTM includes: S1.1, data processing: Divide the standard driving cycle into data segments according to the vehicle start and stop points; S1.

2. Model framework construction: Build the model framework of the driving condition recognition model based on the long short-term memory network LSTM; The constructed model framework includes at least two memory units, each of which uses the gating mechanism shown in the following formula: Where f, i, o represent the opening of the forget gate, input gate, and output gate respectively, W and b are trainable matrix parameters and biases, and h is the implicit vector, which is expressed as: h t =σ(W o ·[h t-1 ,x t ]+b o )·tanh(C t ), C is the cell state containing long-term information; S1.

3. Model training: During the training process, the glorot normal distribution is used to initialize the parameters, the Adam optimization algorithm is used as the optimizer, and the cross entropy loss function is used for training.

3. The hybrid vehicle energy management method according to claim 2, characterized in that: There are 12 standard driving cycles. The data segments are divided by using a time window of fixed length to segment the labeled driving cycle. The segment size is generally 60s to 120s.

4. The hybrid vehicle energy management method according to claim 1, characterized in that: When the deep reinforcement learning algorithm is the DDPG algorithm, the construction of the energy optimization control model includes: S2.

1. Modeling of hybrid vehicle simulation model: Modeling of vehicle longitudinal dynamics, engine, motor, and power battery; S2.2, based on the DDPG algorithm, the vehicle energy management problem is modeled to obtain the corresponding state vector, action vector and real-time reward function; S2.

3. Classify vehicle driving conditions according to the characteristics of vehicle speed and required power changes, and adjust the multi-objective reward function weight selection and training data in the deep reinforcement learning training process according to the characteristics of different categories.

5. The hybrid vehicle energy management method according to claim 4, characterized in that: Adjusting the selection of multi-objective reward function weights specifically includes: training multiple controllers using different reward function weights, using these controllers in suburban conditions and urban conditions, and comparing the control results to select more appropriate multi-objective reward function weights for different driving conditions; Adjusting the training data specifically includes: using driving cycle data that can represent corresponding driving conditions as training data for deep reinforcement learning.

6. The hybrid vehicle energy management method according to claim 4, characterized in that: The longitudinal dynamics of the vehicle can be modeled as: The engine is modeled as a function of efficiency r with respect to torque T and speed ω: r f =f e (ω eng ,,T eng ), The power P and efficiency η of the two motors are calculated and modeled as follows: The battery is modeled based on equivalent circuit theory as: Set the state vector s, action vector a, and real-time reward function r as follows:

7. The hybrid vehicle energy management method according to claim 2, characterized in that: During the model training process, the maximum number of training times is set for iterative calculation, and the loss value is calculated using the following loss function:

8. A hybrid vehicle energy management system, characterized in that: include: Driving condition recognition layer and instantaneous energy optimization control layer; The driving condition recognition layer is used to: input vehicle speed information into a trained driving condition recognition model, and output a driving condition recognition result, where the driving condition recognition result is an urban driving condition or a suburban driving condition; The driving condition recognition model is built based on a sequence network, and the parameters are initialized using the glorot normal distribution. The Adam optimization algorithm is used as the optimizer, and the cross entropy loss function is used for model training. The sequence network includes a recurrent neural network RNN, a long short-term memory network LSTM, or a neural network based on the attention mechanism Transformer. The instantaneous energy optimization control layer is used to: select an energy optimization controller according to the driving condition recognition result, input the driving condition recognition result into the selected energy optimization controller, and output the instantaneous energy optimization control result; The energy optimization controller achieves instantaneous energy optimization control through the energy management strategy of the hybrid vehicle under different driving conditions. The energy management strategy is obtained by training the driving cycle data using a deep reinforcement learning algorithm to adjust the weights of the multi-objective reward function. The multi-objective reward function is established based on the fuel consumption and battery SOC changes.

9. The hybrid vehicle energy management system according to claim 8, further used to execute the hybrid vehicle energy management method as described in any one of claims 2 to 7 above.

Images