Adaptive multi-working-condition recursive deep reinforcement learning hybrid electric vehicle energy management strategy

Abstract

This invention discloses an adaptive multi-condition recursive deep reinforcement learning energy management strategy for hybrid vehicles, comprising: generating random operating conditions as a training set; acquiring the state variables, action variables, and reward function of the hybrid vehicle, and establishing a reward function based on the reward function; creating an RDQN algorithm and training it based on the training set, wherein the RDQN network includes an evaluation network and a target network, the evaluation network and the target network having the same structure; and using the trained RDQN algorithm for energy management, controlling the vehicle by maximizing the corresponding reward, state, and action according to the reward function. This invention introduces LSTM units into existing deep reinforcement learning networks, designs the RDQN algorithm, and trains it on a simulated multi-condition database, fully utilizing the temporal information contained in the operating conditions, thereby improving fuel economy and stabilizing the battery state during hybrid vehicle operation.

Description

Technical Field

[0001] This invention relates to the field of energy management technology for new energy vehicles such as hybrid electric vehicles and hydrogen fuel cell vehicles, and in particular to an adaptive multi-condition recursive deep reinforcement learning energy management strategy for hybrid electric vehicles. Background Technology

[0002] Energy management strategies (EMS) for hybrid electric vehicles (HEVs) can be categorized into three main types: rule-based, optimization-based, and learning-based methods. While rule-based methods are simple, they lack flexibility and optimality under varying operating conditions. Optimization-based methods can yield optimal solutions but require significant computational resources and are typically used as offline benchmarks to explore potential fuel economy. In contrast, learning-based methods, particularly deep reinforcement learning (DRL) algorithms, have shown potential in addressing the energy management strategy problem for HEVs due to their greater adaptability and learning capabilities under complex driving cycles, as well as lower computational resource consumption.

[0003] Liu et al. proposed an adaptive energy management method for hybrid electric tracked vehicles based on Q-learning. Results show that, compared with stochastic dynamic programming, this method has stronger adaptability, optimality, and learning ability, and can effectively reduce computation time. However, maintaining the Q-table is very difficult when the action and state spaces are huge.

[0004] Wu J, He H, Peng J, and others employed the Deep Q-Learning (DQL) algorithm for energy management, which not only solved the dimensionality curse encountered in Q-learning but also achieved better fuel economy than Q-learning. However, the discrete state limitation of Q-learning restricts its application in the energy management of hybrid electric vehicles.

[0005] Compared to DQL, Wu Y and Tan H et al. further explored the deep deterministic policy gradient algorithm (DDPG) with continuous state and action representations, and simultaneously used a priority experience replay algorithm to improve the learning efficiency of EMS, thus achieving performance almost identical to DP. However, its training process involves a trial-and-error approach, requiring a large number of training segments to achieve superior performance.

[0006] Lian R, Peng J, and others proposed a Rule Insertion Deep Reinforcement Learning (RIDRL) ​​energy management system based on a comprehensive analysis of expertise in hybrid electric vehicles. Their research improves the learning efficiency of intelligent agents, reduces the search space, and demonstrates that embedding expert knowledge of HEVs into HEV models can improve learning efficiency and fuel economy.

[0007] Despite the many advantages of DRL, most existing DRL methods are developed and evaluated under standard driving cycles. However, real-world conditions are often variable and differ from the standard conditions under which DRL models are trained. This difference poses a significant challenge to the adaptability and generality of DRL in adapting to new situations. Specifically, when models are deployed in real-world scenarios, the lack of robustness of DRL models can lead to poor performance or even catastrophic failures. Therefore, there is an urgent need to develop new DRL methods that are better suited to the challenges posed by diverse driving conditions. Summary of the Invention

[0008] The purpose of this invention is to overcome the lack of adaptability of energy management strategies under different operating conditions in the prior art, and to provide an adaptive multi-operating-condition recursive deep reinforcement learning energy management strategy for hybrid vehicles.

[0009] To achieve the above-mentioned objectives, the present invention provides the following technical solution:

[0010] An adaptive multi-condition recursive deep reinforcement learning-based energy management strategy for hybrid vehicles includes:

[0011] S1. Generate random operating conditions as the training set;

[0012] S2. Establish the state variables, action variables, and reward function required for the algorithm network. Obtain the state S based on the state variables, obtain the action A based on the action variables, and obtain the reward R based on the reward function. At the same time, establish the reward function based on the reward function.

[0013] S3. Create an RDQN algorithm network, which includes an evaluation network, a target network, and LSTM units. The evaluation network and the target network have the same structure.

[0014] S4. Initialize the parameters of each network;

[0015] S5. Fill the memory pool with random data;

[0016] S6. Under one of the random operating conditions, read in state S;

[0017] S7. Initialize the hidden state and create a temporary memory pool;

[0018] S8. Select a random action A with probability ε, or select the action A corresponding to the maximum Q value of the evaluation network;

[0019] S9. Execute action A and obtain the returned state S′ and reward R;

[0020] S10. Use the LSTM unit to obtain the next hidden state.

[0021] S11, the experience gained Stored in a temporary memory pool;

[0022] S12. Update the evaluation network parameters through backpropagation, update the target network parameters after a specified number of steps, and update the state S at the same time;

[0023] S13. Repeat steps S6-S12 until the set number of iterations is reached, then stop training the RDQN algorithm network.

[0024] S14. Use the trained RDQN algorithm for energy management, and control the system based on maximizing the corresponding reward R, state S, and action A according to the reward function.

[0025] Preferably, when the RDQN algorithm is created, hyperparameters and ε need to be set. The hyperparameters include learning rate, discount factor, learning batch size, specified number of steps for updating the target network, maximum number of iterations, number of neurons in the fully connected layer, and memory pool size. The initial value of ε is 0, and the set value is incremented every step.

[0026] Preferably, the LSTM unit has 100 hidden units and a time step of 32.

[0027] Preferably, the experience Includes time step information.

[0028] Preferably, in the evaluation network and the target network, the Q-function of policy π is:

[0029] Q π (S,A)=E[R t |S t =S,A t =A],

[0030] Among them, S t Let S be the current state, and A be the current state. t Let A and R be the actions at the current moment. t R is the reward at the current moment.

[0031] Preferably, the evaluation network is represented by a θ-parameterized neural network, denoted as Q(S,A;θ), which is optimized by minimizing a loss function, wherein the loss function is:

[0032] L(θ)=E S,A,S′,R [( yDQ NQ(S,A;θ)) 2 ],

[0033] y DQN =R+γMax A′Q(S′,A′;θ′),

[0034] Where L(θ) is the loss function value, E S,A,S′,R Let y be the expected value of the reward after the current state and the action is performed. DQN γMax represents the Q-value of the target network. A′ Let θ be the discount factor for the next state, and Q(S′,A′;θ′) be the target network, where θ′ is the weight fixed for multiple iterations.

[0035] Preferably, the RQDN algorithm can add a hidden state H t-1 To model, thereby generating Q(S, H) t-1 ,A;θ).

[0036] Preferably, in step S2,

[0037] The state variables are selected from the vehicle's state of charge, power speed, and acceleration.

[0038] Action variables are selected from the changes in engine power of the vehicle model.

[0039] The reward function is established based on the vehicle model's fuel consumption and state of charge deviation.

[0040] Preferably, the reward function is:

[0041] G(t)=R t +γR t+1 +γ 2 R t+2 2 ,

[0042] Where γ∈[0,1) is the discount factor, R t R is the reward for the current moment. t+1 R t+2 This is for long-term rewards in the future.

[0043] Preferably, S1 is a random working condition generated using MATLAB's RDE tool, while setting constraints during the generation process. These constraints include the duration of the working condition, sampling interval, speed boundary, tolerance range, minimum travel distance for each type of road segment, maximum speed, number of stops, stop duration, RPA constraint line, and VA95 constraint line.

[0044] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0045] 1. This invention introduces LSTM units into existing deep reinforcement learning networks and designs the RDQN algorithm to perform energy management training under random operating conditions. At the same time, based on the designed reward function and corresponding reward, the hybrid vehicle obtains the optimal state and action in the simulated random operating conditions, thereby improving fuel economy and stabilizing the battery state during the hybrid vehicle's operation.

[0046] 2. By generating random driving conditions, the agent is provided with more realistic operating conditions in addition to standard operating conditions, enabling the agent to learn from multiple environments and improving the adaptability and versatility of energy management strategies to any real operating conditions to a certain extent. Attached Figure Description

[0047] Figure 1 This is a schematic diagram of the process of the present invention;

[0048] Figure 2 This is a flowchart of the RDE generation process of the present invention;

[0049] Figure 3 This is a structural diagram of the Prius power system of the present invention;

[0050] Figure 4 This is a flowchart illustrating the training process of the RDQN algorithm of this invention.

[0051] Figure 5 This is a diagram of the RDQN network model of the present invention;

[0052] Figure 6 This is a graph illustrating the fuel consumption test results of the present invention.

[0053] Figure 7 This is a distribution diagram of the engine operating points in this invention;

[0054] Figure 8 This is a battery state trajectory diagram in this invention. Detailed Implementation

[0055] The present invention will be further described in detail below with reference to experimental examples and specific embodiments. However, this should not be construed as limiting the scope of the above-mentioned subject matter of the present invention to the following embodiments; all technologies implemented based on the content of the present invention fall within the scope of the present invention.

[0056] Example 1

[0057] This invention provides an adaptive multi-condition recursive deep reinforcement learning energy management strategy for hybrid vehicles, such as... Figure 1 The diagram shown is a schematic representation of the process of this invention, specifically including:

[0058] S1. Generate random operating conditions as the training set;

[0059] S2. Establish the state variables, action variables, and reward function required for the algorithm network. Obtain the state S based on the state variables, obtain the action A based on the action variables, and obtain the reward R based on the reward function. At the same time, establish the reward function based on the reward function.

[0060] S3. Create an RDQN algorithm network, which includes an evaluation network, a target network, and LSTM units. The evaluation network and the target network have the same structure.

[0061] S4. Initialize the parameters of each network;

[0062] S5. Fill the memory pool with random data;

[0063] S6. Under one of the random operating conditions, read in state S;

[0064] S7. Initialize the hidden state and create a temporary memory pool;

[0065] S8. Select a random action A with probability ε, or select the action A corresponding to the maximum Q value of the evaluation network;

[0066] S9. Execute action A and obtain the returned state S′ and reward R;

[0067] S10. Use the LSTM unit to obtain the next hidden state.

[0068] S11, the experience gained Stored in a temporary memory pool;

[0069] S12. Update the evaluation network parameters through backpropagation, update the target network parameters after a specified number of steps, and update the state S at the same time;

[0070] S13. Repeat steps S6-S12 until the set number of iterations is reached, then stop training the RDQN algorithm network.

[0071] S14. Use the trained RDQN algorithm for energy management, and maximize the corresponding reward according to the reward function.

[0072] Example 2

[0073] An adaptive multi-condition recursive deep reinforcement learning-based energy management strategy for hybrid vehicles includes:

[0074] 1. As described in S1 above, the generated random driving scenarios serve as the training set, while constraints are established. The training dataset includes the standard driving scenario New European Cyclic Circular (NEDC) and a library of 400 random driving scenarios generated using MATLAB's RDE tool. When generating driving scenarios, the duration is set to 90-120 minutes, and the sampling interval is set to 1 second. Compared to the standard driving scenario, the longer duration is more realistic and provides more comprehensive training and testing results. Purely based on speed limits, 60 km / h and 90 km / h are used as speed boundaries to distinguish between urban, rural, and highway sections. All speeds below 60 km / h are considered urban, while speeds above 90 km / h are considered highway. To better reflect real-world conditions, each trip should mix all three types of road sections, with each operation evenly distributed within a 10% tolerance range. The minimum travel distance for each category should be no less than 16 km. Simultaneously, the maximum speed for a single trip is set to 160 km / h, requiring at least two stops in urban sections, each lasting at least 10 seconds. Constraint lines were set for relative positive acceleration (RPA) and the vehicle speed product per positive acceleration (VA95). For effectiveness, each urban, rural, and highway segment of the RDE trip must be below the VA95 constraint line and above the RPA constraint line. These parameters are defined in the RDE legislation. Since the parameter settings are mostly range-based, 100 random driving conditions were generated as the first test dataset under the same parameter settings. Another set of 100 random driving cycles was used as the second test dataset. Due to the change in parameter settings, the second test dataset differs significantly from the training dataset, such as… Figure 2 The diagram illustrates the process of generating a set of operating conditions.

[0075] 2. As described in S2 above, establish the state variables, action variables, and reward function required for the algorithm network, and establish the reward function based on the reward function. Specifically, in this embodiment, the Toyota Prius is selected as the experimental vehicle for this study. The Prius is an experimental platform widely used in EMS research because of its unique power splitting component called the planetary gear (PG), which enables power coupling between the engine, electric motor, and generator. The engine and generator are connected to the planetary carrier and sun gear of the PG, respectively, while the ring gear is connected to both the electric motor and the output shaft. After power is transmitted through the final drive, it is ultimately delivered to the wheels. The PG enables power coupling between the engine, electric motor, and generator, allowing for a seamless transition between electric and gasoline drive modes. In the research process of this invention, the Prius environment is used for experiments to verify the stronger adaptability of the RDQN method to different operating conditions. Figure 3 A structural diagram of the powertrain system for the Prius model is shown.

[0076] The formula for the power demand of a vehicle during operation is:

[0077]

[0078] Where m is the vehicle mass, g is the acceleration caused by gravity, f is the rolling resistance coefficient, θ is the road slope (not considered in this embodiment), and C D A is the air drag coefficient. F Let ρ be the front area of ​​the vehicle, ρ be the air density, v be the vehicle speed, and a be the vehicle acceleration.

[0079] The relationship between the speeds and torques of the sun gear, ring gear, and planetary carrier in a planetary system is as follows:

[0080]

[0081] Where α represents the characteristic parameter, n g n out and n e T represents the rotational speeds of the generator, gear ring, and engine, respectively. g T put and T e These represent their torques, respectively.

[0082] The equivalent circuit model of the battery, ignoring temperature changes and battery aging, is as follows:

[0083]

[0084] Among them, V oc R0 is the open-circuit voltage, P is the internal resistance, and P is the open-circuit voltage. batt The output power during the charge-discharge cycle is SoC, the battery state of charge is SoC, the battery initial capacity is Q0, and the battery nominal capacity is Q.

[0085] The state variables include the battery's state of charge (SoC), the Prius's velocity and acceleration, and the state S = {SOC, velocity, acceleration} is obtained.

[0086] The increase or decrease of engine power is set as a discrete action variable, including no change, increase or decrease of 1kW, increase or decrease of 2kW, increase or decrease of 4kW, increase or decrease of 6kW, increase or decrease of 8kW, increase or decrease of 10kW, and set to 0kW (engine off), for a total of 14 options. The action A = {0, ±1, ±2, ±4, ±6, ±8 ±10, set0} is obtained.

[0087] The reward function is:

[0088] R = -{α[fuel(t)] + β[SoC] ref -SoC(t)] 2},

[0089] The reward in the formula aims to encourage hybrid vehicles to maintain a stable state of charge (SoC) and fuel efficiency. The reward R is a function of two terms: the first penalizes fuel consumption, and the second penalizes deviations of the current SOC from a reference SOC. The weights of these two terms are determined by parameters α and β. By minimizing this reward function, hybrid vehicles are incentivized to strike a balance between efficient fuel use and battery-friendly operation, crucial considerations for hybrid vehicle operation.

[0090] More specifically, the reference value of SoC. ref The value of α is set to 0.6 to maintain battery charge sustainment based on minimum charge / discharge internal resistance. If α is set too high, the agent might prioritize reducing fuel consumption at all costs, which may not be ideal for maintaining battery state. On the other hand, if β is set too high, it might increase fuel consumption to charge the battery. Therefore, a balance needs to be struck between fuel economy and battery maintenance. In this case, the weight of α in the multi-objective reward function is set to 1, while β is set to 350, which has proven to yield good results for both fuel consumption and battery maintenance.

[0091] Based on the definitions of state, action, and reward, the goal is to maximize the expected value of the reward function G(t):

[0092] G(t)=R t +γR t+1 +γ 2 R t+2 2 ,

[0093] Where γ∈[0,1) is a discount factor, R t R is the reward for the current moment. t+1 R t+2 This is for long-term rewards in the future.

[0094] 3. As described in S3 above, create an RDQN network and train it based on the training set. Specifically, this invention utilizes a recursive DQN (RDQN) algorithm, a variant of DRL, which includes a recursive policy network. The entire algorithm framework includes the training set generation process, and differs slightly from existing DRQN ​​algorithms in terms of application domain and overall system structure. The Q-function for a selected random policy π is:

[0095] Q π (S,A)=E[R t |S t =S,A t =A],

[0096] Among them, S tLet S be the current state, and A be the current state. t Let A and R be the actions at the current moment. t R is the reward at the current moment.

[0097] Deep Q-Networks (DQNs) are represented using neural networks parameterized by θ, to evaluate the network Q(S,A;θ). DQNs are optimized by minimizing the following loss function:

[0098] L(θ)=E S,A,s′,R [(y DQN -Q(S,A;θ)) 2 ],

[0099] y DQN =R+γMax A′ Q(S′,A′;θ′),

[0100] Where L(θ) is the loss function value, E S,A,S′,R Let y be the expected value of the current state and the immediate reward after the action is performed. DQN Let rMax be the Q-value of the target network. A′ Let be the discount factor for the next state, R be the value of the reward function, and Q(S′,A′;θ′) be the target network, where θ′ is the weight fixed for multiple iterations.

[0101] Unlike the traditional DQN algorithm, RDQN preserves temporal information and long-term dependencies in the input sequence, which is crucial when dealing with random and unpredictable scenarios. Specifically, instead of approximating Q(S,A) with a feedforward network, RDQN approximates Q(S,A) with a recurrent neural network that maintains its internal state and aggregates observations over time. This is achieved by adding a hidden state H. t-1 To model, thereby generating Q(S, H) t-1 This makes RDQN particularly suitable for applications such as energy management in hybrid vehicles, where it is difficult to predict future driving conditions based solely on the current state, and historical information is crucial for energy management systems to better identify current driving conditions. Figure 4 The complete RDQN algorithm flowchart is shown.

[0102] Specifically, the RDQN algorithm flow is as described in S4-S13 above:

[0103] First, initialize the parameters of each network and fill the memory pool with random data. Under one random operating condition, read in state S, initialize the hidden states, and create a temporary memory pool. Select a random action A with probability ε, or select the action A corresponding to the maximum Q value of the evaluation network. Execute action A and obtain the returned state S′ and reward R. Use the LSTM unit to obtain the next hidden state. To acquire experience with time step information Store the data in a temporary memory pool; then update the evaluation network parameters through backpropagation, update the target network parameters after a specified number of steps, and update the state S simultaneously; repeat the above process until the set number of iterations is reached, and then stop training the RDQN algorithm network.

[0104] Combining the advantages of RNN and DQN, LSTM units are introduced into the RDQN algorithm, resulting in an RDQN model that can effectively handle sequential states and long-term dependencies. (H) t+1 =LSTM(H t Compared to traditional RNNs, LSTM offers greater memory capacity and better long-term dependency modeling capabilities, making it a suitable choice. The RDQN algorithm updates the recurrent network with continuous observation sequences and reward values ​​over a period of time, enabling the model to remember previous states and better utilize sequence information. For ease of training, a stochastic update learning method is employed, randomly selecting an episode (a random driving scenario generated by the RDE toolkit) from the experience replay memory and learning from it over several steps. Figure 5 This is the network model diagram of RDQN.

[0105] During the creation of RDQN, hyperparameters and ε need to be set. In this optional embodiment, multiple calibrations and screenings are used to select the hyperparameters that have the greatest impact on the algorithm network output, which can reduce the amount of subsequent data processing, improve efficiency, and ensure the accuracy of the algorithm network. The hyperparameters are set as follows: learning rate of 0.001, discount factor of 0.9, learning batch size of 64, target network updated every 300 steps, maximum iteration of 1500, and the number of neurons in the first two fully connected layers of the network is 200 and 100 respectively. The value of ε for the action policy is initially set to 0 and incremented by 0.00001 each step during training. In the early stages of training, the agent explores and then tries to utilize the knowledge it has learned. The number of hidden units in the LSTM layer is 100, and the time step is 32. Since the memory pool of RDQN stores information about the entire series, the memory pool capacity is set to 100 episodes, so that it can quickly utilize the knowledge gained from exploration and continuously update the memory pool.

[0106] 4. As described in S14 above, the trained RDQN algorithm is used for energy management, and control is performed based on maximizing the corresponding reward R, state S, and action A according to the reward function. The recurrent neural network obtained by this invention can effectively improve EMS performance under different driving conditions. Specifically, in this embodiment, the performance of two model algorithms, namely the traditional DQN algorithm and the RDQN algorithm proposed in this invention, was compared on NEDC and 400 randomly generated driving cycles. The obtained models were named DQN NEDC, RDQN NEDC, DQN Random, and RDQN Random. They were tested and verified on two test datasets respectively.

[0107] Fuel economy measures for different models are shown in [link / reference]. Figure 6 The RDQN Random model outperformed the other three models in both mean and variance for both Cost Engine and Fuel 100km. Specifically, compared to the DQN NEDC model, the average Fuel 100km was reduced by 0.793 and 0.79 L / 100km in the first and second tests, respectively. The results also indicate that the RDQN Random model is more robust, with a significantly lower variance in Fuel 100km compared to the other three models. The DQN Random model had the second-highest mean and variance among the four models. These results suggest that using recurrent neural networks and random driving conditions can improve vehicle performance under varying driving cycles. Notably, all four models performed poorly in the second test for fuel economy. This is because the distribution in the second test differed more significantly from the standard NEDC and random training data used to train the DQN model. Therefore, it is clear that the difference between the training and testing environments leads to a decline in the performance of DRL-based EMS.

[0108] Figure 7 The distribution of engine operating points for the four models is shown. The results indicate that the operating points generated by the RDQN random algorithm are more concentrated in the high-efficiency region. The research results suggest that combining recurrent neural networks with various driving conditions may improve the performance of the EMS. The results also show that RDQN outperforms DQN in terms of engine operating point distribution and efficiency.

[0109] Figure 8Two examples of State of Charge (SoC) trajectories for four models are described. Results show that models trained on the New European Driving Cycle (NEDC) fail to produce consistent SoC trajectories. This inconsistency can be attributed to the significant deviation of the tested driving cycle from the standard NEDC. Furthermore, due to the lack of environmental memory in the Deep Q-Network (DQN) model, which observes only a single state at each time step, it may fail to fully capture the dynamics of the driving environment, leading to unreliable action generation by the Energy Management System (EMS). Although the RDQN NEDC outperforms the DQN NEDC in the first example, it exhibits significant oscillations in the final stage of the SoC trajectory. Figure 8 In the second example in (b), the RDQN NEDC SoC trajectory consumed power during the tripping process. Even with the memory of historical trajectories, repeated strategies can still produce erroneous behavior due to differences in training and testing environments. Interestingly, the model trained using a random driving bicycle showed a more stable SoC trajectory than the model trained using NEDC. This finding suggests that they are more battery-friendly in HEVs.

[0110] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An adaptive multi-condition recursive deep reinforcement learning energy management strategy for hybrid vehicles, characterized in that, include: S1. Generate random operating conditions as the training set; S2. Establish the state variables, action variables, and reward function required for the algorithm network. Obtain the state S based on the state variables, the action A based on the action variables, and the reward R based on the reward function. At the same time, establish the reward function based on the reward function. Among them, the state of charge, power speed, and acceleration of the vehicle are selected as the state variables, the engine power change of the vehicle is selected as the action variables, and the reward function is established based on the fuel consumption and state of charge deviation of the vehicle. S3. Create an RDQN algorithm network, which includes an evaluation network, a target network, and LSTM units. The evaluation network and the target network have the same structure. S4. Initialize the parameters of each network; S5. Fill the memory pool with random data; S6. Under one of the random operating conditions, read in state S; S7. Initialize the hidden state and create a temporary memory pool; S8, with The probability is to select a random action A, or to select the action A corresponding to the maximum Q value of the evaluation network; S9. Execute action A and obtain the returned state. And reward R; S10. Use the LSTM unit to obtain the next hidden state. ; S11, the experience gained (S, A, R Stored in a temporary memory pool; S12. Update the evaluation network parameters through backpropagation, update the target network parameters after a specified number of steps, and update the state S at the same time; S13. Repeat steps S6-S12 until the set number of iterations is reached, then stop training the RDQN algorithm network. S14. Use the trained RDQN algorithm for energy management, and control the system based on maximizing the corresponding reward R, state S, and action A according to the reward function.

2. The adaptive multi-condition recursive deep reinforcement learning hybrid vehicle energy management strategy according to claim 1, characterized in that, The RDQN algorithm requires setting hyperparameters and... The hyperparameters include learning rate, discount factor, batch size, specified number of steps for updating the target network, maximum number of iterations, number of neurons in the fully connected layer, and memory pool size. The initial value is 0, and the set value is incremented step by step.

3. The adaptive multi-condition recursive deep reinforcement learning hybrid vehicle energy management strategy according to claim 1, characterized in that, The LSTM unit has 100 hidden units and a time step of 32.

4. The adaptive multi-condition recursive deep reinforcement learning hybrid vehicle energy management strategy according to claim 1, characterized in that, The experience (S, A, R It contains time step information.

5. The adaptive multi-condition recursive deep reinforcement learning hybrid vehicle energy management strategy according to claim 1, characterized in that, In the evaluation network and the target network, the strategy of The function is: ， in, The state at the current moment , Actions at the current moment , Rewards for the current moment .

6. The adaptive multi-condition recursive deep reinforcement learning hybrid vehicle energy management strategy according to claim 1, characterized in that, The evaluation network consists of Parameterized neural network representation, for It optimizes by minimizing a loss function, which is: ， ， in, The value of the loss function. This represents the expected value of the reward after the current state and the action is performed. For the target network value, The discount factor for the next state and, For the target network, where The weights are fixed for multiple iterations.

7. The adaptive multi-condition recursive deep reinforcement learning hybrid vehicle energy management strategy according to claim 1, characterized in that, The RDQN algorithm can add a hidden state. To model, thereby generating .

8. The adaptive multi-condition recursive deep reinforcement learning hybrid vehicle energy management strategy according to claim 1, characterized in that, The return function is: ， in, As a discount factor, The reward for the current moment. This is for long-term rewards in the future.

9. The adaptive multi-condition recursive deep reinforcement learning hybrid vehicle energy management strategy according to claim 1, characterized in that, S1 is a random driving condition generated using MATLAB's RDE tool. During the generation process, constraints are set, including the duration of the driving condition, sampling interval, speed boundary, tolerance range, minimum travel distance for each type of road segment, maximum speed, number of stops, stop duration, RPA constraint line, and VA95 constraint line.

Images