System control method of hybrid electric automobile based on deep layer reinforcement learning
The system control method for hybrid electric vehicles using deep reinforcement learning addresses the limitations of conventional energy management methods by optimizing energy management in real-time, achieving accurate and efficient results.
Patent Information
- Application Number
- JP2024152203
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-10-20
- Filing Date
- 2024-09-04
- Publication Date
- 2025-05-02
- Estimated Expiration
- 2044-09-04
AI Technical Summary
Conventional energy management methods for hybrid electric vehicles are not suitable for wide-ranging applications due to inaccurate results and slow calculations, making it difficult to develop effective and accurate energy management policies quickly.
A system control method for hybrid electric vehicles based on deep reinforcement learning, which includes acquiring historical driving data, creating power system and battery models, performing initial optimization, dynamic planning, and training reinforcement learning algorithms to optimize energy management in real-time.
The method achieves optimal and accurate energy management, significantly speeding up the convergence of reinforcement learning algorithms, and improving computational efficiency, thereby enhancing fuel efficiency and reducing operational costs.
Smart Images
Figure 2025070969000001_ABST
Abstract
Description
[Technical field]
[0001] The present invention relates to the technical field of new energy vehicles, and in particular to a system control method for hybrid electric vehicles based on deep reinforcement learning. [Background technology]
[0002] In recent years, industry and academia have been researching and developing new power systems to drive automobiles. The demands to reduce fuel consumption and pollutant emissions are becoming more stringent. Although electric vehicles (EVs) have the potential to achieve zero emissions, their widespread adoption is limited by deficiencies in driving range, charging infrastructure, and battery costs. Plug-in hybrid electric vehicles (PHEVs) have increasingly attracted the attention of automobile manufacturers, as they combine the advantages of traditional internal combustion engines and new electric motors.
[0003] Currently, energy management policies are important for improving the economy of fuel combustion in hybrid vehicles. Conventional energy management policies (Energy Management Strategies, EMS) are generally divided into three types: rule-based policies, optimization-based policies, and learning-based policies. Although rule-based policies are effective and easy to implement, their control performance is limited by different vehicle models and vehicle configurations, and therefore various driving needs cannot be met. Optimization-based policies include dynamic programming (DP), Pontryagin's Minimum Principle (PMP), etc., and in actual application, they basically depend on the overall operating state information. Reinforcement learning (RL)-based policies can make up for the shortcomings of the conventional control techniques mentioned above. Compared with dynamic programming, reinforcement learning-based energy management policies can save a lot of time and cost in calculations when operating online, and can obtain results that are equivalent to the overall optimum. These properties make it an effective and stable approach to develop policies for managing energy in hybrid systems. Summary of the Invention [Problem to be solved by the invention]
[0004] However, the conventional methods are not suitable for widespread application due to their inaccurate results and slow calculation speed. Therefore, it is particularly desirable to be able to quickly research and develop effective and accurate energy management methods. [Means for solving the problem]
[0005] The present invention aims to provide a system control method for a hybrid electric vehicle based on deep reinforcement learning in view of the problems existing in the prior art. It can ensure that the optimization result is optimal, and can be applied immediately, with a fast processing process, good effectiveness, and accurate results. It can also realize the potential for fuel saving in multiple driving modes of the vehicle, and improve the economy of the entire fuel cell vehicle. At the same time, in the optimization, experience generated by dynamic planning of the historical operating state can be introduced, and the speed of calculation by the reinforcement learning algorithm can be increased, and the accuracy of the calculation result can be further improved. Specifically, the present invention provides a system control method for a hybrid electric vehicle based on deep reinforcement learning, which includes the following steps:
[0006] Step S1 obtains fixed-line multi-dimensional path status information in a history driving process for a plug-in hybrid electric vehicle.
[0007] Step S2 creates a full-vehicle power system model and a power battery model for a plug-in hybrid electric vehicle, and specifically includes the following sub-steps:
[0008] Sub-step S21 creates a vehicle power system model as follows: JPEG2025070969000002.jpg43170 Substep S22 creates a power battery model as follows: JPEG2025070969000003.jpg58170
[0009] Step S3 performs an initial optimization process for the two electric motors to reduce the dimensions to be optimized, and specifically includes the following sub-steps: In sub-step S31, an objective function for performing initial optimization is created for the two electric motors with the overall energy conversion efficiency as a goal, as follows: JPEG2025070969000004.jpg31170 Sub-step S32 discretizes the vehicle speed and the equivalent electric motor power to form network nodes to be traversed. Substep S33 calculates the optimal power allocation of the two electric motors in each network node formed in substep S32 according to the objective function created in substep S31 in each mode, including series mode, electric only mode, parallel one speed mode and parallel two speed mode.
[0010] Step S4 performs dynamic programming calculations on the multi-dimensional path situation information for the plug-in hybrid electric vehicle collected in step S1 based on the initial optimization result in step S3, to generate a state transfer data set. Step S4 specifically includes the following sub-steps: A sub-step S41 selects one or more operating states as multi-dimensional passageway situation information from the historical operating states in the same line section collected in the step S1. Sub-step S42 calculates the minimum intervention speed in each mode, and traverses the feasible modes according to the minimum intervention speed condition at the network node composed of time and SOC. Sub-step S43 allocates the power of the two electric motors according to the initial optimization results in step S3 when traversing the electric-only mode, the parallel 1st speed mode, and the parallel 2nd speed mode.
[0011] Step S5 specifies the state variables, action variables, and incentive functions required for the reinforcement learning algorithm, where the state variable s, action variable a, and incentive function r are as follows: JPEG2025070969000005.jpg72170
[0012] Step S6 initially trains a network of critics and actors using the state transfer dataset generated in step S4.
[0013] Step S7 constructs an ambient intelligence model, transfers the state transfer dataset generated in step S4 to an experience pool, and takes the two neural networks trained in step S6 as the initial network model, repeatedly trains the energy management policy through a deep reinforcement learning algorithm until the algorithm converges, and finally obtains the ambient intelligence model. Step S7 specifically includes the following sub-steps: Substep S71 constructs an alternating algorithm based on the whole-vehicle environment module and the DDPG algorithm module. Substep S72 transfers the state transfer dataset generated in step S4 to an experience pool using an alternating algorithm, and constructs an intelligence module using the two initially trained neural network models in step S6 as initial neural network models for the critic and actor. Substep S73 defines, for each training session, the vehicle real-time state parameters and corresponding incentive values as input parameters to the neural network in the intelligence module, and the control variables output from the neural network as input parameters to the vehicle model in the environment module, respectively, generates new incentive values when the vehicle executes a control command, and stores the obtained experience in the experience pool. Sub-step S74 realizes a learning update step of the neural network by the intelligent body updating the policy gradient. Sub-step S75 iteratively recovers the algorithm until it converges, and finally obtains the ambient intelligence model.
[0014] Step S8 introduces the converged ambient intelligence model into the HCU controller to execute online applications on the fixed line.
[0015] Preferably, the fixed-rail multi-dimensional path situation information acquired in step S1 includes one or more historical vehicle speed curves and path gradient curves collected while the electric vehicle is traveling on the fixed rail.
[0016] Preferably, step S6 specifically includes the following sub-steps: JPEG2025070969000006.jpg20170 Sub-step S62 constructs a neural network configuration of critics and actors, including an input layer, a hidden layer and an output layer, and specifies the number of network neurals in each layer. S63 constructs the output sample set for the critic network as follows. JPEG2025070969000007.jpg50170
[0017] Preferably, the neural network configuration of the critic and actor in step S62 outputs the four mode values of engine power and selects the actual operating variables as follows: JPEG2025070969000008.jpg23170
[0018] Preferably, the formula for updating the specific policy gradient in sub-step S74 is as follows: JPEG2025070969000009.jpg31170JPEG2025070969000010.jpg58170 Effect of the Invention
[0019] Compared with the prior art, the present invention has the following advantageous effects: (1) By managing energy using a deep reinforcement learning algorithm, the present invention can achieve real-time and optimal energy management policies while making maximum use of energy. (2) The present invention changes the problem of simultaneously optimizing three power sources based on the initial optimization calculation when two electric motors are running simultaneously into a problem of layered optimization. First, the power allocation of the two electric motors in the power system is regarded as a static optimization problem, and initial optimization of the power allocation and shift position is performed. Then, a mode is selected according to deep reinforcement learning to allocate power to the engine and the equivalent electric motor, thereby reducing the difficulty of optimization.
[0020] (3) The present invention transfers the results of dynamic programming collected on a fixed line for one or more operating states as expert knowledge to an intelligent entity performing reinforcement learning, thereby significantly accelerating the convergence speed of the reinforcement learning algorithm and significantly improving the computational efficiency.
[0021] (4) By making full use of historical data and topographical information about the scene of an electric vehicle delivering luggage along a fixed track, the method can effectively learn the characteristics of the electric vehicle traveling along the fixed track, and is more applicable to the scene of delivery along the fixed track than general energy management policies. [Brief description of the drawings]
[0022] [Figure 1] FIG. 1 is a schematic diagram showing the flow of a system control method for a hybrid electric vehicle based on deep reinforcement learning according to the present invention. [Diagram 2] 4 is a flowchart of an energy management policy algorithm in the first embodiment according to the present invention. [Diagram 3] 1 is a schematic diagram showing a P2P3 plug-in type hybrid light truck configuration and an explanation of mode classification in a first embodiment of the present invention; [Figure 4] 1 is a flowchart showing initial optimization in the first embodiment according to the present invention. [Diagram 5] FIG. 2 is a schematic diagram showing a network configuration between a critic and an actor in the first embodiment according to the present invention. [Figure 6] FIG. 11 is a schematic diagram showing a portion of the route section used in the 6th IFAC E-COSM 2021 Challenge Match in Example 2 of the present invention as a fixed track on which a light truck ran. [Figure 7A] FIG. 11 is a diagram showing the efficiency of two electric motors in a light truck in Example 2 according to the present invention. [Figure 7B] FIG. 11 is a diagram showing the efficiency of two electric motors in a light truck in Example 2 according to the present invention. [Figure 8] FIG. 11 is a schematic diagram showing a result of initial optimization performed for mode 2 in Example 2 according to the present invention. [Figure 9] FIG. 11 is a schematic diagram showing a result of initial optimization performed for mode 3 in the second embodiment according to the present invention. [Figure 10] FIG. 11 is a schematic diagram showing a result of initial optimization performed on mode 4 in the second embodiment according to the present invention. [Figure 11] FIG. 7 is a schematic diagram showing historical operating states collected in the line section shown in FIG. 6 in a second embodiment according to the present invention; [Figure 12] FIG. 11 is a schematic diagram showing a result of allocating energy in a certain working state according to a dynamic programming energy management policy in the second embodiment of the present invention; [Figure 13] FIG. 11 is a schematic diagram showing convergence characteristics in the training process in the second embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0023] Hereinafter, the specific real-time method according to the present invention will be described in more detail based on specific embodiments with reference to the drawings. The following examples or drawings are only for illustrating the present invention, and are not intended to limit the scope of the present invention.
[0024] The present invention provides a system control method for a hybrid electric vehicle based on deep reinforcement learning, as shown in Figure 1. The method specifically includes the following steps:
[0025] Step S1 obtains fixed-line multi-dimensional path status information in a history driving process for a plug-in hybrid electric vehicle.
[0026] Step S2 creates a full-vehicle power system model and a power battery model for a plug-in hybrid electric vehicle, and specifically includes the following sub-steps: In sub-step S21, the entire vehicle power system model is JPEG2025070969000011.jpg25170JPEG2025070969000012.jpg79170
[0027] Step S3 performs an initial optimization process for the two electric motors to reduce the dimensions to be optimized, and specifically includes the following sub-steps: In sub-step S31, an objective function for performing initial optimization is created for the two electric motors with the overall energy conversion efficiency as a goal, as follows: JPEG2025070969000013.jpg30170 Substep S32 discretizes the vehicle speed and the equivalent electric motor power to form network nodes to be traversed. Sub-step S33 calculates the optimal power allocation of the two electric motors in each network node formed in step S32 according to the objective function created in step S31 in each mode, where each mode includes a series mode, an electric-only mode, a parallel one-speed mode, and a parallel two-speed mode.
[0028] Step S4: for the multi-dimensional path situation information for the plug-in hybrid electric vehicle collected in step S1, a dynamic programming calculation is performed based on the initial optimization result in step S3 to generate a state transfer data set. Step S4 includes the following sub-steps as follows: A sub-step S41 selects one or more operating conditions as passage status multi-dimensional information from the historical operating conditions in the same line section collected in the step S1. Sub-step S42 calculates the minimum intervention speed in each mode, and traverses each feasible mode according to the minimum intervention speed condition at the network node composed of time and SOC. Sub-step S43 allocates the power of the two electric motors according to the initial optimization results in step S3 when traversing the electric-only mode, the parallel 1st speed mode, and the parallel 2nd speed mode.
[0029] Step S5 specifies the state variables, action variables, and incentive functions required for the reinforcement learning algorithm, where the state variable s, action variable a, and incentive function r are as follows: JPEG2025070969000014.jpg27170JPEG2025070969000015.jpg45170
[0030] Step S6 initially trains a network of critics and actors using the state transfer dataset generated in step S4. JPEG2025070969000016.jpg20170 Sub-step S62 builds a neural network configuration of critics and actors, including an input layer, a hidden layer and an output layer, and specifies the number of network neurals in each layer. Among them, the neural network configuration of critics and actors outputs the engine power of the value of four modes, and selects the actual operating variables as follows: JPEG2025070969000017.jpg23170Substep S63 constructs an output sample set for the critic network as follows. JPEG2025070969000018.jpg42170
[0031] In step S7, the ambient intelligence model is loaded, the state transfer dataset generated in step S4 is transferred to the experience pool, and the two neural networks trained in step S6 are used as the initial network model. The training energy management policy is iteratively recovered by the deep reinforcement learning algorithm until the algorithm converges, and the ambient intelligence model is finally obtained. Step S7 specifically includes the following sub-steps: Substep S71 constructs an alternating algorithm based on the whole-vehicle environment module and the DDPG algorithm module. Substep S72 transfers the state transfer dataset generated in step S4 to the experience pool of the alternating algorithm, and constructs an intelligence module using the two initially trained neural network models, critic and actor, as initial neural network models in step S6. Substep S73 defines, for each training session, the vehicle real-time state parameters and corresponding incentive values as input parameters to the neural network in the intelligence module, and the control variables output from the neural network as input parameters to the vehicle model in the environment module, respectively, generates new incentive values when the vehicle executes a control command, and stores the obtained experience in the experience pool. Sub-step S74 realizes a learning update step of the neural network by updating the intelligent policy gradient. The specific policy gradient update formula is as follows: JPEG2025070969000019.jpg87170 Sub-step S75 iteratively recovers until the algorithm converges, and finally obtains the ambient intelligence model.
[0032] Step S8 introduces the converged ambient intelligence model into the HCU controller to execute online applications on the fixed line.
[0033] Example 1 In this embodiment, the method according to the present invention is applied to a hybrid light truck configured as P2P3, as shown in FIG.
[0034] Step S1 obtains fixed-rail multi-dimensional passage situation information during a historical travel process of a plug-in hybrid light logistics truck, which specifically includes at least one historical vehicle speed curve and passage gradient curve collected while the light logistics truck is traveling on a fixed rail.
[0035] Step S2 builds a full-vehicle power system model for a plug-in hybrid light logistics truck based on the power system configuration shown in FIG. 3, and specifically includes the following steps: Sub-step S21 is a process in which the on-board vehicle dynamics model is JPEG2025070969000020.jpg62170JPEG2025070969000021.jpg17170Substep S22 is a process for determining whether the installed power battery model is JPEG2025070969000022.jpg58170
[0036] Step S3 performs an initial optimization process for the mode in which the two electric motors are coupled offline, as shown in FIG. 4, and specifically includes the following sub-steps: In sub-step S31, an objective function for performing initial optimization is created for the two electric motors with the overall energy conversion efficiency as a goal, as follows: JPEG2025070969000023.jpg30170 Substep S32 discretizes the vehicle speed and equivalent electric motor power to form network nodes to be traversed. Sub-step S33 calculates, for each mode, the optimal power allocation of the two electric motors in each network node formed in step S32 according to the objective function created in step S31.
[0037] Step S4 performs dynamic programming calculation based on the historical operating state of the same line section collected in step S1 and the initial optimization result obtained in step S3, and specifically includes the following steps: Sub-step S41 selects one or more operating conditions from the historical operating conditions in the same line section collected in step S1. Sub-step S42 calculates the minimum intervention vehicle speed in each mode based on the determination that the minimum engine speed is always greater than idling, and traverses each feasible mode according to the minimum intervention vehicle speed condition at a network node consisting of time and SOC. Sub-step S43 allocates the power of the two electric motors according to the initial optimization results obtained in step S3 when traversing the electric-only mode (mode 2), the parallel one-speed mode (mode 3), and the parallel two-speed mode (mode 4).
[0038] Step S5 identifies the state variables, action variables, and incentive functions required for the reinforcement learning algorithm. JPEG2025070969000024.jpg56170JPEG2025070969000025.jpg18170
[0039] Step S6 initially trains a network of critics and actors using the state transfer dataset generated in step S4, and specifically includes the following steps: JPEG2025070969000026.jpg18170 Sub-step S62 constructs a network configuration of critics and actors including an input layer, a hidden layer and an output layer, as shown in FIG. 5, and includes the number of network neurals in each layer. In addition, the actor network outputs the value and engine power in the four modes, and the actual operation selection is as follows: The JPEG2025070969000027.jpg23170actor network outputs five values, V1, V2, V3, V4, and Peng. The first four quantities are the mode values. The maximum of these is selected as the selected mode, and the above action variable a = (mode, Peng) is finally formed. S63 constructs an output sample set for the critic network, JPEG2025070969000028.jpg44170
[0040] Step S7 builds an ambient intelligence model and iteratively trains energy management policies through a deep reinforcement learning algorithm, which includes the following steps: Substep S71 constructs an alternating algorithm based on the whole-vehicle environment module and the DDPG algorithm module. Substep S72 transfers the state transfer dataset generated in step S4 to an experience pool, and constructs an intelligence module by using the two neural networks initially trained in step S6 as an initial network of critic and actor. Substep S73 divides the one or more historical vehicle speed curves collected in step S1 into a training set and a test set. Step S74 defines, for each training session, the vehicle real-time state parameters and the corresponding incentive values as input parameters to the neural network in the intelligence module, and the control variables output from the neural network as input parameters to the vehicle model in the environment module, respectively, generates new incentive values when the vehicle executes a control command, and stores the experience gained thereby in the experience pool. In step S75, the intelligent body updates the policy gradient to realize the learning update step of the neural network. Specifically, the formula for updating the policy gradient is as follows: JPEG2025070969000029.jpg9170JPEG2025070969000030.jpg77170 Substep S76 iteratively recovers to validate based on the measurement set until the learning has an ideal effect, and memorizes the persistence model of the entire neural network when training is over.
[0041] Step S8 integrates the converged actor neural network model into the light truck HCU controller and executes the online application in the fixed line. In hardware-in-the-loop simulation, when the actor network built based on Tensorflow converges, the network configuration is reproduced on the Simulink platform and its parameters are extracted to a network built by Simulink. The network built by Simulink is packaged as an HCU module and added to the SpeedGoat controller, and in the hardware-in-the-loop simulation, it is alternately simulated with the light truck model as the real-time output control amount by the controller.
[0042] Example 2 In this embodiment, a part of the line section used in the 6th IFAC E-COSM 2021 Challenge Match is used as the fixed track on which the light truck traveled to optimize the plug-in hybrid logistics light truck. The overall method includes the following steps:
[0043] Step S1 acquires fixed-rail multi-dimensional passageway status information in the history travel process for a plug-in hybrid logistics light truck. In this embodiment, as shown in FIG. 6, a part of the route section used in the 6th IFAC E-COSM 2021 Challenge Match is used as the fixed rail on which the light truck traveled. The route section has a total length of 10.16 km, and in total, there are 11 intersections and the distance between traffic lights is 1288 m to 247 m. The phase cycle time of each traffic light is 120 seconds, and if the yellow light is not taken into account, the duration of the red light and the green light is half.
[0044] Step S2 builds a full-vehicle power system model for a plug-in hybrid logistics light truck based on the full-vehicle parameters and power system configuration. In this embodiment, the main parameters of the light truck are shown in Table 1.
[0045] Table 1 Main parameters of P2-P3 hybrid light truck JPEG2025070969000031.jpg77170
[0046] Step S3 performs initial optimization processing for the mode in which the two electric motors are coupled. In this embodiment, the map diagrams of the efficiency of two electric motors for a light truck are shown in Figures 7A and 7B, and the results of initial optimization in different modes are shown in Figures 8, 9, and 10, respectively.
[0047] Step S4 collects historical operating states in the same line section shown in Fig. 6, and performs dynamic programming calculations based on the results of the initial optimization in step S3. In this embodiment, the historical operating states collected in the line section shown in Fig. 6 are shown in Fig. 11. The dynamic programming energy management policy allocates energy in a certain operating state as shown in Fig. 12.
[0048] Step S5 identifies the state variables, action variables, and incentive functions required for the reinforcement learning algorithm, respectively. JPEG2025070969000032.jpg21170JPEG2025070969000033.jpg54170
[0049] Step S6 initially trains a network of critics and actors using the state transition data set generated by the dynamic programming calculation.
[0050] Step S7 builds an ambient intelligence model and iteratively trains the energy management policy through a deep reinforcement learning algorithm. In this embodiment, the convergence characteristics of the training process are shown in FIG.
[0051] Step S8 integrates the converged actor neural network model into the light truck HCU controller to execute the online application in the fixed line. In this embodiment, the HCU controller is used as the Speedgoat real-time controller.
[0052] It should be noted that the above embodiments are merely for illustrating the technical means of the present invention, and are not intended to limit the same. Although the present invention has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that the technical means of the present invention can be amended or replaced with equivalents, and should be included in the patent scope of the present invention as long as they do not exceed the spirit and scope of the technical means.
Claims
1. Step S1 of acquiring fixed line multi-dimensional path status information in a history travel process for a plug-in hybrid electric vehicle; A step S2 of creating a full vehicle power system model and a power battery model for a plug-in hybrid electric vehicle, All vehicle power system models are When equipped with a power battery model, Step S3 of initially optimizing the two electric motors and reducing the dimension of the optimization, For two electric motors, the overall energy conversion efficiency is the target, and the initial optimization objective function is created as follows: a sub-step S32 of discretizing the vehicle speed and the equivalent electric motor power to form network nodes that require traversal; Step S3 specifically includes: in each mode, calculating an optimal power allocation of the two electric motors in each network node formed in sub-step S32 according to the objective function created in sub-step S31, the modes including series mode, electric only mode, parallel one-speed mode and parallel two-speed mode; and sub-step S33. A step S4 of performing a dynamic programming calculation based on the result of the initial optimization in step S3 for the multi-dimensional path situation information for the plug-in hybrid electric vehicle collected in step S1 to generate a state transfer data set, A sub-step S41 of selecting one or more operation states as multi-dimensional passageway situation information from the historical operation states in the same line section collected in the step S1; A sub-step S42 of calculating a minimum intervention speed in each mode and traversing each executable mode according to the minimum intervention speed condition at a network node composed of time and SOC; Step S4 specifically includes a sub-step S43 of allocating the power of the two electric motors according to the initial optimization result in step S3 when traversing the electric-only mode, the parallel 1-speed mode, and the parallel 2-speed mode; Step S5 of identifying state variables, action variables and incentive functions required for the reinforcement learning algorithm, the state variable s, the action variable a and the incentive function r being respectively: Step S6 of initially training a network of critics and actors using the state transfer dataset generated in step S4; Step S7: construct an ambient intelligence model, transfer the state transfer data set generated in step S4 to an experience pool, and use the two neural networks trained in step S6 as an initial network model to recover and train an energy management policy through a deep reinforcement learning algorithm until the algorithm converges, and finally obtain an ambient intelligence model; Sub-step S71: constructing an alternating algorithm based on the whole-vehicle environment module and the DDPG algorithm module; A sub-step S72 of transferring the state transfer data set generated in step S4 to an experience pool in the alternating algorithm, and loading the intelligence module with the two initially trained neural network models as the initial neural network models of critic and actor in step S6; sub-step S73: for each training, define the whole-vehicle real-time state parameters and corresponding incentive values as input parameters to the neural network in the intelligence module, and define the control variables output from the neural network as input parameters to the whole-vehicle model in the environment module, respectively, generate new incentive values when the vehicle executes a control command, and store the obtained experience in an experience pool; A sub-step S74 of implementing a learning update step of the neural network by updating the policy gradient by the intelligence; Step S7 specifically includes a sub-step S75 of iteratively recovering the algorithm until it converges, and finally obtaining the ambient intelligence model; and step S8 of introducing the converged ambient intelligence model into the HCU controller and executing an online application on a fixed line.
2. 2. The system control method for a hybrid electric vehicle based on deep reinforcement learning as described in claim 1, wherein the fixed-track multi-dimensional path situation information acquired in step S1 includes at least one historical vehicle speed curve and path gradient curve collected while the electric vehicle is traveling on the fixed track.
3. Step S6 is A sub-step S62 of constructing a neural network configuration of critics and actors including an input layer, a hidden layer and an output layer, and specifying the number of neural networks in each layer; When constructing the output sample set of the critic network,
4. The neural network configuration of critic and actor in sub-step S62 outputs the values of the four modes and the engine power, and selects the actual operating variables.
5. In sub-step S74, a specific formula for updating the policy gradient is