Parallel hybrid vehicle energy management method and apparatus

Abstract

The application discloses a parallel hybrid vehicle energy management method and device, and the method comprises the following steps: acquiring an initial vehicle state signal; constructing a vehicle dynamics equation; constructing a parallel hybrid vehicle structure model according to the vehicle dynamics equation; constructing a battery thermal model and a battery life model according to the parallel hybrid vehicle structure model; constructing a parameter-free return function according to the battery thermal model and the battery life model; generating a reinforcement learning space state and a reinforcement learning control action according to the parameter-free return function; constructing a target double-delay deep deterministic policy gradient network; constructing a vehicle energy management learning strategy; and determining a target engine equivalent torque by using the vehicle energy management learning strategy according to the initial vehicle state signal, the target engine equivalent torque being used for controlling engine operation. The application realizes parallel hybrid vehicle energy management, improves accuracy, applicability and efficiency, and can be widely applied to the technical field of vehicle engineering.

Description

Technical Field

[0001] The present invention relates to the field of vehicle engineering technology, and in particular to an energy management method and device for a parallel hybrid vehicle. Background Art

[0002] Among environmentally friendly vehicles, hybrid electric vehicles (HEVs) offer longer driving ranges than pure electric vehicles and lower fuel consumption than traditional fuel vehicles, making them more environmentally friendly. However, HEV energy management systems are far more complex than those of traditional fuel vehicles and pure electric vehicles. Traditional rule-based energy management methods rely primarily on expert experience and experimental calibration, but these methods suffer from low accuracy due to the complexity of actual operating conditions. Optimization-based energy management methods optimize energy by predicting road conditions in advance, but they rely too heavily on high-performance models and have low applicability. Learning-based energy management methods rely primarily on reinforcement learning strategies, but they struggle to cope with high-dimensional nonlinear features, and the hyperparameter adjustment process is cumbersome and inefficient.

[0003] In summary, the technical problems existing in the relevant technologies need to be improved. Summary of the Invention

[0004] The embodiments of the present invention provide a method and device for energy management of a parallel hybrid vehicle, which effectively improves accuracy, applicability and efficiency.

[0005] In one aspect, an embodiment of the present invention provides a method for energy management of a parallel hybrid vehicle, comprising the following steps:

[0006] Obtaining an initial vehicle status signal;

[0007] Construct vehicle dynamics equations;

[0008] According to the vehicle dynamics equation, a parallel hybrid vehicle structural model is constructed;

[0009] Constructing a battery thermal model and a battery life model based on the parallel hybrid vehicle structural model;

[0010] Constructing a parameter-free reward function according to the battery thermal model and the battery life model;

[0011] generating a reinforcement learning space state and a reinforcement learning control action according to the parameter-free reward function;

[0012] Construct a target dual-delay deep deterministic policy gradient network;

[0013] constructing a vehicle energy management learning policy according to the parallel hybrid vehicle structure model, the parameter-free return function, the reinforcement learning space state, the reinforcement learning control action, an experience replay area, and the target double-delay deep deterministic policy gradient network;

[0014] determining a target engine equivalent torque for controlling engine operation to achieve vehicle energy management according to the initial vehicle state signal and using the vehicle energy management learning policy.

[0015] In some embodiments, the constructing a vehicle dynamics equation includes:

[0016] calculating a vehicle running rolling resistance according to a vehicle mass, a gravitational acceleration, a rolling resistance coefficient, and a vehicle road slope;

[0017] calculating a vehicle running slope resistance according to the vehicle mass, the gravitational acceleration, and the vehicle road slope;

[0018] calculating a vehicle running air resistance according to an air density, a vehicle frontal area, an air resistance coefficient, and a vehicle running speed;

[0019] calculating a vehicle running acceleration resistance according to a rotational mass conversion coefficient, the vehicle mass, and a vehicle running acceleration;

[0020] constructing the vehicle dynamics equation according to the vehicle running rolling resistance, the vehicle running slope resistance, the vehicle running air resistance, and the vehicle running acceleration resistance.

[0021] In some embodiments, the constructing a battery thermal model according to the parallel hybrid vehicle structure model includes:

[0022] extracting a battery mass, a battery specific heat capacity, and a battery surface area from the parallel hybrid vehicle structure model;

[0023] constructing the battery thermal model according to a battery temperature, an ambient temperature, the battery mass, the battery specific heat capacity, a natural heat convection constant, the battery surface area, a battery operating current, a battery open-circuit voltage, a battery operating voltage, and an integration time.

[0024] In some embodiments, the constructing a battery life model according to the parallel hybrid vehicle structure model includes:

[0025] extracting a battery discharge rate from the parallel hybrid vehicle structure model;

[0026] calculating an activation energy according to the battery discharge rate;

[0027] Calculating a battery throughput based on an exponential factor, the battery discharge rate, the activation energy, a universal gas constant, and a battery temperature;

[0028] Calculating a battery capacity loss percentage based on the exponential factor, the battery discharge rate, the activation energy, the universal gas constant, the battery temperature, the battery throughput, and a power law coefficient;

[0029] Calculating the equivalent number of cycles before the end of the battery life according to the battery throughput and the battery capacity loss percentage;

[0030] The battery life model is constructed based on the battery capacity loss percentage, the equivalent number of cycles before the end of the battery life, the battery life state, the battery operating current and the integration time.

[0031] In some embodiments, constructing a parameter-free reward function based on the battery thermal model and the battery life model includes:

[0032] determining a battery health state according to the battery thermal model and the battery life model;

[0033] Calculate the loss associated with the deviation between the current state of charge value and the reference value based on the battery state of charge and the battery state of charge reference value;

[0034] The parameter-free reward function is constructed according to the fuel consumption, the battery state of charge, the battery state of health, and the loss associated with the deviation between the current value of the state of charge and a reference value.

[0035] In some embodiments, generating a reinforcement learning space state and a reinforcement learning control action according to the parameter-free reward function includes:

[0036] Dividing the vehicle state according to the parameter-free reward function to obtain a direct state, wherein the direct state includes vehicle torque demand, power demand, vehicle speed, vehicle acceleration or fuel consumption;

[0037] performing a refining process on the direct state;

[0038] Generating the reinforcement learning space state according to the refined direct state;

[0039] Calculating an initial engine equivalent torque based on the current engine output torque, the maximum engine output torque, and the minimum engine output torque;

[0040] The initial engine equivalent torque is used as the reinforcement learning control action.

[0041] In some embodiments, constructing a target dual-delay deep deterministic policy gradient network includes:

[0042] Acquire a vehicle standard operating condition data set, wherein the vehicle standard operating condition data set includes parameters and observation values ​​related to energy management during simulated driving of the vehicle standard operating condition;

[0043] Based on the value network model and the policy network model, an initial double-delay deep deterministic policy gradient network is constructed;

[0044] The vehicle standard operating condition data set is input into the initial double-delay deep deterministic policy gradient network to train the initial double-delay deep deterministic policy gradient network to obtain the target double-delay deep deterministic policy gradient network.

[0045] In some embodiments, determining the target engine equivalent torque using the vehicle energy management learning strategy based on the initial vehicle state signal includes:

[0046] Initializing the first initial value network weight, the second initial value network weight, the initial policy network weight, and model training parameters, wherein the model training parameters include experience replay area storage information, learning rate, random noise, discount factor, batch gradient descent sample number, and preset maximum number of iterations;

[0047] Calculating a first target value network weight based on the first initial value network weight;

[0048] Calculating a second target value network weight based on the second initial value network weight;

[0049] Calculating a target policy network weight based on the initial policy network weight;

[0050] Refining the initial vehicle state signal to obtain a current state;

[0051] generating a current control action according to the current state, the random noise and the control strategy;

[0052] Execute the current control action to obtain the reward value and the next state;

[0053] Storing the current state, the current control action, the reward value, and the next moment state in the experience playback area;

[0054] Randomly sampling the experience playback area to obtain a number of samples;

[0055] generating a control action at the next moment according to the plurality of samples;

[0056] Calculating a timing difference error according to the state at the next moment and the control action at the next moment;

[0057] Calculating a minimized loss based on the timing difference error;

[0058] Based on the model training parameters, updating the value network parameters using the minimized loss;

[0059] Based on the model training parameters, update the policy network parameters using gradient descent;

[0060] updating the first target value network weight according to the value network parameter, the first initial value network weight, and the first target value network weight;

[0061] updating the second target value network weight according to the value network parameter, the second initial value network weight, and the second target value network weight;

[0062] According to the strategy network parameters, the initial strategy network weight and the target strategy network weight, the target strategy network weight is updated until the preset maximum number of iterations is reached to obtain the target engine equivalent torque.

[0063] In another aspect, an embodiment of the present invention provides an energy management device for a parallel hybrid vehicle, comprising:

[0064] The first module is used to obtain an initial vehicle status signal;

[0065] The second module is used to construct the vehicle dynamics equations;

[0066] A third module is used to construct a parallel hybrid vehicle structural model based on the vehicle dynamics equation;

[0067] A fourth module is used to construct a battery thermal model and a battery life model based on the parallel hybrid vehicle structural model;

[0068] A fifth module is configured to construct a parameter-free reward function based on the battery thermal model and the battery life model;

[0069] A sixth module is used to generate a reinforcement learning space state and a reinforcement learning control action according to the parameter-free reward function;

[0070] The seventh module is used to build a target dual-delay deep deterministic policy gradient network;

[0071] an eighth module for constructing a vehicle energy management learning strategy based on the parallel hybrid vehicle structural model, the parameter-free reward function, the reinforcement learning space state, the reinforcement learning control action, the experience replay area, and the target dual-delay deep deterministic policy gradient network;

[0072] The ninth module is used to determine the target engine equivalent torque based on the initial vehicle state signal using the vehicle energy management learning strategy, and the target engine equivalent torque is used to control the engine operation to achieve vehicle energy management.

[0073] In another aspect, an embodiment of the present invention provides a computer device, comprising:

[0074] at least one processor;

[0075] at least one memory for storing at least one program;

[0076] When the at least one program is executed by the at least one processor, the at least one processor implements the method.

[0077] The beneficial effects of the present invention are as follows:

[0078] The embodiment of the present invention first obtains an initial vehicle state signal, then constructs a vehicle dynamics equation, and constructs a parallel hybrid vehicle structural model based on the vehicle dynamics equation. Based on the parallel hybrid vehicle structural model, a battery thermal model and a battery life model are constructed, and based on the battery thermal model and the battery life model, a parameter-free reward function is constructed. Then, based on the parameter-free reward function, a reinforcement learning space state and a reinforcement learning control action are generated, a target dual-delay deep deterministic policy gradient network is constructed, and a vehicle energy management learning strategy is constructed. Finally, based on the initial vehicle state signal, the target engine equivalent torque is determined using the vehicle energy management learning strategy, so that the engine operation can be controlled by the target engine equivalent torque to achieve parallel hybrid vehicle energy management, thereby improving accuracy, applicability and efficiency.

[0079] Other features and advantages of the present invention will be described in the following description, and in part will become apparent from the description, or will be understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained through the structures particularly pointed out in the description and the drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0080] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0081] Figure 1 This is a flow chart of an energy management method for a parallel hybrid vehicle according to an embodiment of the present invention;

[0082] Figure 2A schematic diagram of a parallel hybrid vehicle structure model according to an embodiment of the present invention;

[0083] Figure 3 A schematic diagram of an initial double-delayed deep deterministic policy gradient network architecture according to an embodiment of the present invention;

[0084] Figure 4 This is a schematic diagram of a speed curve for a training condition according to an embodiment of the present invention;

[0085] Figure 5 A schematic diagram of a vehicle energy management learning strategy architecture according to an embodiment of the present invention;

[0086] Figure 6 A schematic diagram of a speed curve of a composite working condition according to an embodiment of the present invention;

[0087] Figure 7 This is a schematic structural diagram of an energy management device for a parallel hybrid vehicle according to an embodiment of the present invention;

[0088] Figure 8 The figure is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0089] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the embodiments of the present application. They are merely examples of devices and methods consistent with some aspects of the embodiments of the present application as detailed in the appended claims.

[0090] It will be understood that the terms "first", "second", etc. used in this application may be used herein to describe various concepts, but unless otherwise specified, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another. For example, without departing from the scope of the embodiments of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the words "if" and "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0091] The terms "at least one", "plurality", "each", "any", etc. used in this application include "at least one", "two" or more, "plurality" or "each", "any" or "any one", "each" or "any one" as used herein.

[0092] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application pertains. The terms used herein are for the purpose of describing the embodiments of this application only and are not intended to limit this application.

[0093] Before explaining the embodiments of the present application in detail, some of the nouns and terms involved in the embodiments of the present application are first explained. The nouns and terms involved in the embodiments of the present application are subject to the following explanations.

[0094] A parallel hybrid electric vehicle (HEV) is a hybrid electric vehicle that combines a traditional fuel engine with an electric motor to provide power. HEV mode (hybrid power mode) is the primary operating mode of a parallel hybrid vehicle. Its core goal is to improve fuel efficiency and reduce exhaust emissions by using the electric motor and fuel engine to work together.

[0095] Battery State of Charge (SOC): refers to the ratio of the remaining power of the battery to its capacity in a fully charged state, usually expressed as a percentage.

[0096] Among all environmentally friendly vehicles, hybrid electric vehicles (HEVs) enjoy longer driving ranges than pure electric vehicles and lower fuel consumption than conventional fuel vehicles, making them more environmentally friendly. However, HEV energy management systems are far more complex than those of conventional fuel vehicles and pure electric vehicles. Consequently, HEV energy management strategies (EMS) have become a hot topic in the automotive field. Research on HEV EMS can be categorized into three main categories: rule-based, optimization-based, and learning-based. While rule-based EMS are easy to implement, their excessive reliance on expert experience and experimental calibration makes it difficult to formulate reasonable rules for complex operating conditions. Optimization-based EMS strategies include global optimization and real-time optimization. Typical global optimization strategies include dynamic programming, genetic algorithms, and particle swarm optimization. These algorithms are computationally expensive and are typically performed offline. They are often used as benchmarks to evaluate the effectiveness of other online EMSs. Real-time optimization control transforms the global optimization problem into an instantaneous optimization problem to improve the feasibility of online execution. Examples include Pontryagin's minimum principle (PMP), equivalent consumption minimization strategy (ECMS), and model predictive control (MPC). However, the key to the success of MPC is rapid prediction and optimization strategies. Road conditions must be predicted in advance, which relies heavily on high-performance models. PMP offers excellent optimization efficiency, but the common state (co-state) is difficult to obtain and the computational complexity is relatively high. Equivalent fuel consumption minimization strategies offer excellent real-time performance, but the historical road information used to calculate equivalent fuel consumption often fails to represent future driving conditions, resulting in poor algorithm robustness.

[0097] Learning-based EMS, particularly recently developed reinforcement learning techniques, has opened up new avenues for addressing this challenge. Energy management strategies based on the reinforcement learning algorithm Q-learning can significantly improve vehicle fuel economy compared to traditional rule-based strategies. However, due to the "curse of dimensionality," reinforcement learning strategies like Q-learning are infeasible for systems with high-dimensional, continuous state and action spaces. The Deep Deterministic Policy Gradient (DDPG) strategy is an online actor-critic reinforcement learning strategy. The DDPG agent computes an optimal policy that maximizes long-term reward and can be trained in environments with continuous or discrete state spaces and continuous action spaces. Unfortunately, overestimation of the value function, leading to incremental bias and suboptimal policies, is a common drawback of DDPG. Energy management strategies based on deep reinforcement learning have achieved fruitful results, but several issues require further investigation. First, the high-dimensional, nonlinear nature of hybrid vehicle energy management systems inevitably leads to redundancy in the deep reinforcement learning state space, which severely impairs the learning efficiency and real-time performance of deep reinforcement learning. Secondly, most energy management strategy research focuses primarily on fuel economy, while ignoring the impact on the vehicle's lithium-ion battery. Finally, the extensive hyperparameter tuning required for deep reinforcement learning has become a significant pain point for most researchers and engineers, and the multi-objective weighting factors in the reward function undoubtedly exacerbate this pain. Furthermore, in rapidly changing traffic scenarios, balancing multiple optimization objectives through weighting factors is difficult, resulting in reduced vehicle fuel economy.

[0098] In view of this, this embodiment makes up for the problem of over-estimation of the DDPG strategy through the following five improvements: (1) Using two critic networks, the two networks estimate the action value function, and during the training process, the smaller value of the two critic network estimates (Q value) is selected as the estimated value to avoid unlimited increase of the Q value; (2) Using a soft update method instead of directly copying the parameters of the current network as the parameters of the target network; (3) Using policy noise, Epsilon-Greedy exploration noise is used during exploration, which can appropriately expand the exploration range and increase the accuracy of estimation; (4) Using delayed learning, the frequency of critic network updates is greater than that of actor network updates; (5) Using gradient clipping, the gradient of actor parameter update is clipped to a certain range. The target engine equivalent torque is determined by the improved dual-delay deep deterministic policy gradient network and vehicle energy management learning strategy to control engine operation, thereby realizing energy management of parallel hybrid vehicles and improving accuracy, applicability and efficiency.

[0099] The parallel hybrid vehicle energy management method provided by the embodiment of the application relates to the technical field of vehicle engineering. The parallel hybrid vehicle energy management method provided by the embodiment of the application can be applied to a terminal, can be applied to a server, and can also be software running in the terminal or the server. In some embodiments, the terminal can be a smart phone, a tablet computer, a notebook computer, a desktop computer, a smart speaker, a smart watch, a vehicle-mounted terminal, and the like, but is not limited thereto. The server end can be configured as a stand-alone physical server, can be configured as a server cluster or a distributed system formed by multiple physical servers, can be configured as a cloud server providing basic cloud computing services such as cloud service, cloud database, cloud computing, cloud function, cloud storage, network service, cloud communication, middleware service, domain name service, security service, CDN, and big data and artificial intelligence platform, and can also be a node server in a blockchain network. The software can be an application that implements a parallel hybrid vehicle energy management method, and the like, but is not limited to the above forms.

[0100] The application can be used in many general or specific computer system environments or configurations. For example: personal computers, server computers, handheld devices or portable devices, tablet devices, multi-processor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, and the like. The application can be described in the general context of computer-executable instructions executed by a computer, such as a program module. Generally, program modules include routines, programs, objects, components, data structures, and the like that perform specific tasks or implement specific abstract data types. The application can also be practiced in a distributed computing environment in which tasks are performed by remote processing devices connected by a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media, including storage devices.

[0101] The embodiments of the application will be specifically explained below in combination with the drawings:

[0102] Figure 1 is an optional flowchart of the parallel hybrid vehicle energy management method provided by the embodiment of the application, Figure 1 The method in can include but is not limited to steps S101 to S109.

[0103] Step S101, acquiring an initial vehicle state signal;

[0104] Step S102, constructing a vehicle dynamics equation;

[0105] Step S103, constructing a parallel hybrid vehicle structure model according to the vehicle dynamics equation;

[0106] Step S104: constructing a battery thermal model and a battery life model based on the parallel hybrid vehicle structure model;

[0107] Step S105: construct a parameter-free reward function based on the battery thermal model and the battery life model;

[0108] Step S106: Generate reinforcement learning space state and reinforcement learning control action according to the parameter-free reward function;

[0109] Step S107: constructing a target double-delay deep deterministic policy gradient network;

[0110] Step S108: Constructing a vehicle energy management learning strategy based on the parallel hybrid vehicle structure model, the parameter-free reward function, the reinforcement learning space state, the reinforcement learning control action, the experience replay area, and the target double-delay deep deterministic policy gradient network;

[0111] Step S109: Determine the target engine equivalent torque using the vehicle energy management learning strategy based on the initial vehicle state signal. The target engine equivalent torque is used to control engine operation to achieve vehicle energy management.

[0112] Steps S101 to S109 shown in the embodiment of the present application implement energy management of parallel hybrid vehicles, improving accuracy, applicability and efficiency.

[0113] In step S101 of some embodiments, the initial vehicle state signal may be acquired through a vehicle sensor device, or may be acquired through other means, without limitation.

[0114] In some embodiments, in step S102, constructing the vehicle dynamics equation may include but is not limited to the following steps:

[0115] Calculate the vehicle's rolling resistance based on the vehicle's mass, gravitational acceleration, rolling resistance coefficient, and the vehicle's road slope;

[0116] Calculate the vehicle's slope resistance based on the vehicle's mass, gravitational acceleration, and the vehicle's road slope;

[0117] Calculate the vehicle's air resistance based on air density, vehicle frontal area, air resistance coefficient and vehicle speed;

[0118] Calculate the vehicle's acceleration resistance based on the rotational mass conversion coefficient, vehicle mass, and vehicle acceleration;

[0119] The vehicle dynamics equation is constructed based on the vehicle's rolling resistance, vehicle's slope resistance, vehicle's air resistance and vehicle's acceleration resistance.

[0120] In some embodiments, the rolling resistance of the vehicle can be calculated based on the vehicle mass, gravity acceleration, rolling resistance coefficient and vehicle road slope, wherein the calculation formula of the vehicle rolling resistance is: f =mgfcon(α), where F f is the rolling resistance of the vehicle, m is the vehicle mass, g is the acceleration of gravity, f is the rolling resistance coefficient, and α is the vehicle road slope. According to the vehicle mass, gravity acceleration and vehicle road slope, the vehicle slope resistance is calculated. The calculation formula of the vehicle slope resistance is: F i = mg sin(α), where F i is the vehicle's slope resistance. The vehicle's air resistance is calculated based on air density, vehicle frontal area, air resistance coefficient, and vehicle speed. The calculation formula for vehicle air resistance is: Where, F ω is the air resistance of the vehicle, ρ is the air density, A is the frontal area of ​​the vehicle, C D is the air resistance coefficient, and v is the vehicle speed. The vehicle acceleration resistance is calculated based on the rotational mass conversion coefficient, vehicle mass, and vehicle acceleration. The calculation formula for the vehicle acceleration resistance is: F j =δma, where F j is the vehicle acceleration resistance, and δ is the rotation mass conversion coefficient. Based on the vehicle rolling resistance, vehicle slope resistance, vehicle air resistance and vehicle acceleration resistance, the vehicle dynamics equation is constructed, where the expression of the vehicle dynamics equation is: F t =F f +F i +F ω +F j , where F t It is the driving force for the vehicle. It is understandable that the vehicle power system needs to provide the traction required for the vehicle to move. This traction can be calculated using the vehicle dynamics equation. The vehicle dynamics equation is the basis for building MATLAB / Simulink models and mainly describes the behavior of the vehicle under different operating conditions.

[0121] In some embodiments, in step S103, a parallel hybrid vehicle structure model may be constructed according to the vehicle dynamics equation. For example, the parallel hybrid vehicle structure model may be constructed according to the vehicle dynamics equation through MATLAB / Simulink. The parallel hybrid vehicle structure model may be as follows: Figure 2As shown in the figure, the engine and electric motor are connected in parallel, and a clutch can be used to engage and disengage the engine and wheels. At low speeds or idling, the clutch is disengaged, prioritizing the electric motor to drive the vehicle, reducing fuel consumption. When the vehicle requires high power output, the clutch is engaged, and the engine and electric motor work together to provide the required power. This design allows for flexible switching of power sources under different operating conditions to optimize energy utilization and enhance vehicle performance.

[0122] In some embodiments, in step S104, building a battery thermal model based on the parallel hybrid vehicle structure model may include but is not limited to the following steps:

[0123] Extract battery mass, battery specific heat capacity, and battery surface area from a parallel hybrid vehicle structural model;

[0124] A battery thermal model is constructed based on battery temperature, ambient temperature, battery mass, battery specific heat capacity, natural thermal convection constant, battery surface area, battery operating current, battery open circuit voltage, battery operating voltage and integration time.

[0125] In some embodiments, the battery mass, battery specific heat capacity, and battery surface area may be first extracted from the parallel hybrid vehicle structural model. Then, a battery thermal model may be constructed based on the battery temperature, ambient temperature, battery mass, battery specific heat capacity, natural thermal convection constant, battery surface area, battery operating current, battery open circuit voltage, battery operating voltage, and integration time. The mathematical expression of the battery thermal model is: Where T is the general temperature, t is the integration time, m D is the battery mass, c is the battery specific heat capacity, I is the battery operating current, OCV is the battery open circuit voltage, V is the battery operating voltage, h is the natural thermal convection constant, A b is the battery surface area, T emp is the battery temperature, T amb It is understandable that the subsequent parameter-free reward function includes an evaluation of the battery life SOH (battery state of health). The change of battery SOH is related to the battery temperature and usage status, so it is necessary to establish a battery thermal model for analysis.

[0126] In some embodiments, in step S104, building a battery life model based on the parallel hybrid vehicle structure model may include but is not limited to the following steps:

[0127] Extracting battery discharge rate from parallel hybrid vehicle structural model;

[0128] Calculate activation energy based on battery discharge rate;

[0129] Calculate battery throughput based on the exponential factor, battery discharge rate, activation energy, universal gas constant, and battery temperature;

[0130] Calculate the battery capacity loss percentage based on the exponential factor, battery discharge rate, activation energy, universal gas constant, battery temperature, battery throughput, and power law coefficient;

[0131] Calculate the equivalent number of cycles before the end of battery life based on battery throughput and battery capacity loss percentage;

[0132] A battery life model is constructed based on the battery capacity loss percentage, the equivalent number of cycles before the end of the battery life, the battery life state, the battery operating current and the integration time.

[0133] In some embodiments, the battery discharge rate can be extracted from the parallel hybrid vehicle structure model, and the activation energy can be calculated based on the battery discharge rate. The activation energy calculation formula is: E a (c r )=(31700-370.3·c r ), where E a is the activation energy, c r is the battery discharge rate. Then, the battery throughput is calculated based on the exponential factor, battery discharge rate, activation energy, universal gas constant, and battery temperature. The calculation formula for battery throughput is: Where A b h is the battery throughput, T emp is the battery temperature, B is the exponential factor, R is the universal gas constant, and z is the power law coefficient. The battery capacity loss percentage is calculated based on the exponential factor, battery discharge rate, activation energy, universal gas constant, battery temperature, battery throughput, and power law coefficient. The calculation formula for the battery capacity loss percentage is: Where C n is the battery capacity loss percentage. Then, based on the battery throughput and the battery capacity loss percentage, the equivalent number of cycles before the end of the battery life is calculated. The calculation formula for the equivalent number of cycles before the end of the battery life is: Where N is the equivalent number of cycles before the end of the battery life. Finally, a battery life model is constructed based on the battery capacity loss percentage, the equivalent number of cycles before the end of the battery life, the battery life state, the battery operating current and the integration time. The expression of the battery life model is: Where SOH is the battery life state, t is the integration time, and I is the battery operating current. It is understandable that the subsequent parameter-free reward function is provided with an evaluation of the battery life SOH (battery health state), and the change of the battery SOH is related to the temperature and usage status of the battery, so it is necessary to establish a battery life model for analysis. Furthermore, the relationship between the exponential factor and the battery discharge rate can include: when the exponential factor is 31630, the battery discharge rate is 0.5; when the exponential factor is 21681, the battery discharge rate is 2; when the exponential factor is 12934, the battery discharge rate is 6; when the exponential factor is 15512, the battery discharge rate is 10.

[0134] In some embodiments, in step S105, constructing a parameter-free reward function based on the battery thermal model and the battery life model may include but is not limited to the following steps:

[0135] Determine the battery health status based on the battery thermal model and battery life model;

[0136] Calculate the loss associated with the deviation between the current state of charge value and the reference value based on the battery state of charge and the battery state of charge reference value;

[0137] A parameter-free reward function is constructed based on the losses related to fuel consumption, battery state of charge, battery health state, and the deviation between the current state of charge and the reference value.

[0138] In some embodiments, the optimization goal of the hybrid vehicle energy management strategy is to improve fuel economy, keep the battery SOC within a reasonable range to avoid overcharging and over-discharging, and suppress the battery life degradation. To achieve this optimization goal, the battery health state can be determined based on the battery thermal model and the battery life model. It is understandable that the change in battery health state is related to the temperature and usage status of the battery, and can be analyzed based on the battery thermal model and the battery life model. Then, based on the battery state of charge and the battery state of charge reference value, the loss related to the deviation between the current state of charge value and the reference value is calculated, where the calculation formula for the loss related to the deviation between the current state of charge value and the reference value is: C soc =(SOC ref -SOC) 2 , where C soc The loss related to the deviation between the current state of charge and the reference value, SOC ref is the reference value of battery state of charge, SOC is the battery state of charge. Then, according to the fuel consumption, battery state of charge, battery health status and the loss related to the deviation between the current state of charge and the reference value, a parameter-free reward function is constructed, where the expression of the parameter-free reward function is: R = -[m f ·SOC+C soc·(1-SOC)]·(1-SOH), where R is a parameter-free reward function, m f is the fuel consumption (kg), SOH is the battery health state (State of Health). It can be understood that the three parts of the parameter-free reward function {m f ·SOC, C soc (1-SOC) and 1-SOH are on the same order of magnitude. When the battery is fully charged, the parameter-free reward function is primarily driven by fuel consumption, whereas when the battery SOC is low, the parameter-free reward function is primarily driven by the battery SOC. Therefore, this parameter-free reward function does not require additional parameter tuning.

[0139] In some embodiments, in step S106, generating the reinforcement learning space state and the reinforcement learning control action according to the parameter-free reward function may include but is not limited to the following steps:

[0140] According to the parameter-free reward function, the vehicle state is divided into direct states, where the direct states include vehicle torque demand, power demand, vehicle speed, vehicle acceleration or fuel consumption;

[0141] Refining the direct state;

[0142] Generate reinforcement learning space state based on the direct state after refinement;

[0143] Calculating an initial engine equivalent torque based on the current engine output torque, the maximum engine output torque, and the minimum engine output torque;

[0144] The initial engine equivalent torque is used as the reinforcement learning control action.

[0145] In some embodiments, the vehicle state can be first divided according to the parameter-free reward function to obtain direct states. Direct states usually directly affect the vehicle's optimization objectives (such as fuel economy, battery SOC and battery SOH), wherein direct states include vehicle torque demand, power demand, vehicle speed, vehicle acceleration or fuel consumption, and battery SOC, battery SOH and battery current directly affect the battery state and life. Furthermore, after the division, indirect states are also obtained. Indirect states do not directly affect the optimization objectives, mainly including clutch state or transmission gear. It can be understood that the design of the state space needs to be closely centered around the vehicle optimization objectives and the parameter-free reward function. According to the parameter-free reward function, the vehicle state is divided into direct states and indirect states. The direct state directly affects the vehicle's optimization objectives, and the indirect state affects the vehicle's optimization objectives in other ways.

[0146] Then the direct state is refined, and the reinforcement learning space state is generated according to the refined direct state. Exemplarily, the direct state can be introduced into the reinforcement learning state space, and states that are correlated or have the same effect are screened, combined and deleted to complete state refinement. For example, vehicle speed and vehicle acceleration are both kinetic energy indicators of the vehicle and the acceleration can be calculated by the vehicle speed, so the reinforcement learning state space only retains the vehicle speed; the vehicle demand torque, the vehicle demand power and the vehicle speed are correlated with each other, and the third state can be calculated by any two states, so the reinforcement learning state space eliminates the vehicle power demand; the battery SOC, the battery SOH and the battery current are closely related and can be calculated by the current, which are repeated states, so the reinforcement learning state space only retains the battery current. The reinforcement learning state space after state refinement is S={V spd ,Trq dmd ,m f ,I batt}, wherein V spd represents the vehicle speed, Trq dmd indicates the vehicle torque demand, I batt represents the battery current, and m f is the fuel consumption. It can be understood that the three elements of reinforcement learning are state space, action space and reward function, and the state refinement of the embodiment is an improvement to the state space of the reinforcement learning algorithm.

[0147] According to the current output torque of the engine, the maximum output torque of the engine and the minimum output torque of the engine, the initial engine equivalent torque is calculated, wherein the calculation formula of the initial engine equivalent torque is: In the formula, is the initial engine equivalent torque, T ice is the current output torque of the engine, is the maximum output torque of the engine, is the minimum output torque of the engine. Finally, the initial engine equivalent torque is taken as the reinforcement learning control action to determine the deep reinforcement learning action signal. It can be understood that the deep reinforcement learning algorithm outputs a control action, which can control the engine output torque through the control action, and then drive the vehicle to travel.

[0148] In some embodiments, in step S107, constructing the target double-delay deep deterministic policy gradient network can include but is not limited to the following steps:

[0149] Obtaining a vehicle standard working condition data set, the vehicle standard working condition data set including parameters and observation values related to energy management in vehicle standard working condition simulation driving;

[0150] According to the value network model and the policy network model, an initial double-delay deep deterministic policy gradient network is constructed;

[0151] The vehicle standard operating condition dataset is input into the initial double-delay deep deterministic policy gradient network to train the initial double-delay deep deterministic policy gradient network and obtain the target double-delay deep deterministic policy gradient network.

[0152] In some embodiments, a vehicle standard operating condition dataset can be obtained first, wherein the vehicle standard operating condition dataset includes parameters and observations related to energy management during vehicle standard operating condition simulation driving. Then, based on the value network model Critic and the policy network model Actor, an initial double-delay deep deterministic policy gradient network is constructed. The initial double-delay deep deterministic policy gradient network architecture is as follows: Figure 3 As shown in the figure, deep neural networks can be used to build the Critic network and the Actor network respectively, and jointly construct the initial double-delay deep deterministic policy gradient network, namely the Actor-Critic network, and initialize the Actor-Critic network parameters and normalize the state data. Figure 3 It consists of an initial actor network, two initial critic networks, a target actor network, and two target critic networks. In the initial double-delayed deep deterministic policy gradient network, the state S is input to the initial actor network and the initial critic network respectively. The initial actor network outputs an action A, which is input to the initial critic network. The two initial critic networks each output the corresponding Q value, and the TD error is calculated to update the parameters of the initial critic network. Simultaneously, the target critic network is updated based on the output of the initial critic network, and the target actor network is updated based on the output of the initial actor network. Tanh and ReLU are two commonly used activation functions in neural networks. It can be understood that the actor-critic algorithm is a reinforcement learning method involving a policy network (actor) and a value network (critic). TD error (Temporal Difference Error) is a metric used to evaluate the accuracy of a prediction model and is widely used in reinforcement learning algorithms. TD error measures model performance by comparing the difference between the model's predicted value and the actual value. Finally, the vehicle standard working condition data set is input into the initial double-delay deep deterministic policy gradient network to train the initial double-delay deep deterministic policy gradient network and obtain the target double-delay deep deterministic policy gradient network. For example, the New European Driving Cycle (NEDC) working condition can be selected as the training set to train the initial double-delay deep deterministic policy gradient network. The selected training working condition speed curve is as follows: Figure 4It can be understood that the working condition data (ie, the training set) is a 2-dimensional vector, the first dimension is time, and the second dimension is working condition speed.

[0153] In some embodiments, in step S108, a vehicle energy management learning strategy can be constructed based on the parallel hybrid vehicle structure model, the parameter-free reward function, the reinforcement learning space state, the reinforcement learning control action, the experience replay area, and the target double-delay deep deterministic policy gradient network. The vehicle energy management learning strategy architecture is as follows: Figure 5 As shown, the parallel hybrid vehicle structural model outputs the initial state to the state refinement module to obtain the refined state s, calculates the reward value through the parameter-free reward function, and calculates the state at the next moment through reinforcement learning. The reinforcement learning space state, reinforcement learning control action a, reward value r and the state at the next moment are all stored in the experience replay area, and the experience replay area is randomly sampled to obtain samples. The samples are input into the target double-delay deep deterministic policy gradient network, and the network outputs the control action as the engine equivalent torque to control the engine operation.

[0154] In some embodiments, in step S109, the target engine equivalent torque is determined based on the initial vehicle state signal using the vehicle energy management learning strategy, which may include but is not limited to the following steps:

[0155] Initialize the first initial value network weight, the second initial value network weight, the initial policy network weight, and the model training parameters. The model training parameters include the experience replay area storage information, the learning rate, the random noise, the discount factor, the batch gradient descent sample number, and the preset maximum number of iterations.

[0156] Calculating a first target value network weight based on the first initial value network weight;

[0157] Calculating a second target value network weight based on the second initial value network weight;

[0158] Calculate the target strategy network weight based on the initial strategy network weight;

[0159] Refining the initial vehicle state signal to obtain the current state;

[0160] Generate current control action based on current state, random noise and control strategy;

[0161] Execute the current control action to obtain the reward value and the next state;

[0162] The current state, current control action, reward value and next moment state are stored in the experience playback area;

[0163] Randomly sample the experience playback area to obtain several samples;

[0164] Generate the next moment control action based on several samples;

[0165] Calculate the timing difference error based on the next moment state and the next moment control action;

[0166] Calculate the minimization loss based on the time series difference error;

[0167] Based on the model training parameters, the value network parameters are updated by minimizing the loss;

[0168] Based on the model training parameters, the policy network parameters are updated using gradient descent;

[0169] updating the first target value network weight according to the value network parameter, the first initial value network weight, and the first target value network weight;

[0170] updating the second target value network weight according to the value network parameter, the second initial value network weight and the second target value network weight;

[0171] According to the strategy network parameters, the initial strategy network weights and the target strategy network weights, the target strategy network weights are updated until the preset maximum number of iterations is reached to obtain the target engine equivalent torque.

[0172] In some embodiments, the target engine equivalent torque can be determined based on the initial vehicle state signal using a vehicle energy management learning strategy, wherein the target engine equivalent torque is used to control engine operation to achieve vehicle energy management. The first initial value network weight can be initialized first. Second initial value network weight Initial policy network weight θ μ And model training parameters, where the model training parameters include experience replay area storage information, learning rate τ, random noise ε, discount factor γ, batch gradient descent sample number M and preset maximum number of iterations T. Then, based on the first initial value network weight, the first target value network weight is calculated Calculate the second target value network weight based on the second initial value network weight According to the initial policy network weight, calculate the target policy network weight θ μ′ ←θ μ , where ← represents assignment.

[0173] Then iterate from t=1:T (i.e., iterate T times), refine the initial vehicle state signal, and obtain the current state s t ←S t ={V spd (t),Trq dmd (t),m f (t),Ibatt (t)}, where s t is the state at time t, that is, the current state, S t is the general state space, V spd Indicates vehicle speed, Trq dmd Refers to the vehicle torque demand, I batt Represents the battery current, m f is the fuel consumption. According to the current state, random noise and control strategy, the current control action a is generated. t ←π(a t |s t )=μ(s t )+ε, where a t is the current control action, μ is the actor network, and ε is the random noise. Execute the current control action and get the reward value r t and the next moment state s t Then the current state, current control action, reward value and next moment state are stored in the experience playback area D, and the experience playback area is randomly sampled to obtain several samples. For example, M samples {s j ,a j ,r j ,s j ′},j=1,2,...,M, where s j is the current state of the jth sample, a j is the current control action of the jth sample, r j is the return value of the jth sample, s j ' is the next moment state of the jth sample. Based on several samples, the next moment control action a' is generated. Based on the next moment state and the next moment control action, the timing difference error y is calculated. j , where if s′ is a terminal state, then y j =r j Otherwise, y j =r j +γmin j＝1,2 Q j′ (s′, a′). According to the time series difference error, the minimum loss is calculated, where the calculation formula for minimizing the loss is: Where, L ki To minimize the loss, M is the total number of samples. According to the model training parameters, the value network parameters are updated by minimizing the loss. After the Critic network (value network parameters) are updated C times, the policy network parameters (Actor network) are updated using gradient descent according to the model training parameters. The gradient calculation formula is: Where, Indicates that for θ μGradient calculation, J is the loss function, J = -R, Represents the gradient calculation for a, Q(s,a) is the Critic network value function. Finally, based on the value network parameters, the first initial value network weight and the first target value network weight, the first target value network weight is updated. Update the second target value network weight according to the value network parameter, the second initial value network weight and the second target value network weight According to the policy network parameters, the initial policy network weights and the target policy network weights, update the target policy network weights θ μ′ =τθ μ +(1-τ)θ μ′ , until the preset maximum number of iterations is reached, the target engine equivalent torque is obtained to control the engine operation.

[0174] In some embodiments, performance experiments can be conducted to obtain the parameters and observations that affect energy management when the vehicle is driving in a more complex and complex working condition with more road elements, and based on the parameters that affect energy management in the complex working condition, the optimization performance of the trained dual-delay deep deterministic policy gradient network can be verified. For example, multiple standard working conditions such as the Highway Fuel Economy Test (HWFET), the Urban Dynamometer Driving Schedule (UDDS), the World Light Vehicle Test Procedure Class 3 (WLTP Class 3) and the China Light-duty Vehicle Test Cycle (CLTC) can be combined to form a complex working condition with more complex road conditions and more road elements, and then imported into the vehicle driver model to test the dual-delay deep deterministic policy gradient network with no parameter return. Among them, the speed curve of the complex working condition is as follows: Figure 6 shown.

[0175] In some embodiments, this embodiment proposes a parameter-free reward function for an energy management strategy based on TD3 (double-delay deep deterministic policy gradient algorithm), and uses battery SOC as a weight parameter. The parameter-free reward function does not require experts to perform parameter tuning, and can implement the energy management strategy and perform adaptive adjustment according to the vehicle state, thereby improving the optimality and adaptability of the energy management strategy. This embodiment uses state space (vehicle speed, torque demand, fuel consumption and battery current), action space (engine output torque), parameter-free reward function, and state refinement to optimize the management of vehicle status. The method of this embodiment improves data-driven and logical consistency, reduces the influence of human subjective factors on the design process, and thus improves the effectiveness and reliability of the design.

[0176] The beneficial effects of implementing the embodiments of the present invention include: first, acquiring an initial vehicle state signal, then constructing a vehicle dynamics equation, constructing a parallel hybrid vehicle structural model based on the vehicle dynamics equation, constructing a battery thermal model and a battery life model based on the parallel hybrid vehicle structural model, constructing a parameter-free reward function based on the battery thermal model and the battery life model, then generating a reinforcement learning spatial state and reinforcement learning control action based on the parameter-free reward function, constructing a target dual-delay deep deterministic policy gradient network, and constructing a vehicle energy management learning strategy. Finally, based on the initial vehicle state signal, the vehicle energy management learning strategy is used to determine a target engine equivalent torque, thereby controlling engine operation to achieve parallel hybrid vehicle energy management, thereby improving accuracy, applicability, and efficiency. Furthermore, this embodiment uses a parameter-free reward dual-delay deep deterministic policy gradient algorithm combined with state space refinement technology to address the problems of weight factors being difficult to balance multiple optimization objectives and the heavy parameter adjustment task, as well as the problem of deep reinforcement learning state redundancy, thereby improving the real-time, adaptability, and optimality of the energy management strategy. This embodiment uses state refinement technology to optimize the state space of deep reinforcement learning strategies to address the issue of redundant state information and improve the learning efficiency of a deep deterministic policy gradient method based on state refinement and parameter-free double-delayed rewards. This embodiment not only optimizes vehicle fuel consumption and maintains battery SOC within a reasonable range during operation, but also prevents battery life degradation.

[0177] like Figure 7 As shown, an embodiment of the present invention further provides an energy management device for a parallel hybrid vehicle, comprising:

[0178] The first module 801 is used to obtain an initial vehicle status signal;

[0179] The second module 802 is used to construct the vehicle dynamics equation;

[0180] The third module 803 is used to construct a parallel hybrid vehicle structure model based on the vehicle dynamics equation;

[0181] The fourth module 804 is used to construct a battery thermal model and a battery life model based on the parallel hybrid vehicle structure model;

[0182] The fifth module 805 is used to construct a parameter-free reward function based on the battery thermal model and the battery life model;

[0183] The sixth module 806 is used to generate a reinforcement learning space state and a reinforcement learning control action according to the parameter-free reward function;

[0184] The seventh module 807 is used to construct a target dual-delay deep deterministic policy gradient network;

[0185] An eighth module 808 is configured to construct a vehicle energy management learning strategy based on a parallel hybrid vehicle structural model, a parameter-free reward function, a reinforcement learning spatial state, a reinforcement learning control action, an experience replay area, and a target dual-delay deep deterministic policy gradient network;

[0186] The ninth module 809 is used to determine the target engine equivalent torque based on the initial vehicle state signal using the vehicle energy management learning strategy. The target engine equivalent torque is used to control the engine operation to achieve vehicle energy management.

[0187] The contents of the above method embodiments are all applicable to the present device embodiments. The functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0188] like Figure 8 As shown, an embodiment of the present invention further provides a computer device, including:

[0189] at least one processor 901;

[0190] At least one memory 902, configured to store at least one program;

[0191] When at least one program is executed by at least one processor, the at least one processor implements Figure 1 The method shown.

[0192] The contents of the above method embodiments are all applicable to the present device embodiments. The functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0193] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but are not intended to limit the scope of the present invention. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and essence of the present invention should be within the scope of the present invention.

Claims

1. A parallel hybrid vehicle energy management method, characterized in that: The following steps are involved: Obtaining an initial vehicle status signal; Construct vehicle dynamics equations; According to the vehicle dynamics equation, a parallel hybrid vehicle structural model is constructed; Constructing a battery thermal model and a battery life model based on the parallel hybrid vehicle structural model; Constructing a parameter-free reward function according to the battery thermal model and the battery life model; generating a reinforcement learning space state and a reinforcement learning control action according to the parameter-free reward function; Construct a target dual-delay deep deterministic policy gradient network; Constructing a vehicle energy management learning strategy based on the parallel hybrid vehicle structural model, the parameter-free reward function, the reinforcement learning space state, the reinforcement learning control action, the experience replay area, and the target double-delay deep deterministic policy gradient network; According to the initial vehicle state signal, the target engine equivalent torque is determined using the vehicle energy management learning strategy. The target engine equivalent torque is used to control engine operation to achieve vehicle energy management.

2. The method according to claim 1, characterized in that The constructing of the vehicle dynamics equation includes: Calculate the vehicle's rolling resistance based on the vehicle's mass, gravitational acceleration, rolling resistance coefficient, and the vehicle's road slope; Calculating the vehicle's driving slope resistance according to the vehicle mass, the gravitational acceleration, and the vehicle's road slope; Calculate the vehicle's air resistance based on air density, vehicle frontal area, air resistance coefficient and vehicle speed; Calculating the vehicle acceleration resistance based on the rotational mass conversion coefficient, the vehicle mass, and the vehicle acceleration; The vehicle dynamics equation is constructed according to the vehicle rolling resistance, the vehicle slope resistance, the vehicle air resistance and the vehicle acceleration resistance.

3. The method according to claim 1, characterized in that The step of constructing a battery thermal model based on the parallel hybrid vehicle structural model includes: extracting battery mass, battery specific heat capacity, and battery surface area from the parallel hybrid vehicle structural model; The battery thermal model is constructed according to the battery temperature, the ambient temperature, the battery mass, the battery specific heat capacity, the natural thermal convection constant, the battery surface area, the battery operating current, the battery open circuit voltage, the battery operating voltage and the integration time.

4. The method according to claim 1, wherein The step of constructing a battery life model based on the parallel hybrid vehicle structure model includes: extracting a battery discharge rate from the parallel hybrid vehicle structural model; Calculating activation energy according to the battery discharge rate; Calculating a battery throughput based on an exponential factor, the battery discharge rate, the activation energy, a universal gas constant, and a battery temperature; Calculating a battery capacity loss percentage based on the exponential factor, the battery discharge rate, the activation energy, the universal gas constant, the battery temperature, the battery throughput, and a power law coefficient; Calculating the equivalent number of cycles before the end of the battery life according to the battery throughput and the battery capacity loss percentage; The battery life model is constructed based on the battery capacity loss percentage, the equivalent number of cycles before the end of the battery life, the battery life state, the battery operating current and the integration time.

5. The method according to claim 1, wherein The constructing of a parameter-free reward function according to the battery thermal model and the battery life model includes: determining a battery health state according to the battery thermal model and the battery life model; Calculate the loss associated with the deviation between the current state of charge value and the reference value based on the battery state of charge and the battery state of charge reference value; The parameter-free reward function is constructed according to the fuel consumption, the battery state of charge, the battery state of health, and the loss associated with the deviation between the current value of the state of charge and a reference value.

6. The method according to claim 1, characterized in that Generating the reinforcement learning space state and the reinforcement learning control action according to the parameter-free reward function includes: Dividing the vehicle state according to the parameter-free reward function to obtain a direct state, wherein the direct state includes vehicle torque demand, power demand, vehicle speed, vehicle acceleration or fuel consumption; performing a refining process on the direct state; Generating the reinforcement learning space state according to the refined direct state; Calculating an initial engine equivalent torque based on the current engine output torque, the maximum engine output torque, and the minimum engine output torque; The initial engine equivalent torque is used as the reinforcement learning control action.

7. The method according to claim 1, characterized in that The target dual-delay deep deterministic policy gradient network is constructed, including: Acquire a vehicle standard operating condition data set, wherein the vehicle standard operating condition data set includes parameters and observation values ​​related to energy management during simulated driving of the vehicle standard operating condition; Based on the value network model and the policy network model, an initial double-delay deep deterministic policy gradient network is constructed; The vehicle standard operating condition data set is input into the initial double-delay deep deterministic policy gradient network to train the initial double-delay deep deterministic policy gradient network to obtain the target double-delay deep deterministic policy gradient network.

8. The method according to claim 1, characterized in that The determining of the target engine equivalent torque by using the vehicle energy management learning strategy according to the initial vehicle state signal includes: Initializing the first initial value network weight, the second initial value network weight, the initial policy network weight, and model training parameters, wherein the model training parameters include experience replay area storage information, learning rate, random noise, discount factor, batch gradient descent sample number, and preset maximum number of iterations; Calculating a first target value network weight based on the first initial value network weight; Calculating a second target value network weight based on the second initial value network weight; Calculating a target policy network weight based on the initial policy network weight; Refining the initial vehicle state signal to obtain a current state; generating a current control action according to the current state, the random noise and the control strategy; Execute the current control action to obtain the reward value and the next state; Storing the current state, the current control action, the reward value, and the next moment state in the experience playback area; Randomly sampling the experience playback area to obtain a number of samples; generating a control action at the next moment according to the plurality of samples; Calculating a timing difference error according to the state at the next moment and the control action at the next moment; Calculating a minimized loss based on the timing difference error; Based on the model training parameters, updating the value network parameters using the minimized loss; Based on the model training parameters, update the policy network parameters using gradient descent; updating the first target value network weight according to the value network parameter, the first initial value network weight, and the first target value network weight; updating the second target value network weight according to the value network parameter, the second initial value network weight, and the second target value network weight; According to the strategy network parameters, the initial strategy network weight and the target strategy network weight, the target strategy network weight is updated until the preset maximum number of iterations is reached to obtain the target engine equivalent torque.

9. A parallel hybrid vehicle energy management device, characterized in that: include: The first module is used to obtain an initial vehicle status signal; The second module is used to construct the vehicle dynamics equations; A third module is used to construct a parallel hybrid vehicle structural model based on the vehicle dynamics equation; A fourth module is used to construct a battery thermal model and a battery life model based on the parallel hybrid vehicle structural model; A fifth module is configured to construct a parameter-free reward function based on the battery thermal model and the battery life model; A sixth module is used to generate a reinforcement learning space state and a reinforcement learning control action according to the parameter-free reward function; The seventh module is used to build a target dual-delay deep deterministic policy gradient network; an eighth module for constructing a vehicle energy management learning strategy based on the parallel hybrid vehicle structural model, the parameter-free reward function, the reinforcement learning space state, the reinforcement learning control action, the experience replay area, and the target dual-delay deep deterministic policy gradient network; The ninth module is used to determine the target engine equivalent torque based on the initial vehicle state signal using the vehicle energy management learning strategy, and the target engine equivalent torque is used to control the engine operation to achieve vehicle energy management.

10. A computer device, characterized in that: include: at least one processor; at least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the method according to any one of claims 1 to 8.

Images