Parallel hybrid power vehicle energy management method and device

Abstract

The invention discloses an energy management method and device for a parallel hybrid power vehicle. The method comprises the steps that an initial vehicle state signal is obtained; constructing a vehicle kinetic equation; according to the vehicle kinetic equation, a parallel hybrid power vehicle structure model is constructed; constructing a battery thermal model and a battery life model according to the parallel hybrid power vehicle structure model; constructing a parameter-free return function according to the battery thermal model and the battery life model; generating a reinforcement learning space state and a reinforcement learning control action according to the parameter-free return function; constructing a target double-delay depth deterministic strategy gradient network; constructing a vehicle energy management learning strategy; according to the initial vehicle state signal, target engine equivalent torque is determined through a vehicle energy management learning strategy, and the target engine equivalent torque is used for controlling operation of an engine. Energy management of the parallel hybrid power vehicle is achieved, and accuracy, applicability and efficiency are improved. The method can be widely applied to the technical field of vehicle engineering.

Description

Technical Field

[0001] The present invention relates to the field of vehicle engineering technology, and in particular to an energy management method and device for a parallel hybrid vehicle. Background Art

[0002] The automotive industry is one of the main areas of greenhouse gas emissions in the world. The development of energy-saving and new energy vehicles is an important measure to promote green development. Among environmentally friendly vehicles, hybrid electric vehicles (HEVs) have a longer driving range than pure electric vehicles, and lower fuel consumption than traditional fuel vehicles, making them more environmentally friendly. However, the energy management system of hybrid electric vehicles is much more complex than that of traditional fuel vehicles and pure electric vehicles. Traditional rule-based energy management methods mainly rely on expert experience and experimental calibration, but the actual working conditions are complex, resulting in low accuracy. Optimization-based energy management methods optimize energy by predicting road conditions in advance, but they rely too much on models with superior performance and have low applicability. Learning-based energy management methods mainly rely on reinforcement learning strategies, but they are difficult to deal with high-dimensional nonlinear features and the hyperparameter adjustment process is cumbersome and inefficient.

[0003] In summary, the technical problems existing in the relevant technologies need to be improved. Summary of the invention

[0004] The embodiments of the present invention provide a method and device for energy management of a parallel hybrid vehicle, which effectively improves accuracy, applicability and efficiency.

[0005] In one aspect, an embodiment of the present invention provides a method for energy management of a parallel hybrid vehicle, comprising the following steps:

[0006] Obtaining an initial vehicle status signal;

[0007] Construct vehicle dynamics equations;

[0008] According to the vehicle dynamics equation, a parallel hybrid vehicle structural model is constructed;

[0009] Constructing a battery thermal model and a battery life model according to the parallel hybrid vehicle structural model;

[0010] Constructing a parameter-free reward function according to the battery thermal model and the battery life model;

[0011] Generate a reinforcement learning space state and a reinforcement learning control action according to the parameter-free reward function;

[0012] Construct a target dual-delay deep deterministic policy gradient network;

[0013] Constructing a vehicle energy management learning strategy according to the parallel hybrid vehicle structural model, the parameter-free reward function, the reinforcement learning space state, the reinforcement learning control action, the experience replay area, and the target dual-delay deep deterministic policy gradient network;

[0014] According to the initial vehicle state signal, the target engine equivalent torque is determined using the vehicle energy management learning strategy, and the target engine equivalent torque is used to control the engine operation to achieve vehicle energy management.

[0015] In some embodiments, constructing the vehicle dynamics equations includes:

[0016] Calculate the vehicle's rolling resistance based on the vehicle's mass, gravitational acceleration, rolling resistance coefficient and the vehicle's road slope;

[0017] Calculating the vehicle driving slope resistance according to the vehicle mass, the gravitational acceleration and the vehicle road slope;

[0018] Calculate the vehicle's air resistance based on air density, vehicle's frontal area, air resistance coefficient and vehicle's speed;

[0019] Calculating the vehicle's driving acceleration resistance according to the rotational mass conversion coefficient, the vehicle's mass and the vehicle's driving acceleration;

[0020] The vehicle dynamics equation is constructed according to the vehicle running rolling resistance, the vehicle running slope resistance, the vehicle running air resistance and the vehicle running acceleration resistance.

[0021] In some embodiments, constructing a battery thermal model according to the parallel hybrid vehicle structural model includes:

[0022] extracting battery mass, battery specific heat capacity, and battery surface area from the parallel hybrid vehicle structural model;

[0023] The battery thermal model is constructed according to the battery temperature, the ambient temperature, the battery mass, the battery specific heat capacity, the natural thermal convection constant, the battery surface area, the battery operating current, the battery open circuit voltage, the battery operating voltage and the integration time.

[0024] In some embodiments, constructing a battery life model according to the parallel hybrid vehicle structure model includes:

[0025] extracting a battery discharge rate from the parallel hybrid vehicle structural model;

[0026] Calculating activation energy according to the battery discharge rate;

[0027] Calculating a battery throughput based on an exponential factor, the battery discharge rate, the activation energy, a universal gas constant, and a battery temperature;

[0028] Calculating a battery capacity loss percentage according to the exponential factor, the battery discharge rate, the activation energy, the universal gas constant, the battery temperature, the battery throughput, and a power law coefficient;

[0029] Calculating the equivalent number of cycles before the end of battery life according to the battery throughput and the battery capacity loss percentage;

[0030] The battery life model is constructed according to the battery capacity loss percentage, the equivalent number of cycles before the end of the battery life, the battery life state, the battery operating current and the integration time.

[0031] In some embodiments, constructing a parameter-free reward function according to the battery thermal model and the battery life model includes:

[0032] Determining a battery health state according to the battery thermal model and the battery life model;

[0033] Calculate the loss associated with the deviation between the current state of charge value and the reference value according to the battery state of charge and the battery state of charge reference value;

[0034] The parameter-free reward function is constructed according to the fuel consumption, the battery state of charge, the battery health state and the loss related to the deviation between the current value of the state of charge and the reference value.

[0035] In some embodiments, generating a reinforcement learning space state and a reinforcement learning control action according to the parameter-free reward function includes:

[0036] According to the parameter-free reward function, the vehicle state is divided to obtain a direct state, wherein the direct state includes a vehicle torque demand, a power demand, a vehicle speed, a vehicle acceleration or a fuel consumption;

[0037] performing a refining process on the direct state;

[0038] Generating the reinforcement learning space state according to the direct state after the refinement process;

[0039] Calculating an initial engine equivalent torque according to the current engine output torque, the maximum engine output torque and the minimum engine output torque;

[0040] The initial engine equivalent torque is used as the reinforcement learning control action.

[0041] In some embodiments, constructing a target double-delay deep deterministic policy gradient network includes:

[0042] Acquire a vehicle standard operating condition data set, wherein the vehicle standard operating condition data set includes parameters and observation values ​​related to energy management during vehicle standard operating condition simulation driving;

[0043] According to the value network model and the policy network model, an initial double-delayed deep deterministic policy gradient network is constructed;

[0044] The vehicle standard operating condition data set is input into the initial double-delay deep deterministic policy gradient network so that the initial double-delay deep deterministic policy gradient network is trained to obtain the target double-delay deep deterministic policy gradient network.

[0045] In some embodiments, determining the target engine equivalent torque using the vehicle energy management learning strategy according to the initial vehicle state signal includes:

[0046] Initialize the first initial value network weight, the second initial value network weight, the initial strategy network weight and model training parameters, wherein the model training parameters include experience replay area storage information, learning rate, random noise, discount factor, batch gradient descent sample number and preset maximum number of iterations;

[0047] Calculating a first target value network weight according to the first initial value network weight;

[0048] Calculating a second target value network weight according to the second initial value network weight;

[0049] Calculating target strategy network weights according to the initial strategy network weights;

[0050] Refining the initial vehicle state signal to obtain a current state;

[0051] Generate a current control action according to the current state, the random noise and the control strategy;

[0052] Execute the current control action to obtain the reward value and the state at the next moment;

[0053] The current state, the current control action, the reward value and the next moment state are stored in the experience playback area;

[0054] Randomly sampling the experience playback area to obtain a number of samples;

[0055] Generate a control action at the next moment according to the plurality of samples;

[0056] Calculating a timing difference error according to the state at the next moment and the control action at the next moment;

[0057] Calculate the minimization loss according to the timing difference error;

[0058] According to the model training parameters, updating the value network parameters using the minimized loss;

[0059] Based on the model training parameters, updating the policy network parameters using gradient descent;

[0060] updating the first target value network weight according to the value network parameter, the first initial value network weight and the first target value network weight;

[0061] updating the second target value network weight according to the value network parameter, the second initial value network weight and the second target value network weight;

[0062] According to the strategy network parameters, the initial strategy network weights and the target strategy network weights, the target strategy network weights are updated until the preset maximum number of iterations is reached to obtain the target engine equivalent torque.

[0063] On the other hand, an embodiment of the present invention provides an energy management device for a parallel hybrid vehicle, comprising:

[0064] The first module is used to obtain an initial vehicle status signal;

[0065] The second module is used to construct the vehicle dynamics equations;

[0066] A third module is used to construct a parallel hybrid vehicle structure model according to the vehicle dynamics equation;

[0067] A fourth module is used to construct a battery thermal model and a battery life model according to the parallel hybrid vehicle structural model;

[0068] A fifth module is used to construct a parameter-free reward function according to the battery thermal model and the battery life model;

[0069] A sixth module, used to generate a reinforcement learning space state and a reinforcement learning control action according to the parameter-free reward function;

[0070] The seventh module is used to build a target dual-delay deep deterministic policy gradient network;

[0071] An eighth module is used to construct a vehicle energy management learning strategy according to the parallel hybrid vehicle structure model, the parameter-free reward function, the reinforcement learning space state, the reinforcement learning control action, the experience playback area and the target dual-delay deep deterministic policy gradient network;

[0072] The ninth module is used to determine the target engine equivalent torque according to the initial vehicle state signal using the vehicle energy management learning strategy, and the target engine equivalent torque is used to control the engine operation to achieve vehicle energy management.

[0073] In another aspect, an embodiment of the present invention provides a computer device, comprising:

[0074] at least one processor;

[0075] at least one memory for storing at least one program;

[0076] When the at least one program is executed by the at least one processor, the at least one processor implements the described method.

[0077] The beneficial effects of the present invention are as follows:

[0078] The embodiment of the present invention first obtains an initial vehicle state signal, then constructs a vehicle dynamics equation, and constructs a parallel hybrid vehicle structural model based on the vehicle dynamics equation. Based on the parallel hybrid vehicle structural model, a battery thermal model and a battery life model are constructed. Based on the battery thermal model and the battery life model, a parameter-free reward function is constructed, and then based on the parameter-free reward function, a reinforcement learning space state and a reinforcement learning control action are generated, a target dual-delay deep deterministic policy gradient network is constructed, and a vehicle energy management learning strategy is constructed. Finally, based on the initial vehicle state signal, the target engine equivalent torque is determined using the vehicle energy management learning strategy, so that the engine operation can be controlled by the target engine equivalent torque to achieve parallel hybrid vehicle energy management, thereby improving accuracy, applicability and efficiency.

[0079] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the description and the drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0080] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0081] Figure 1 This is a flow chart of a parallel hybrid vehicle energy management method according to an embodiment of the present invention;

[0082] Figure 2A schematic diagram of a parallel hybrid vehicle structure model according to an embodiment of the present invention;

[0083] Figure 3 A schematic diagram of an initial double-delayed deep deterministic policy gradient network architecture according to an embodiment of the present invention;

[0084] Figure 4 A schematic diagram of a training condition speed curve according to an embodiment of the present invention;

[0085] Figure 5 A schematic diagram of a vehicle energy management learning strategy architecture according to an embodiment of the present invention;

[0086] Figure 6 A schematic diagram of a composite operating condition speed curve according to an embodiment of the present invention;

[0087] Figure 7 This is a schematic structural diagram of an energy management device for a parallel hybrid vehicle according to an embodiment of the present invention;

[0088] Figure 8 The figure is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0089] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is further described in detail below in conjunction with the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the embodiments of the present application. They are only examples of devices and methods consistent with some aspects of the embodiments of the present application as detailed in the attached claims.

[0090] It is understood that the terms "first", "second", etc. used in this application can be used to describe various concepts in this article, but unless otherwise specified, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another concept. For example, without departing from the scope of the embodiment of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the words "if" and "if" as used herein can be interpreted as "at the time of" or "when" or "in response to determination".

[0091] The terms "at least one", "multiple", "each", "any", etc. used in this application, at least one includes one, two or more, multiple includes two or more, each refers to each of the corresponding multiple, and any refers to any one of the multiple.

[0092] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0093] Before describing the embodiments of the present application in detail, some nouns and terms involved in the embodiments of the present application are first described. The nouns and terms involved in the embodiments of the present application are subject to the following explanations.

[0094] Parallel hybrid vehicle (HEV): refers to a hybrid electric vehicle that combines a traditional fuel engine and an electric motor to provide power. HEV mode (hybrid mode) is the main working mode of parallel hybrid vehicles. Its core is to improve fuel efficiency and reduce exhaust emissions through the coordinated work of the electric motor and the fuel engine.

[0095] Battery State of Charge (SOC): refers to the ratio of the remaining power of the battery to its capacity when fully charged, usually expressed as a percentage.

[0096] In the related technologies, the automotive industry is one of the main areas of global greenhouse gas emissions. Among all environmentally friendly vehicles, hybrid electric vehicles (HEVs) enjoy longer driving distances than pure electric vehicles, lower fuel consumption than traditional fuel vehicles, and are more environmentally friendly. However, the energy management system of hybrid electric vehicles is much more complex than that of traditional fuel vehicles and pure electric vehicles. Therefore, the energy management strategy (EMS) of hybrid vehicles has become a research hotspot in the automotive field. Research on the energy management strategy of hybrid vehicles can be divided into three main categories: rule-based strategies, optimization-based strategies, and learning-based strategies. Although rule-based energy management strategies are easy to implement, over-reliance on expert experience and experimental calibration makes it difficult to formulate reasonable rules for very complex working conditions. Optimization-based energy management strategies include global optimization strategies and real-time optimization strategies. Typical global optimization strategies mainly include dynamic programming, genetic algorithms, and particle swarm optimization. These algorithms have high computational costs and are usually performed offline. They are often used as benchmarks for evaluating the effectiveness of other online EMS. Real-time optimization control converts the global optimization problem into an instantaneous optimization problem to improve the feasibility of online execution, such as Pontryagin's minimum principle (PMP), equivalent consumption minimization strategy (ECMS), and model predictive control (MPC). However, the key to the success of MPC is fast prediction and fast optimization strategy. It is necessary to predict the road conditions in advance, which depends largely on the model with superior performance. PMP has good optimization efficiency, but the co-state is difficult to obtain and the calculation amount is relatively large. The equivalent fuel consumption minimization strategy has good real-time characteristics, but the historical road information used to calculate the equivalent fuel consumption often cannot represent the future driving conditions, resulting in poor robustness of the algorithm.

[0097] Learning-based EMS, especially the reinforcement learning technology developed in recent years, has opened up a new direction to meet this challenge. Energy management strategies based on the reinforcement learning algorithm Q-learning can greatly improve the fuel economy performance of vehicles compared with traditional rule-based strategies. However, due to the "curse of dimensionality" problem, reinforcement learning strategies like Q-learning are not feasible for systems with high-dimensional, continuous state and action spaces. The Deep Deterministic Policy Gradient (DDPG) strategy is an online "actor-critic" reinforcement learning strategy. The DDPG agent calculates an optimal strategy that maximizes long-term rewards and can be trained in environments with continuous or discrete state spaces and continuous action spaces. Unfortunately, overestimation of the value function leads to incremental bias and suboptimal strategies, which is a common defect of DDPG. Energy management strategies based on deep reinforcement learning have achieved fruitful results, but some issues need to be further studied. First, the high-dimensional nonlinear characteristics of the hybrid vehicle energy management system inevitably lead to redundancy in the deep reinforcement learning state space, which seriously weakens the learning efficiency and real-time performance of deep reinforcement learning. Secondly, most energy management strategy research focuses on fuel economy, while ignoring the impact on vehicle lithium-ion batteries. Finally, the large amount of deep reinforcement learning hyperparameter adjustment has made most researchers and engineers miserable, and the multi-objective weight factors of the reward function have undoubtedly exacerbated this pain. In addition, in rapidly changing traffic scenarios, it is difficult to balance multiple optimization objectives through weight factors, which reduces the fuel economy of the vehicle.

[0098] In view of this, this embodiment makes up for the problem of over-estimation of the DDPG strategy through the following five improvements: (1) Using two Critic networks, the two networks estimate the action value function, and select the smaller value of the two Critic network estimates (Q value) as the estimated value during the training process to avoid unlimited increase of the Q value; (2) Using the soft update method instead of directly copying the parameters of the current network as the parameters of the target network; (3) Using policy noise, using Epsilon-Greedy exploration noise during exploration, the exploration range can be appropriately expanded to increase the accuracy of the estimation; (4) Using delayed learning, the frequency of the Critic network update is greater than the frequency of the Actor network update; (5) Using gradient clipping, the gradient of the Actor parameter update is clipped within a certain range. The target engine equivalent torque is determined by the improved dual-delay deep deterministic policy gradient network and the vehicle energy management learning strategy to control the engine operation, thereby realizing the energy management of the parallel hybrid vehicle, thereby improving the accuracy, applicability and efficiency.

[0099] A parallel hybrid vehicle energy management method provided in an embodiment of the present application relates to the field of vehicle engineering technology. A parallel hybrid vehicle energy management method provided in an embodiment of the present application can be applied to a terminal, can also be applied to a server, and can also be software running in a terminal or a server. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, and a vehicle-mounted terminal, etc., but is not limited to this; the server side can be configured as an independent physical server, or can be configured as a server cluster or distributed system composed of multiple physical servers, and can also be configured as a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The server can also be a node server in a blockchain network; the software can be an application that implements a parallel hybrid vehicle energy management method, etc., but is not limited to the above forms.

[0100] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0101] The following is a detailed explanation of the embodiments of the present application in conjunction with the accompanying drawings:

[0102] Figure 1 is an optional flow chart of a parallel hybrid vehicle energy management method provided in an embodiment of the present application. Figure 1 The method may include but is not limited to steps S101 to S109.

[0103] Step S101, obtaining an initial vehicle status signal;

[0104] Step S102, constructing a vehicle dynamics equation;

[0105] Step S103, constructing a parallel hybrid vehicle structure model according to the vehicle dynamics equation;

[0106] Step S104, constructing a battery thermal model and a battery life model according to the parallel hybrid vehicle structure model;

[0107] Step S105: construct a parameter-free reward function according to the battery thermal model and the battery life model;

[0108] Step S106: Generate reinforcement learning space state and reinforcement learning control action according to the parameter-free reward function;

[0109] Step S107, constructing a target double-delay deep deterministic policy gradient network;

[0110] Step S108, constructing a vehicle energy management learning strategy according to the parallel hybrid vehicle structure model, the parameter-free reward function, the reinforcement learning space state, the reinforcement learning control action, the experience playback area and the target double-delay deep deterministic policy gradient network;

[0111] Step S109: According to the initial vehicle state signal, a target engine equivalent torque is determined using a vehicle energy management learning strategy. The target engine equivalent torque is used to control engine operation to achieve vehicle energy management.

[0112] Steps S101 to S109 shown in the embodiment of the present application implement energy management of parallel hybrid vehicles, improving accuracy, applicability and efficiency.

[0113] In step S101 of some embodiments, the initial vehicle state signal may be acquired through a vehicle sensor device or other methods, without limitation thereto.

[0114] In some embodiments, in step S102, constructing the vehicle dynamics equation may include but is not limited to the following steps:

[0115] Calculate the vehicle's rolling resistance based on the vehicle's mass, gravitational acceleration, rolling resistance coefficient and the vehicle's road slope;

[0116] Calculate the vehicle's slope resistance based on the vehicle's mass, gravity acceleration and the vehicle's road slope;

[0117] Calculate the vehicle's air resistance based on air density, vehicle's frontal area, air resistance coefficient and vehicle's speed;

[0118] Calculate the vehicle's acceleration resistance based on the rotational mass conversion factor, the vehicle's mass and the vehicle's acceleration;

[0119] The vehicle dynamics equation is constructed based on the vehicle's rolling resistance, vehicle's slope resistance, vehicle's air resistance and vehicle's acceleration resistance.

[0120] In some embodiments, the rolling resistance of the vehicle can be calculated based on the vehicle mass, gravity acceleration, rolling resistance coefficient and vehicle road slope, wherein the calculation formula of the rolling resistance of the vehicle is: f = mgfcon(α), where F f is the rolling resistance of the vehicle, m is the vehicle mass, g is the acceleration of gravity, f is the rolling resistance coefficient, and α is the slope of the vehicle road. The slope resistance of the vehicle is calculated based on the vehicle mass, the acceleration of gravity and the slope of the vehicle road. The calculation formula of the slope resistance of the vehicle is: F i = mg sin(α), where F i is the vehicle's driving slope resistance. The vehicle's driving air resistance is calculated based on the air density, the vehicle's frontal area, the air resistance coefficient and the vehicle's driving speed. The calculation formula for the vehicle's driving air resistance is: In the formula, F ω is the air resistance of the vehicle, ρ is the air density, A is the frontal area of ​​the vehicle, C D is the air resistance coefficient, and v is the vehicle speed. The vehicle acceleration resistance is calculated based on the rotational mass conversion coefficient, vehicle mass, and vehicle acceleration. The calculation formula for the vehicle acceleration resistance is: F j =δma, where F j is the vehicle driving acceleration resistance, δ is the rotation mass conversion coefficient. According to the vehicle driving rolling resistance, vehicle driving slope resistance, vehicle driving air resistance and vehicle driving acceleration resistance, the vehicle dynamics equation is constructed, where the expression of the vehicle dynamics equation is: F t =F f +F i +F ω +F j , where F t It is the driving force for the car. It is understandable that the vehicle power system needs to provide the traction required for the vehicle to travel, which can be calculated by the vehicle dynamics equation. The vehicle dynamics equation is the basis for establishing the MATLAB / Simulink model, which mainly describes the behavior of the vehicle under different operating conditions.

[0121] In some embodiments, in step S103, a parallel hybrid vehicle structure model may be constructed according to the vehicle dynamics equation. For example, the parallel hybrid vehicle structure model may be constructed according to the vehicle dynamics equation through MATLAB / Simulink. The parallel hybrid vehicle structure model may be as follows: Figure 2As shown. The engine and motor are connected in parallel, and the clutch can be used to engage or disengage the engine and wheels. At low speed or idle, the clutch is disengaged and the vehicle is driven by the motor first to reduce fuel consumption. When the vehicle requires high power output, the clutch is engaged, and the engine and motor work together to provide the required power. This design allows flexible switching of power sources under different working conditions to optimize energy utilization and improve vehicle performance.

[0122] In some embodiments, in step S104, a battery thermal model is constructed according to the parallel hybrid vehicle structure model, which may include but is not limited to the following steps:

[0123] Extract battery mass, battery specific heat capacity, and battery surface area from a parallel hybrid vehicle structural model;

[0124] A battery thermal model is constructed based on battery temperature, ambient temperature, battery mass, battery specific heat capacity, natural thermal convection constant, battery surface area, battery operating current, battery open circuit voltage, battery operating voltage and integration time.

[0125] In some embodiments, the battery mass, battery specific heat capacity and battery surface area may be first extracted from the parallel hybrid vehicle structural model, and then a battery thermal model may be constructed based on the battery temperature, ambient temperature, battery mass, battery specific heat capacity, natural thermal convection constant, battery surface area, battery operating current, battery open circuit voltage, battery operating voltage and integration time, wherein the mathematical expression of the battery thermal model is: In the formula, T is the general temperature, t is the integration time, m D is the battery mass, c is the battery specific heat capacity, I is the battery operating current, OCV is the battery open circuit voltage, V is the battery operating voltage, h is the natural thermal convection constant, A b is the battery surface area, T emp is the battery temperature, T amb It is understandable that the subsequent parameter-free feedback function is set with an evaluation of the battery life SOH (battery health state), and the change of battery SOH is related to the temperature and usage status of the battery, so it is necessary to establish a battery thermal model for analysis.

[0126] In some embodiments, in step S104, a battery life model is constructed according to the parallel hybrid vehicle structure model, which may include but is not limited to the following steps:

[0127] Extracting battery discharge rate from parallel hybrid vehicle structural model;

[0128] Calculate activation energy based on battery discharge rate;

[0129] Calculate battery throughput based on exponential factor, battery discharge rate, activation energy, universal gas constant and battery temperature;

[0130] Calculate the battery capacity loss percentage based on the exponential factor, battery discharge rate, activation energy, universal gas constant, battery temperature, battery throughput and power law coefficient;

[0131] Calculate the equivalent number of cycles before the end of battery life based on battery throughput and battery capacity loss percentage;

[0132] A battery life model is constructed based on the battery capacity loss percentage, the equivalent number of cycles before the end of the battery life, the battery life state, the battery operating current and the integration time.

[0133] In some embodiments, the battery discharge rate can be extracted from the parallel hybrid vehicle structure model, and the activation energy can be calculated according to the battery discharge rate, wherein the activation energy is calculated as follows: E a (c r )=(31700-370.3·c r ), where E a is the activation energy, c r is the battery discharge rate. Then, the battery throughput is calculated based on the exponential factor, battery discharge rate, activation energy, universal gas constant and battery temperature, where the calculation formula of the battery throughput is: In the formula, A b h is the battery throughput, T emp is the battery temperature, B is the exponential factor, R is the universal gas constant, and z is the power law coefficient. The battery capacity loss percentage is calculated based on the exponential factor, battery discharge rate, activation energy, universal gas constant, battery temperature, battery throughput, and power law coefficient. The calculation formula for the battery capacity loss percentage is: In the formula, C n is the battery capacity loss percentage. Then, the equivalent number of cycles before the end of the battery life is calculated based on the battery throughput and the battery capacity loss percentage. The calculation formula for the equivalent number of cycles before the end of the battery life is: In the formula, N is the equivalent number of cycles before the end of the battery life. Finally, the battery life model is constructed according to the battery capacity loss percentage, the equivalent number of cycles before the end of the battery life, the battery life state, the battery operating current and the integration time. The expression of the battery life model is: In the formula, SOH is the battery life state, t is the integration time, and I is the battery operating current. It can be understood that the subsequent parameter-free return function is set with an evaluation of the battery life SOH (battery health state), and the change of the battery SOH is related to the temperature and usage status of the battery, so it is necessary to establish a battery life model for analysis. In addition, the relationship between the exponential factor and the battery discharge rate can include: when the exponential factor is 31630, the battery discharge rate is 0.5; when the exponential factor is 21681, the battery discharge rate is 2; when the exponential factor is 12934, the battery discharge rate is 6; when the exponential factor is 15512, the battery discharge rate is 10.

[0134] In some embodiments, in step S105, a parameter-free reward function is constructed according to the battery thermal model and the battery life model, which may include but is not limited to the following steps:

[0135] Determine the battery health status based on the battery thermal model and battery life model;

[0136] Calculate the loss associated with the deviation between the current state of charge value and the reference value according to the battery state of charge and the battery state of charge reference value;

[0137] A parameter-free reward function is constructed based on the fuel consumption, battery state of charge, battery health state, and the losses associated with the deviation between the current value of the state of charge and the reference value.

[0138] In some embodiments, the optimization goal of the hybrid vehicle energy management strategy is to improve fuel economy, keep the battery SOC within a reasonable range to avoid overcharging and overdischarging, and inhibit battery life degradation. To achieve this optimization goal, the battery health state can be determined based on the battery thermal model and the battery life model. It can be understood that the change in the battery health state is related to the temperature and usage status of the battery, and can be analyzed based on the battery thermal model and the battery life model. Then, based on the battery state of charge and the battery state of charge reference value, the loss related to the deviation between the current state of charge value and the reference value is calculated, where the calculation formula for the loss related to the deviation between the current state of charge value and the reference value is: C soc =(SOC ref -SOC) 2 , where C soc The loss related to the deviation between the current state of charge and the reference value, SOC ref is the reference value of the battery state of charge, and SOC is the battery state of charge. Then, according to the fuel consumption, battery state of charge, battery health status and the loss related to the deviation between the current value of the state of charge and the reference value, a parameter-free reward function is constructed, where the expression of the parameter-free reward function is: R = -[m f ·SOC+C soc·(1-SOC)]·(1-SOH), where R is a parameter-free reward function, m f is the fuel consumption (kg), SOH is the battery health state (State of Health). It can be understood that the three parts of the parameter-free reward function {m f ·SOC, C soc ·(1-SOC), 1-SOH} are in the same order of magnitude. When the battery is fully charged, the parameter-free reward function is mainly calculated by fuel consumption, while when the battery SOC is low, the parameter-free reward function is mainly dominated by the battery SOC. Therefore, the parameter-free reward function does not require additional parameter adjustment.

[0139] In some embodiments, in step S106, generating a reinforcement learning space state and a reinforcement learning control action according to a parameter-free reward function may include but is not limited to the following steps:

[0140] According to the parameter-free reward function, the vehicle state is divided to obtain the direct state, and the direct state includes vehicle torque demand, power demand, vehicle speed, vehicle acceleration or fuel consumption;

[0141] Refining of the direct state;

[0142] Generate reinforcement learning space state according to the direct state after refinement processing;

[0143] Calculating an initial engine equivalent torque according to the current engine output torque, the maximum engine output torque and the minimum engine output torque;

[0144] The initial engine equivalent torque is used as the reinforcement learning control action.

[0145] In some embodiments, the vehicle state can be first divided according to the parameter-free reward function to obtain direct states. The direct states usually directly affect the optimization objectives of the vehicle (such as fuel economy, battery SOC and battery SOH), where the direct states include vehicle torque demand, power demand, vehicle speed, vehicle acceleration or fuel consumption, and battery SOC, battery SOH and battery current directly affect the state and life of the battery. Furthermore, after the division, indirect states are also obtained. Indirect states do not directly affect the optimization objectives, mainly including clutch states or transmission gears. It can be understood that the design of the state space needs to be closely centered around the vehicle optimization objectives and the parameter-free reward function. According to the parameter-free reward function, the vehicle state is divided into direct states and indirect states. The direct state directly affects the optimization objectives of the vehicle, and the indirect state affects the optimization objectives of the vehicle in other ways.

[0146] The direct state is then refined, and the reinforcement learning space state is generated based on the refined direct state. Exemplarily, the direct state can be introduced into the reinforcement learning state space, and the states that are related to each other or have the same function can be screened, merged and deleted to complete the state refinement. For example, vehicle speed and vehicle acceleration are both indicators of vehicle kinetic energy and the acceleration can be calculated from the vehicle speed, so the reinforcement learning state space only retains the vehicle speed; the vehicle required torque, vehicle required power and vehicle speed are interrelated, and the third state can be calculated from any two states, so the reinforcement learning state space excludes the vehicle power demand; the battery SOC, battery SOH and battery current are closely related, and can be calculated from the current, which is a repeated state, so the reinforcement learning state space only retains the battery current. The reinforcement learning state space after state refinement is S={V spd ,Trq dmd ,m f ,I batt}, where V spd Indicates vehicle speed, Trq dmd Refers to the vehicle torque demand, I batt Represents the battery current, m f It is understandable that the three elements of reinforcement learning are state space, action space and reward function, and the state refinement in this embodiment is an improvement on the state space of the reinforcement learning algorithm.

[0147] Then, the initial engine equivalent torque is calculated according to the current engine output torque, the maximum engine output torque and the minimum engine output torque. The calculation formula of the initial engine equivalent torque is: In the formula, is the initial engine equivalent torque, T ice is the current output torque of the engine, is the maximum output torque of the engine, is the minimum output torque of the engine. Finally, the initial engine equivalent torque is used as the reinforcement learning control action to determine the deep reinforcement learning action signal. It can be understood that the deep reinforcement learning algorithm outputs the control action, and the engine output torque can be controlled by the control action to drive the vehicle.

[0148] In some embodiments, in step S107, constructing a target double-delay deep deterministic policy gradient network may include but is not limited to the following steps:

[0149] Acquire a vehicle standard operating condition data set, where the vehicle standard operating condition data set includes parameters and observation values ​​related to energy management during vehicle standard operating condition simulation driving;

[0150] According to the value network model and the policy network model, an initial double-delayed deep deterministic policy gradient network is constructed;

[0151] The vehicle standard operating condition data set is input into the initial double-delay deep deterministic policy gradient network so that the initial double-delay deep deterministic policy gradient network is trained to obtain the target double-delay deep deterministic policy gradient network.

[0152] In some embodiments, a vehicle standard operating condition data set may be first obtained, wherein the vehicle standard operating condition data set includes parameters and observations related to energy management during vehicle standard operating condition simulation driving. Then, based on the value network model Critic and the policy network model Actor, an initial double-delayed deep deterministic policy gradient network is constructed. The initial double-delayed deep deterministic policy gradient network architecture is as follows: Figure 3 As shown in the figure, deep neural networks can be used to build Critic networks and Actor networks respectively, and jointly construct an initial double-delayed deep deterministic policy gradient network, namely the Actor-Critic network, and initialize the Actor-Critic network parameters and normalize the state data. Figure 3 It includes an initial Actor network, two initial Critic networks, a target Actor network and two target Critic networks. In the initial double-delayed deep deterministic policy gradient network, the state S is input into the initial Actor network and the initial Critic network respectively, the initial Actor network outputs action A and inputs the initial Critic network, the two initial Critic networks output the corresponding Q values ​​respectively, and calculate the TD error, which is used to update the parameters of the initial Critic network. At the same time, the target Critic network is updated according to the output of the initial Critic network, and the target Actor network is updated according to the output of the initial Actor network. Among them, Tanh and Relu are two commonly used activation functions for neural networks. It can be understood that the Actor-Critic algorithm is a reinforcement learning method involving a policy network (Actor) and a value network (Critic). TD error (Temporal Difference Error) is an indicator used to evaluate the accuracy of a prediction model and is widely used in reinforcement learning algorithms. TD error measures the performance of a model by comparing the difference between the predicted value and the actual value of the model. Finally, the vehicle standard operating condition data set is input into the initial double-delay deep deterministic policy gradient network to train the initial double-delay deep deterministic policy gradient network and obtain the target double-delay deep deterministic policy gradient network. For example, the New European Driving Cycle (NEDC) operating condition can be selected as the training set to train the initial double-delay deep deterministic policy gradient network. The selected training operating condition speed curve is as follows: Figure 4It can be understood that the operating condition data (ie, training set) is a 2-dimensional vector, the first dimension is time, and the second dimension is operating condition speed.

[0153] In some embodiments, in step S108, a vehicle energy management learning strategy can be constructed based on the parallel hybrid vehicle structure model, the parameter-free reward function, the reinforcement learning space state, the reinforcement learning control action, the experience playback area and the target double-delay deep deterministic policy gradient network. The vehicle energy management learning strategy architecture is as follows: Figure 5 As shown, the parallel hybrid vehicle structural model outputs the initial state to the state refinement module to obtain the refined state s, calculates the reward value through the parameter-free reward function, and calculates the state at the next moment through reinforcement learning. The reinforcement learning space state, reinforcement learning control action a, reward value r and the state at the next moment are stored in the experience replay area, and the experience replay area is randomly sampled to obtain samples. The samples are input into the target double-delay deep deterministic policy gradient network, and the network outputs the control action as the engine equivalent torque to control the engine operation.

[0154] In some embodiments, in step S109, according to the initial vehicle state signal, the target engine equivalent torque is determined using the vehicle energy management learning strategy, which may include but is not limited to the following steps:

[0155] Initialize the first initial value network weight, the second initial value network weight, the initial strategy network weight and the model training parameters, the model training parameters include the experience playback area storage information, the learning rate, the random noise, the discount factor, the batch gradient descent sample number and the preset maximum number of iterations;

[0156] Calculating a first target value network weight according to the first initial value network weight;

[0157] Calculating a second target value network weight according to the second initial value network weight;

[0158] According to the initial strategy network weight, calculate the target strategy network weight;

[0159] Refining the initial vehicle state signal to obtain the current state;

[0160] Generate current control action according to current state, random noise and control strategy;

[0161] Execute the current control action to obtain the reward value and the next state;

[0162] The current state, current control action, reward value and next moment state are stored in the experience playback area;

[0163] Randomly sample the experience playback area to obtain several samples;

[0164] Generate the next moment control action based on several samples;

[0165] Calculate the timing difference error according to the state at the next moment and the control action at the next moment;

[0166] Calculate the minimization loss based on the time difference error;

[0167] Based on the model training parameters, the value network parameters are updated by minimizing the loss;

[0168] Based on the model training parameters, the policy network parameters are updated using gradient descent;

[0169] updating the first target value network weight according to the value network parameter, the first initial value network weight and the first target value network weight;

[0170] updating the second target value network weight according to the value network parameter, the second initial value network weight and the second target value network weight;

[0171] According to the strategy network parameters, the initial strategy network weights and the target strategy network weights, the target strategy network weights are updated until the preset maximum number of iterations is reached to obtain the target engine equivalent torque.

[0172] In some embodiments, the target engine equivalent torque can be determined based on the initial vehicle state signal using the vehicle energy management learning strategy, wherein the target engine equivalent torque is used to control the engine operation to achieve vehicle energy management. The first initial value network weight can be initialized Second initial value network weight Initial policy network weights θ μ And model training parameters, where the model training parameters include experience playback area storage information, learning rate τ, random noise ε, discount factor γ, batch gradient descent sample number M and preset maximum number of iterations T. Then, based on the first initial value network weight, calculate the first target value network weight According to the second initial value network weight, calculate the second target value network weight According to the initial policy network weight, calculate the target policy network weight θ μ′ ←θ μ , where ← indicates assignment.

[0173] Then iterate from t=1:T (i.e., loop iteration T times), refine the initial vehicle state signal, and obtain the current state s t ←S t = {V spd (t),Trq dmd (t),m f (t),Ibatt (t)}, where s t is the state at time t, that is, the current state, S t is the general state space, V spd Indicates vehicle speed, Trq dmd Refers to the vehicle torque demand, I batt Represents the battery current, m f is the fuel consumption. According to the current state, random noise and control strategy, the current control action a is generated. t ←π(a t |s t )=μ(s t )+ε, where a t is the current control action, μ is the actor network, and ε is random noise. Execute the current control action and get the reward value r t and the next state s t Then the current state, current control action, reward value and next moment state are stored in the experience playback area D, and the experience playback area is randomly sampled to obtain several samples. For example, M samples {s j ,a j ,r j ,s j ′},j=1,2,...,M, where s j is the current state of the jth sample, a j is the current control action of the jth sample, r j is the return value of the jth sample, s j ' is the next moment state of the jth sample. Based on several samples, the next moment control action a' is generated, and the timing difference error y is calculated based on the next moment state and the next moment control action. j , where if s′ is a terminal state, then y j =r j , otherwise, y j =r j +γmin j＝1,2 Q j′ (s′, a′). According to the time difference error, the minimization loss is calculated, where the calculation formula for minimizing the loss is: Where, L ki To minimize the loss, M is the total number of samples. According to the model training parameters, the value network parameters are updated by minimizing the loss. After the Critic network (value network parameters) is updated C times, the policy network parameters (Actor network) are updated by gradient descent according to the model training parameters. The gradient calculation formula is: In the formula, Indicates that for θ μGradient calculation, J is the loss function, J = -R, represents the gradient calculation for a, Q(s,a) is the Critic network value function. Finally, according to the value network parameters, the first initial value network weight and the first target value network weight, the first target value network weight is updated. Update the second target value network weight according to the value network parameter, the second initial value network weight and the second target value network weight According to the policy network parameters, the initial policy network weights and the target policy network weights, update the target policy network weights θ μ′ =τθ μ +(1-τ)θ μ′ , until the preset maximum number of iterations is reached, the target engine equivalent torque is obtained to control the engine operation.

[0174] In some embodiments, performance experiments can be conducted to obtain the parameters and observations that affect energy management when the vehicle is driving in more complex and complex conditions with more road elements. Based on the parameters that affect energy management in the complex conditions, the optimization performance of the trained dual-delay deep deterministic policy gradient network can be verified. For example, multiple standard conditions such as highway conditions (Highway Fuel Economy Test, HWFET), urban dynamometer conditions (Urban Dynamometer Driving Schedule, UDDS), world light vehicle test conditions (Worldwide Light Vehicle Test Procedure Class3, WLTPClass3) and China Light-duty Vehicle Test Cycle (China Light-duty Vehicle Test Cycle, CLTC) can be combined to form a composite condition with more complex road conditions and more road elements, and then imported into the vehicle driver model to test the dual-delay deep deterministic policy gradient network with no parameter return. Among them, the speed curve of the composite condition is as follows: Figure 6 shown.

[0175] In some embodiments, this embodiment proposes a parameter-free reward function for an energy management strategy based on TD3 (double-delayed deep deterministic policy gradient algorithm), and uses battery SOC as a weight parameter. The parameter-free reward function can implement the energy management strategy and realize adaptive adjustment according to the vehicle state without the need for experts to set parameters, thereby improving the optimality and adaptability of the energy management strategy. This embodiment uses state space (vehicle speed, torque demand, fuel consumption and battery current), action space (engine output torque), parameter-free reward function, and state refinement to optimize the management of vehicle status. The method of this embodiment improves data-driven and logical consistency, reduces the influence of human subjective factors on the design process, and thus improves the effectiveness and reliability of the design.

[0176] The beneficial effects of implementing the embodiments of the present invention include: the embodiments of the present invention first obtain the initial vehicle state signal, then construct the vehicle dynamics equation, construct the parallel hybrid vehicle structure model according to the vehicle dynamics equation, construct the battery thermal model and the battery life model according to the parallel hybrid vehicle structure model, construct the parameter-free reward function according to the battery thermal model and the battery life model, and then generate the reinforcement learning space state and reinforcement learning control action according to the parameter-free reward function, construct the target double-delay deep deterministic policy gradient network, construct the vehicle energy management learning strategy, and finally determine the target engine equivalent torque according to the initial vehicle state signal using the vehicle energy management learning strategy, so that the engine operation can be controlled by the target engine equivalent torque to achieve parallel hybrid vehicle energy management, thereby improving accuracy, applicability and efficiency. At the same time, this embodiment uses the parameter-free reward double-delay deep deterministic policy gradient algorithm and combines it with the state space refinement technology to solve the problem that the weight factor is difficult to balance multiple optimization objectives and the parameter adjustment task is heavy, as well as the problem of deep reinforcement learning state redundancy, thereby improving the real-time, adaptability and optimality of the energy management strategy. This embodiment uses state refinement technology to optimize the state space of deep reinforcement learning strategies to solve the problem of state information redundancy and improve the learning efficiency of the deep deterministic policy gradient method based on state refinement and parameter-free return double delay. This embodiment not only optimizes the fuel consumption during vehicle operation and keeps the battery SOC within a reasonable range, but also can suppress the decline of battery life.

[0177] like Figure 7 As shown, an embodiment of the present invention further provides an energy management device for a parallel hybrid vehicle, comprising:

[0178] The first module 801 is used to obtain an initial vehicle state signal;

[0179] The second module 802 is used to construct the vehicle dynamics equation;

[0180] The third module 803 is used to construct a parallel hybrid vehicle structure model according to the vehicle dynamics equation;

[0181] The fourth module 804 is used to construct a battery thermal model and a battery life model according to the parallel hybrid vehicle structure model;

[0182] The fifth module 805 is used to construct a parameter-free reward function according to the battery thermal model and the battery life model;

[0183] The sixth module 806 is used to generate a reinforcement learning space state and a reinforcement learning control action according to the parameter-free reward function;

[0184] The seventh module 807 is used to construct a target double-delay deep deterministic policy gradient network;

[0185] An eighth module 808 is used to construct a vehicle energy management learning strategy based on a parallel hybrid vehicle structural model, a parameter-free reward function, a reinforcement learning space state, a reinforcement learning control action, an experience playback area, and a target dual-delay deep deterministic policy gradient network;

[0186] The ninth module 809 is used to determine the target engine equivalent torque according to the initial vehicle state signal by using the vehicle energy management learning strategy. The target engine equivalent torque is used to control the engine operation to achieve vehicle energy management.

[0187] The contents of the above method embodiments are all applicable to the present device embodiments. The functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0188] like Figure 8 As shown, an embodiment of the present invention further provides a computer device, including:

[0189] at least one processor 901;

[0190] At least one memory 902, used to store at least one program;

[0191] When at least one program is executed by at least one processor, the at least one processor implements Figure 1 The method shown.

[0192] The contents of the above method embodiments are all applicable to the present device embodiments. The functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0193] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but the scope of the rights of the present invention is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present invention should be within the scope of the rights of the present invention.

Claims

1. A parallel hybrid vehicle energy management method, characterized in that: The following steps are involved: Obtaining an initial vehicle status signal; Construct vehicle dynamics equations; According to the vehicle dynamics equation, a parallel hybrid vehicle structural model is constructed; Constructing a battery thermal model and a battery life model according to the parallel hybrid vehicle structural model; Constructing a parameter-free reward function according to the battery thermal model and the battery life model; Generate a reinforcement learning space state and a reinforcement learning control action according to the parameter-free reward function; Construct a target dual-delay deep deterministic policy gradient network; Constructing a vehicle energy management learning strategy according to the parallel hybrid vehicle structural model, the parameter-free reward function, the reinforcement learning space state, the reinforcement learning control action, the experience replay area, and the target dual-delay deep deterministic policy gradient network; According to the initial vehicle state signal, the target engine equivalent torque is determined using the vehicle energy management learning strategy, and the target engine equivalent torque is used to control the engine operation to achieve vehicle energy management.

2. The method according to claim 1, characterized in that The constructing of the vehicle dynamics equation comprises: Calculate the vehicle's rolling resistance based on the vehicle's mass, gravitational acceleration, rolling resistance coefficient and the vehicle's road slope; Calculating the vehicle driving slope resistance according to the vehicle mass, the gravitational acceleration and the vehicle road slope; Calculate the vehicle's air resistance based on air density, vehicle's frontal area, air resistance coefficient and vehicle's speed; Calculating the vehicle's driving acceleration resistance according to the rotational mass conversion coefficient, the vehicle's mass and the vehicle's driving acceleration; The vehicle dynamics equation is constructed according to the vehicle running rolling resistance, the vehicle running slope resistance, the vehicle running air resistance and the vehicle running acceleration resistance.

3. The method according to claim 1, characterized in that The step of constructing a battery thermal model according to the parallel hybrid vehicle structural model includes: extracting battery mass, battery specific heat capacity, and battery surface area from the parallel hybrid vehicle structural model; The battery thermal model is constructed according to the battery temperature, the ambient temperature, the battery mass, the battery specific heat capacity, the natural thermal convection constant, the battery surface area, the battery operating current, the battery open circuit voltage, the battery operating voltage and the integration time.

4. The method according to claim 1, characterized in that The step of constructing a battery life model according to the parallel hybrid vehicle structure model includes: extracting a battery discharge rate from the parallel hybrid vehicle structural model; Calculating activation energy according to the battery discharge rate; Calculating a battery throughput based on an exponential factor, the battery discharge rate, the activation energy, a universal gas constant, and a battery temperature; Calculating a battery capacity loss percentage according to the exponential factor, the battery discharge rate, the activation energy, the universal gas constant, the battery temperature, the battery throughput, and a power law coefficient; Calculating the equivalent number of cycles before the end of battery life according to the battery throughput and the battery capacity loss percentage; The battery life model is constructed according to the battery capacity loss percentage, the equivalent number of cycles before the end of the battery life, the battery life state, the battery operating current and the integration time.

5. The method according to claim 1, characterized in that The constructing a parameter-free reward function according to the battery thermal model and the battery life model comprises: Determining a battery health state according to the battery thermal model and the battery life model; Calculate the loss associated with the deviation between the current state of charge value and the reference value according to the battery state of charge and the battery state of charge reference value; The parameter-free reward function is constructed according to the fuel consumption, the battery state of charge, the battery health state and the loss related to the deviation between the current value of the state of charge and the reference value.

6. The method according to claim 1, characterized in that The step of generating a reinforcement learning space state and a reinforcement learning control action according to the parameter-free reward function includes: According to the parameter-free reward function, the vehicle state is divided to obtain a direct state, wherein the direct state includes a vehicle torque demand, a power demand, a vehicle speed, a vehicle acceleration or a fuel consumption; performing a refining process on the direct state; Generating the reinforcement learning space state according to the direct state after the refinement process; Calculating an initial engine equivalent torque according to the current engine output torque, the maximum engine output torque and the minimum engine output torque; The initial engine equivalent torque is used as the reinforcement learning control action.

7. The method according to claim 1, characterized in that The method of constructing a target dual-delay deep deterministic policy gradient network includes: Acquire a vehicle standard operating condition data set, wherein the vehicle standard operating condition data set includes parameters and observation values ​​related to energy management during vehicle standard operating condition simulation driving; According to the value network model and the policy network model, an initial double-delayed deep deterministic policy gradient network is constructed; The vehicle standard operating condition data set is input into the initial double-delay deep deterministic policy gradient network so that the initial double-delay deep deterministic policy gradient network is trained to obtain the target double-delay deep deterministic policy gradient network.

8. The method according to claim 1, characterized in that The step of determining the target engine equivalent torque by using the vehicle energy management learning strategy according to the initial vehicle state signal comprises: Initialize the first initial value network weight, the second initial value network weight, the initial strategy network weight and model training parameters, wherein the model training parameters include experience replay area storage information, learning rate, random noise, discount factor, batch gradient descent sample number and preset maximum number of iterations; Calculating a first target value network weight according to the first initial value network weight; Calculating a second target value network weight according to the second initial value network weight; Calculating target strategy network weights according to the initial strategy network weights; Refining the initial vehicle state signal to obtain a current state; Generate a current control action according to the current state, the random noise and the control strategy; Execute the current control action to obtain the reward value and the state at the next moment; The current state, the current control action, the reward value and the next moment state are stored in the experience playback area; Randomly sampling the experience playback area to obtain a number of samples; Generate a control action at the next moment according to the plurality of samples; Calculating a timing difference error according to the state at the next moment and the control action at the next moment; Calculate the minimization loss according to the timing difference error; According to the model training parameters, updating the value network parameters using the minimized loss; Based on the model training parameters, updating the policy network parameters using gradient descent; updating the first target value network weight according to the value network parameter, the first initial value network weight and the first target value network weight; updating the second target value network weight according to the value network parameter, the second initial value network weight and the second target value network weight; According to the strategy network parameters, the initial strategy network weights and the target strategy network weights, the target strategy network weights are updated until the preset maximum number of iterations is reached to obtain the target engine equivalent torque.

9. A parallel hybrid vehicle energy management device, characterized in that: include: The first module is used to obtain an initial vehicle status signal; The second module is used to construct the vehicle dynamics equations; A third module is used to construct a parallel hybrid vehicle structure model according to the vehicle dynamics equation; A fourth module is used to construct a battery thermal model and a battery life model according to the parallel hybrid vehicle structural model; A fifth module is used to construct a parameter-free reward function according to the battery thermal model and the battery life model; A sixth module, used to generate a reinforcement learning space state and a reinforcement learning control action according to the parameter-free reward function; The seventh module is used to build a target dual-delay deep deterministic policy gradient network; An eighth module is used to construct a vehicle energy management learning strategy according to the parallel hybrid vehicle structure model, the parameter-free reward function, the reinforcement learning space state, the reinforcement learning control action, the experience playback area and the target double-delay deep deterministic policy gradient network; The ninth module is used to determine the target engine equivalent torque according to the initial vehicle state signal using the vehicle energy management learning strategy, and the target engine equivalent torque is used to control the engine operation to achieve vehicle energy management.

10. A computer device, characterized in that: include: at least one processor; at least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the method according to any one of claims 1 to 8.

Images