Electric vehicle energy management method and system considering power battery degradation inhibition

Abstract

The present application relates to a kind of electric vehicle energy management method and system considering power battery recession inhibition.The method includes determining action space;According to the state data of electric vehicle, determine state space;According to the real-time error of power battery state of charge and the recession degree of power battery, construct reward function;According to the action space and the state space and reward function, determine trained deep Q network;The trained deep Q network includes: Actor action network and Critic evaluation network;According to the real-time state data in state space, using trained deep Q network determines real-time action variable value, and then according to the real-time action variable value, the power distribution control of electric vehicle is carried out.The present application can solve the energy management problem of coaxial parallel hybrid powertrain of mixed action space, and comprehensively improve the equivalent economy of electric vehicle.

Description

Technical Field

[0001] This invention relates to the field of electric vehicle energy management, and in particular to an electric vehicle energy management method and system that takes into account the suppression of power battery degradation. Background Technology

[0002] Electric vehicles (EVs), with their diverse energy sources, long driving range, high fuel economy, and low carbon emissions, have become a hot research topic. However, during operation, the power battery needs frequent charging and discharging to meet the vehicle's instantaneous power demands, which accelerates battery aging. Given that one-third of the overall cost of an EV currently comes from its power battery, EVs face significant challenges in battery cost control. Especially without breakthroughs in battery cycle life and pricing, the cost of battery degradation over time will greatly increase the overall operating cost of the vehicle. Therefore, how to mitigate battery degradation is one of the critical issues that the EV industry urgently needs to address.

[0003] With the increasing cross-category integration and fusion of various algorithms in the field of machine learning, and the improvement of computing power brought about by hardware advancements, learning algorithms represented by deep reinforcement learning have developed rapidly and demonstrated good application results in many types of problem domains. Furthermore, in the field of electric vehicle energy management, represented by hybrid power systems with a dual energy supply source of power battery and engine, several successful application examples have been achieved by setting the action space of deep reinforcement learning according to different coupling forms of hybrid energy sources and selecting specific types of deep reinforcement learning algorithms: ① Series hybrid electric vehicles: In this type, the engine's operating state is completely decoupled from the driving conditions, while the driving motor's operating state is completely coupled with the road conditions. The engine energy is completely converted into electrical energy by the generator, used to charge the power battery or provide power to the driving motor. To improve fuel efficiency, the engine is usually set to operate on the optimal fuel curve. Given the engine power, the overall vehicle operating state can be uniquely determined. To simplify the network structure and improve computational efficiency, the action space is set to discrete, which can be solved using the Deep Q-Networks (DQN) algorithm. ② Parallel hybrid electric vehicles: In some operating conditions, the engine directly drives the vehicle together with the driving motor. At this point, the engine, starter / generator motor, and drive motor operate coaxially, with the drive shaft speed directly coupled to the vehicle speed on the road. By setting variables such as the engine's output torque and the starter / generator motor's output torque as action quantities, the overall vehicle operating state can be uniquely determined. For this type of continuous action space problem, the Deep Deterministic Policy Gradient (DDPG) algorithm has been well applied.

[0004] However, for such Figure 1 The coaxial series-parallel hybrid electric vehicle shown combines characteristics of both series and parallel vehicles: when the clutch is engaged, the vehicle can be considered a parallel vehicle; when the clutch is disengaged, the vehicle operates in a series mode. Therefore, solving the learning-based energy management strategy for this vehicle type is a hybrid action space problem. Directly applying the Deep Q-Networks (DQN) algorithm and the Deep Deterministic Policy Gradient (DDPG) algorithm to obtain the optimal control strategy does not yield ideal results.

[0005] Based on the above problems, there is an urgent need for a new energy management method or system for electric vehicles to solve the energy management problem of coaxial series-parallel hybrid vehicles involving a hybrid action space. Summary of the Invention

[0006] The purpose of this invention is to provide an energy management method and system for electric vehicles that takes into account the suppression of power battery degradation, which can solve the energy management problem of coaxial series-parallel hybrid electric vehicles with mixed operating space and comprehensively improve the equivalent economy of electric vehicles.

[0007] To achieve the above objectives, the present invention provides the following solution:

[0008] An energy management method for electric vehicles that considers the suppression of battery degradation includes:

[0009] Determine the action space; the action space includes: clutch state and corresponding action variable values; the action variable values ​​include: engine power change, engine output torque, output torque of the starter / generator integrated motor, and the ratio of mechanical braking torque to required torque; the clutch state includes: clutch disengagement and clutch engagement.

[0010] The state space is determined based on the state data of the electric vehicle; the state data includes: current driving speed, historical speed, vehicle acceleration, vehicle torque demand, vehicle power demand, engine status, power battery state of charge, and power battery power.

[0011] A reward function is constructed based on the real-time error of the state of charge of the power battery and the degree of degradation of the power battery;

[0012] The trained deep Q-network is determined based on the action space, the state space, and the reward function; the trained deep Q-network includes an Actor action network and a Critic evaluation network; the Actor action network takes the state data in the state space as input and the action variable value as output; the Critic evaluation network takes the state data and action variable value in the state space as input and the clutch state as output.

[0013] Based on the real-time state data in the state space, a trained deep Q-network is used to determine the real-time action variable values, and then the power distribution control of the electric vehicle is performed based on the real-time action variable values.

[0014] Optionally, determining the action space specifically includes the following formula:

[0015] W={0,{ΔP Engine i Brake}}∪{1,{T Engine ,T ISG i Brake}};

[0016] Where W represents the action space, 0 represents clutch disengagement, 1 represents clutch engagement, and ΔP Engine i represents the change in engine power. Brake T represents the ratio of mechanical braking torque to the required torque. Engine T is the engine output torque. ISG This refers to the output torque of the integrated starter / generator motor.

[0017] Optionally, the step of determining the state space based on the state data of the electric vehicle further includes:

[0018] Using formula s∈S standardizes the state data in the state space;

[0019] in, For each standardized state data point, s represents a single state data point, S represents the state space, u represents the average value of each state data point, and σ represents the standard deviation of each state data point.

[0020] Optionally, the step of constructing a reward function based on the real-time error of the power battery's state of charge and the degree of battery degradation specifically includes:

[0021] Using the formula r=-α·(p1·ΔSoC ref +p2·m fuel +p3·loss Battery Determine the reward function;

[0022] Among them, SoC ref(t) represents the reference trajectory of the power battery's state of charge at time t, SoC ref (t)=SoC0-λ·D(t), where SoC0 is the initial value of the state of charge of the power battery directly measured by the on-board sensor at the initial moment. SoC f The final values ​​of the state of charge of the power battery, D(t) and D, are determined by a global optimization method based on dynamic programming at the last moment. Total These represent the vehicle's current distance traveled and total mileage, respectively, ΔSoC ref ΔSoC is the real-time error of the state of charge of the power battery. ref (t) = SoC(t) - SoC ref (t), where SoC(t) is the current value of the power battery's state of charge directly measured by the onboard sensors at time t, m fuel For engine fuel consumption, m fuel =MAP Power (ΔP Engine ,n Engine ) = MAP Torque (T Engine ,n Engine ), MAP Power (ΔP Engine ,n Engine ) is the change in engine power ΔP Engine and engine speed n Engine m determined by referring to the fuel consumption MAP chart fuel With ΔP Engine and n Engine The relationship between MAP Torque (T Engine ,n Engine ) is determined by engine power T Engine and engine speed n Engine m determined by referring to the fuel consumption MAP chart fuel With T Engine and n Engine The relationship between Q loss For the normalized capacity loss of the power battery, A is the pre-coefficient factor, which is a constant; T Battery The temperature of the power battery is measured by a temperature sensor in the vehicle's power battery management system. C rate The charge / discharge rate of the power battery. Q0 is the rated capacity of the power battery, a fixed value for a given electric vehicle, and a normal quantity. Battery The charging / discharging current of the power battery is given by [value], and the power battery power P is given by [value]. Battery With output voltage U Battery The ratio of the output voltage U of the power battery BatteryMeasured by voltage sensors in the vehicle's power battery management system; B is a compensation factor, which is a constant; E a The activation energy of the power battery is a constant value for a specific type of power battery and is considered a normal quantity; R is the gas constant, which is a defined normal quantity; A h For the ampere-hour throughput of the power battery, z is the exponential factor, e is the natural logarithm, and loss is... Battery The degree of degradation of the power battery is represented by α, the scaling factor is α, p1, p2, and p3 are the weighting factors, and r is the reward function.

[0023] Optionally, determining the trained deep Q-network based on the action space, the state space, and the reward function specifically includes:

[0024] Construct the Actor action network and the Critic evaluation network respectively, and obtain the parameter sets of the Actor action network and the Critic evaluation network respectively;

[0025] Determine the depth Q-network based on the Actor action network and the Critic evaluation network;

[0026] The deep Q-network is used as the target neural network and the evaluation neural network, respectively.

[0027] The Adam gradient descent method and reward function are used to train the parameter sets in the target neural network and the evaluation neural network respectively, and the trained deep Q network is determined.

[0028] An electric vehicle energy management system that considers the suppression of battery degradation includes:

[0029] An action space determination module is used to determine the action space; the action space includes: clutch status and corresponding action variable values; the action variable values ​​include: engine power change, engine output torque, output torque of the starter / generator integrated motor, and the ratio of mechanical braking torque to required torque; the clutch status includes: clutch disengagement and clutch engagement.

[0030] The state space determination module is used to determine the state space based on the state data of the electric vehicle; the state data includes: current driving speed, historical speed, vehicle acceleration, vehicle torque demand, vehicle power demand, engine status, power battery state of charge, and power battery power.

[0031] The reward function construction module is used to construct a reward function based on the real-time error of the power battery's state of charge and the degree of degradation of the power battery.

[0032] The trained deep Q-network determination module is used to determine the trained deep Q-network based on the action space, the state space, and the reward function; the trained deep Q-network includes: an Actor action network and a Critic evaluation network; the Actor action network takes the state data in the state space as input and the action variable value as output; the Critic evaluation network takes the state data and action variable value in the state space as input and the clutch state as output;

[0033] The power distribution control module is used to determine the real-time action variable values ​​using a trained deep Q-network based on the real-time state data in the state space, and then perform power distribution control of the electric vehicle based on the real-time action variable values.

[0034] Optionally, the action space determination module specifically includes the following formula:

[0035] W={0,{ΔP Engine i Brake}}∪{1,{T Engine ,T ISG i Brake}};

[0036] Where W represents the action space, 0 represents clutch disengagement, 1 represents clutch engagement, and ΔP Engine i represents the change in engine power. Brake T represents the ratio of mechanical braking torque to the required torque. Engine T is the engine output torque. ISG This refers to the output torque of the integrated starter / generator motor.

[0037] Optionally, it also includes:

[0038] The standardization module is used to utilize formulas s∈S standardizes the state data in the state space;

[0039] in, For each standardized state data point, s represents a single state data point, S represents the state space, u represents the average value of each state data point, and σ represents the standard deviation of each state data point.

[0040] Optionally, the reward function construction module specifically includes:

[0041] The reward function building unit is used to utilize the formula r = -α·(p1·ΔSoC) ref +p2·m fuel +p3·loss Battery Determine the reward function;

[0042] Among them, SoC ref(t) represents the reference trajectory of the power battery's state of charge at time t, SoC ref (t)=SoC0-λ·D(t), where SoC0 is the initial value of the state of charge of the power battery directly measured by the on-board sensor at the initial moment. SoC f The final values ​​of the state of charge of the power battery, D(t) and D, are determined by a global optimization method based on dynamic programming at the last moment. Total These represent the vehicle's current distance traveled and total mileage, respectively, ΔSoC ref ΔSoC is the real-time error of the state of charge of the power battery. ref (t) = SoC(t) - SoC ref (t), where SoC(t) is the current value of the power battery's state of charge directly measured by the onboard sensors at time t, m fuel For engine fuel consumption, m fuel =MAP Power (ΔP Engine ,n Engine ) = MAP Torque (T Engine ,n Engine ), MAP Power (ΔP Engine ,n Engine ) is the change in engine power ΔP Engine and engine speed n Engine m determined by referring to the fuel consumption MAP chart fuel With ΔP Engine and n Engine The relationship between MAP Torque (T Engine ,n Engine ) is determined by engine power T Engine and engine speed n Engine m determined by referring to the fuel consumption MAP chart fuel With T Engine and n Engine The relationship between Q loss For the normalized capacity loss of the power battery, A is the pre-coefficient factor, which is a constant; T Battery The temperature of the power battery is measured by a temperature sensor in the vehicle's power battery management system. C rate The charge / discharge rate of the power battery. Q0 is the rated capacity of the power battery, a fixed value for a given electric vehicle, and a normal quantity. Battery The charging / discharging current of the power battery is given by [value], and the power battery power P is given by [value]. Battery With output voltage U Battery The ratio of the output voltage U of the power battery BatteryMeasured by voltage sensors in the vehicle's power battery management system; B is a compensation factor, which is a constant; E a The activation energy of the power battery is a constant value for a specific type of power battery and is considered a normal quantity; R is the gas constant, which is a defined normal quantity; A h For the ampere-hour throughput of the power battery, z is the exponential factor, e is the natural logarithm, and loss is... Battery The degree of degradation of the power battery is represented by α, the scaling factor is α, p1, p2, and p3 are the weighting factors, and r is the reward function.

[0043] Optionally, the trained deep Q-network determination module specifically includes:

[0044] The parameter set acquisition unit is used to construct the Actor action network and the Critic evaluation network respectively, and to acquire the parameter sets of the Actor action network and the Critic evaluation network respectively.

[0045] A deep Q-network determination unit is used to determine the deep Q-network based on the Actor network and the Critic evaluation network.

[0046] A target neural network and an evaluation neural network determination unit is used to determine the deep Q-network as the target neural network and the evaluation neural network, respectively.

[0047] The trained deep Q-network determines the unit, which is used to train the parameter sets in the target neural network and the evaluation neural network respectively using Adam gradient descent and reward function, and determines the trained deep Q-network.

[0048] According to specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0049] This invention provides an energy management method and system for electric vehicles that considers the suppression of battery degradation. It constructs a reward function based on the real-time error of the battery's state of charge and the degree of battery degradation; determines a trained deep Q-network based on the action space, the state space, and the reward function; determines real-time action variable values ​​using the trained deep Q-network based on real-time state data in the state space; and then performs power distribution control of the electric vehicle based on these real-time action variable values. This solves the energy management problem of coaxial series-parallel hybrid vehicles involving a mixed action space. Furthermore, it considers the suppression of battery degradation, and has significant application prospects for comprehensively improving the equivalent economic efficiency of electric vehicles, particularly coaxial series-parallel hybrid vehicles. Attached Figure Description

[0050] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0051] Figure 1 This is a schematic diagram of a coaxial series-parallel hybrid electric vehicle.

[0052] Figure 2 A schematic diagram of an electric vehicle energy management method considering the suppression of power battery degradation provided by the present invention;

[0053] Figure 3 This is a schematic diagram of the SoC following results;

[0054] Figure 4 This is a schematic diagram of the power allocation results;

[0055] Figure 5 This diagram illustrates the results of power battery power consumption, fuel consumption, and power battery wear.

[0056] Figure 6 This is a schematic diagram of an electric vehicle energy management system that considers the suppression of power battery degradation, provided by the present invention. Detailed Implementation

[0057] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0058] The purpose of this invention is to provide an energy management method and system for electric vehicles that takes into account the suppression of power battery degradation, which can solve the energy management problem of coaxial series-parallel hybrid electric vehicles with mixed operating space and comprehensively improve the equivalent economy of electric vehicles.

[0059] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0060] Figure 2 This is a schematic diagram of a method for energy management of electric vehicles that considers the suppression of battery degradation, as provided by the present invention. Figure 2 As shown, the present invention provides an electric vehicle energy management method that considers the suppression of power battery degradation, comprising:

[0061] S201, Determine the action space; the action space includes: clutch status and corresponding action variable values; the action variable values ​​include: engine power change, engine output torque, output torque of the starter / generator integrated motor, and the ratio of mechanical braking torque to required torque; the clutch status includes: clutch disengagement and clutch engagement.

[0062] The clutch state, which determines whether the engine of a hybrid electric vehicle directly participates in the vehicle's drive, is set as a discrete action variable a(1); when the clutch is disengaged, a(1) = 0, and the vehicle's operating state can be determined based on the determined engine operating state. To fully utilize the advantages of the series hybrid electric vehicle configuration and improve engine fuel efficiency, the engine operating point is restricted to the optimal fuel efficiency curve. At this time, given the engine output power P... Engine Therefore, the engine output torque and speed can be uniquely determined. Thus, the engine power change ΔP is selected. Engine For continuous action variables a(2)∈[-ΔP Engine,max ,+ΔP Engine,max ], ΔP Engine,max This represents the maximum value of the engine power variation, which is a normal value. When the clutch is engaged, a(1) = 1, the speed of the power source (engine, drive motor) is coupled with the road surface, and the torque relationship of each power component satisfies the equation: T Deman d = T Engine +T ISG +T Motor In the formula, T Demand Vehicle torque requirement; T Engine T ISG and T Motor These represent the output torques of the engine, the starter / generator motor, and the drive motor, respectively. When T... ISG and T Motor A negative value indicates that the starter / generator motor and drive motor are in generator mode. Demand-side power T Demand Given a supply side T Engine T ISG and T Motor Any two of these three factors can determine the power distribution state of the entire vehicle. Therefore, in this mode, the engine torque output T is selected. Engine The output torque of the starter / generator integrated motor is a continuous motion variable a(3)∈[T] Engine,m i n ,T Engine,max ], T Engine,m i n and T Engine,max T represents the engine output torque respectively. ISG Minimum and maximum values, a(4)∈[T]ISG,min ,T ISG,max ], T ISG,min and T ISG,max These represent the minimum and maximum output torque of the integrated starter / generator motor, respectively. Furthermore, for both vehicle operating modes, a regenerative braking ratio needs to be set, which is closely related to battery aging and fuel consumption. The vehicle's braking system needs to dissipate the non-regenerative braking energy; therefore, the ratio i of mechanical braking torque to the required torque needs to be set. Brake Let a be a continuous action variable (5). That is:

[0063] W={0,{ΔP Engine i Brake}}∪{1,{T Engine ,T ISG i Brake}};

[0064] Specifically, it is expressed as follows:

[0065] W={0,{ΔP Engine i Brake}}∪{1,{T Engine ,T ISG i Brake}};

[0066] Where W represents the action space, 0 represents clutch disengagement, 1 represents clutch engagement, and ΔP Engine i represents the change in engine power. Brake T represents the ratio of mechanical braking torque to the required torque. Engine T is the engine output torque. ISG This refers to the output torque of the integrated starter / generator motor.

[0067] S202, determine the state space based on the electric vehicle's state data; the state data includes: current vehicle speed, historical vehicle speed, vehicle acceleration, vehicle torque demand, vehicle power demand, engine status, power battery state of charge, and power battery power; the historical vehicle speed is specifically the vehicle speed v from 3 seconds ago. -1 ,v -2 ,v -3 ;

[0068] That is, select the current driving speed v and the historical speed v3 seconds ago. -1 ,v -2 ,v -3 Vehicle acceleration a, vehicle required torque T Demand Vehicle power demand P Demand Engine status S Engine Power battery state of charge SoC, power battery power P Battery The state space is structured as follows:

[0069] S={v,v -1 ,v -2 ,v -3 ,a,T Demand ,P Demand ,S Engine ,SoC,P Battery};

[0070] Following S202 are:

[0071] Using formula s∈S standardizes the state data in the state space;

[0072] in, For each standardized state data point, s represents a single state data point, S represents the state space, u represents the average value of each state data point, and σ represents the standard deviation of each state data point.

[0073] S203 constructs a reward function based on the real-time error of the power battery's state of charge and the degree of power battery degradation; that is, it suppresses power battery degradation while reducing the equivalent fuel consumption cost of the whole vehicle.

[0074] S203 specifically includes:

[0075] Using the formula r=-α·(p1·ΔSoC ref +p2·m fuel +p3·loss Battery Determine the reward function;

[0076] Among them, SoC ref (t) represents the reference trajectory of the power battery's state of charge at time t, SoC ref (t)=SoC0-λ·D(t), where SoC0 is the initial value of the state of charge of the power battery directly measured by the on-board sensor at the initial moment. SoC f The final values ​​of the state of charge of the power battery, D(t) and D, are determined by a global optimization method based on dynamic programming at the last moment. Total These represent the vehicle's current distance traveled and total mileage, respectively, ΔSoC ref ΔSoC is the real-time error of the state of charge of the power battery. ref (t) = SoC(t) - SoC ref (t), where SoC(t) is the current value of the power battery's state of charge directly measured by the onboard sensors at time t, m fuel For engine fuel consumption, m fuel =MAP Power (ΔP Engine ,n Engine ) = MAPTorque (T Engine ,n Engine ), MAP Power (ΔP Engine ,n Engine ) is the change in engine power ΔP Engine and engine speed n Engine m determined by referring to the fuel consumption MAP chart fuel With ΔP Engine and n Engine The relationship between MAP Torque (T Engine ,n Engine ) is determined by engine power T Engine and engine speed n Engine m determined by referring to the fuel consumption MAP chart fuel With T Engine and n Engine The relationship between Q loss For the normalized capacity loss of the power battery, A is the pre-coefficient factor, which is a constant; T Battery The temperature of the power battery is measured by a temperature sensor in the vehicle's power battery management system. C rate The charge / discharge rate of the power battery. Q0 is the rated capacity of the power battery, a fixed value for a given electric vehicle, and a normal quantity. Battery The charging / discharging current of the power battery is given by [value], and the power battery power P is given by [value]. Battery With output voltage U Battery The ratio of the output voltage U of the power battery Battery Measured by voltage sensors in the vehicle's power battery management system; B is a compensation factor, which is a constant; E a The activation energy of the power battery is a constant value for a specific type of power battery and is considered a normal quantity; R is the gas constant, which is a defined normal quantity; A h For the ampere-hour throughput of the power battery, z is the exponential factor, e is the natural logarithm, and loss is... Battery The degree of degradation of the power battery is represented by α, the scaling factor is α, p1, p2, and p3 are the weighting factors, and r is the reward function.

[0077] In addition, engine speed n Engine The relationship between the vehicle speed and the current speed v is: In the formula r roll Represents the radius of the wheel, a normal value, i0 and i g These represent the final drive ratio and the transmission ratio of the current gear, respectively, and are normal values.

[0078] Ah The specific calculation process is as follows: within any extremely short time interval [t, t+1], the charge / discharge rate C of the power battery rate If it can be considered to remain unchanged, then Based on the normalized capacity loss Q loss Throughput A h The expression for the differential is: The power battery capacity loss within any extremely short time interval [t, t+1] In the formula ΔA h The ampere-hour throughput of the power battery within any extremely short time interval [t, t+1] is defined as follows: Finally, the degree of degradation of the power battery loss Battery It can be calculated as

[0079] S204, determine the trained deep Q-network based on the action space, the state space, and the reward function; the trained deep Q-network includes: an Actor action network and a Critic evaluation network; the Actor action network takes the state data in the state space as input and the action variable value as output; the Critic evaluation network takes the state data and action variable value in the state space as input and the clutch state as output;

[0080] S204 specifically includes:

[0081] Construct the Actor action network and the Critic evaluation network respectively, and obtain the parameter set θ of the Actor action network and the parameter set ω of the Critic evaluation network respectively;

[0082] Determine the depth Q-network based on the Actor action network and the Critic evaluation network;

[0083] The deep Q-network is used as the target neural network and the evaluation neural network, respectively.

[0084] The Adam gradient descent method and reward function are used to train the parameter sets in the target neural network and the evaluation neural network respectively, and the trained deep Q network is determined.

[0085] Actor Action Network:

[0086] Set the number of neurons in the input layer: according to the state space expression S={v,v} in section 2. -1 ,v -2 ,v -3 ,a,T Demand ,P Demand ,S Engine ,SoC,P BatteryIt can be seen that the state space is composed of v, v -1 ,v -2 ,v -3 ,a,T Demand ,P Demand ,S Engine ,SoC,P Battery These 10 state variables constitute a space with a dimension of 10. Therefore, the number of neurons in the input layer is set to be equal to the dimension of the state space, which is 10 in total.

[0087] Set the number of neurons in the output layer: According to the expression for the action space in section 1, A = {0, {ΔP} Engine i Brake}}∪{1,{T Engine ,T ISG i Brake It can be seen that the action space is determined by ΔP when the clutch is disengaged, i.e., a(1) = 0. Engine i Brake These two action quantities constitute; when the clutch is closed, i.e., a(1) = 1, it is determined by T. Engine ,T ISG i Brake These three action variables constitute the input space. Based on the union of the mathematical relationships between these two cases, the dimension of the action space is 3. Therefore, the number of neurons in the output layer is set to be the same as the dimension of the continuous action space, a total of 3.

[0088] Setting the output layer activation function: The output layer activation function adopts the tanh function (a common existing activation function in the field of machine learning, whose expression is y = (e^(-tanh)) / (e^(-tanh))). x -e -x ) / (e x -e -x In the formula, x represents the generalized input and y represents the generalized output.

[0089] Set the number of hidden layers and the number of neurons in each layer: Set the number of hidden layers between the input and output layers to 3, and set 100 neurons in each layer;

[0090] Setting the activation functions for the hidden layers: All three hidden layer activation functions use the ReLU function (a common existing activation function in the field of machine learning, whose expression is y = max(0,x), where x represents the generalized input, y represents the generalized output, and max represents the maximum value sign).

[0091] This completes all the settings for the Actor action network, whose parameter set is denoted as θ.

[0092] Critic rating network:

[0093] Setting the number of input layer neurons: Based on the aforementioned Actor action network setup process, the Critic evaluation network has a state space dimension of 10 and an action space dimension of 3. Summing the two, the number of input layer neurons in the Critic evaluation network is set to 13.

[0094] Setting the number of output layer neurons: Based on the clutch state set in 1. which includes two states, disengagement and closure, the number of output layer neurons of the Critic evaluation network is set to 2.

[0095] Set the output layer activation function: The output layer activation function is tanh;

[0096] Set the number of hidden layers and the number of neurons in each layer: Set the number of hidden layers between the input and output layers to 3, and set 100 neurons in each layer;

[0097] Set the activation functions for the hidden layers: all three activation functions for the hidden layers use the ReLU function.

[0098] This completes all the settings for the Critic evaluation network, whose parameter set is denoted as ω.

[0099] Replication and updating of neural networks:

[0100] After the above neural network is constructed, let it be called the TargetNetwork. A copy of the exact same network is made and set as the EvaluationNetwork. The parameters of the target neural network (ω) are then alternately calculated using the Adam gradient descent method (a technical term). T ,θ T ) and the evaluation neural network parameter set (ω E ,θ E The update of the target neural network parameter set (ω) T ,θ T ) by the target Actor action network parameter set θ T And the target Critic evaluation network parameter set ω T Composition, evaluation of neural network parameter set (ω) E ,θ E The evaluation of the Actor action network parameter set θ E And evaluate the Critic evaluation network parameter set ω E composition.

[0101] Specifically, let the symbol Q(s,a)s∈S,a∈A denote the state-action value function. For the coaxial series-parallel hybrid vehicle problem involving a mixed action space as described in this invention, its state-action value function can be denoted as Q(s,k,x). k )s∈S,k∈{0,1},xk ∈{{ΔP Engine i Brake},{T Engine ,T ISG i Brake}}, then the symbol Q T (s,k,x k ) and Q E (s,k,x k ) represent the target neural network state-action value function and the evaluation neural network state-action value function, respectively.

[0102] The parameter set update process of a neural network is as follows:

[0103] First, at the current time t, based on the target neural network state-action value function Q... T (s,k,x k Calculate the target state action value y Targe t, In the formula, r(t) represents the value of the reward function at the current time t, γ is an adjustable parameter, s(t-1) represents the value of the state variable at the previous time t-1, and due to the short-time memory of the neural network, its value can be directly obtained from the storage module of the vehicle control unit, and k(t) represents the value of the discrete action variable k at the current time t. This indicates that the parameter set of the target neural network at the current time t is (ω T (t),θ T The value of the continuous action variable when (t)).

[0104] Next, the parameter set θ of the target Actor action network at the current time t is calculated. T The target neural network loss is determined by (t). and the network parameter set ω evaluated by the target Critic T The target neural network loss is determined by (t).

[0105]

[0106]

[0107] In the formula This indicates that the parameter set of the target Critic evaluation network at the current time t is ω. T The values ​​of the continuous action variables at time (t). θ represents the parameter set of the target Actor action network at the current time t. T The value of the continuous action variable at time (t).

[0108] Furthermore, the gradient for updating the neural network parameter set at the current time t is calculated and denoted as follows:

[0109]

[0110] In the two formulas This indicates that at the previous time t-1, when the evaluation neural network parameter set is (ω E (t-1),θ E The value of the continuous action variable at time (t-1).

[0111] Furthermore, the parameter set of the evaluation neural network at the current time t (ω) is calculated. E (t),θ E (t)):

[0112]

[0113]

[0114] Finally, update the parameter set of the target neural network:

[0115]

[0116] This completes the update of the parameter sets for both the target neural network and the evaluation neural network.

[0117] Set a stability threshold for the neural network parameter set when updating the parameter sets (ω) of both the target neural network and the evaluation neural network. T ,θ T ), (ω E ,θ E When the parameters remain stable within the threshold range, stop updating and set the optimal set of dual network parameters (ω) at this point. * T ,θ * T ), (ω * E ,θ * E The data is stored in the data recording module of the vehicle controller.

[0118] S205, based on the real-time state data in the state space, a trained deep Q-network is used to determine the real-time action variable values, and then the power distribution control of the electric vehicle is performed based on the real-time action variable values.

[0119] Based on the vehicle speed v(t) measured at the current time t by the vehicle speed sensor and the acceleration a(t) measured at the current time by the acceleration sensor, the following formula is used:

[0120]

[0121]

[0122] F r (t)=m(t)·g·f

[0123] F g (t)=m(t)·g·θ

[0124] In the formula, P Demand (t) represents the total power demand of the vehicle, v(t) represents the speed of the vehicle at the current moment, and F a (t) represents the air resistance experienced by the vehicle at the current moment, F r (t) represents the rolling resistance experienced by the vehicle at the current moment, F g (t) represents the gradient resistance experienced by the vehicle at the current moment, η Motor The efficiency of the motor is represented by m(t), the total mass of the vehicle at present is represented by C. D The aerodynamic drag coefficient of a vehicle is represented by A, the frontal area by f, the rolling resistance coefficient by θ, and the gradient drag coefficient by g, all in m / s². 2 .

[0125] During vehicle operation, the short-term change in its mass is mainly due to the consumption of fuel, so the mass change is negligible, and m(t) is treated as a constant. Therefore, the above formula changes as follows:

[0126] The symbols in the formula are the same as those described above.

[0127] Based on the vehicle's required torque T Demand (t) and demand power P Demand The relationship between (t) r roll The radius of the wheel is a normal value, and the required torque T of the entire vehicle at the current time t is determined. Demand (t).

[0128] Engine Status S Engine The throttle position sensor in the engine assembly can be used to obtain the battery's state of charge (SoC), which can be fed back in real time by the vehicle's battery management system (BMS). The battery power (P) is also provided. Battery (t) can also be obtained from the real-time voltage U collected by the power battery management system (BMS). Battery (t) and current I Battery (t), based on equation P Battery (t)=U Battery (t)·I Battery (t) is the result of calculation.

[0129] The highest speed in history in the first 3 seconds v -1 v -2v -3 It can directly read the stored values ​​from the data recording module in the vehicle controller.

[0130] Based on the values ​​of the aforementioned state variables and the optimal set of dual network parameters, the vehicle controller can output the values ​​of the action variables, thereby achieving optimal power distribution control during vehicle operation. This achieves optimal energy management of the vehicle while considering the suppression of power battery degradation, thus comprehensively improving the vehicle's economy.

[0131] The present invention will be further illustrated by the following embodiments. A certain coaxial series-parallel hybrid electric vehicle has the parameters shown in Table 1:

[0132] Table 1

[0133]

[0134]

[0135] SoC follow-up results as follows Figure 3 As shown, the power distribution results between the engine and the power battery are as follows: Figure 4 As shown, the cost results for the vehicle's power battery power consumption, fuel consumption, and power battery loss are as follows: Figure 5 As shown, based on the energy management strategy of the present invention that considers the degradation suppression of the power battery, the optimal power distribution control during vehicle operation is achieved, thereby comprehensively improving the vehicle's economy.

[0136] Figure 6 This is a schematic diagram of an electric vehicle energy management system structure that considers the suppression of power battery degradation, as provided by the present invention. Figure 6 As shown, the present invention provides an electric vehicle energy management system that considers the suppression of power battery degradation, comprising:

[0137] The motion space determination module 601 is used to determine the motion space; the motion space includes: clutch state and corresponding motion variable values; the motion variable values ​​include: engine power change, engine output torque, output torque of the starter / generator integrated motor, and the ratio of mechanical braking torque to required torque; the clutch state includes: clutch disengagement and clutch engagement.

[0138] The state space determination module 602 is used to determine the state space based on the state data of the electric vehicle; the state data includes: current driving speed, historical speed, vehicle acceleration, vehicle torque demand, vehicle power demand, engine status, power battery state of charge, and power battery power.

[0139] The reward function construction module 603 is used to construct a reward function based on the real-time error of the state of charge of the power battery and the degree of degradation of the power battery.

[0140] The trained deep Q-network determination module 604 is used to determine the trained deep Q-network based on the action space, the state space, and the reward function; the trained deep Q-network includes: an Actor action network and a Critic evaluation network; the Actor action network takes the state data in the state space as input and the action variable value as output; the Critic evaluation network takes the state data and action variable value in the state space as input and the clutch state as output;

[0141] The power distribution control module 605 is used to determine the real-time action variable value using a trained deep Q-network based on the real-time state data in the state space, and then perform power distribution control of the electric vehicle based on the real-time action variable value.

[0142] The motion space determination module 601 specifically includes the following formula:

[0143] W={0,{ΔP Engine i Brake}}∪{1,{T Engine ,T ISG i Brake}};

[0144] Where W represents the action space, 0 represents clutch disengagement, 1 represents clutch engagement, and ΔP Engine i represents the change in engine power. Brake T represents the ratio of mechanical braking torque to the required torque. Engine T is the engine output torque. ISG This refers to the output torque of the integrated starter / generator motor.

[0145] The electric vehicle energy management system considering the suppression of power battery degradation provided by the present invention further includes:

[0146] The standardization module is used to utilize formulas s∈S standardizes the state data in the state space;

[0147] in, For each standardized state data point, s represents a single state data point, S represents the state space, u represents the average value of each state data point, and σ represents the standard deviation of each state data point.

[0148] The reward function construction module 603 specifically includes:

[0149] The reward function building unit is used to utilize the formula r = -α·(p1·ΔSoC) ref +p2·m fuel +p3·lossBattery Determine the reward function;

[0150] Among them, SoC ref (t) represents the reference trajectory of the power battery's state of charge at time t, SoC ref (t)=SoC0-λ·D(t), where SoC0 is the initial value of the state of charge of the power battery directly measured by the on-board sensor at the initial moment. SoC f The final values ​​of the state of charge of the power battery, D(t) and D, are determined by a global optimization method based on dynamic programming at the last moment. Total These represent the vehicle's current distance traveled and total mileage, respectively, ΔSoC ref ΔSoC is the real-time error of the state of charge of the power battery. ref (t) = SoC(t) - SoC ref (t), where SoC(t) is the current value of the power battery's state of charge directly measured by the onboard sensors at time t, m fuel For engine fuel consumption, m fuel =MAP Power (ΔP Engine ,n Engine ) = MAP Torque (T Engine ,n Engine ), MAP Power (ΔP Engine ,n Engine ) is the change in engine power ΔP Engine and engine speed n Engine m determined by referring to the fuel consumption MAP chart fuel With ΔP Engine and n Engine The relationship between MAP Torque (T Engine ,n Engine ) is determined by engine power T Engine and engine speed n Engine m determined by referring to the fuel consumption MAP chart fuel With T Engine and n Engine The relationship between Q loss For the normalized capacity loss of the power battery, A is the pre-coefficient factor, which is a constant; T Battery The temperature of the power battery is measured by a temperature sensor in the vehicle's power battery management system. C rate The charge / discharge rate of the power battery. Q0 is the rated capacity of the power battery, a fixed value for a given electric vehicle, and a normal quantity. BatteryThe charging / discharging current of the power battery is given by [value], and the power battery power P is given by [value]. Battery With output voltage U Battery The ratio of the output voltage U of the power battery Battery Measured by voltage sensors in the vehicle's power battery management system; B is a compensation factor, which is a constant; E a The activation energy of the power battery is a constant value for a specific type of power battery and is considered a normal quantity; R is the gas constant, which is a defined normal quantity; A h For the ampere-hour throughput of the power battery, z is the exponential factor, e is the natural logarithm, and loss is... Battery The degree of degradation of the power battery is represented by α, the scaling factor is α, p1, p2, and p3 are the weighting factors, and r is the reward function.

[0151] The trained deep Q-network determination module 604 specifically includes:

[0152] The parameter set acquisition unit is used to construct the Actor action network and the Critic evaluation network respectively, and to acquire the parameter sets of the Actor action network and the Critic evaluation network respectively.

[0153] A deep Q-network determination unit is used to determine the deep Q-network based on the Actor network and the Critic evaluation network.

[0154] A target neural network and an evaluation neural network determination unit is used to determine the deep Q-network as the target neural network and the evaluation neural network, respectively.

[0155] The trained deep Q-network determines the unit, which is used to train the parameter sets in the target neural network and the evaluation neural network respectively using Adam gradient descent and reward function, and determines the trained deep Q-network.

[0156] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the systems disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple; relevant parts can be referred to the method section.

[0157] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. An energy management method for electric vehicles that considers the suppression of battery degradation, characterized in that, include: Define the action space; The action space includes: clutch status and corresponding action variable values; the action variable values ​​include: engine power change, engine output torque, output torque of the starter / generator integrated motor, and the ratio of mechanical braking torque to required torque; the clutch status includes: clutch disengagement and clutch engagement; the electric vehicle model is a coaxial series-parallel hybrid vehicle with a mixed action space. The state space is determined based on the state data of the electric vehicle; the state data includes: current driving speed, historical speed, vehicle acceleration, vehicle torque demand, vehicle power demand, engine status, power battery state of charge, and power battery power. A reward function is constructed based on the real-time error of the state of charge of the power battery and the degree of degradation of the power battery; The trained deep Q-network is determined based on the action space, the state space, and the reward function; the trained deep Q-network includes an Actor action network and a Critic evaluation network; the Actor action network takes the state data in the state space as input and the action variable value as output; the Critic evaluation network takes the state data and action variable value in the state space as input and the clutch state as output. Based on the real-time state data in the state space, a trained deep Q-network is used to determine the real-time action variable values, and then the power distribution control of the electric vehicle is performed based on the real-time action variable values.

2. The electric vehicle energy management method considering the suppression of power battery degradation according to claim 1, characterized in that, The determination of the action space specifically includes the following formula: ； in, W This represents the operating space, where 0 indicates clutch disengagement and 1 indicates clutch engagement. This represents the change in engine power. The ratio of mechanical braking torque to required torque. For engine output torque, This refers to the output torque of the integrated starter / generator motor.

3. The electric vehicle energy management method considering the suppression of power battery degradation according to claim 1, characterized in that, The process of determining the state space based on the electric vehicle's state data then includes: Using formula The state data in the state space is standardized; among which, For standardized single-state data, For single state data, Representing the state space, This is the average value of the data for each state. denoted as the standard deviation of the data for each state.

4. The electric vehicle energy management method considering the suppression of power battery degradation according to claim 1, characterized in that, The reward function, constructed based on the real-time error of the power battery's state of charge and the degree of battery degradation, specifically includes: Using formula Determine the reward function; where, for t Reference trajectory of the state of charge of the power battery at any given time. , The initial value of the state of charge of the power battery is directly measured by the on-board sensors at the initial moment. , This is the final value of the state of charge of the power battery determined by a global optimization method based on dynamic programming at the last moment. and These are the vehicle's current distance traveled and total mileage, respectively. This refers to the real-time error of the power battery's state of charge. , The onboard sensors at time t directly measure the current value of the power battery's state of charge. For engine fuel consumption, , For the change in engine power and engine speed Determined by checking the fuel consumption MAP chart and and The relationship between them For the engine power and engine speed Determined by checking the fuel consumption MAP chart and and The relationship between them The degree of degradation of the power battery, , For the normalized capacity loss of the power battery, , The coefficient factor is a constant. The power battery temperature is measured by a temperature sensor in the vehicle's power battery management system. The charge / discharge rate of the power battery. ; This refers to the rated capacity of the power battery, which is a fixed value for a given electric vehicle and is considered a normal quantity. The charging / discharging current of the power battery is [value], and the power of the power battery is [value]. With output voltage The ratio of the output voltage of the power battery Measured by voltage sensors in the vehicle's power battery management system; , where is the compensation factor and is a constant; The activation energy of the power battery is a fixed value for a specific type of power battery, and it is a normal quantity. is the gas constant, and is a defined normal quantity; For the ampere-hour throughput of the power battery, , As an exponential factor, e It is the natural logarithm. This is the scaling factor. p 1, p 2, p 3 represents the weighting coefficients. This is the reward function.

5. The electric vehicle energy management method considering the suppression of power battery degradation according to claim 1, characterized in that, The step of determining the trained deep Q-network based on the action space, the state space, and the reward function specifically includes: Construct the Actor action network and the Critic evaluation network respectively, and obtain the parameter sets of the Actor action network and the Critic evaluation network respectively; Determine the depth Q-network based on the Actor action network and the Critic evaluation network; The deep Q-network is used as the target neural network and the evaluation neural network, respectively. The Adam gradient descent method and reward function are used to train the parameter sets in the target neural network and the evaluation neural network respectively, and the trained deep Q network is determined.

6. An electric vehicle energy management system considering the suppression of power battery degradation, characterized in that, include: The motion space determination module is used to determine the motion space; The action space includes: clutch status and corresponding action variable values; the action variable values ​​include: engine power change, engine output torque, output torque of the starter / generator integrated motor, and the ratio of mechanical braking torque to required torque; the clutch status includes: clutch disengagement and clutch engagement; the electric vehicle model is a coaxial series-parallel hybrid vehicle with a mixed action space. The state space determination module is used to determine the state space based on the state data of the electric vehicle; the state data includes: current driving speed, historical speed, vehicle acceleration, vehicle torque demand, vehicle power demand, engine status, power battery state of charge, and power battery power. The reward function construction module is used to construct a reward function based on the real-time error of the power battery's state of charge and the degree of degradation of the power battery. The trained deep Q-network determination module is used to determine the trained deep Q-network based on the action space, the state space, and the reward function; the trained deep Q-network includes: an Actor action network and a Critic evaluation network; the Actor action network takes the state data in the state space as input and the action variable value as output; the Critic evaluation network takes the state data and action variable value in the state space as input and the clutch state as output; The power distribution control module is used to determine the real-time action variable values ​​using a trained deep Q-network based on the real-time state data in the state space, and then perform power distribution control of the electric vehicle based on the real-time action variable values.

7. An electric vehicle energy management system considering the suppression of power battery degradation according to claim 6, characterized in that, The motion space determination module specifically includes the following formulas: ； in, W This represents the operating space, where 0 indicates clutch disengagement and 1 indicates clutch engagement. This represents the change in engine power. The ratio of mechanical braking torque to required torque. For engine output torque, This refers to the output torque of the integrated starter / generator motor.

8. An electric vehicle energy management system considering the suppression of power battery degradation according to claim 6, characterized in that, Also includes: The standardization module is used to utilize formulas Standardize the state data in the state space; in, For standardized single-state data, For single state data, Representing the state space, This is the average value of the data for each state. denoted as the standard deviation of the data for each state.

9. An electric vehicle energy management system considering the suppression of power battery degradation according to claim 6, characterized in that, The reward function construction module specifically includes: Reward function building blocks, used to utilize formulas Determine the reward function; in, Let t be the reference trajectory of the state of charge of the power battery. , The initial value of the state of charge of the power battery is directly measured by the on-board sensors at the initial moment. , This is the final value of the state of charge of the power battery determined by a global optimization method based on dynamic programming at the last moment. and These are the vehicle's current distance traveled and total mileage, respectively. This refers to the real-time error of the power battery's state of charge. , The onboard sensors at time t directly measure the current value of the power battery's state of charge. For engine fuel consumption, , For the change in engine power and engine speed Determined by checking the fuel consumption MAP chart and and The relationship between them For the engine power and engine speed Determined by checking the fuel consumption MAP chart and and The relationship between them The degree of degradation of the power battery, , For the normalized capacity loss of the power battery, , The coefficient factor is a constant. The temperature of the power battery is measured by temperature sensors in the vehicle's power battery management system. The charge / discharge rate of the power battery. , This refers to the rated capacity of the power battery, a fixed value for a given electric vehicle, and a normal value. The charging / discharging current of the power battery is [value], and the power of the power battery is [value]. With output voltage The ratio of the output voltage of the power battery Measured by voltage sensors in the vehicle's power battery management system; , where is the compensation factor and is a constant; The activation energy of the power battery is a fixed value for a specific type of power battery, and it is a normal quantity. is the gas constant, and is a defined normal quantity; For the ampere-hour throughput of the power battery, , Here, e is the exponential factor, and e is the natural logarithm. p1, p2, and p3 are scaling factors, and p1, p2, and p3 are weighting factors, respectively. This is the reward function.

10. An electric vehicle energy management system considering the suppression of power battery degradation according to claim 6, characterized in that, The trained deep Q-network determination module specifically includes: The parameter set acquisition unit is used to construct the Actor action network and the Critic evaluation network respectively, and to acquire the parameter sets of the Actor action network and the Critic evaluation network respectively. A deep Q-network determination unit is used to determine the deep Q-network based on the Actor network and the Critic evaluation network. A target neural network and an evaluation neural network determination unit is used to determine the deep Q-network as the target neural network and the evaluation neural network, respectively. The trained deep Q-network determines the unit, which is used to train the parameter sets in the target neural network and the evaluation neural network respectively using Adam gradient descent and reward function, and determines the trained deep Q-network.

Images