Lithium ion battery low-temperature heating and charging collaborative optimization control method and system based on reinforcement learning

Through deep reinforcement learning technology, the low-temperature heating and charging of lithium-ion batteries are optimized in a coordinated manner, which solves the problems of slowing charging speed and poor safety in low-temperature environments, and achieves a safe, efficient and fast charging effect.

CN119009203AActive Publication Date: 2024-11-22UNIV OF JINAN

Patent Information

Application Number
CN202411049071.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-01
Publication Date
2024-11-22
Estimated Expiration
2044-08-01

AI Technical Summary

Technical Problem

The charging speed of lithium-ion batteries slows down in low-temperature environments, and traditional charging methods cannot effectively control the temperature rise, resulting in poor safety and the inability to achieve coordinated optimization of heating and charging, limiting the charging efficiency in low-temperature environments.

Method used

Adopt the coordinated optimization control method of low-temperature heating and charging of lithium-ion batteries based on deep reinforcement learning. By constraining multiple physical processes such as battery current, voltage, lithium-ion boundary, core temperature, etc., and combining the reward and punishment mechanism, the allocation of heating AC current and charging DC is optimized. Current, achieve safe and fast charging.

Benefits of technology

In a low-temperature environment, the charging efficiency is significantly improved, the battery life is extended, the safety and speed of the charging process is ensured, and the coordinated optimization of heating and charging is achieved, avoiding the slowdown in charging speed and lithium ion precipitation problems caused by low temperatures.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119009203A_ABST
    Figure CN119009203A_ABST
Patent Text Reader

Abstract

The invention relates to a lithium ion battery low-temperature heating and charging collaborative optimization control method and system based on reinforcement learning. The method comprises the following steps: establishing an electrothermal coupling model of a lithium ion battery; initializing an action space and a state space of the reinforcement learning model, and initializing network parameters of a strategy network and a value network of the reinforcement learning model; multi-state sensing: acquiring state information of the lithium ion battery in real time; target function, reward function and constraint design: the reward function is set, upper and lower limits of action and state variables are designed, and targets include charging speed and temperature rise heat production; according to the current state information, the optimal heating alternating current and the optimal charging direct current are calculated through a trained reinforcement learning model and a learning algorithm; and controlling the heater and the charger to work and adjusting the temperature and the charging speed of the battery according to optimization results, namely the optimal heating alternating current and the optimal charging direct current which are calculated. According to the invention, a safe, efficient and rapid charging effect is realized, and the method has significant application prospects and technical advantages.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to a collaborative optimization control method and system for low-temperature heating and charging of a lithium-ion battery based on reinforcement learning, and belongs to the technical field of battery management. Background Art

[0002] One of the main bottlenecks in the development of lithium-ion battery charging technology is the safe and fast charging of lithium-ion batteries in low-temperature environments. Ambient temperature has a significant impact on the charging performance of lithium-ion batteries. Especially under low-temperature conditions, the chemical reaction rate inside the battery is significantly reduced, resulting in a slower charging speed. If the current is increased blindly, lithium ions will be precipitated, accelerating the aging and life reduction of the battery. In charging technology, the three core indicators of high efficiency, safety and speed often have competitive contradictions, especially the contradiction between safety and speed.

[0003] Traditional battery charging methods at room temperature include constant current and constant voltage, multi-stage constant current, and constant power. Among them, the constant current and constant voltage method is widely used due to its simple design, but it fails to effectively control the temperature rise during fast charging, resulting in poor safety, which is also one of the main causes of fires in electric vehicles. Performance deteriorates at low temperatures. Existing studies usually preheat the battery before low-temperature charging, and charge the battery after heating it to a specific temperature. However, this charging strategy divides the preheating and charging processes into two independent control links, and cannot achieve simultaneous output of AC and DC currents during the charging stage, and fails to achieve coordinated optimization of heating and charging control, which limits the charging efficiency in low-temperature environments.

[0004] In addition, traditional charging methods are mostly constrained by macroscopic characteristics such as battery terminal voltage and surface temperature, ignoring the internal electrical-thermal coupling mechanism and core temperature characteristics of the battery, making it difficult to achieve comprehensive optimization of charging speed and safety performance. Summary of the invention

[0005] In view of the shortcomings of the prior art, the present invention provides a collaborative optimization control method for low-temperature heating and charging of lithium-ion batteries based on reinforcement learning;

[0006] The present invention comprehensively considers the constraints of the core temperature state and multi-physical processes inside the battery, and proposes a collaborative optimization control method for heating and charging based on deep reinforcement learning. The present invention aims to achieve safe and fast charging. By constraining multi-physical processes such as battery current, voltage, lithium precipitation boundary, core temperature, etc., and combining a reward and punishment mechanism, setting safety thresholds for voltage, current and temperature, and optimizing the allocation of heating AC current and charging DC current components, the optimal safe and fast charging method at low temperatures is obtained.

[0007] The present invention also provides a collaborative optimization control system for low-temperature heating and charging of lithium-ion batteries based on reinforcement learning.

[0008] Terminology explanation:

[0009] 1. SOC (State of Charge) refers to the current state of charge of the battery, usually expressed as a percentage (0% to 100%). It describes how much energy the battery stores or how much energy it can still provide.

[0010] 2. DDPG (Deep Deterministic Policy Gradient), deep deterministic policy gradient algorithm, DDPG is a reinforcement learning algorithm based on deep learning, which combines deep neural networks and policy gradient methods. DDPG consists of two main parts: a policy network (Actor) and a value function network (Critic). The Actor network is used to output actions, while the Critic network is used to evaluate the value of actions.

[0011] The technical solution of the present invention is:

[0012] A collaborative optimization control method for low-temperature heating and charging of lithium-ion batteries based on reinforcement learning, including:

[0013] Model equation establishment: Establish the electric-thermal coupling model of lithium-ion batteries;

[0014] Initialization: Initialize the action space and state space of the reinforcement learning model, and initialize the network parameters of the policy network and value network of the reinforcement learning model;

[0015] Multi-state sensing: Real-time acquisition of lithium-ion battery status information, including AC current, DC current, voltage, SOC, and internal core temperature;

[0016] Objective function, reward function and constraint design: Set the reward function and design the upper and lower limits of actions and state variables. The goals include charging speed and temperature rise heat generation.

[0017] Strategy optimization: Input the current state variable into the strategy network. The strategy network is responsible for selecting two actions including AC action and DC action according to the current state, and inputting the action into the electric-thermal coupling model of the lithium-ion battery, obtaining the next state and outputting the corresponding reward. At the same time, the current experience is stored in the experience replay pool, and samples are randomly drawn from the experience replay pool for training; the currently selected action is evaluated through the value network, the state and network parameters are updated, and the reinforcement learning model is optimized until the cyclic training reaches the maximum number of iterations; the action value that maximizes the reward return under the constraints of the reward function is output;

[0018] Control execution: According to the current state of the battery and under the constraints of the reward function, the heater and charger are controlled to work according to the optimally calculated heating AC and charging DC, and the heating AC and charging DC are output, so that the lithium-ion battery is charged according to the optimal heating and charging current.

[0019] Further preferably, the action space includes an alternating current component and a direct current component of the battery, and the state space includes a state of charge (SOC) of the battery, a battery voltage, and an internal core temperature of the battery.

[0020] Further preferably, each state variable s in the state space t is defined as:

[0021] s t ={SOC,[T c ] nor ,[U O ] nor}

[0022] Among them, SOC represents the state of charge of the battery, [T c ] nor Indicates the internal core temperature of the battery, [U O ] nor Indicates the battery voltage.

[0023] Further preferably, each action variable a in the action space t is defined as:

[0024] a t ={[I AC ] nor ,[I DC ] nor}

[0025] Among them, [I AC ] nor represents the AC current component of the battery, [I DC ] nor Represents the DC current component of the battery.

[0026] Further preferably, both the state variables and the action variables are normalized to the interval range of [0,1], and the normalization calculation is shown as follows:

[0027]

[0028] Among them, X represents the current state variable or action variable value, X max Indicates the maximum value of the state variable or action variable to be set, X min Indicates the minimum value of the state variable or action variable, X norm Represents the normalized value of a state variable or action variable.

[0029] Further preferably, two objective functions are set, including a charging speed objective function and a temperature rise heat generation objective function;

[0030] The charging speed objective function J1 is expressed as:

[0031] J1=minNt

[0032] Among them, t refers to the sampling period, N refers to the number of sampling steps to complete charging; the charging speed objective function J1 is to achieve charging in the shortest time;

[0033] The objective function J2 of heating and heat generation is expressed as:

[0034] J2=maxQ gen

[0035] Among them, Q gen Refers to the total heat generated during the battery heating and charging process.

[0036] Further preferably, the reward function is expressed as:

[0037]

[0038] Among them, ω i are the weight coefficients of different objective functions, i = 1 to 6; r soc (t) is the battery SOC reward function, r Qgen (t) is the battery heat generation reward function, r volt (t) is the battery voltage reward function, r temp (t) is the battery core temperature reward function, r IDC (t) is the battery DC current reward function, r IAC (t) is the battery AC current reward function;

[0039] r soc (t) The reward function is expressed as:

[0040] r soc (t)=|SOC tar -SOC(t)|

[0041] Among them, SOC tar represents the target SOC value, SOC(t) represents the current battery SOC;

[0042] r Qgen (t) The reward function is expressed as:

[0043]

[0044] r volt (t), r temp(t), r IDC (t), r IAC (t) is a soft constraint reward function, which is specifically expressed as:

[0045]

[0046]

[0047] Among them, U O (t) represents the current battery voltage, U max Indicates the maximum threshold of voltage; T c (t) represents the current battery core temperature, T max Indicates the maximum threshold of temperature; I DC (t) represents the current battery DC component, I DC_max Indicates the maximum threshold of the DC component; I AC (t) represents the current battery AC component, I AC_max Indicates the maximum threshold of the AC component.

[0048] According to the preferred embodiment of the present invention, the electrothermal coupling model of the lithium-ion battery includes an electrical model and a two-state thermal model. The electrical model includes an ohmic impedance R o , polarization impedance R ct and the charge transfer process capacitance C dl , voltage source U OCV , inductance L and Warburg impedance Z w ;

[0049] The two-state thermal model is specifically expressed as:

[0050]

[0051] Among them, T c 、T s and T a is the battery core, surface and ambient temperature; R i and R o are heat conduction resistance and convection resistance respectively; C c and C s Represents the internal heat capacity and surface heat capacity of lithium-ion batteries.

[0052] Preferably, according to the present invention, the reinforcement learning model includes a policy network and a value network; the policy network is used to output actions, and the value network is used to evaluate the value of actions.

[0053] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of a method for coordinated optimization control of low-temperature heating and charging of a lithium-ion battery based on reinforcement learning are implemented.

[0054] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of a method for coordinated optimization control of low-temperature heating and charging of a lithium-ion battery based on reinforcement learning.

[0055] A collaborative optimization control system for low-temperature heating and charging of lithium-ion batteries based on reinforcement learning, including:

[0056] The model building module is configured to: build an electric-thermal coupling model of a lithium-ion battery;

[0057] The status acquisition module is configured to: acquire status information of the lithium-ion battery in real time;

[0058] The reinforcement learning module is configured to: calculate the optimal heating power and charging current through a learning algorithm using a trained reinforcement learning model according to current state information;

[0059] The control execution module is configured to: control the operation of the heater and the charger and adjust the battery temperature and the charging speed according to the optimization result, i.e. the calculated optimal heating power and charging current.

[0060] The beneficial effects of the present invention are:

[0061] The present invention proposes a solution with significant advantages and innovation in the low-temperature charging technology of lithium-ion batteries, and specifically has the following advantages:

[0062] 1. Multi-physical process constraint optimization: By comprehensively constraining the battery's current, voltage, lithium precipitation boundary, core temperature and other multi-physical processes, the present invention takes into account the microscopic and macroscopic characteristics of the battery. In a low-temperature environment, the heating and charging process of the battery is fully optimized, effectively avoiding the slowdown of charging speed and lithium ion precipitation caused by low temperature, thereby extending the battery life and improving the charging efficiency.

[0063] 2. Deep reinforcement learning collaborative control strategy: The present invention uses a collaborative optimization control method for heating and charging based on deep reinforcement learning, which can achieve the optimal allocation of AC heating components and charging DC components, so that charging can be performed simultaneously during the heating process. This method not only improves the charging speed, but also ensures the safety of the charging process, effectively solving the safety hazards caused by improper temperature rise control in traditional methods.

[0064] 3. Innovative low-temperature charging method: There is no low-temperature charging method on the market or in existing research that combines heating of the AC charging component with charging DC optimization. The present invention has achieved a breakthrough in this regard, which can significantly improve the charging efficiency in a low-temperature environment, and simultaneously perform thermal management during the charging process, ensuring the safety and efficiency of the overall process.

[0065] 4. Safe and fast charging adaptation: The present invention can construct a reinforcement learning reward function according to the current state of the battery. Under the constraints of battery current, voltage, lithium deposition and core temperature, it can optimize and calculate the heating AC and charging DC, control the heater and charger to output heating AC and charging DC, and realize the adaptive adjustment of heating and charging current with the battery state, ensuring that the charging temperature rises quickly in a low temperature environment, and charging is safe and fast;

[0066] In summary, the present invention, in the lithium-ion battery charging technology at low temperature, achieves safe, efficient and fast charging effects through comprehensive physical process constraints, the application of deep reinforcement learning and innovative heating and charging collaborative optimization strategies, and has significant application prospects and technical advantages. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] Figure 1 This is a schematic diagram of the training process of the heating and charging optimization algorithm based on DDPG;

[0068] FIG2( a ) is a schematic diagram showing the core temperature comparison results of the collaborative optimization control method of the present invention and the traditional preheating-before-charging strategy;

[0069] FIG2( b ) is a schematic diagram showing the comparison results of the maximum allowable DC current and the DC current of the collaborative optimization control method of the present invention and the traditional preheating-before-charging strategy;

[0070] FIG2( c ) is a schematic diagram showing the SOC comparison results of the collaborative optimization control method of the present invention and the traditional preheating-then-charging strategy. DETAILED DESCRIPTION

[0071] The present invention will be further defined below in conjunction with the accompanying drawings and embodiments, but is not limited thereto.

[0072] Example 1

[0073] A collaborative optimization control method for low-temperature heating and charging of lithium-ion batteries based on reinforcement learning, including:

[0074] Model equation establishment: Establish the electric-thermal coupling model of lithium-ion batteries;

[0075] Initialization: Initialize the action space and state space of the reinforcement learning model, and initialize the network parameters of the policy network and value network of the reinforcement learning model;

[0076] Multi-state sensing: Real-time acquisition of lithium-ion battery status information, including AC current, DC current, voltage, SOC, and internal core temperature;

[0077] Objective function, reward function and constraint design: Set the reward function and design the upper and lower limits of actions and state variables. The goals include charging speed and temperature rise heat generation.

[0078] Strategy optimization: Input the current state variable into the strategy network. The strategy network is responsible for selecting two actions including AC action and DC action according to the current state, and inputting the action into the electric-thermal coupling model of the lithium-ion battery, obtaining the next state and outputting the corresponding reward. At the same time, the current experience (state, action, reward, next action) is stored in the experience replay pool, and samples are randomly drawn from the experience replay pool for training; the currently selected action is evaluated through the value network, the state and network parameters are updated, and the reinforcement learning model is optimized until the cyclic training reaches the maximum number of iterations; the action value that maximizes the reward return under the constraints of the reward function is output;

[0079] Control execution: According to the current state of the battery and under the constraints of the reward function, the heater and charger are controlled to work according to the optimally calculated heating AC and charging DC, and the heating AC and charging DC are output, so that the lithium-ion battery is charged according to the optimal heating and charging current.

[0080] Example 2

[0081] The method for coordinated optimization control of low-temperature heating and charging of a lithium-ion battery based on reinforcement learning according to Embodiment 1 includes:

[0082] This embodiment establishes an electrothermal coupling model of a lithium-ion battery, such as Figure 1 As shown, it includes an electrical model and a two-state thermal model. The electrical model includes an ohmic impedance R o , polarization impedance R ct and the charge transfer process capacitance C dl , voltage source U OCV , inductance L and Warburg impedance Z w ;

[0083] The output voltage of a lithium-ion battery can be calculated as follows:

[0084] U O =U OCV +I·Z total

[0085] Among them, U O is the output voltage of the lithium-ion battery. OCV is the OCV of a lithium-ion battery. I is the current flowing through a lithium-ion battery and is positive when charging.

[0086] The charging current I can be calculated by superimposing the AC and DC components as follows:

[0087] I=I DC +I AC sin(2πf*t)

[0088] Among them I DCis the DC component, I AC is the peak-to-peak value of the sinusoidal AC component.

[0089] When an AC component is applied to a lithium-ion battery, the heat generated is mainly determined by its real impedance. The total heat generated by the battery is Q gen and real impedance Z Re It can be expressed as:

[0090]

[0091] A two-state thermal model is used to capture the thermal characteristics of lithium-ion batteries. The two-state thermal model is specifically described as:

[0092]

[0093] Among them, T c , T s and T a is the battery core, surface and ambient temperature; R i and R o are heat conduction resistance and convection resistance respectively; C c and C s Represents the internal heat capacity and surface heat capacity of lithium-ion batteries.

[0094] The reinforcement learning model consists of a policy network and a value network; the policy network is used to output actions, while the value network is used to evaluate the value of actions.

[0095] The deep deterministic policy gradient algorithm DDPG (Deep Deterministic Policy Gradient) is used to optimize the battery heating AC and charging DC current.

[0096] The action space includes the AC current component and the DC current component of the battery, and the state space includes the battery's state of charge (SOC), battery voltage, and battery internal core temperature.

[0097] Each state variable s in the state space t is defined as:

[0098] s t ={SOC,[T c ] nor ,[U O ] nor}

[0099] Among them, SOC represents the state of charge of the battery, [T c ] nor Indicates the internal core temperature of the battery, [U O ] nor Indicates the battery voltage.

[0100] Each action variable a in the action space t is defined as:

[0101] a t ={[I AC ] nor ,[I DC ] nor}

[0102] Among them, [I AC ] nor represents the AC current component of the battery, [I DC ] nor Represents the DC current component of the battery.

[0103] Both state variables and action variables are normalized to the interval range of [0,1]. The normalization calculation is shown as follows:

[0104]

[0105] Among them, X represents the current state variable or action variable value, X max Indicates the maximum value of the state variable or action variable to be set, X min Indicates the minimum value of the state variable or action variable, X norm Represents the normalized value of a state variable or action variable.

[0106] Two objective functions are set, including a charging speed objective function and a temperature rise heat generation objective function;

[0107] The charging speed objective function J1 is expressed as:

[0108] J1=minNt

[0109] Among them, t refers to the sampling period, N refers to the number of sampling steps to complete charging; the charging speed objective function J1 is to achieve charging in the shortest time;

[0110] The objective function J2 of heating and heat generation is expressed as:

[0111] J2=maxQ gen

[0112] Among them, Q gen It refers to the total heat generated during the battery heating and charging process. The heating and heat generation objective function J2 is to maximize the heat generation during the heating and charging process to increase the battery temperature.

[0113] The reward function is expressed as:

[0114]

[0115] Among them, ωi are the weight coefficients of different objective functions, i = 1 to 6; r soc (t) is the battery SOC reward function, r Qgen (t) is the battery heat generation reward function, r volt (t) is the battery voltage reward function, r temp (t) is the battery core temperature reward function, r IDC (t) is the battery DC current reward function, r IAC (t) is the battery AC current reward function;

[0116] r soc (t) The reward function is expressed as:

[0117] r soc (t)=|SOC tar -SOC(t)|

[0118] Among them, SOC tar Represents the target SOC value, SOC(t) represents the current battery SOC; the optimization target of the battery charging speed is expressed as the gap between the current SOC and the target SOC. When charging at a lower current or keeping it uncharged, the lithium-ion battery will suffer a lower penalty, while charging at a higher current will produce a greater reward. This design is intended to encourage the battery to pursue a higher charging current during training, thereby reducing the overall charging time.

[0119] r Qgen (t) The reward function is expressed as:

[0120]

[0121] As the battery temperature increases, the DC safety current allowed by the battery will increase. Setting the heat generation reward function is intended to encourage the battery to pursue a higher battery temperature during training to increase the charging current, thereby speeding up the charging speed of lithium-ion batteries at low temperatures.

[0122] r volt (t), r temp (t), r IDC (t), r IAC (t) is a soft constraint reward function, which is specifically expressed as:

[0123]

[0124]

[0125] Among them, U O (t) represents the current battery voltage. In this embodiment, U max Indicates the maximum voltage threshold, which is 4.2V; T c(t) represents the current battery core temperature, T max Indicates the maximum temperature threshold M, which is 30°C; I DC (t) represents the current battery DC component, I DC_max It represents the maximum threshold of the DC component, which is obtained by real-time table lookup based on the lithium precipitation boundary curve of the battery as the SOC and temperature change; I AC (t) represents the current battery AC component, I AC_max Indicates the maximum threshold of the AC component, which is 15A; when the battery's voltage, temperature, DC and AC components do not exceed their respective maximum allowable thresholds, the battery will not be penalized. When the threshold is exceeded, the battery will be penalized, thereby guiding the battery to ensure battery safety during charging.

[0126] Figure 2 (a) is a schematic diagram of the core temperature comparison results of the collaborative optimization control method of the present invention and the traditional preheating before charging strategy; Figure 2 (b) is a schematic diagram of the maximum allowable DC current and DC current comparison results of the collaborative optimization control method of the present invention and the traditional preheating before charging strategy; Figure 2 (c) is a schematic diagram of the SOC comparison results of the collaborative optimization control method of the present invention and the traditional preheating before charging strategy. Including comparisons with two cases where the switching temperature is 15°C and 10°C. By observing Figures 2 (a) and 2 (b), it can be clearly seen that in the initial stage of charging, the proposed low-temperature fast charging optimization control strategy focuses on increasing the battery temperature. At this stage, the AC component dominates to achieve the goal of rapid temperature rise. When the battery temperature reaches about -6°C, the DC current component is gradually introduced. As the heating and charging process proceeds, when the battery temperature reaches about 14°C, the battery heat generation and heat dissipation reach a balance. In general, although the proposed strategy introduces a DC component in the early stage, the heating speed is slower than the traditional method, but since the AC component always exists, compared with the traditional preheating and then charging strategy, when the battery heat generation and heat dissipation reach a balance, the battery temperature is still high, and the DC current component occupies a large proportion. According to Figure 2(c), the charging speed increases significantly in the later stage. In the traditional strategy, since there is no AC heating to maintain the temperature, the DC component also decreases after the temperature drops, thereby increasing the charging time. It can be seen that increasing the battery temperature as much as possible in a low temperature environment is crucial to increasing the charging speed.

[0127] In order to verify the effectiveness of the low-temperature heating charging system optimization control proposed by the present invention, a series of control tests were conducted to comprehensively evaluate the performance of the charging control scheme. These control tests were designed to be carried out under the same test platform to ensure that the reliability and adaptability of the scheme can be fully verified. Through the systematic analysis and comparison of the test results, the present invention can gain an in-depth understanding of the performance of the control scheme under different conditions and provide guidance for further optimization and adjustment.

[0128] In order to quantitatively compare the charging speed of the present invention with that of the traditional preheating-before-charging strategy, the increase in the battery charging SOC within the same time period is used to measure the charging speed. The specific calculation is as follows:

[0129]

[0130] Among them, σ SOC Indicates the percentage increase of SOC change in the same time period, ΔSOC i It represents the SOC change of the strategy proposed in the present invention within a specified time period, ΔSOC j It represents the SOC change of the preheating strategy before charging within a specified time period. The SOC results of different charging strategies are shown in Table 1:

[0131] Table 1

[0132]

[0133] Through comparative analysis, it can be seen that the optimal strategy for low-temperature safe and fast charging proposed by the present invention increases the charging capacity by 66.67% compared to the heating charging strategy with a switching temperature of 15°C, significantly improving the low-temperature charging speed. Similarly, compared to the heating charging strategy with switching temperatures of 10°C, 5°C, and 0°C, the charging capacity increases by 81.82%, 100%, and 122%, respectively, while ensuring that the voltage, current, and temperature do not exceed the threshold, ensuring the safety of the battery and improving the charging performance of lithium-ion batteries in low-temperature environments.

[0134] Example 3

[0135] A computer device includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the reinforcement learning-based coordinated optimization control method for low-temperature heating and charging of a lithium-ion battery described in Example 1 or 2 are implemented.

[0136] Example 4

[0137] A computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the reinforcement learning-based coordinated optimization control method for low-temperature heating and charging of a lithium-ion battery described in Example 1 or 2.

[0138] Example 5

[0139] A collaborative optimization control system for low-temperature heating and charging of lithium-ion batteries based on reinforcement learning, including:

[0140] The model building module is configured to: build an electric-thermal coupling model of a lithium-ion battery;

[0141] The status acquisition module is configured to: acquire status information of the lithium-ion battery in real time;

[0142] The reinforcement learning module is configured to: calculate the optimal heating power and charging current through a learning algorithm using a trained reinforcement learning model according to current state information;

[0143] The control execution module is configured to: control the operation of the heater and the charger and adjust the battery temperature and the charging speed according to the optimization result, i.e. the calculated optimal heating power and charging current.

Claims

1. A collaborative optimization control method for low-temperature heating and charging of lithium-ion batteries based on reinforcement learning. It is characterized in that include: Model equation establishment: Establish the electric-thermal coupling model of lithium-ion batteries; Initialization: Initialize the action space and state space of the reinforcement learning model, and initialize the network parameters of the policy network and value network of the reinforcement learning model; Multi-state sensing: Real-time acquisition of lithium-ion battery status information, including AC current, DC current, voltage, SOC, and internal core temperature; Objective function, reward function and constraint design: Set the reward function and design the upper and lower limits of actions and state variables. The goals include charging speed and temperature rise heat generation. Strategy optimization: The current state variable is input into the strategy network. The strategy network is responsible for selecting two actions including AC action and DC action according to the current state, and inputting the action into the electric-thermal coupling model of the lithium-ion battery, obtaining the next state and outputting the corresponding reward. At the same time, the current experience is stored in the experience replay pool, and samples are randomly drawn from the experience replay pool for training; the currently selected action is evaluated through the value network, the state and network parameters are updated, and the reinforcement learning model is optimized until the cyclic training reaches the maximum number of iterations; Output the action value that maximizes the reward return under the constraints of the reward function; Control execution: According to the current state of the battery and under the constraints of the reward function, the heater and charger are controlled to work according to the optimally calculated heating AC and charging DC, and the heating AC and charging DC are output, so that the lithium-ion battery is charged according to the optimal heating and charging current; Further preferably, the action space includes the alternating current component and the direct current component of the battery, and the state space includes the state of charge of the battery, the battery voltage and the internal core temperature of the battery.

2. According to claim 1, the lithium-ion battery low-temperature heating and charging coordinated optimization control method based on reinforcement learning, It is characterized in that Each state variable s in the state space t is defined as: s t ={SOC,[T c ] nor ,[U O ] nor } Among them, SOC represents the state of charge of the battery, [T c ] nor Indicates the internal core temperature of the battery, [U O ] nor Indicates battery voltage; Further preferably, each action variable a in the action space t is defined as: a t ={[I AC ] nor ,[I DC ] nor } Among them, [I AC ] nor represents the AC current component of the battery, [I DC ] nor Represents the DC current component of the battery.

3. According to claim 2, the lithium-ion battery low-temperature heating and charging coordinated optimization control method based on reinforcement learning, It is characterized in that Both state variables and action variables are normalized to the interval range of [0,1]. The normalization calculation is shown as follows: Among them, X represents the current state variable or action variable value, X max Indicates the maximum value of the state variable or action variable to be set, X min Indicates the minimum value of the state variable or action variable, X norm Represents the normalized value of a state variable or action variable.

4. According to claim 1, the lithium-ion battery low-temperature heating and charging coordinated optimization control method based on reinforcement learning, It is characterized in that Two objective functions are set, including a charging speed objective function and a temperature rise heat generation objective function; Charging speed objective function J 1 It is expressed as: J 1 =minNt Among them, t refers to the sampling period, N refers to the number of sampling steps to complete charging; the charging speed objective function J 1 Charging is achieved in the shortest time; Heating and heat generation objective function J 2 It is expressed as: J 2 =maxQ gen Among them, Q gen Refers to the total heat generated during the battery heating and charging process.

5. According to claim 1, the lithium-ion battery low-temperature heating and charging coordinated optimization control method based on reinforcement learning, It is characterized in that The reward function is expressed as: Among them, ω i are the weight coefficients of different objective functions, i = 1 to 6; r soc (t) is the battery SOC reward function, r Qgen (t) is the battery heat generation reward function, r volt (t) is the battery voltage reward function, r temp (t) is the battery core temperature reward function, is the battery DC current reward function, is the battery AC current reward function; r soc (t) The reward function is expressed as: r soc (t)=|SOC tar -SOC(t)| Among them, SOC tar represents the target SOC value, SOC(t) represents the current battery SOC; r Qgen (t) The reward function is expressed as: r volt (t), r temp (t), As a soft constraint reward function, it is specifically expressed as: Among them, U O (t) represents the current battery voltage, U max Indicates the maximum threshold of voltage; T c (t) represents the current battery core temperature, T max Indicates the maximum threshold of temperature; I DC (t) represents the current battery DC component, I DC_max Indicates the maximum threshold of the DC component; I AC (t) represents the current battery AC component, I AC_max Indicates the maximum threshold of the AC component.

6. The method for coordinated optimization control of low-temperature heating and charging of lithium-ion batteries based on reinforcement learning according to claim 1, It is characterized in that The electrothermal coupling model of lithium-ion batteries includes an electrical model and a two-state thermal model; the electrical model includes the ohmic impedance R o , polarization impedance R ct and the charge transfer process capacitance C dl , voltage source U OCV , inductance L and Warburg impedance Z w ; The two-state thermal model is specifically expressed as: Among them, T c 、T s and T a is the battery core, surface and ambient temperature; R i and R o are heat conduction resistance and convection resistance respectively; C c and C s Represents the internal heat capacity and surface heat capacity of lithium-ion batteries.

7. The method for coordinated optimization control of low temperature heating and charging of lithium-ion batteries based on reinforcement learning according to any one of claims 1 to 6, It is characterized in that The reinforcement learning model consists of a policy network and a value network; the policy network is used to output actions, while the value network is used to evaluate the value of actions.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program. It is characterized in that When the processor executes the computer program, the steps of the reinforcement learning-based coordinated optimization control method for low-temperature heating and charging of a lithium-ion battery are implemented as described in any one of claims 1-7.

9. A computer-readable storage medium having a computer program stored thereon, It is characterized in that When the computer program is executed by a processor, the steps of the reinforcement learning-based coordinated optimization control method for low-temperature heating and charging of a lithium-ion battery are implemented.

10. Coordinated optimization control system for low-temperature heating and charging of lithium-ion batteries based on reinforcement learning, It is characterized in that include: The model building module is configured to: build an electric-thermal coupling model of a lithium-ion battery; The status acquisition module is configured to: acquire status information of the lithium-ion battery in real time; The reinforcement learning module is configured to: calculate the optimal heating power and charging current through a learning algorithm using a trained reinforcement learning model according to current state information; The control execution module is configured to: control the operation of the heater and the charger and adjust the battery temperature and the charging speed according to the optimization result, i.e. the calculated optimal heating power and charging current.

Citation Information

Patent Citations

  • Preheating charging loss optimization battery pack charging method in low-temperature environment

    CN104409788A

  • Quick battery charging control method based on PPO algorithm and considering charging electric charge

    CN115447431A

  • Low-temperature environment battery heating cooperative charging method

    CN115642673A

  • Power battery charging heat management method and device, computer equipment and storage medium

    CN117901722A

  • Lithium ion battery low-temperature heating-charging collaborative topology, control method and controller

    CN118157261A

Cited By

  • Mathematical model-based vehicle-mounted charger component parameter optimization method and system

    CN119808674A

  • Lithium ion battery low-temperature preheating charging segmented optimization method and system based on reinforcement learning

    CN120087183A

  • Energy storage battery heating method, equipment and medium

    CN121663036A