Energy storage system thermal management method and thermal management regulation model training method

Abstract

The present specification relates to a thermal management method of an energy storage system, and a training method of a thermal management regulation model. The thermal management method comprises: obtaining charge and discharge power data and temperature data of the energy storage system; inputting the charge and discharge power data and the temperature data into a target thermal management regulation model to obtain a regulation action parameter for thermal management of the energy storage system; wherein the regulation action parameter is used to regulate a temperature adjusting device of the energy storage system, so that the power consumption of the temperature adjusting device meets a preset condition and the temperature of the energy storage system reaches a target temperature. The embodiments of the present specification can simultaneously consider the temperature state and the charge and discharge state of the energy storage system, use the target thermal management regulation model to predict the regulation parameter, and achieve the purpose of reducing the power consumption of the temperature regulation device and smoothly and accurately regulating the temperature of the energy storage system.

Description

Technical Field

[0001] This specification relates to the field of thermal management technology, and in particular to a thermal management method for an energy storage system and a training method for a thermal management control model. Background Technology

[0002] The large-scale and commercial development of electrochemical energy storage is inevitable. However, the safe and efficient operation of electrochemical energy storage depends on the temperature control of the thermal management system.

[0003] In related technologies, the temperature control strategy of thermal management systems typically determines whether to cool or heat based on the highest and lowest temperatures of the battery pack in the energy storage system. However, this strategy suffers from problems such as control lag and overshoot. Summary of the Invention

[0004] This specification aims to at least partially address one of the technical problems in the related art. To this end, one objective of this specification is to propose a thermal management method for an energy storage system that considers the operating and temperature states of the energy storage system to perform thermal management, thereby ensuring temperature stability while reducing the power consumption of the temperature control device.

[0005] The second objective of this specification is to propose a training method for a thermal management control model.

[0006] The third objective of this specification is to provide a thermal management device for an energy storage system.

[0007] The fourth objective of this specification is to propose a training device for a thermal management control model.

[0008] The fifth objective of this specification is to propose an energy storage system.

[0009] The sixth objective of this specification is to provide a computer-readable storage medium.

[0010] To achieve the above objectives, a thermal management method for an energy storage system is proposed in the first aspect of this specification. The method includes: acquiring charge / discharge power data and temperature data of the energy storage system; inputting the charge / discharge power data and the temperature data into a target thermal management control model to obtain control action parameters for thermal management of the energy storage system; wherein the control action parameters are used to control the temperature regulation device of the energy storage system so that the power consumption of the temperature regulation device meets preset conditions and the temperature of the energy storage system reaches a target temperature.

[0011] In some embodiments of this specification, the target thermal management control model is trained in the following manner:

[0012] The environmental state variables and action variables of the energy storage system at different times within a historical preset time period are obtained. The environmental state variables include the charging and discharging power variables and the temperature variables of the energy storage system. The action variables include the control action variables for thermal management of the energy storage system. A reward function for the thermal management control model is constructed with the goal of the temperature regulation device's power consumption meeting the preset conditions and the energy storage system's temperature reaching the target temperature within the historical preset time period. The parameters of the thermal management control model are updated based on the value of the reward function, using the environmental state variables and the action variables as inputs, to obtain the target thermal management control model.

[0013] In some embodiments of this specification, the reward function of the thermal management control model is constructed with the goal of the power consumption of the temperature regulation device meeting the preset conditions and the temperature of the energy storage system reaching the target temperature within the preset historical time period. This includes: constructing a first principal reward function based on the difference between the historical maximum total power consumption of the temperature regulation device and the total power consumption of the temperature regulation device within the preset historical time period; constructing a second principal reward function based on the absolute difference between the temperature data of the energy storage system at different times within the preset historical time period and the corresponding target temperature at different times; and constructing the reward function based on the first principal reward function and the second principal reward function.

[0014] In some embodiments of this specification, the reward function includes a main reward function and a branch penalty term. Constructing the reward function based on the first main reward function and the second main reward function includes: performing a weighted summation of the first main reward function and the second main reward function based on a first weight coefficient corresponding to the first main reward function and a second weight coefficient corresponding to the second main reward function to obtain the main reward function; determining the branch penalty term based on the product of the number of times the temperature of the energy storage system exceeds a preset temperature range within the preset historical time period and a third weight coefficient; and calculating the difference between the main reward function and the branch penalty term to obtain the reward function.

[0015] In some embodiments of this specification, a second principal reward function is constructed based on the absolute difference between the temperature data of the energy storage system at different times within a historical preset time period and the target temperature at those different times. This includes: determining a first branch term of the second principal reward function based on the sum of squares of the absolute differences between the temperature data of the energy storage system at different times within a historical preset time period and the target temperature at those different times; determining the sum of squares of the target temperature at different times as a second branch term of the second principal reward function; and calculating the difference between the second branch term and the first branch term to obtain the second principal reward function.

[0016] In some embodiments of this specification, the method further includes: acquiring historical operating data of the energy storage system and historical operating parameters of the temperature regulation device; training a constructed physical field simulation model based on the historical operating data and the historical operating parameters to obtain a target physical field simulation model; wherein the target physical field simulation model is used to solve for the temperature data of the energy storage system at the next moment based on the operating data of the energy storage system at the current moment and the operating parameters of the temperature regulation device.

[0017] In some embodiments of this specification, the method further includes: obtaining the environmental state variables and action variables of the energy storage system at different times within a preset time period through the target physical field simulation model.

[0018] To achieve the above objectives, a training method for a thermal management control model is proposed in the second aspect of this specification. The training method includes: acquiring environmental state variables and action variables of the energy storage system at different times within a preset time period; wherein the environmental state variables include the charging and discharging power variables of the energy storage system and the temperature variables of the energy storage system; the action variables include control action variables for thermal management of the energy storage system; constructing an objective function for the thermal management control model with the objectives of the power consumption of the temperature regulation device used for thermal management meeting preset conditions and the temperature of the energy storage system reaching a target temperature within the preset time period; and updating the parameters of the thermal management control model based on the value of the objective function, using the environmental state variables and the action variables as inputs to the thermal management control model, to obtain a target thermal management control model.

[0019] To achieve the above objectives, a third aspect of this specification provides a thermal management device for an energy storage system. The thermal management device includes: an acquisition module for acquiring charge / discharge power data and temperature data of the energy storage system; and a control action determination module for inputting the charge / discharge power data and temperature data into a target thermal management control model to obtain control action parameters for thermal management of the energy storage system. The control action parameters are used to control the temperature regulation device of the energy storage system to ensure that the power consumption of the temperature regulation device meets preset conditions and that the temperature of the energy storage system reaches a target temperature.

[0020] To achieve the above objectives, a training device for a thermal management control model is proposed in the fourth aspect of this specification. The training device for the thermal management control model includes: a sample acquisition module, used to acquire environmental state variables and action variables of the energy storage system at different times within a preset time period; wherein the environmental state variables include the charging and discharging power variables of the energy storage system and the temperature variables of the energy storage system; the action variables include control action variables for thermal management of the energy storage system; an objective function determination module, used to construct an objective function for the thermal management control model with the objectives of the power consumption of the temperature regulation device used for thermal management within the preset time period meeting preset conditions and the temperature of the energy storage system reaching a target temperature; and a training module, used to update the parameters of the thermal management control model based on the value of the objective function, using the environmental state variables and the action variables as inputs to the thermal management control model, to obtain a target thermal management control model.

[0021] To achieve the above objectives, a fifth aspect of this specification provides an energy storage system. The energy storage system includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements the method described in any of the embodiments of the first aspect described above.

[0022] To achieve the above objectives, a computer-readable storage medium is provided in a sixth aspect of this specification. The computer-readable storage medium includes a stored computer program, wherein, when the computer program is executed, it controls the device on which the computer-readable storage medium is located to perform the method described in any of the embodiments of the first aspect above.

[0023] Through the above embodiments, when performing thermal management on an energy storage system, temperature data and charge / discharge power data are used to predict the control parameters for thermal management of the energy storage system using a target thermal management control model. These control parameters are then used to regulate the temperature control device, ensuring that the power consumption of the temperature control device meets preset conditions and that the energy storage system reaches the target temperature. The embodiments in this specification simultaneously consider the temperature state and charge / discharge state of the energy storage system, using the target thermal management control model to predict control parameters, thereby achieving the goal of reducing the power consumption of the temperature control device and ensuring a smooth and accurate temperature control of the energy storage system.

[0024] Additional aspects and advantages of this specification will be set forth in part in the description which follows, and in part will be obvious from the description or may be learned by practice of this specification. Attached Figure Description

[0025] Figure 1a This is a schematic diagram of the thermal management system of the energy storage system provided in the embodiments of this specification.

[0026] Figure 1b This is a schematic diagram of an energy storage system provided in one embodiment of this specification.

[0027] Figure 2 This is a flowchart of a thermal management method for an energy storage system according to an embodiment of this specification.

[0028] Figure 3a This is a line graph illustrating the thermal management simulation of an energy storage system according to one embodiment of this specification.

[0029] Figure 3b This is a line graph illustrating the thermal management of an energy storage system according to one embodiment of this specification.

[0030] Figure 4 This is a flowchart of the training method for the thermal management control model provided in the embodiments of this specification.

[0031] Figure 5 This is a structural block diagram of the thermal management device of the energy storage system provided in the embodiments of this specification.

[0032] Figure 6 This is a structural block diagram of the training device for the thermal management control model provided in the embodiments of this specification.

[0033] Figure 7 This is a schematic block diagram of the energy storage system according to an embodiment of this specification. Detailed Implementation

[0034] The embodiments of this specification are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this specification, and should not be construed as limiting this specification.

[0035] The clean energy substitution of the world is imminent, and the large-scale and commercial development of electrochemical energy storage is unstoppable. With changes in raw material supply and demand and the deepening of energy storage participation in the electricity market, the economics and demand for energy storage will undergo significant transformations. The safe and efficient operation of electrochemical energy storage relies on the temperature control of its thermal management system.

[0036] In related technologies, the temperature control strategy for thermal management is relatively simple. Based on the highest and / or lowest temperatures of the battery pack, it is determined whether the temperature control device should heat, circulate (compressor off), or cool. Here, "battery pack" generally refers to a combination of batteries, primarily the processing and assembly of lithium battery packs. This involves combining battery cells, battery protection boards, battery connectors, labels, etc., using battery pack technology to create the product required by the customer. Energy storage systems may include multiple battery packs.

[0037] Temperature control strategies in related technologies, which adjust the chiller unit's operating status based on changes in battery pack temperature after energy storage starts up, typically have the following problems:

[0038] Firstly, there are issues with control lag and overshoot. After the energy storage system stops charging and discharging and enters standby mode, the cell temperature is still relatively high. Even after temperature detection, the system may still activate the compressor of the temperature control device for cooling, resulting in unnecessary waste. Similarly, using the same start-up and cut-off temperatures at different charge / discharge rates of the energy storage system will cause unnecessary overshoot and lead to temperature fluctuations.

[0039] Secondly, overshoot leads to rapid temperature fluctuations in the battery cells, which increases the maximum temperature difference between cells, exacerbates inconsistency, and reduces cell lifespan.

[0040] Third, due to the imprecise temperature control, a large margin needs to be reserved in the temperature control target to prevent temperature exceedance alarms. For example, the battery pack temperature control target could be set at an upper limit of 35°C, but to reduce the risk of temperature exceeding the limit, it can only be set at 32°C.

[0041] To address the problems in the aforementioned related technologies, this specification proposes a thermal management method for energy storage systems and a training method for a thermal management control model. This specification can simultaneously consider the charging and discharging operating states of the energy storage system and the temperature state of the energy storage battery to perform thermal management, thereby reducing the power consumption of the temperature control device while maintaining a stable temperature for the energy storage system.

[0042] This specification provides a scenario example of a thermal management method for an energy storage system, which is applied to... Figure 1a The energy storage system thermal management system shown includes an energy storage system 102, a client 104, and a server 106. The server 106 is used to train the thermal management control model.

[0043] For details on the specific components of the energy storage system 102, please refer to [link / reference]. Figure 1b The energy storage system 102 includes multiple battery modules, each consisting of multiple battery packs 110. Each battery pack 110 is equipped with a temperature measuring point to measure its temperature. The energy storage system 102 also includes a temperature control device 120, such as a water-cooled unit. Figure 1b As shown, the temperature control device 120 can be arranged on the left side of the energy storage system container, with an inlet-outlet and outlet-outlet airflow configuration. The system flow channel is designed with bottom inlet and top outlet. The coolant flows from the inlet pipe to six branch lines, and each branch line is further divided into eight smaller branches flowing to the liquid-cooled PACK to cool or heat the battery cells. After flowing out of the PACK, the coolant is collected through the branch pipes and returned to the water-cooled unit via the return pipe, thus forming a cycle.

[0044] The energy storage system 102 is usually also equipped with a local controller, which can collect the charging and discharging power data of the energy storage system 102.

[0045] In this scenario example, the energy storage system 102 is communicatively connected to both the client 104 and the server 106. The energy storage system 102 can provide the server 106 with training sample data for a thermal management control model. The target thermal management control model trained by the server 106 can be configured on the client 104, allowing the client 104 to perform thermal management control on the energy storage system. Alternatively, the trained target thermal management control model can be directly configured in the energy storage system 102, enabling the energy storage system 102 to automatically perform thermal management control.

[0046] The thermal management control model can be built using the proximal policy optimization algorithm in deep reinforcement learning. The training process of the thermal management control model on server 106 includes: acquiring the environmental state variables and action variables of energy storage system 102 at different times within a historical preset time period. The environmental state variables include the charging and discharging power variables of energy storage system 102 and the temperature variables of the energy storage system. The action variables include the control action variables for thermal management of the energy storage system. With the goal of the temperature regulation device power consumption meeting preset conditions and the energy storage system temperature reaching the target temperature within the historical preset time period, a reward function for the thermal management control model is constructed. Using the environmental state variables and action variables as inputs to the thermal management control model, the parameters of the thermal management control model are updated based on the value of the reward function to obtain the target thermal management control model.

[0047] Taking the target thermal management control model configured on client 104 as an example, firstly, client 104 acquires the charging / discharging power data and temperature data of energy storage system 102. This data is then input into the target thermal management control model, which processes the data to obtain control parameters for thermal management of the energy storage system. Based on these parameters, client 104 regulates the temperature control device of energy storage system 104, enabling it to cool or heat the battery modules of the energy storage system, ensuring that the power consumption of the temperature control device meets preset conditions and that the energy storage system reaches the target temperature. Through this control process, the thermal management of energy storage system 102 considers both the charging / discharging operating state and the temperature state. It can adjust the operating state of the temperature control device in a timely manner according to the charging / discharging power of the energy storage system, maintaining a stable overall temperature for the energy storage system. Furthermore, by reducing control lag or overshoot, the power consumption of the temperature control device is also reduced accordingly.

[0048] Figure 2 This is a flowchart illustrating the thermal management method of an energy storage system according to an embodiment of this specification. Please refer to [link / reference]. Figure 2 The thermal management method of this energy storage system includes:

[0049] S210 acquires charging and discharging power data and temperature data of the energy storage system.

[0050] S220 inputs charging and discharging power data and temperature data into the target thermal management control model to obtain control action parameters for thermal management of the energy storage system.

[0051] Among them, the control action parameters are used to control the temperature regulation device of the energy storage system so that the power consumption of the temperature regulation device meets the preset conditions and the temperature of the energy storage system reaches the target temperature.

[0052] In the embodiments of this specification, the target thermal management control model is used to calculate the control parameters for thermal management of the energy storage system based on the charging and discharging power data and temperature data of the energy storage system, with the goal of the power consumption of the thermal management temperature regulation device meeting preset conditions and the temperature of the energy storage system reaching the target temperature.

[0053] Temperature data includes temperature data from different battery packs in the energy storage system. Generally, the thermal management control cycle for the energy storage system can be aligned with the system's power dispatch cycle, or a separate step size can be set, such as 1 minute or 5 minutes. One step size constitutes one control cycle, and one control cycle can be recorded as one moment.

[0054] When predicting control action parameters using a target thermal management control model, the charging / discharging power data and temperature data of the energy storage system at the current moment can be obtained. These data are then input into the target thermal management control model to obtain the control action parameters for thermal management of the energy storage system at the next moment. Based on these obtained control action parameters, the temperature regulation device of the energy storage system is controlled at the next moment. By continuously controlling the temperature regulation device according to the predicted control action parameters at each moment, the power consumption of the temperature regulation device can meet preset conditions, and the temperature of the energy storage system can reach the target temperature. The temperature of the energy storage system can be the temperature of a specific battery pack, the highest temperature among all battery packs, or the average temperature of the battery packs; specifically, it is determined according to the temperature control requirements of the energy storage system.

[0055] To reduce power consumption waste, the preset condition for the temperature control device can be to minimize the total power consumption of the temperature control device within a preset time period. That is, at each moment within the preset time period, the temperature control device is controlled based on the control action parameters predicted by the target thermal management control model, which can minimize the total power consumption of the temperature control device within the preset time period.

[0056] Within a preset time period, adjusting the temperature control device based on the control action parameters predicted by the target thermal management control model for the next moment ensures that the temperature of the energy storage system at each moment is close to the target temperature. This results in a smoother and more precise overall temperature of the energy storage system within the preset time period, reducing the risk of overshoot leading to larger temperature differences and shortened cell lifespan. Furthermore, more precise temperature control capabilities provide the energy storage system with a more flexible temperature control range. Operating at the upper temperature limit as much as possible without triggering an over-limit alarm in the battery pack temperature reduces the power consumption of the temperature regulation device, effectively improving the efficiency of the energy storage system and increasing its return on investment.

[0057] Through the above embodiments, when performing thermal management on an energy storage system, temperature data and charge / discharge power data are used to predict the control parameters for thermal management of the energy storage system using a target thermal management control model. These control parameters are then used to regulate the temperature control device, ensuring that the power consumption of the temperature control device meets preset conditions and that the energy storage system reaches the target temperature. The embodiments in this specification simultaneously consider the temperature state and charge / discharge state of the energy storage system, using the target thermal management control model to predict control parameters, thereby achieving the goal of reducing the power consumption of the temperature control device and ensuring a smooth and accurate temperature control of the energy storage system.

[0058] In some embodiments of this specification, the target thermal management control model is trained in the following manner:

[0059] The environmental state variables and action variables of the energy storage system at different times within a historical preset time period are obtained. The environmental state variables include the charging and discharging power variable and the temperature variable of the energy storage system; the action variables include the control action variables for thermal management of the energy storage system. A reward function for the thermal management control model is constructed with the objectives of ensuring the power consumption of the temperature regulation device meets preset conditions and the energy storage system temperature reaches the target temperature within the historical preset time period. Using the environmental state variables and action variables as inputs to the thermal management control model, the parameters of the thermal management control model are updated based on the value of the reward function to obtain the target thermal management control model.

[0060] For example, using the actual power environment of the energy storage system undergoing thermal management as the theoretical basis, an initial thermal management control model is constructed based on a near-end strategy optimization algorithm. Simultaneously, the control cycle for the temperature regulation device is determined according to actual control requirements. For example, the control cycle for the temperature regulation device can be consistent with the scheduling cycle for the energy storage system's charge and discharge operations, or a separate step size can be set. For instance, if the step size is set to 5 minutes, then one control cycle is 5 minutes, and a day includes 288 control cycles. One moment corresponds to one control cycle.

[0061] Based on the established initial thermal management control model, a minimum energy consumption objective function for the temperature regulation device used for thermal management is established, and constraints for the thermal management control model are constructed. Simultaneously, the state observation space, action space, and reward function required for the thermal management control model are established. The reward function is designed based on the objective function and constraints. Using the state and action variables at time t as inputs to the model's policy and action networks, the thermal management control model is iteratively trained using policy gradient and temporal difference methods to obtain the optimal real-time control model for thermal management of the energy storage system, i.e., the target thermal management control model.

[0062] In the embodiments of this specification, the power consumption of the temperature regulation device meeting preset conditions can be considered as minimizing the total power consumption of the temperature regulation device. The energy storage system reaching the target temperature can be understood as minimizing the difference between the energy storage system's temperature and the target temperature. That is, the objective function can be established with the goals of minimizing the total power consumption of the temperature regulation device for thermal management within a certain period and minimizing the difference between the energy storage system's temperature and the target temperature. Based on this objective function, and considering the preset upper and lower limit temperatures of the energy storage system's battery PACK as constraints, a reward function for the thermal management control model is constructed.

[0063] Obtain the environmental state variables s of the energy storage system at different times within a historical preset time period. t and action variable a t Among them, the environmental state variable s t Including the charging and discharging power variable p of the energy storage system t Temperature variable T of energy storage system t Action variable a t This includes control action variables for thermal management of the energy storage system. The control action variable represents the power of the temperature regulating device at time t, and its value can be positive or negative. A positive value represents cooling, a negative value represents heating, and 0 indicates that the compressor of the temperature regulating device stops working, and only the circulation pump is on. The environmental state variable is the state variable corresponding to the aforementioned state observation space. The action variable is the action variable corresponding to the aforementioned action space. In the embodiments of this specification, the charge / discharge power variable represents the charge / discharge power data corresponding to different times; the temperature variable represents the temperature data corresponding to different times.

[0064] Specifically, when training the thermal management control model, the energy storage system can act as an agent in a deep reinforcement learning environment. The agent executes the optimal action and interacts with the agent's environment, obtaining immediate rewards and the next environmental state variable. The current environmental state variable, the optimal action variable, the immediate reward, and the next environmental state variable constitute the transition experience, which is stored in a buffer.

[0065] For example, the historical preset time period can typically be a specific day in history, with one day constituting one training cycle to train the thermal management control model. If the step size is set to 5 minutes, the historical preset time period includes 288 control cycles, corresponding to 288 moments. Since excessively high battery pack temperatures can cause safety issues, while excessively low temperatures only affect the performance of the energy storage system, to reduce the occurrence of safety problems, the highest temperature among multiple battery packs in the energy storage system can be used as the temperature data for the energy storage system.

[0066] Obtain the environmental state variables s of the energy storage system at 288 times on a specific day in history. t {p t ,T t} and action variable at Among them, p t T represents the charging and discharging power of the energy storage system at time t; t T represents the highest temperature T of the battery pack in the energy storage system at time t. max A reward function is constructed with the objective of minimizing the total power consumption of the temperature regulation device and the difference between the energy storage system's temperature and the target temperature on a given historical day. The environmental state variables s corresponding to 288 time points are then used. t {p t ,T t} and action variable a t As inputs to the policy network and action network of the thermal management control model, based on the value of the reward function, {s} is obtained from multiple interactions between the agent and the environment. t ,a t, s t+1 ,r t The proximal policy optimization algorithm is used to update the model parameters in the policy neural network. This process is iterated until the model converges, resulting in the target thermal management control model. Where r... t s represents the value of the reward function at time t; t+1 This represents the environmental state variable at time t+1.

[0067] In some embodiments of this specification, the reward function of the thermal management control model is constructed with the goal of ensuring that the power consumption of the temperature regulation device meets preset conditions and that the temperature of the energy storage system reaches the target temperature within a historical preset time period. This includes: constructing a first principal reward function based on the difference between the historical maximum total power consumption of the temperature regulation device and the total power consumption of the temperature regulation device within a historical preset time period; constructing a second principal reward function based on the absolute difference between the temperature data of the energy storage system at different times within the historical preset time period and the corresponding target temperature at different times; and constructing a reward function based on the first and second principal reward functions.

[0068] In the embodiments of this specification, the objective function is established based on two objectives: first, to minimize the power consumption of the temperature regulation device; and second, to achieve the target temperature for the energy storage system. Therefore, corresponding principal reward functions can be constructed for each of these two objectives.

[0069] For one objective, if the model is trained on a daily basis, then the historical maximum total power consumption of the temperature control device is determined as the historical maximum daily power consumption of the temperature control device, which is defined as the maximum total power consumption W. max The total power consumption of the temperature control device during the historical preset time period is the power consumption (W) at each moment within the historical preset time period. i The sum of the two. Based on the difference between the two, the first principal reward function W is determined. k Therefore, the first main reward function Where k represents the number of moments within the historical preset time period; i represents the i-th moment (the i-th control cycle).

[0070] To address the second objective, a second principal reward function is constructed based on the absolute difference between the temperature data of the energy storage system at different times within a historical preset time period and the corresponding target temperature at those times. Since temperature is continuously changing, a target curve can be constructed, which determines the target temperature at different times. The closer the actual temperature of the energy storage system is to the target temperature, the smaller the absolute difference between the two. Therefore, the absolute difference between the temperature data of the energy storage system at different times within a historical preset time period and the corresponding target temperature can be determined. Based on the absolute difference at each time point within the historical preset time period, the second principal reward function is constructed. For example, since the temperature data of the energy storage system may be greater than or less than the target temperature, the square of the difference can be used to represent the magnitude of the absolute difference. The second principal reward function... Where k represents the number of moments within a preset historical time period; TR i T represents the temperature data of the energy storage system at time i; i This represents the target temperature at time i. ΔT is the sum of squares of the differences between the historical temperature data and the target temperature over a preset time period; the smaller the sum, the closer the temperature data is to the target temperature.

[0071] In some embodiments of this specification, a second principal reward function is constructed based on the absolute difference between the temperature data of the energy storage system at different times within a historical preset time period and the corresponding target temperature at those times. This includes: determining a first branch term of the second principal reward function based on the sum of squares of the absolute differences between the temperature data of the energy storage system at different times within a historical preset time period and the corresponding target temperature at those times; determining the sum of squares of the target temperatures at different times as a second branch term of the second principal reward function; and calculating the difference between the second branch term and the first branch term to obtain the second principal reward function.

[0072] Due to the near-end policy optimization algorithm, the reward function is typically designed to be as large as possible. However, the constructed ΔT above is closer to the target temperature as its value decreases. Therefore, the sum of squares of the target temperature can be introduced as a second branch term. The difference between the second branch term and the first branch term ΔT is used to construct a second principal reward function, such that a larger value of the second principal reward function indicates that the temperature data of the energy storage system is closer to the target temperature.

[0073] Specifically, the first branch term ΔT of the second principal reward function is determined based on the sum of squares of the absolute differences between the temperature data of the energy storage system at different times within a historical preset time period and the corresponding target temperature at those times. The construction process of the first branch term ΔT can refer to the construction process of ΔT described above, and will not be repeated here. The sum of squares of the target temperatures at different times is determined as the second branch term of the second principal reward function. The second branch term can be expressed as... The second main reward function is obtained by subtracting the second branch term from the first branch term.

[0074] In some embodiments of this specification, the reward function includes a main reward function and a secondary penalty term. Constructing the reward function based on a first main reward function and a second main reward function includes: weighting and summing the first and second main reward functions using a first weighting coefficient corresponding to the first main reward function and a second weighting coefficient corresponding to the second main reward function to obtain the main reward function. Determining the secondary penalty term by multiplying the number of times the energy storage system's temperature exceeded a preset temperature range within a historical preset time period by a third weighting coefficient. Subtracting the main reward function from the secondary penalty term yields the reward function.

[0075] In the embodiments of this specification, to achieve faster convergence and higher accuracy of the thermal management control model, the reward function, in addition to the two objectives reflected in the objective function, also needs to consider unexpected situations that may occur in the energy storage system within a preset historical period. Therefore, a branch penalty term is added to reflect these unexpected situations. During the thermal management of the energy storage system, the most common unexpected situations include the system temperature exceeding preset upper and lower temperature limits. Exceeding the upper temperature limit may cause safety issues, while exceeding the lower temperature limit may degrade the performance of the energy storage system.

[0076] For example, the number of times the temperature of the energy storage system exceeds a preset temperature range within a preset historical time period can be used as a penalty item to construct a branch penalty item.

[0077] The reward function consists of a main reward function and secondary penalty terms. The main reward function is constructed using a first main reward function and a second main reward function. Since the primary objectives of thermal management for energy storage systems differ under different environments, corresponding weight coefficients can be assigned to the first main reward function, the second main reward function, and the penalty terms.

[0078] Specifically, based on the first weight coefficient corresponding to the first principal reward function and the second weight coefficient corresponding to the second principal reward function, a weighted sum is performed on the first and second principal reward functions to obtain the principal reward function. The principal reward function can be expressed as r. master =α1×W k +α2×T kWhere α1 represents the first weighting coefficient; α represents the first main reward function; α2 represents the second weighting coefficient; This represents the second main reward function.

[0079] The branch penalty term r is determined by multiplying the number of times the energy storage system's temperature exceeded the preset temperature range within a historical preset time period by the third weighting coefficient. branch = α3 × K. Where α3 represents the third weighting coefficient; K represents the number of times the temperature of the energy storage system exceeds the upper and lower limits within the preset historical time period.

[0080] Therefore, the reward function r t =r master -r branch =α1×W k +α2×T k -α3×K. It is understandable that during the training of the thermal management control model, the three weight coefficients can be adjusted accordingly based on the model training results.

[0081] In some embodiments of this specification, the method further includes: acquiring historical operating data of the energy storage system and historical operating parameters of the temperature regulation device. The constructed physical field simulation model is trained based on the historical operating data and historical operating parameters to obtain a target physical field simulation model. The target physical field simulation model is used to calculate the temperature data of the energy storage system at the next moment based on the current operating data of the energy storage system and the operating parameters of the temperature regulation device.

[0082] Training thermal management control models requires a large amount of sample data, while the actual historical operating data of energy storage systems is unlikely to reach the required sample data volume. Therefore, energy storage system simulation environments are typically established. These simulation environments model the real thermal management processes of energy storage systems to obtain a large dataset.

[0083] In related technologies, the cell-level electrochemical-thermal coupling model of a battery actually calculates the battery's reaction and heat generation using an electrochemical model, and then calculates the battery temperature based on heat transfer and dissipation principles. The calculated temperature is then transferred to the temperature-dependent kinetics transfer parameters of the electrochemical model as the reaction temperature of the lithium-ion battery. Container-level thermal simulation systems typically use finite element analysis software such as ANSYS Fluent, which can provide relatively accurate results of cell temperature changes. However, the simulation environment requirements for reinforcement learning differ from those of general models, demanding extremely high computational speeds, with each round ideally completed within seconds, preferably milliseconds. The thermal simulation systems in related technologies have slow computation speeds, making it difficult to meet these performance requirements.

[0084] Therefore, to address the problem of slow computation speed in related technologies, the embodiments in this specification employ deep learning algorithms to construct a deep learning neural network numerical simulation model, i.e., a physics field simulation model. This model is trained using limited data and can quickly solve for the environmental state parameters at the next moment.

[0085] Specifically, historical operating data of the energy storage system and historical operating parameters of the temperature control device are obtained as training sample data. Both the historical operating data and historical operating parameter data are measured data. When the measured data cannot meet the training requirements, historical simulation data obtained through thermal simulation systems in related technologies can be used to train the physical field simulation model.

[0086] The historical operational data of the energy storage system includes historical charge and discharge power data and historical temperature data. Additionally, more simulation operational data and parameters under different conditions can be obtained based on thermal simulation systems from related technologies. Combining these two sets of data constitutes the basic training sample dataset.

[0087] The basic training sample dataset is divided into a training set and a test set. Based on various deep learning algorithms, the constructed physical field simulation model is trained using the training set portion until the physical field simulation model converges, resulting in a regression model between different system states and temperature change prediction data, i.e., the target physical field simulation model. The deep learning algorithms can include, but are not limited to, recurrent neural networks (RNNs), convolutional neural networks (convnets, CNNs), and attention-based neural networks such as Transformers.

[0088] The target physics simulation model is used to solve for the temperature data of the energy storage system at the next moment based on the current operating data of the energy storage system and the working parameters of the temperature regulation device.

[0089] In some embodiments of this specification, the thermal management method for the energy storage system further includes: obtaining the environmental state variables and action variables of the energy storage system at different times within a preset time period through a target physical field simulation model.

[0090] Since the target physics simulation model can achieve millisecond-level computation speed, while the thermal management control model requires extremely high computation speed and a large sample size for training, training sample data for training the thermal management control model can be obtained through the target physics simulation model. This data consists of the environmental state variables and action variables of the energy storage system at different times within a preset time period.

[0091] Furthermore, the patterns learned from the physical field simulation model regarding the charging and discharging power, temperature, and operating status of the temperature regulation device in the energy storage system are solidified in the training of the thermal management control model, which can reduce hysteresis and overshoot phenomena. This allows for more precise and stable temperature control of the energy storage system.

[0092] It is understood that the thermal management method in the embodiments of this specification is not limited to liquid-cooled energy storage systems; the method is also applicable to energy storage systems using other cooling methods such as air cooling. Accordingly, the temperature regulating device can be a water-cooled unit in a liquid-cooled energy storage system, or an air-cooled unit in an air-cooled energy storage system.

[0093] In one specific embodiment of this specification, the thermal management method for an energy storage system may include the following process:

[0094] A physical field simulation model of the energy storage system is built, which supports single-step simulation. The step size can be set arbitrarily, such as 1 minute, 5 minutes, etc. The thermal management process of the energy storage system is simulated using the trained target physical field simulation model, and the results are as follows: Figure 3a The graph shown is a line graph illustrating the simulation effect of thermal management of the energy storage system. Figure 3a The horizontal axis represents time. The first straight line represents the highest temperature of the battery pack in the energy storage system; the second straight line represents the lowest temperature of the battery pack in the energy storage system; the third straight line represents the power consumption of the temperature regulation device (cooling system power consumption); and the fourth straight line represents the power of the energy storage system (active power).

[0095] A deep reinforcement learning environment for a thermal management control model is constructed, which embeds a method for rapidly implementing single-step invocation of the target physics simulation model.

[0096] The initial thermal management control model is established, including but not limited to the Proximal Policy Optimization (PPO) algorithm, the Asynchronous Advantage Actor-Critic (A3C) algorithm, and the policy gradient deep reinforcement learning neural network model framework such as Twin Delayed Deep Deterministic Policy Gradient (TD3).

[0097] Based on the established initial thermal management control model, a minimum energy consumption objective function for the temperature regulation device performing thermal management is established, and constraints for the thermal management control model are constructed. Simultaneously, the state observation space, action space, and reward function required for the thermal management control model are established. The reward function is designed based on the objective function and constraints.

[0098] Based on historical data from the test operation of the energy storage system or simulation data obtained from the simulation model of the target physical field, data preprocessing and normalization are performed, and a multi-threaded method is used to train the thermal management control model.

[0099] When the training of the thermal management control model meets the termination condition, training is stopped and the optimal model file is saved. Simultaneously, a model conversion tool is used to convert the optimal model file, making the converted model file adaptable to various platform environments, such as CentOS Linux and ARM.

[0100] Real-time acquisition of charging / discharging power and temperature data from the energy storage system. Inputting this data into a pre-trained target thermal management control model yields control parameters for thermal management of the energy storage system. These parameters can be represented as commands; for example, command 1 indicates heating; command 2 indicates pump circulation (compressor shutdown); and command 3 indicates cooling.

[0101] The temperature regulation device of the energy storage system is controlled based on the control action parameters. The temperature regulation device includes actuators such as water pumps, valves, and compressors. For example, the actual actions of the compressor controller controlled by the commands represented by the control action parameters can be divided into three categories: 1. Cooling point 40℃, heating point 40℃; 2. Water pump circulation; 3. Cooling point 18℃, heating point 10℃. Cooling begins when the coolant temperature is higher than the cooling point, and heating begins when it is lower than the heating point. Therefore, the equivalent logic of the above three actions is: 1. Heating; 2. Water pump circulation; 3. Cooling. The thermal management system only outputs the cooling target to the compressor controller; the specific compressor power consumption is automatically controlled by the frequency converter in a closed loop, and the thermal management system does not intervene.

[0102] Through the above embodiments, the energy storage system can achieve efficient and stable thermal management by executing thermal management scheduling commands represented by control action parameters. This reduces hysteresis and overshoot, and allows for more precise and stable temperature control of the energy storage system. Figure 3b The graph shown is a line graph illustrating the thermal management effect of the energy storage system after implementing the above-mentioned thermal management methods. Figure 3b The horizontal axis represents time; the fifth line represents the power consumption of the temperature control device (cooling system power consumption); the sixth line represents the temperature of the battery pack in the energy storage system (battery temperature); and the seventh line represents the power of the energy storage system (active power). Figure 3b The temperature of the energy storage system is the highest temperature of the battery pack in the energy storage system. By comparison... Figure 3a The first straight line and Figure 3bAs shown by the sixth line, after implementing the above thermal management method, the maximum temperature of the energy storage system's battery pack becomes smoother and closer to the preset temperature limit. While steadily controlling the energy storage system temperature, the power consumption of the temperature regulation device is reduced, effectively improving the efficiency of the energy storage system.

[0103] Corresponding to the above embodiments, this specification also proposes a training method for a thermal management control model. Please refer to [link / reference]. Figure 4 Training methods for thermal management control models include:

[0104] S410: Obtain the environmental state variables and action variables of the energy storage system at different times within a preset time period.

[0105] Among them, environmental state variables include the charging and discharging power variables of the energy storage system and the temperature variables of the energy storage system; action variables include the control action variables for thermal management of the energy storage system.

[0106] S420 is a target function for constructing a thermal management control model, with the goal of ensuring that the power consumption of the temperature regulation device used for thermal management meets preset conditions and that the temperature of the energy storage system reaches the target temperature within a preset time period.

[0107] S430 uses environmental state variables and action variables as inputs to the thermal management control model, updates the parameters of the thermal management control model based on the value of the objective function, and obtains the target thermal management control model.

[0108] For specific limitations on the training method of the thermal management control model, please refer to the limitations on the training process of the thermal management control model mentioned above, which will not be repeated here.

[0109] Through the above embodiments, a target thermal management control model is obtained by training the thermal management control model based on environmental state variables and action variables. The target thermal management control model is then used to predict control parameters, achieving the goal of reducing the power consumption of the temperature control device and ensuring smooth and precise temperature control of the energy storage system. This ensures that the overall temperature of the energy storage system tends to be smooth and precise within a preset time period, reducing the risk of overshoot leading to larger temperature differences and reduced cell lifespan. Furthermore, more precise temperature control capabilities provide the energy storage system with a more flexible temperature control range. Without triggering an over-limit alarm in the battery pack temperature, the system operates as close to the upper temperature limit as possible, thereby reducing the power consumption of the temperature regulation device, effectively improving the efficiency of the energy storage system, and increasing the return on investment.

[0110] Corresponding to the above embodiments, this specification also proposes a thermal management device for an energy storage system. Please refer to... Figure 5 The thermal management device of the energy storage system includes:

[0111] The acquisition module 510 is used to acquire the charging and discharging power data and temperature data of the energy storage system.

[0112] The control action determination module 520 is used to input the charge and discharge power data and temperature data into the target thermal management control model to obtain the control action parameters for thermal management of the energy storage system.

[0113] Among them, the control action parameters are used to control the temperature regulation device of the energy storage system so that the power consumption of the temperature regulation device meets the preset conditions and the temperature of the energy storage system reaches the target temperature.

[0114] Through the above embodiments, when performing thermal management on an energy storage system, temperature data and charge / discharge power data are used to predict the control parameters for thermal management of the energy storage system using a target thermal management control model. These control parameters are then used to regulate the temperature control device, ensuring that the power consumption of the temperature control device meets preset conditions and that the energy storage system reaches the target temperature. The embodiments in this specification simultaneously consider the temperature state and charge / discharge state of the energy storage system, using the target thermal management control model to predict control parameters, thereby achieving the goal of reducing the power consumption of the temperature control device and ensuring a smooth and accurate temperature control of the energy storage system.

[0115] Specific limitations regarding the thermal management device of the energy storage system can be found in the limitations of the thermal management method for energy storage systems mentioned above, and will not be repeated here. Each module in the aforementioned thermal management device of the energy storage system can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in hardware or independent of the processor in a computer device, or stored in software in the memory of a computer device, so that the processor can call and execute the corresponding operations of each module.

[0116] Corresponding to the above embodiments, this specification also proposes a training device for a thermal management control model. Please refer to [link / reference]. Figure 6 The training device for the thermal management control model includes:

[0117] The sample acquisition module 610 is used to acquire the environmental state variables and action variables of the energy storage system at different times within a preset time period.

[0118] Among them, environmental state variables include the charging and discharging power variables of the energy storage system and the temperature variables of the energy storage system; action variables include the control action variables for thermal management of the energy storage system.

[0119] The objective function determination module 620 is used to construct the objective function of the thermal management control model with the objectives of the power consumption of the temperature regulation device used for thermal management meeting preset conditions and the temperature of the energy storage system reaching the target temperature within a preset time period.

[0120] The training module 630 is used to update the parameters of the thermal management control model based on the value of the objective function, using environmental state variables and action variables as inputs to the thermal management control model, thereby obtaining the target thermal management control model.

[0121] Through the above embodiments, a target thermal management control model is obtained by training the thermal management control model based on environmental state variables and action variables. The target thermal management control model is then used to predict control parameters, achieving the goal of reducing the power consumption of the temperature control device and ensuring smooth and precise temperature control of the energy storage system. This ensures that the overall temperature of the energy storage system tends to be smooth and precise within a preset time period, reducing the risk of overshoot leading to larger temperature differences and reduced cell lifespan. Furthermore, more precise temperature control capabilities provide the energy storage system with a more flexible temperature control range. Without triggering an over-limit alarm in the battery pack temperature, the system operates as close to the upper temperature limit as possible, thereby reducing the power consumption of the temperature regulation device, effectively improving the efficiency of the energy storage system, and increasing the return on investment.

[0122] Specific limitations regarding the training device for the thermal management control model can be found in the limitations on the training method for the thermal management control model described above, and will not be repeated here. Each module in the aforementioned training device for the thermal management control model can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0123] Corresponding to the above embodiments, this specification also provides an energy storage system.

[0124] Figure 7 This is a structural block diagram of an energy storage system according to one embodiment of this specification. Figure 7 As shown, the energy storage system 700 includes a memory 704, a processor 702, and a computer program 706 stored in the memory 704 and executable on the processor 702. When the processor 702 executes the computer program 706, it implements the thermal management method of the energy storage system in any of the above embodiments.

[0125] According to the embodiments of this specification, when the processor 702 executes the computer program 706, the electronic device can simultaneously consider the temperature state and charging / discharging state of the energy storage system, and use the target thermal management control model to predict the control parameters, thereby achieving the purpose of reducing the power consumption of the temperature control device and ensuring smooth and accurate temperature control of the energy storage system.

[0126] Corresponding to the above embodiments, this specification also proposes a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the thermal management method of the energy storage system of any of the above embodiments.

[0127] According to embodiments of this specification, the computer-readable storage medium, when the computer program is executed by the processor, can simultaneously consider the temperature state and charge / discharge state of the energy storage system, and use the target thermal management control model to predict the control parameters, thereby achieving the purpose of reducing the power consumption of the temperature control device and ensuring smooth and accurate temperature control of the energy storage system.

[0128] It should be noted that the logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0129] It should be understood that various parts of this specification can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0130] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this specification. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0131] In the description of this specification, it should be understood that the terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," "clockwise," "counterclockwise," "axial," "radial," and "circumferential" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing this specification and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this specification.

[0132] Furthermore, the terms "first," "second," etc., used in the embodiments of this specification are for descriptive purposes only and should not be construed as indicating or implying relative importance, or implicitly specifying the number of technical features indicated in this embodiment. Therefore, features defined with terms such as "first" and "second" in the embodiments of this specification can explicitly or implicitly indicate that the embodiment includes at least one of those features. In the description of this specification, the word "multiple" means at least two or more, such as two, three, four, etc., unless otherwise explicitly specified in the embodiments.

[0133] In this specification, unless otherwise explicitly specified or limited in the embodiments, the terms "installation," "connection," "joining," and "fixing," etc., appearing in the embodiments, should be interpreted broadly. For example, a connection can be a fixed connection, a detachable connection, or an integral part; it can also be a mechanical connection, an electrical connection, etc. Of course, it can also be a direct connection, or an indirect connection through an intermediate medium, or it can be the internal communication between two components, or the interaction between two components. Those skilled in the art will be able to understand the specific meaning of the above terms in this specification based on the specific implementation.

[0134] In this specification, unless otherwise expressly specified and limited, "above" or "below" the second feature can mean that the first and second features are in direct contact, or that the first and second features are in indirect contact through an intermediate medium. Furthermore, "above," "over," and "on top" of the second feature can mean that the first feature is directly above or diagonally above the second feature, or simply that the first feature is at a higher horizontal level than the second feature. "Below," "below," and "under" the second feature can mean that the first feature is directly below or diagonally below the second feature, or simply that the first feature is at a lower horizontal level than the second feature.

[0135] Although embodiments of this specification have been shown and described above, it is to be understood that the above embodiments are exemplary and should not be construed as limiting this specification. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of this specification.

Claims

1. A thermal management method for an energy storage system, characterized in that, The method includes: Acquire charging and discharging power data and temperature data of the energy storage system; The charging and discharging power data and the temperature data are input into the target thermal management control model to obtain control action parameters for thermal management of the energy storage system; wherein, the control action parameters are used to control the temperature regulation device of the energy storage system so that the power consumption of the temperature regulation device meets the preset conditions and the temperature of the energy storage system reaches the target temperature. The target thermal management control model is trained in the following way: The system acquires environmental state variables and action variables of the energy storage system at different times within a historical preset time period; wherein, the environmental state variables include the charging and discharging power variables of the energy storage system and the temperature variables of the energy storage system; the action variables include the control action variables for thermal management of the energy storage system. The reward function of the thermal management control model is constructed with the goal of the power consumption of the temperature regulation device meeting the preset conditions and the temperature of the energy storage system reaching the target temperature within the preset historical time period. Using the environmental state variables and the action variables as inputs to the thermal management control model, the parameters of the thermal management control model are updated based on the value of the reward function to obtain the target thermal management control model; The reward function for the thermal management control model is constructed with the objectives of ensuring that the power consumption of the temperature regulation device meets the preset conditions and that the temperature of the energy storage system reaches the target temperature within the preset historical time period. The reward function includes: A first mainline reward function is constructed based on the difference between the historical maximum total power consumption of the temperature regulation device and the total power consumption of the temperature regulation device during the historical preset time period. A second principal reward function is constructed based on the absolute difference between the temperature data of the energy storage system at different times within the historical preset time period and the target temperature at different times. The reward function is constructed based on the first main reward function and the second main reward function.

2. The method according to claim 1, characterized in that, The reward function includes a main reward function and branch penalty terms; the reward function is constructed based on the first main reward function and the second main reward function, including: Based on the first weight coefficient corresponding to the first mainline reward function and the second weight coefficient corresponding to the second mainline reward function, the first mainline reward function and the second mainline reward function are weighted and summed to obtain the mainline reward function; The branch penalty term is determined by multiplying the number of times the temperature of the energy storage system exceeds the preset temperature range within the preset historical time period by the third weighting coefficient. The reward function is obtained by subtracting the main reward function from the branch penalty term.

3. The method according to claim 1, characterized in that, Based on the absolute difference between the temperature data of the energy storage system at different times within the historical preset time period and the corresponding target temperature at different times, a second principal reward function is constructed, including: The first branch term of the second main reward function is determined based on the sum of the squares of the absolute differences between the temperature data of the energy storage system at different times within the historical preset time period and the target temperature at different times. The sum of squares of the target temperature at different times is determined as the second branch term of the second principal reward function; The second main reward function is obtained by subtracting the second branch term from the first branch term.

4. The method according to claim 1, characterized in that, The method further includes: Acquire historical operating data of the energy storage system and historical operating parameters of the temperature regulation device; The physical field simulation model is trained based on the historical operating data and the historical operating parameters to obtain the target physical field simulation model; wherein, the target physical field simulation model is used to solve for the temperature data of the energy storage system at the next moment based on the operating data of the energy storage system at the current moment and the operating parameters of the temperature regulation device.

5. The method according to claim 4, characterized in that, The method further includes: obtaining the environmental state variables and action variables of the energy storage system at different times within a preset time period through the target physical field simulation model.

6. A training method for a thermal management control model, characterized in that, The method includes: The system acquires environmental state variables and action variables of the energy storage system at different times within a historical preset time period; wherein, the environmental state variables include the charging and discharging power variables of the energy storage system and the temperature variables of the energy storage system; the action variables include the control action variables for thermal management of the energy storage system. The reward function of the thermal management control model is constructed with the goal of the power consumption of the temperature regulation device used for thermal management meeting the preset conditions and the temperature of the energy storage system reaching the target temperature within the preset historical time period. Using the environmental state variables and the action variables as inputs to the thermal management control model, the parameters of the thermal management control model are updated based on the value of the reward function to obtain the target thermal management control model; The reward function for the thermal management control model is constructed with the objectives of ensuring that the power consumption of the temperature regulation device meets the preset conditions and that the temperature of the energy storage system reaches the target temperature within the preset historical time period. The reward function includes: A first mainline reward function is constructed based on the difference between the historical maximum total power consumption of the temperature regulation device and the total power consumption of the temperature regulation device during the historical preset time period. A second principal reward function is constructed based on the absolute difference between the temperature data of the energy storage system at different times within the historical preset time period and the target temperature at different times. The reward function is constructed based on the first main reward function and the second main reward function.

7. A thermal management device for an energy storage system, characterized in that, The device includes: The acquisition module is used to acquire charging and discharging power data and temperature data of the energy storage system; The control action determination module is used to input the charge / discharge power data and the temperature data into the target thermal management control model to obtain control action parameters for thermal management of the energy storage system; wherein, the control action parameters are used to control the temperature regulation device of the energy storage system so that the power consumption of the temperature regulation device meets the preset conditions and the temperature of the energy storage system reaches the target temperature. The target thermal management control model is trained in the following way: The system acquires environmental state variables and action variables of the energy storage system at different times within a historical preset time period; wherein, the environmental state variables include the charging and discharging power variables of the energy storage system and the temperature variables of the energy storage system; the action variables include the control action variables for thermal management of the energy storage system. The reward function of the thermal management control model is constructed with the goal of the power consumption of the temperature regulation device meeting the preset conditions and the temperature of the energy storage system reaching the target temperature within the preset historical time period. Using the environmental state variables and the action variables as inputs to the thermal management control model, the parameters of the thermal management control model are updated based on the value of the reward function to obtain the target thermal management control model; The reward function for the thermal management control model is constructed with the objectives of ensuring that the power consumption of the temperature regulation device meets the preset conditions and that the temperature of the energy storage system reaches the target temperature within the preset historical time period. The reward function includes: A first mainline reward function is constructed based on the difference between the historical maximum total power consumption of the temperature regulation device and the total power consumption of the temperature regulation device during the historical preset time period. A second principal reward function is constructed based on the absolute difference between the temperature data of the energy storage system at different times within the historical preset time period and the target temperature at different times. The reward function is constructed based on the first main reward function and the second main reward function.

8. An energy storage system, characterized in that, The method includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor, when executing the computer program, implements the method as claimed in any one of claims 1 to 5.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored computer program, wherein, when the computer program is executed, it controls the device on which the computer-readable storage medium is located to perform the method as described in any one of claims 1 to 6.

Images