Reinforcement learning-based soc dynamic balancing control method and device for energy storage system

Abstract

The application relates to the technical field of SOC dynamic balance control of an energy storage system, and provides an energy storage system SOC dynamic balance control method and device based on reinforcement learning, which solves the problems of poor balance and poor operation stability of the energy storage system. The method comprises the following steps: dynamically adjusting battery operation parameters of a target area of the energy storage system; collecting first temperature and humidity values and SOC deviation values of the target area and other areas; combining the data of all areas, calculating optimal balance current parameters corresponding to each area, including the size and direction of the balance current; and transmitting the optimal balance current parameters to a balance circuit of the corresponding area, so that energy is transferred between the batteries of the target area and the other areas through the balance circuit, thereby reducing the SOC deviation between the areas, making the battery states of the energy storage system more balanced, and improving the overall operation stability. The application improves the balance and operation stability of the energy storage system.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of SOC dynamic balancing control of energy storage systems, and particularly relates to an energy storage system SOC dynamic balancing control method and device based on reinforcement learning. BACKGROUND

[0002] In the application scenarios of reinforcement learning-based SOC dynamic balancing control of energy storage systems such as distributed energy grid connection and outdoor energy storage base, factors such as environmental temperature and humidity and load fluctuation often present complex dynamic changes. These changes will cause the SOC (State of Charge) of battery modules in different regions of the energy storage system to be unbalanced, which not only affects the charging and discharging efficiency of the system, but also shortens the service life of the battery and even causes safety hazards. Therefore, a control method that can perceive the dynamic characteristics of the scene in real time and quickly respond to SOC deviation is needed to achieve accurate balancing of the SOC of battery modules in different regions and to ensure stable and efficient operation of the energy storage system in complex scenarios.

[0003] At present, there is an energy storage system SOC balancing control scheme based on model predictive control to meet the above technical needs. This scheme establishes a mathematical model of the energy storage system, predicts the SOC change trend in the future period of time combined with historical operation data, and then generates a control strategy to adjust the charging and discharging current of each region to achieve SOC balancing. The core is to use the model to describe the dynamic characteristics of the system to plan the control action in advance, so as to cope with the dynamic changes in the scene to some extent.

[0004] However, the existing scheme based on model predictive control has obvious defects. On the one hand, its control effect is highly dependent on the accuracy of the established mathematical model, while in actual complex scenarios, the dynamic characteristics of the energy storage system will change due to factors such as sudden changes in environmental temperature and humidity and battery aging, causing deviations between the model and the actual system, and thus affecting the accuracy of SOC balancing control; on the other hand, model predictive control has high computational complexity when dealing with multi-variable coupling and fast dynamic changes, making it difficult to achieve real-time response, unable to timely compensate for the SOC deviation caused by sudden load fluctuations and other situations, and unable to meet the efficiency and real-time requirements of SOC dynamic balancing control in reinforcement learning scenarios. SUMMARY

[0005] The present application provides an energy storage system SOC dynamic balancing control method and device based on reinforcement learning to solve the problem of poor balancing and operation stability of the energy storage system caused by model dependence deviation, high computational complexity and response lag in the prior art.

[0006] In a first aspect, the present application provides an energy storage system SOC dynamic balancing control method based on reinforcement learning, comprising:

[0007] The operation parameters of the battery modules in the target region in the energy storage system are dynamically adjusted based on reinforcement learning;

[0008] After the adjustment of the operation parameters is completed, the first temperature and humidity values and SOC deviation values of the target region and other regions are collected;

[0009] Based on the first temperature and humidity values and the SOC deviation values of all regions, optimal balancing current parameters corresponding to each region are calculated using an adaptive control algorithm, the optimal balancing current parameters including the size and direction of the balancing current;

[0010] The optimal balancing current parameters are transmitted to the balancing circuit of the corresponding region, and energy is transferred between the battery modules in the target region and other regions through the balancing circuit to reduce the SOC deviation between regions.

[0011] Optionally, the dynamic adjustment of the operation parameters of the battery modules in the target region in the energy storage system based on reinforcement learning comprises:

[0012] The second temperature and humidity values of multiple regions in the energy storage system and the SOC values of the battery modules in each region are collected, the collection time of the second temperature and humidity values being earlier than that of the first temperature and humidity values;

[0013] The SOC values of the battery modules in all regions are aggregated to form an energy storage system SOC distribution, and the daily temperature and humidity trend is obtained based on the second temperature and humidity values and historical meteorological data;

[0014] Based on the daily temperature and humidity trend and the energy storage system SOC distribution, a global balancing strategy is generated using reinforcement learning, and based on the global balancing strategy, a target region whose temperature and humidity value exceeds a set temperature and humidity threshold and whose SOC deviation exceeds a preset deviation threshold is identified;

[0015] The operation parameters of the battery modules in the corresponding region are dynamically adjusted by the controller corresponding to the target region, the operation parameters including the charge-discharge rate and the working parameters of the temperature and humidity adjustment unit.

[0016] Optionally, based on the daily temperature and humidity trend and the energy storage system SOC distribution, a global balancing strategy is generated using reinforcement learning, and based on the global balancing strategy, a target region whose temperature and humidity value exceeds a set temperature and humidity threshold and whose SOC deviation exceeds a preset deviation threshold is identified, comprising:

[0017] The daily temperature and humidity trend and the energy storage system SOC distribution are taken as input features of reinforcement learning, and temperature and humidity feature weights and SOC feature weights are configured respectively;

[0018] weighting and fusing the daytime temperature and humidity trend and the energy storage system SOC distribution based on the temperature and humidity characteristics and the SOC characteristic weight to generate an initial strategy parameter;

[0019] Based on the initial strategy parameter, combined with the real-time operation feedback data of all regions, the target strategy parameter containing the temperature and humidity and SOC associated control coefficients is obtained through multiple rounds of iteration adjustment by reinforcement learning algorithm.

[0020] Based on the target strategy parameter, a global equilibrium strategy containing the temperature and humidity allowable fluctuation range and the SOC allowable deviation interval of all regions is constructed.

[0021] The second temperature and humidity value of all regions is compared with the temperature and humidity allowable range to screen out temperature and humidity abnormal regions, and the actual SOC value of all regions is compared with the SOC allowable fluctuation interval to screen out SOC imbalance regions.

[0022] The overlapping part of the temperature and humidity abnormal region and the SOC imbalance region is determined as the target region.

[0023] Optionally, based on the first temperature and humidity value and the SOC deviation value of all regions, the optimal balancing current parameter corresponding to each region is calculated by using an adaptive control algorithm, and the optimal balancing current parameter includes the size and direction of the balancing current, including:

[0024] The first temperature and humidity value and the SOC deviation value of all regions are associated to obtain a dynamic influence relationship of temperature and humidity value on SOC deviation.

[0025] According to the dynamic influence relationship, the corresponding adjustment coefficient is configured for all regions, and based on the adjustment coefficient, the SOC deviation value of all regions is quantitatively converted to obtain the deviation amount that needs to be compensated for each region.

[0026] According to the numerical size of the deviation amount, the size of the balancing current is determined, and according to the positive and negative attributes of the deviation amount, the direction of the balancing current is determined, and based on the size and the direction, the preliminary current parameter is generated.

[0027] Through the adaptive control algorithm, the preliminary current parameter is dynamically corrected to obtain the optimal balancing current parameter corresponding to each region.

[0028] Optionally, according to the dynamic influence relationship, the corresponding adjustment coefficient is configured for all regions, and based on the adjustment coefficient, the SOC deviation value of all regions is quantitatively converted to obtain the deviation amount that needs to be compensated for each region, including:

[0029] The dynamic influence relationship divides the temperature and humidity value range into multiple continuous intervals, and configures a corresponding adjustment coefficient for each interval;

[0030] According to the first temperature and humidity value of each region, the corresponding adjustment coefficient is matched;

[0031] According to the number of battery modules in each region and the SOC consistency between the modules, the adjustment coefficient is corrected to obtain a module correlation coefficient of each region;

[0032] The module correlation coefficient is multiplied by the SOC deviation value of the corresponding region to obtain a preliminary deviation amount of each region;

[0033] According to the current operating power of each region, the preliminary deviation amount is dynamically scaled to obtain a deviation amount that needs to be compensated for each region.

[0034] Optionally, the optimal balancing current parameter is transmitted to the balancing circuit of the corresponding region, and energy is transferred between the battery modules of the target region and other regions through the balancing circuit to reduce the SOC deviation between regions, including:

[0035] The optimal balancing current parameter is sent to the balancing circuit of the corresponding region through a signal transmission channel;

[0036] The optimal balancing current parameter is analyzed by the balancing circuit to determine the starting region and the receiving region of energy transfer;

[0037] According to the position distribution of the starting region and the receiving region, a corresponding energy transmission path is selected;

[0038] Based on the current size in the optimal balancing current parameter, the energy transmission intensity of the energy transmission path is adjusted to make the energy transmitted between the battery modules of the starting region and the receiving region according to the set intensity;

[0039] During the energy transmission process, the SOC deviation value of each region is continuously monitored, and when the SOC deviation value is reduced to a preset range, the balancing circuit is controlled to stop energy transmission.

[0040] Optionally, according to the position distribution of the starting region and the receiving region, a corresponding energy transmission path is selected, including:

[0041] The position information of the starting region and the receiving region is obtained, and the relative position relationship is determined based on the position information;

[0042] Based on the relative position relationship, all potential transmission paths that can connect the starting region and the receiving region are listed;

[0043] checking current use states of the potential transmission paths, and screening out a standby transmission path in an available state;

[0044] configuring priorities for all standby transmission paths, and selecting a first standby transmission path as the energy transmission path according to the priorities from high to low.

[0045] In a second aspect, the present application provides a SOC dynamic balancing control device for an energy storage system based on reinforcement learning, comprising:

[0046] an adjusting module configured to dynamically adjust operating parameters of battery modules in a target region in the energy storage system based on reinforcement learning;

[0047] a collecting module configured to collect first temperature and humidity values and SOC deviation values of the target region and other regions after the adjustment of the operating parameters is completed;

[0048] a calculating module configured to calculate optimal balancing current parameters corresponding to each region based on the first temperature and humidity values and the SOC deviation values of all regions by using an adaptive control algorithm, wherein the optimal balancing current parameters include sizes and directions of balancing currents;

[0049] an optimizing module configured to transmit the optimal balancing current parameters to balancing circuits of the corresponding regions, and perform energy transfer between the battery modules in the target region and other regions by the balancing circuits to reduce the SOC deviation between the regions.

[0050] In a third aspect, the present application provides a computing device comprising a processing component and a storage component; the storage component stores one or more computer instructions; the one or more computer instructions are used to be called and executed by the processing component to implement the SOC dynamic balancing control method for the energy storage system based on reinforcement learning according to any one of the first aspect.

[0051] In a fourth aspect, the present application provides a computer storage medium storing a computer program, wherein the computer program is executed by a computer to implement the SOC dynamic balancing control method for the energy storage system based on reinforcement learning according to any one of the first aspect.

[0052] In the present application, a SOC dynamic balancing control method for energy storage system based on reinforcement learning is provided, which comprises: dynamically adjusting the operating parameters of the battery modules in the target area of the energy storage system based on reinforcement learning; after the adjustment of the operating parameters is completed, collecting the first temperature and humidity values and SOC deviation values of the target area and other areas; based on the first temperature and humidity values and SOC deviation values of all areas, calculating the optimal balancing current parameters corresponding to each area using an adaptive control algorithm, the optimal balancing current parameters including the size and direction of the balancing current; transmitting the optimal balancing current parameters to the balancing circuit of the corresponding area, and transferring energy between the battery modules in the target area and other areas through the balancing circuit to reduce the SOC deviation between areas.

[0053] The present application has the following advantages:

[0054] By dynamically adjusting the operating parameters of the battery modules in the target area of the energy storage system based on reinforcement learning, an initial control basis can be provided for subsequent SOC balancing control, ensuring that the control direction matches the actual state of the target area; by collecting the first temperature and humidity values and SOC deviation values of the target area and other areas after the adjustment of the operating parameters is completed, real-time state data after adjustment can be obtained, providing accurate input for the calculation of subsequent optimal balancing current parameters; by calculating the optimal balancing current parameters corresponding to each area using an adaptive control algorithm based on the first temperature and humidity values and SOC deviation values of all areas, including the size and direction of the balancing current, adaptive current control parameters can be generated in combination with real-time state differences, ensuring the accuracy and pertinence of balancing control; by transmitting the optimal balancing current parameters to the balancing circuit of the corresponding area and transferring energy between the battery modules in the target area and other areas through the balancing circuit, directional and quantitative energy transfer can be achieved, directly reducing the SOC deviation between areas and achieving the goal of dynamic balancing control.

[0055] Further, the second temperature and humidity values and SOC values of multiple areas of the energy storage system are collected to form an SOC distribution and obtain the daytime temperature and humidity trend in combination with historical meteorological data; both are used as input features of reinforcement learning, and after being weighted and fused to generate initial strategy parameters after being configured with corresponding weights, target strategy parameters are obtained by iterative adjustment in combination with real-time operation feedback, a global balancing strategy containing the allowed fluctuation range of temperature and humidity and the allowed deviation interval of SOC is constructed, and the overlapping part of the temperature and humidity abnormal area and the SOC unbalanced area is selected as the target area, and the charge-discharge rate and the working parameters of the temperature and humidity adjusting unit thereof are adjusted by the controller. The corresponding technical effect is: by fusing historical and real-time data and dynamically adjusting strategy parameters, the target area with abnormal temperature and humidity and SOC is accurately identified, and the operating parameters are adjusted accordingly, improving the accuracy of target area positioning and the adaptability of control parameters, and providing more accurate early-stage guarantee for global SOC dynamic balancing control.

[0056] These aspects or other aspects of the present application will be made clearer in the following description of embodiments. BRIEF DESCRIPTION OF DRAWINGS

[0057] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the description of embodiments or prior art. Obviously, the drawings described below are only some embodiments of the present application, and those skilled in the art can obtain other drawings according to these drawings without creative labor.

[0058] Figure 1 A flow chart of a SOC dynamic balancing control method of an energy storage system based on reinforcement learning provided by an embodiment of the present application;

[0059] Figure 2 A structural schematic diagram of a SOC dynamic balancing control device of an energy storage system based on reinforcement learning provided by an embodiment of the present application;

[0060] Figure 3 A structural schematic diagram of a computing device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0061] In order to make the person skilled in the art better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application.

[0062] In some processes described in the specification and claims of the present application and the above-mentioned drawings, a plurality of operations appearing in a specific order are included, but it should be clearly understood that these operations can be executed or in parallel without the order in which they appear in this text. The serial numbers of the operations, such as 11, 12, etc., are only used to distinguish different operations, and the serial numbers themselves do not represent any execution order. In addition, these processes can include more or fewer operations, and these operations can be executed in sequence or in parallel. It should be noted that the "first", "second", etc. described herein are used to distinguish different messages, devices, modules, etc., and do not represent the order of precedence. Also, "first" and "second" are not different types.

[0063] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0064] To solve the problems of model-dependent deviation, complex calculation and response lag in the prior art, an SOC dynamic balancing control method for an energy storage system based on reinforcement learning is provided in the embodiments of the present application, which adopts the following concept: first, the operating parameters of the battery in the key area of the battery system are dynamically adjusted through reinforcement learning; after the parameters are adjusted, the temperature, humidity data of these key areas and other areas, and the difference data of the battery state of charge are collected; then, the current size and direction that can balance the battery state are calculated according to the data by using an adaptive control algorithm; finally, the current parameters are transmitted to the corresponding circuit, and the energy is transferred between the batteries in different areas through the circuit, so as to reduce the difference of the battery state in each area. This method does not need to rely on an accurate mathematical model, can dynamically adjust according to the actual situation, solves the control deviation problem caused by the inaccurate model in the prior art, and reduces the difficulty of processing complex situations through the pre-adjustment of reinforcement learning and the efficient calculation of the adaptive control algorithm, thereby solving the problem of complex calculation. Moreover, the data can be collected in real time and the appropriate current can be calculated quickly, the energy can be transferred in time, the problem of untimely response in the prior art is avoided, and the state of the battery system can be more stably balanced.

[0065] Figure 1 A flowchart of the SOC dynamic balancing control method for an energy storage system based on reinforcement learning provided in the embodiments of the present application is shown in Figure 1 , and the method comprises the following steps.

[0066] S11, dynamically adjusting the operating parameters of the battery module in the target area of the energy storage system based on reinforcement learning.

[0067] The reinforcement learning is a method of achieving a goal by interacting with the environment and continuously optimizing the behavior; the energy storage system is a whole device for storing and releasing electric energy; the target area is a specific area in the energy storage system that needs to be adjusted; the battery module is a group of batteries that constitute the energy storage system; the operating parameters include the speed of battery charging and discharging, the working strength of the temperature adjusting device, etc., and the dynamic adjustment of these parameters means that the operating parameters are changed in real time according to the actual situation, and finally the operating parameters that adapt to the state of the target area are generated.

[0068] In the embodiments of the present application, first, the multiple regions of the energy storage system are determined, the historical operation data of each region including the past temperature, humidity, battery state and current real-time state data are collected, and the target region that needs to be adjusted in priority is found out by analyzing these data through reinforcement learning; then, the reinforcement learning generates an adjustment strategy according to the current state and possible changes of the target region, which considers how to make the battery of the target region operate more reasonably; finally, the operation parameters of the battery modules of the target region are dynamically adjusted according to the strategy, for example, if the charging and discharging of the battery of the target region is too fast and may affect the state balance, the speed is slowed down, and if the temperature is too high and may affect the performance of the battery, the working strength of the heat dissipation equipment is increased, for example, in an outdoor energy storage base station, the reinforcement learning analysis finds that the battery state deviation of region B is large and the temperature is high, and then generates a strategy: because the current charging and discharging speed of region B is fast and may accelerate the deviation, the speed is appropriately slowed down, and because the temperature is high, the power of the heat dissipation equipment is increased to improve the environment.

[0069] S12, after the completion of the adjustment of the operation parameters, the first temperature and humidity values and the SOC deviation values of the target region and other regions are collected.

[0070] Among them, the first temperature and humidity values refer to the current temperature and humidity data collected by the sensor after the completion of the adjustment of the operation parameters of the target region and other regions in the energy storage system; the SOC deviation values refer to the difference data between the charging states of the batteries of different regions, which are recorded after collection and calculation, and can be used to reflect the differences between the environmental states and the battery states of the regions after adjustment.

[0071] In the embodiments of the present application, first, it is confirmed that the adjustment of the operation parameters of the target region has been completed, to ensure that the data collected at this time can truly reflect the state after adjustment; then, the current temperature and humidity of the target region and all other regions are measured by using the sensor to obtain the first temperature and humidity values; then, the charging states of the batteries of the regions, that is, the SOC values, are detected to calculate the SOC deviation values between different regions, for example, the SOC value of the target region is subtracted from the SOC values of other regions to obtain the difference values, which are the deviations of the two; finally, the collected first temperature and humidity values and the calculated SOC deviation values are recorded as reference data for subsequent steps, for example, in an outdoor energy storage base station, after the completion of the adjustment of the operation parameters of region B, the temperature and humidity of region B, the temperature and humidity of region A, and the temperature and humidity of region C are measured by using the sensor; the SOC of region B, the SOC of region A, and the SOC of region C are detected, the SOC deviation between region B and region A is calculated, the SOC deviation between region B and region C is calculated, and these data are recorded.

[0072] S13, based on the first temperature and humidity values and the SOC deviation values of all regions, the optimal balanced current parameters corresponding to each region are calculated by using the adaptive control algorithm, and the optimal balanced current parameters include the size and direction of the balanced current.

[0073] Wherein, the adaptive control algorithm is a method that can automatically adjust the calculation method according to the input real-time data to adapt to different situations; the optimal balanced current parameter refers to the current related data that can make the battery state of each region balanced, wherein the current size represents how much energy transfer, and the current direction represents which region the energy is transferred from and which region the energy is transferred to, and these parameters are generated by algorithm analysis data.

[0074] In the embodiments of the present application, first, the first temperature and humidity values and the SOC deviation values of all regions are collected, and these data are input into the adaptive control algorithm; then the algorithm analyzes these data, determines how much energy each region needs to receive or output, that is, the current size, and from which region the energy should flow to which region, that is, the current direction, in combination with the influence of temperature and humidity on the battery state, such as high humidity affecting energy transfer efficiency; for example, when the battery state of region A is lower than that of the target region, energy needs to flow from the target region to region A, and the current direction is from the target region to region A, and the current size is determined according to the deviation size; finally, the algorithm calculates the optimal balanced current parameters corresponding to each region, for example, in an outdoor energy storage base station, after the adaptive control algorithm analyzes the first temperature and humidity values of regions A, B and C, the humidity of region A is slightly high, which may affect the transfer efficiency, and the SOC deviation values of region A are lower than those of region B and region C are higher than those of region B, it is determined that region B needs to transfer energy to region A, and the current size is appropriately set due to the influence of deviation and humidity, and region C needs to transfer energy to region B, and the current size is appropriately set due to the deviation condition, and the directions are B to A and C to B respectively.

[0075] S14, transmit the optimal balanced current parameter to the balancing circuit of the corresponding region, and transfer energy between the battery modules of the target region and other regions through the balancing circuit to reduce the SOC deviation between regions.

[0076] Wherein, the balancing circuit is a circuit for realizing energy transfer between battery modules of different regions; energy transfer refers to the process of transferring electrical energy from the battery modules of one region to the battery modules of another region; the SOC deviation is the difference in the state of charge of the battery in different regions, and the result generated by energy transfer is that the SOC deviation between regions is reduced.

[0077] In this embodiment, the calculated optimal equalization current parameters for each region, including current magnitude and direction, are first transmitted to the corresponding equalization circuit. For example, the equalization circuit connecting regions B and A receives parameters from B to A, and the equalization circuit connecting regions C and B receives parameters from C to B. Then, the equalization circuit operates according to the received parameters, controlling the amount of energy transferred according to the current magnitude and the path of energy transfer according to the current direction, thereby realizing the transfer of electrical energy between the target region and other regions. During the transfer process, the SOC deviation of each region is continuously monitored until the deviation is reduced to a reasonable range. For example, in an outdoor energy storage base station, the equalization circuit connecting B and A operates according to the set current magnitude and direction, transferring electrical energy from B to A, and the equalization circuit connecting C and B operates according to the set current magnitude and direction, transferring electrical energy from C to B. After a period of time, the SOC deviation between regions B and A decreases, and the SOC deviation between regions B and C decreases.

[0078] This application provides the following specific example: In an outdoor energy storage base station's energy storage system, there are three areas: A, B, and C. Through reinforcement learning analysis, it was discovered that area B has a higher temperature and greater battery state fluctuations, thus identifying it as the target area. Subsequently, its charging and discharging speed was slowed down, and its heat dissipation power was increased. After adjustment, real-time temperature and humidity were collected for areas B, A, and C. The detected charging states were 75%, 70%, and 82%, respectively, and the deviations between B and A were calculated to be 5% and between B and C to be 7%. The data was input into an adaptive algorithm to obtain optimal current parameters: 3A was transferred from B to A, and 2A was transferred from C to B. The corresponding equalization circuit operated according to these parameters. After one hour, the charging state deviations of each area were reduced to approximately 2%, resulting in a more balanced state.

[0079] By executing S11~S14, this embodiment of the application uses reinforcement learning to accurately locate the target area that needs adjustment and dynamically optimize its operating parameters, laying a reasonable foundation for subsequent equalization control; it collects real-time temperature, humidity and SOC deviation data after adjustment to ensure that subsequent calculations are based on the real state; it uses an adaptive control algorithm to generate current parameters that are adapted to the environment and state of each area to ensure the accuracy of energy transfer; and finally, it achieves directional and quantitative energy transfer through the equalization circuit, effectively reducing the SOC deviation between each area, making the overall state of the energy storage system more balanced, and improving the stability and reliability of system operation.

[0080] In one possible embodiment, S11, dynamically adjusting the operating parameters of the battery modules in the target area of ​​the energy storage system based on reinforcement learning, includes:

[0081] Step 111: Collect the second temperature and humidity values ​​of multiple areas in the energy storage system and the SOC value of the battery modules in each area. The second temperature and humidity values ​​are collected earlier than the first temperature and humidity values.

[0082] The second temperature and humidity value is temperature and humidity data of multiple regions of the energy storage system obtained by the sensor before the first temperature and humidity value is planned to be collected, and is used to reflect the environment state of an earlier period. The SOC value is the current state of charge data of the battery module in each region, and reflects the power situation of the battery. These data jointly constitute the early information basis for subsequent analysis, and provide coherent initial data support for subsequent steps.

[0083] In the embodiments of the present application, first, multiple regions of the energy storage system that need to be monitored are determined, and sensors for detecting temperature and humidity and devices for detecting battery power are installed in each region. Then, according to the planned collection time of the first temperature and humidity value, the second temperature and humidity value is collected at an earlier time point (such as several hours in advance), and the SOC value of each region is obtained through the battery detection device. Finally, the collected second temperature and humidity value and SOC value are classified and stored for subsequent use. For example, in an outdoor energy storage base station, the first temperature and humidity value is planned to be collected at 2 pm, the second temperature and humidity value of each region is collected at 10 am, including the temperature and humidity of region A being 18℃ and 45% respectively, the temperature and humidity of region B being 19℃ and 47% respectively, and the temperature and humidity of region C being 20℃ and 50% respectively, and the SOC values of the batteries in each region are 68%, 70% and 72% respectively, and these data are sorted and archived according to regions and types.

[0084] In step 112, the SOC values of the battery modules in all regions are aggregated to form an energy storage system SOC distribution, and a daytime temperature and humidity trend is obtained based on the second temperature and humidity value and combined with historical meteorological data.

[0085] The energy storage system SOC distribution is the overall state presentation formed by aggregating the SOC values of the battery modules in each region, and reflects the distribution of the power of all batteries in the system. The daytime temperature and humidity trend is the general rule of the change of the temperature and humidity with time obtained by analyzing the second temperature and humidity value and the meteorological data of a period of time (such as the past week), and is used to predict the change direction of the environment of the day.

[0086] In the embodiment of the present application, first, the SOC values of each region collected in step 111 are summarized, and the SOC values of each region are listed by region to form an energy storage system SOC distribution that can intuitively display the differences in energy of each region; second, the meteorological data of the past period of time is collected, and the temperature and humidity changes in different time periods of each day are sorted out; finally, the second temperature and humidity values are compared with the historical meteorological data to find out the common law of the change of temperature and humidity with time, and then the daytime temperature and humidity trend of the day is inferred, for example, in an outdoor energy storage base, after summarizing the SOC values, the SOC of regions A to C is 68%, 70%, and 72% respectively, forming an SOC distribution of “energy gradually increasing from A to C”; combined with the data of the past week, it is found that the temperature and humidity rise by an average of 1°C and 3% per hour from 9 am to 2 pm, combined with the second temperature and humidity value, it is inferred that the daytime temperature and humidity will gradually rise.

[0087] Step 113, based on the daytime temperature and humidity trend and the energy storage system SOC distribution, a global equilibrium strategy is generated by reinforcement learning, and based on the global equilibrium strategy, a target region whose temperature and humidity value exceeds a set temperature and humidity threshold and whose SOC deviation exceeds a preset deviation threshold is identified.

[0088] Wherein, the global equilibrium strategy is a scheme for overall regulation of the system generated by reinforcement learning by analyzing the daytime temperature and humidity trend and the SOC distribution, which contains the reasonable temperature and humidity range and the battery state difference standard of each region; the set temperature and humidity threshold is the reasonable upper limit of the temperature and humidity of each region specified in the global strategy, and the preset deviation threshold is the reasonable difference range of the SOC value of each region and the average SOC value of the system; the target region is the region that exceeds both thresholds and needs to be adjusted first.

[0089] In the embodiment of the present application, first, the daytime temperature and humidity trend and the energy storage system SOC distribution are input into the reinforcement learning model, and the model generates a global equilibrium strategy by analyzing the correlation between the two (such as the influence of temperature and humidity rise on battery state), and clearly defines the temperature and humidity threshold and the SOC deviation threshold of each region; second, the average SOC value of the system is calculated, and the deviation is obtained by subtracting the average value from the SOC value of each region; finally, the second temperature and humidity value is compared with the set temperature and humidity threshold, and the SOC deviation of each region is compared with the preset deviation threshold, and the region that exceeds both thresholds at the same time is identified as the target region, for example, in an outdoor energy storage base, the global strategy generated by reinforcement learning sets the temperature and humidity threshold to 25°C and 60%, and the SOC deviation threshold to 3%; the average SOC of the system is (68%+70%+72%) ÷ 3 = 70%, the deviation of region A is 68%-70% =-2%, the deviation of region B is 70%-70% = 0, and the deviation of region C is 72%-70% = 2%, none of which exceeds the deviation threshold; the second temperature and humidity value is lower than the temperature and humidity threshold, so no target region is identified.

[0090] Step 114, dynamically adjusting the operation parameters of the battery modules in the corresponding area through the controller corresponding to the target area, the operation parameters including the charge-discharge rate and the working parameters of the temperature and humidity adjusting unit.

[0091] Wherein, the controller is a device installed in each area for executing the adjustment command; the charge-discharge rate reflects the speed of battery charging or discharging, and the working parameters (such as power) of the temperature and humidity adjusting unit reflect the working intensity of the heat dissipation or humidification equipment; by adjusting these parameters, the battery state and the environmental temperature and humidity of the target area are made to approach the reasonable range, generating a more optimal operating state.

[0092] In the embodiments of the present application, first, when the target area is identified in step 113, the system sends an adjustment instruction to the controller of the area, the instruction is based on the global balancing strategy and clearly indicates the charge-discharge rate and temperature and humidity adjustment parameters that need to be adjusted; second, after the controller receives the instruction, it adjusts the charge-discharge speed of the battery modules and the working intensity of the temperature and humidity adjusting unit; finally, the adjusted state is continuously monitored to ensure that the parameter changes meet the expectations, for example, in an outdoor energy storage base station, if area C is identified as the target area (temperature 26℃ exceeds 25℃, SOC deviation 4% exceeds 3%), after the controller receives the instruction, the charge-discharge rate is adjusted from 1.0 to 0.8 (slow down the charging), and the heat dissipation power is adjusted from 400W to 500W (enhance the heat dissipation), so that the state approaches the reasonable range.

[0093] The present application provides the following specific examples: in the energy storage system of an outdoor energy storage base station, there are three areas A, B and C. In the morning, the staff collects the second temperature and humidity values and the state of charge of the battery modules in each area according to the plan, finds that the temperature and humidity in area A are relatively low, area B is next, and area C is slightly higher; at the same time, it is observed that the power in area A is less, the power in area B is medium, and the power in area C is more, these conditions are recorded and archived according to the area, and another set of data is collected at noon. Then, the state of charge of each area is summarized, showing a distribution of gradually increasing power from A to C; combined with the past weather records for a period of time, it is found that the temperature and humidity usually gradually rises from morning to noon, so it is inferred that the temperature and humidity during this period of the day will also show a similar trend. Then, the temperature and humidity change trend and the power distribution are input into the reinforcement learning model to generate a global balancing strategy, which sets a reasonable upper limit for the temperature and humidity and a reasonable range for the difference in battery state; through comparative analysis, the temperature and humidity and the power difference in each area are within the reasonable range at the beginning, and no area needs to be adjusted. During the afternoon monitoring, it is found that the temperature in area C exceeds the reasonable upper limit, and the difference in power between it and other areas also exceeds the reasonable range, so it is determined as the target area; the controller corresponding to the area receives the adjustment instruction, slows down the charge-discharge speed of the battery, and increases the working intensity of the heat dissipation equipment, and after continuous monitoring, the state of area C gradually approaches the reasonable range.

[0094] By performing steps 111-114, the embodiment of the application provides a coherent information base for the entire process by collecting early data in advance through step 111; step 112 clearly presents the system state and predicts environmental changes by aggregating the SOC value and analyzing the temperature and humidity trend, providing a basis for strategy formulation; step 113 uses reinforcement learning to generate a global strategy and accurately identify target areas, ensuring that the control object is clear; step 114 adjusts the parameters through the controller to make the target area state closer to the reasonable range. The overall process realizes a closed loop from data collection, trend analysis to strategy generation and precise control, improving the balance and stability of the energy storage system operating state.

[0095] In a possible embodiment, step 113, based on the daytime temperature and humidity trend and the energy storage system SOC distribution, uses reinforcement learning to generate a global balance strategy, and based on the global balance strategy, identifies a target area whose temperature and humidity value exceeds the set temperature and humidity threshold and whose SOC deviation exceeds the preset deviation threshold, including:

[0096] a1, taking the daytime temperature and humidity trend and the energy storage system SOC distribution as input features of reinforcement learning, and configuring temperature and humidity feature weights and SOC feature weights respectively.

[0097] Among them, the daytime temperature and humidity trend refers to the general rule of temperature and humidity changing with time in a day, and the energy storage system SOC distribution is the overall distribution of battery power in each area, both of which are the basis information for reinforcement learning analysis; temperature and humidity feature weights and SOC feature weights are numerical values representing the importance of these two types of information in analysis, and the weight size reflects the influence of this type of information on the result, and by configuring the weight, the analysis can be more in line with the actual demand, ultimately providing a focused input basis for subsequent strategy generation.

[0098] In the embodiment of the application, first, the core information that reinforcement learning needs to analyze is determined as the daytime temperature and humidity trend and the energy storage system SOC distribution, and both are determined as input features; second, according to the influence of temperature and humidity and battery power on system balance in actual scenarios, corresponding weights are configured for these two types of features, if temperature and humidity have a greater impact on the battery, the temperature and humidity feature weight is slightly higher, and vice versa, the SOC feature weight is higher; for example, in an outdoor energy storage base station, because the change of environmental temperature and humidity has a significant impact on the battery state, the temperature and humidity feature weight is configured to be slightly higher than the SOC feature weight, to highlight the role of temperature and humidity factors.

[0099] a2, based on the temperature and humidity feature and the SOC feature weight, weighting and fusing the daytime temperature and humidity trend and the energy storage system SOC distribution to generate an initial strategy parameter.

[0100] Wherein, the weighted fusion is a process of comprehensively calculating the daytime temperature and humidity trend and the SOC distribution two types of information by combining the temperature and humidity feature weight and the SOC feature weight, and adjusting the proportion of the two types of information in the fusion result through the weight; the initial strategy parameter is a preliminary control standard obtained after the fusion, which contains the basic data of the reasonable operation state of each region inferred according to the current information, and provides an initial basis for subsequent optimization.

[0101] In the embodiment of the application, first, the temperature and humidity feature weight and the SOC feature weight configured in a1 are taken out, the daytime temperature and humidity trend data is adjusted according to the temperature and humidity weight, and the SOC distribution data is adjusted according to the SOC weight; then the two types of adjusted information are combined and calculated to comprehensively obtain a set of preliminary control parameters, i.e. the initial strategy parameter, which reflects the preliminary judgment of the operation state of each region under the current weight; for example, in an outdoor energy storage base station, the temperature and humidity trend data gradually rising is adjusted by a temperature and humidity weight of 0.6, and the power distribution data of A low, B medium and C high is adjusted by a SOC weight of 0.4, and then the two results are added to obtain the initial strategy parameter containing the preliminary control standard of the temperature and humidity and power of each region.

[0102] a3, based on the initial strategy parameter, combining the real-time operation feedback data of all regions, the target strategy parameter containing the temperature and humidity and SOC associated control coefficient is obtained through the multi-round iterative adjustment of the reinforcement learning algorithm.

[0103] Wherein, the real-time operation feedback data is the actual operation data of the current temperature, humidity and battery power of each region, which is used to reflect the actual effect of strategy execution; the reinforcement learning algorithm adjusts the strategy parameter repeatedly by comparing the actual effect with the expected target; the multi-round iterative adjustment is a process of repeatedly executing the strategy, collecting the feedback and adjusting the parameter, so that the parameter is gradually optimized; the target strategy parameter is the final control standard obtained after multiple adjustments, wherein the temperature and humidity and SOC associated control coefficient represents the degree of mutual influence between the temperature, humidity and battery power, so that the control is more accurate.

[0104] In the embodiment of the application, first, the initial strategy parameter generated in a2 is taken as the starting point, which is applied to the energy storage system control; second, the real-time operation data of each region under the strategy is collected, including the actual temperature and humidity and SOC value, and the difference between these data and the expected effect of the strategy is analyzed; then the parameter is adjusted according to the difference through the reinforcement learning algorithm, the process of repeatedly executing the strategy, collecting the feedback and adjusting the parameter is repeated, and the deviation between the actual and expected is gradually reduced; after multiple adjustments, the target strategy parameter containing the temperature and humidity and SOC associated control coefficient is obtained, for example, in an outdoor energy storage base station, it is found that the temperature and humidity of region C rises too fast after the initial strategy parameter is applied, which leads to the increase of power deviation, and through the multiple adjustments of the reinforcement learning algorithm, the associated control coefficient in the final target strategy parameter can accurately reflect the mutual influence relationship between the two.

[0105] a4, constructing a global balancing strategy containing the temperature and humidity allowable fluctuation range and the SOC allowable deviation interval of all regions based on the target strategy parameter.

[0106] Wherein, the target strategy parameter is the optimized regulation standard, containing the correlation between temperature and humidity and SOC; the temperature and humidity allowable fluctuation range is the reasonable interval of normal fluctuation of temperature and humidity in each region, and the SOC allowable deviation interval is the acceptable difference range between the battery capacity and the average capacity in each region; the global balancing strategy is the overall regulation scheme formed by integrating these ranges and intervals, which is used to guide the balanced operation of the entire energy storage system and to clarify the state in which each region needs to be adjusted.

[0107] In the embodiments of the present application, the correlation between temperature and humidity and SOC is extracted from the target strategy parameter obtained from a3, and the reasonable fluctuation range of temperature and humidity is determined for each region in combination with the system safety operation requirements, such as the temperature not exceeding the limit harmful to the battery and the humidity being within the interval beneficial to the battery operation; at the same time, the allowable deviation interval of SOC of each region and the average SOC of the system is determined to clarify the acceptable range of capacity difference; finally, these ranges and intervals are integrated to form the global balancing strategy for guiding the operation of the entire system, for example, in the outdoor energy storage base station, the temperature and humidity allowable fluctuation range and the SOC allowable deviation interval of regions A, B and C are determined based on the target strategy parameter, and the global balancing strategy is formed after integration to clarify the normal operation boundary of each region.

[0108] a5, comparing the second temperature and humidity values of all regions with the temperature and humidity allowable fluctuation range to screen out the temperature and humidity abnormal regions, and comparing the actual SOC values of all regions with the SOC allowable fluctuation interval to screen out the SOC imbalance regions.

[0109] Wherein, the second temperature and humidity values are the temperature and humidity data collected earlier, the temperature and humidity allowable fluctuation range is the reasonable interval determined in the global balancing strategy, and the regions exceeding the range after comparison are the temperature and humidity abnormal regions; the actual SOC values are the current battery capacity of each region, and the SOC allowable deviation interval is the reasonable capacity difference range, and the regions exceeding the interval after comparison are the SOC imbalance regions; the screening of these two types of regions provides a basis for subsequent determination of the key adjustment object.

[0110] In the embodiments of the present application, first, the second temperature and humidity values of each area are taken out and compared with the corresponding temperature and humidity allowable fluctuation range in the global balancing strategy one by one. If the temperature or humidity of a certain area exceeds the range, it is marked as a temperature and humidity abnormal area. Second, the actual SOC values of each area are taken out, the difference between the SOC value and the average SOC value of the system is calculated, and then compared with the SOC allowable deviation interval. If the difference exceeds the interval, it is marked as an SOC imbalance area. For example, in an outdoor energy storage base station, the second temperature and humidity values of areas A, B and C are compared with the allowable range, and it is found that the humidity of area C exceeds the range, so area C is marked as a temperature and humidity abnormal area. The difference between the SOC of each area and the average value is calculated, and it is found that the difference of area C exceeds the allowable interval, so area C is marked as an SOC imbalance area.

[0111] a6, the overlapping part of the temperature and humidity abnormal area and the SOC imbalance area is determined as the target area.

[0112] Among them, the temperature and humidity abnormal area is an area whose temperature and humidity exceeds the reasonable range, the SOC imbalance area is an area whose power difference exceeds the reasonable range, and the overlapping part of the two is an area that exists both temperature and humidity abnormalities and SOC imbalance. The target area is such an overlapping area, which has more prominent problems and needs to be adjusted first to ensure the overall balance of the system.

[0113] In the embodiments of the present application, first, the temperature and humidity abnormal areas and the SOC imbalance areas screened in a5 are listed, and then the areas appearing in both lists are found. These areas that exist both problems are the target areas that need to be processed first. For example, in an outdoor energy storage base station, the temperature and humidity abnormal area is area C, and the SOC imbalance area is also area C. The two overlap, so area C is determined as the target area. If the temperature and humidity abnormal areas are B and C, and the SOC imbalance areas are C and A, then the overlapping area C is the target area.

[0114] The application provides the following specific examples: in the energy storage system of the outdoor energy storage base station, three regions A, B and C are provided. First, the staff observes that the daytime temperature and humidity have a gradually rising trend, and finds that the power of region A is low, the power of region B is medium, and the power of region C is high, and these information is used as the basis for analysis of reinforcement learning. Considering that the temperature and humidity have a more obvious influence on the battery state, the importance of the temperature and humidity related information is slightly higher than that of the power distribution information. Then, the importance of the two types of information is combined, and the temperature and humidity change trend and the power distribution are comprehensively processed to obtain a preliminary regulation standard for guiding the running state of each region. Then, the preliminary standard is applied to the system, and after a period of operation, it is found that the temperature and humidity rising speed of region C is faster than expected, resulting in a larger difference in power between region C and other regions. Through reinforcement learning, the problem is analyzed and the standard is adjusted, and after multiple optimizations, a regulation standard that accurately reflects the mutual influence of temperature and humidity and power is obtained. Based on the optimized standard, the temperature and humidity range that can normally fluctuate and the reasonable interval of power difference are determined for regions A, B and C, and an overall scheme for guiding the operation of the entire system is formed. Then, the temperature and humidity data collected before in each region is compared with the reasonable range, and it is found that the humidity of region C exceeds the normal range, which is marked as a region with abnormal temperature and humidity. Then, the difference between the power of each region and the average power is calculated, and the difference of region C also exceeds the reasonable interval, which is marked as a region with power imbalance. Finally, by checking the marking results, it is found that the regions with abnormal temperature and humidity and power imbalance are all region C, and region C is determined as the target region that needs to be adjusted first.

[0115] By performing a1-a6, the embodiments of the application generate initial policy parameters by weighted fusion, obtain target policy parameters that fit the actual situation through multiple rounds of iterative optimization, and then construct a global balanced policy, and finally accurately screen out the target region that simultaneously exists temperature and humidity abnormality and SOC imbalance. The synergistic effect of each step highlights the influence of key factors, and through repeated optimization, the regulation standard is more in line with the actual situation of the system, ensuring the accuracy of the target region positioning, providing a clear object for subsequent adjustment, and effectively improving the balance and stability of the energy storage system operation.

[0116] In a possible embodiment, S13, based on the first temperature and humidity values and the SOC deviation values of all regions, calculates the optimal balanced current parameters corresponding to each region using an adaptive control algorithm. The optimal balanced current parameters include the size and direction of the balanced current, including:

[0117] Step 131, associating the first temperature and humidity values and the SOC deviation values of all regions to obtain the dynamic influence relationship of the temperature and humidity values on the SOC deviation.

[0118] The first temperature and humidity value is the temperature and humidity data of each region after the operation parameter adjustment, the SOC deviation value is the difference data of the battery capacity of different regions, the correlation processing is to analyze the mutual influence between the two types of data, and the dynamic influence relationship refers to the rule of how the temperature and humidity change causes the SOC deviation change, and is used to reflect the effect of the environment on the battery state difference. Through this analysis, the relationship between the environmental factors and the battery state difference can be determined.

[0119] In the embodiment of the present application, first, the first temperature and humidity value and the corresponding SOC deviation value of all regions are collected, and then the relationship between the temperature and humidity change and the SOC deviation change is found out through data analysis method, such as whether the temperature rise will cause the SOC deviation to increase, and the influence of humidity change on the SOC deviation. Finally, the influence rule that changes with the temperature and humidity change, that is, the dynamic influence relationship, is summarized. For example, in an outdoor energy storage base station, it is collected that when the temperature of region A rises, the SOC deviation of region A and region B also increases, and when the humidity of region C rises, the SOC deviation change of region C and region A is obvious. Through analysis, it is concluded that the dynamic influence relationship that the temperature and humidity rise will exacerbate the SOC deviation.

[0120] In the embodiment of the present application, first, the first temperature and humidity value and the corresponding SOC deviation value of all regions are collected, and then the relationship between the temperature and humidity change and the SOC deviation change is found out through data analysis method, such as whether the temperature rise will cause the SOC deviation to increase, and the influence of humidity change on the SOC deviation. Finally, the influence rule that changes with the temperature and humidity change, that is, the dynamic influence relationship, is summarized. For example, in an outdoor energy storage base station, it is collected that when the temperature of region A rises, the SOC deviation of region A and region B also increases, and when the humidity of region C rises, the SOC deviation change of region C and region A is obvious. Through analysis, it is concluded that the dynamic influence relationship that the temperature and humidity rise will exacerbate the SOC deviation.

[0121] The adjustment coefficient is determined according to the dynamic influence relationship, and is used to measure the influence degree of temperature and humidity on the SOC deviation of different regions. Different regions may have different adjustment coefficients due to different environmental characteristics. The quantitative conversion is to convert the SOC deviation value into a specific deviation amount that needs to be compensated by combining the adjustment coefficient. The deviation amount reflects the degree of actual adjustment considering the environmental influence, and is the basis for determining the current parameter.

[0122] In the embodiment of the present application, first, the first temperature and humidity value and the corresponding SOC deviation value of all regions are collected, and then the relationship between the temperature and humidity change and the SOC deviation change is found out through data analysis method, such as whether the temperature rise will cause the SOC deviation to increase, and the influence of humidity change on the SOC deviation. Finally, the influence rule that changes with the temperature and humidity change, that is, the dynamic influence relationship, is summarized. For example, in an outdoor energy storage base station, it is collected that when the temperature of region A rises, the SOC deviation of region A and region B also increases, and when the humidity of region C rises, the SOC deviation change of region C and region A is obvious. Through analysis, it is concluded that the dynamic influence relationship that the temperature and humidity rise will exacerbate the SOC deviation.

[0123] Step 133, determining the size of the equalization current according to the numerical value of the deviation amount, determining the direction of the equalization current according to the positive and negative attributes of the deviation amount, and generating the preliminary current parameter based on the size and direction.

[0124] Wherein, the numerical value of the deviation amount reflects the degree of SOC deviation that needs to be made up, the larger the value, the more energy needs to be transferred; the positive and negative attributes of the deviation amount represent the direction of the SOC deviation, the positive attribute may represent that the power of a certain area is too high and needs to output energy, and the negative attribute may represent that the power is too low and needs to input energy; the size of the balancing current is determined by the numerical value of the deviation amount, and the direction is determined by the positive and negative of the deviation amount, and the preliminary current parameter is a preliminary current standard formed by the size and direction for energy transfer.

[0125] In the embodiments of the present application, first, the numerical value of the deviation amount that each area needs to make up is checked, the larger the value, the larger the balancing current size determined to achieve more energy transfer; then according to the positive and negative attributes of the deviation amount, if it is positive, the current direction is determined to flow from the area to other areas, and if it is negative, the current direction is determined to flow from other areas to the area; finally, the preliminary current parameter is generated by combining the current size and direction determined, for example, in the outdoor energy storage base station, the deviation amount that area C needs to make up is positive and the value is large, the balancing current size is determined to be 3A and the direction is to flow out of area C; the deviation amount that area A needs to make up is negative and the value is small, the balancing current size is determined to be 1A and the direction is to flow into area A, thereby generating the preliminary current parameter.

[0126] Step 134, dynamically correct the preliminary current parameter through an adaptive control algorithm to obtain the optimal balancing current parameter corresponding to each area.

[0127] Wherein, the adaptive control algorithm is a method that can automatically adjust parameters according to the real-time state of the system, and can flexibly correct the preliminary current parameter according to the actual operation; dynamic correction means that the algorithm continuously adjusts the size and direction of the current according to the real-time feedback of the system, so that the parameter is more suitable for system changes; the optimal balancing current parameter is the current parameter that is most suitable for the current system state after correction, which can achieve more accurate energy transfer.

[0128] In the embodiments of the present application, first, the preliminary current parameter generated in step 133 is applied to the system, and the real-time operation data of the system is collected at the same time, including the real-time situation of the temperature and humidity changes and the SOC deviation of each area; then the adaptive control algorithm is used to analyze the difference between these real-time data and the expected effect, if it is found that the current size causes the energy transfer to be too fast or too slow, or the direction does not meet the actual demand, the preliminary current parameter is adjusted; repeat this process until the optimal balancing current parameter that can make the SOC deviation stable and small is obtained, for example, in the outdoor energy storage base station, it is found that the current of area C makes the SOC deviation too small after the preliminary current parameter is applied, and the current size is corrected from 3A to 2.5A through the adaptive control algorithm, thereby obtaining a more suitable optimal balancing current parameter.

[0129] The application provides the following specific examples: in the energy storage system of the outdoor energy storage base station, there are regions A, B and C. The adjusted temperature and humidity and the power difference of each region are collected, and it is analyzed that the temperature and humidity change will affect the power difference, for example, the temperature rise may make the difference larger. Accordingly, different regions are provided with adjustment coefficients, and the coefficient of the region affected by the environment is slightly larger. Then, the power difference is converted into a deviation amount to be compensated by combining the coefficient. The current size is determined according to the deviation amount, and the direction is determined according to the positive and negative, to form the preliminary current parameters. After application, it is found that the power difference change of some regions is not as expected, and the optimal current parameters are obtained through adaptive adjustment, so that the power difference is reasonably reduced.

[0130] By performing steps 131-134, the embodiment of the application determines the influence law of temperature and humidity on SOC deviation through correlation processing, provides a reference of environmental factors for subsequent adjustment, configures adjustment coefficients and quantitatively converts deviation amount, so that the adjustment demand is more suitable for the environmental characteristics of each region, determines the preliminary current parameters based on the deviation amount, and provides a basic standard for energy transfer. Through dynamic correction by the adaptive algorithm, the optimal current parameters can accurately adapt to the real-time state of the system, ensure reasonable and efficient energy transfer, effectively reduce the SOC deviation of each region, and improve the balance and stability of the energy storage system.

[0131] In one possible embodiment, step 132, according to the dynamic influence relationship, configures corresponding adjustment coefficients for all regions, quantitatively converts the SOC deviation values of all regions based on the adjustment coefficients, and obtains the deviation amount that needs to be compensated by each region, including:

[0132] b1, the dynamic influence relationship divides the temperature and humidity value range into a plurality of continuous intervals, and configures corresponding adjustment coefficients for each interval.

[0133] Wherein, the dynamic influence relationship refers to the influence law of temperature and humidity change on power difference, the temperature and humidity value range is the sum of possible values of temperature and humidity, the continuous interval is a plurality of continuous parts divided from the sum, and the adjustment coefficient is a value corresponding to each interval for measuring the influence degree of temperature and humidity in the interval on power difference. Through such division and configuration, the influence degree in different temperature and humidity ranges can be accurately quantified, and finally the influence degree standard divided by interval is formed.

[0134] In the embodiments of the present application, first, the critical points at which the temperature and humidity influence changes significantly are found according to the dynamic influence relationship; second, the temperature and humidity value range is divided into multiple continuous intervals according to these critical points; and finally, the adjustment coefficient corresponding to each interval is configured according to the influence degree of temperature and humidity in each interval on the power difference (the greater the influence, the greater the coefficient), for example, in an outdoor energy storage base station, it is found that the influence on the power difference is small when the temperature is low, medium when the temperature is medium, and large when the temperature is high, so the temperature is divided into three continuous intervals, and small to large adjustment coefficients are configured respectively, and the humidity is also divided into intervals and configured in a similar manner.

[0135] b2, determine the interval to which the first temperature and humidity value of each region belongs, and match the corresponding adjustment coefficient.

[0136] Among them, the first temperature and humidity value is the adjusted actual temperature and humidity data of each region, the interval to which it belongs is the specific part of the continuous interval in which the temperature and humidity value is located in b1, and the corresponding adjustment coefficient is the adjustment coefficient configured by b1 in the interval. Through such matching, there is a corresponding quantitative standard for the actual temperature and humidity influence of each region, and finally the influence coefficient that fits the environment of each region is determined.

[0137] In the embodiments of the present application, first, the first temperature and humidity value of each region is obtained; second, the temperature value of each region is determined to belong to which temperature interval and the humidity value to belong to which humidity interval by comparing the continuous temperature and humidity intervals divided in b1; and finally, the adjustment coefficient of the corresponding interval is taken out from the coefficient configured by b1 as the temperature and humidity adjustment coefficient of the region, for example, in an outdoor energy storage base station, the temperature of region A belongs to the medium interval, and the temperature adjustment coefficient of the interval is matched, the humidity belongs to the lower interval, and the humidity adjustment coefficient of the interval is matched, and the two coefficients are used as the adjustment coefficient of region A.

[0138] b3, correct the adjustment coefficient in combination with the number of battery modules in each region and the SOC consistency between the modules to obtain the module correlation coefficient of each region.

[0139] Among them, the number of battery modules is the total number of battery modules contained in each region, the SOC consistency between the modules is the degree of similarity of the power between different battery modules in the same region (poor consistency means large power difference between the modules), the adjustment coefficient is the coefficient matched in b2, and the module correlation coefficient is the coefficient obtained by correcting the number of modules and the consistency, which can make the coefficient reflect the regional environmental influence and internal state characteristics at the same time, and finally form an adjustment coefficient that comprehensively considers.

[0140] In the embodiments of the present application, firstly, the number of battery modules in each region is counted, and the SOC consistency between modules in each region is evaluated; secondly, the adjustment coefficient obtained by b2 is corrected according to the counting and evaluation results; and finally, the corrected module correlation coefficient is obtained, for example, in an outdoor energy storage base station, the number of modules in region B is large and the consistency is poor, the adjustment coefficient is increased, the number of modules in region A is small and the consistency is good, the adjustment coefficient is slightly adjusted, and the module correlation coefficients of the two regions are obtained.

[0141] b4, multiplying the module correlation coefficient and the SOC deviation value of the corresponding region to obtain the preliminary deviation amount of each region.

[0142] Wherein, is the power difference between different regions, and the preliminary deviation amount is the result of multiplication, which is used to preliminarily measure the degree of power difference that needs to be made up. This result integrates the effects of environmental influence and internal state on power difference, and provides a preliminary quantitative demand for subsequent adjustment.

[0143] In the embodiments of the present application, firstly, the module correlation coefficients of each region obtained by b3 and the corresponding SOC deviation values of each region are obtained; secondly, the module correlation coefficient of each region is multiplied by the corresponding SOC deviation value; and finally, the preliminary deviation amount of each region is obtained, for example, in an outdoor energy storage base station, through such calculation, the preliminary deviation amount reflects the effects of environmental influence and internal state on power difference.

[0144] b5, combining the current running power of each region, dynamically scaling the preliminary deviation amount to obtain the deviation amount that needs to be made up.

[0145] Wherein, the current running power is the current power output or input state of each region, the preliminary deviation amount is the preliminary result obtained in b4, the dynamic scaling is to adjust the size of the preliminary deviation amount according to the running power, and the deviation amount that needs to be made up is the final result obtained after scaling, which can fit the actual running state of the region, and finally form an adjustment amount that meets the actual working demand.

[0146] In the embodiments of the present application, firstly, the current running power of each region is obtained; secondly, the scaling ratio is determined according to the running power; then the preliminary deviation amount obtained by b4 is dynamically scaled by the scaling ratio; and finally, the deviation amount that needs to be made up is obtained, for example, in an outdoor energy storage base station, through such scaling, the adjustment amount fits the actual working intensity of the region, and interference to the high-power running region is avoided.

[0147] The application provides the following specific examples: in the energy storage system of an outdoor energy storage base station, there are three regions A, B and C. First, according to the influence law of temperature and humidity on power difference, the range in which temperature and humidity may appear is divided into several continuous parts, each part is configured with a corresponding influence coefficient, and the coefficient of the part with greater influence is larger. Then, see which part the adjusted actual temperature and humidity of each region belong to, and find the corresponding influence coefficient. Then, considering the number of battery modules in each region and the similarity of power between the modules, the previous influence coefficient is corrected, the coefficient of the region with more modules or greater power difference is appropriately increased, and the adjustment coefficient that comprehensively considers the environment and internal state is obtained. Then, multiply the corrected coefficient by the power difference between the regions to obtain the preliminary power difference that needs to be compensated. Finally, according to the current working intensity of each region, adjust the preliminary deviation, and the working intensity of the region is smaller. The adjustment range of the low region is larger, and the final deviation that needs to be compensated is obtained.

[0148] By performing b1~b5, the embodiments of the application divide the temperature and humidity interval and configure the corresponding coefficient, so that the influence of different ranges of environment is accurately quantified; the coefficient corresponding to the actual temperature and humidity of each region is matched to ensure the pertinence of the environmental influence; the coefficient is corrected in combination with the number of battery modules and consistency to comprehensively consider the influence of the internal state of the region; the preliminary deviation is obtained by multiplication to form the preliminary quantification of the adjustment requirement; finally, the adjustment amount is scaled in combination with the running power to make the adjustment amount fit the actual working intensity, so that the deviation that needs to be compensated considers the environment, internal state and running intensity at the same time, so that the subsequent adjustment is more accurate and more in line with the actual situation of the system, effectively improving the rationality and effectiveness of the adjustment.

[0149] In one possible embodiment, S14, the optimal balancing current parameter is transmitted to the balancing circuit of the corresponding region, and energy transfer is performed between the battery modules of the target region and other regions through the balancing circuit to reduce the SOC deviation between regions, including:

[0150] Step 141, the optimal balancing current parameter is sent to the balancing circuit of the corresponding region through a signal transmission channel.

[0151] Wherein, the optimal balancing current parameter is the current setting most suitable for energy transfer after correction, including current size and direction, used to guide how energy is transferred between regions; the signal transmission channel is the path for transmitting these parameters, such as wire or wireless channel, responsible for transmitting parameters from the control system to the execution device; the balancing circuit is a device that receives parameters and actually performs energy transfer, which clearly specifies the specific operation mode by receiving parameters, and finally makes the energy transfer have a clear execution basis.

[0152] In the embodiments of the present application, first, the critical points at which the temperature and humidity influence obviously change are found out according to the dynamic influence relationship; second, the temperature and humidity value range is divided into multiple continuous intervals according to these critical points; and finally, the adjustment coefficient corresponding to each interval is configured according to the influence degree of temperature and humidity on the power difference in each interval (the greater the influence, the greater the coefficient), for example, in an outdoor energy storage base station, it is found that the influence on the power difference is small when the temperature is low, medium when the temperature is medium, and large when the temperature is high, so the temperature is divided into these three continuous intervals, and small to large adjustment coefficients are configured respectively, and the humidity is also divided into intervals and configured with coefficients in a similar manner.

[0153] Step 142, the optimal balancing current parameter is analyzed by the balancing circuit to determine the starting area and the receiving area of energy transfer.

[0154] Among them, the analysis is the process of the balancing circuit interpreting the received optimal balancing current parameter, extracting key information from the parameter; the starting area is the area where energy flows out, and the receiving area is the area where energy flows in, and the two areas are determined by the direction information in the analysis parameter, which ensures the direction of energy transfer is accurate and avoids the problem of reverse transmission.

[0155] In the embodiments of the present application, first, the first temperature and humidity values of each area are obtained; second, the temperature value of each area is determined to belong to which temperature interval and the humidity value to belong to which humidity interval by comparing the temperature and humidity continuous intervals divided in b1; and finally, the adjustment coefficient of the corresponding interval is taken out from the coefficient configured in b1 as the temperature and humidity adjustment coefficient of the area, for example, in an outdoor energy storage base station, the temperature of area A belongs to the medium interval, and the temperature adjustment coefficient of the interval is matched, the humidity belongs to the lower interval, and the humidity adjustment coefficient of the interval is matched, and the two coefficients are used as the adjustment coefficient of area A.

[0156] Step 143, according to the position distribution of the starting area and the receiving area, a corresponding energy transmission path is selected.

[0157] Among them, the position distribution refers to the relative position relationship of the starting area and the receiving area in the energy storage system, such as whether adjacent or separated by other areas; the energy transmission path is the line connecting the two areas for energy transmission, different position distributions correspond to different paths, selecting the appropriate path can reduce the loss of energy in the transmission process, improve the transmission efficiency, and ensure that more energy can be effectively transferred.

[0158] In the embodiments of the present application, firstly, the position distribution of the starting area and the receiving area in the system is determined, and it is understood whether they are directly adjacent or need to pass through other areas, and the relative position relationship between the areas is mastered; secondly, the most suitable energy transmission path is selected from the multiple paths preset in the system according to the position distribution, and the adjacent areas usually select the direct connection path, and the spaced areas select the indirect path with smaller loss to avoid detouring and increasing energy loss; for example, in the outdoor energy storage base station, the starting area A and the receiving area B are adjacent, and the direct line between them is selected as the transmission path; the starting area C and the receiving area A are not adjacent, and there is an area B in between, and a special cross-area line from C to A is selected to reduce the energy loss passing through the intermediate area.

[0159] In step 144, the energy transmission intensity of the energy transmission path is adjusted based on the current size in the optimal balancing current parameter, so that the energy is transmitted between the battery modules of the starting area and the receiving area according to the set intensity.

[0160] Among them, the current size is the information representing the amount of energy transmission in the optimal balancing current parameter, which directly determines how much energy is transmitted per unit time; the energy transmission intensity is the amount of energy passing through the path per unit time, which is related to the current size; adjusting the transmission intensity is to change the energy flow in the path according to the current size, to ensure that the transmitted energy meets the actual demand, neither too much nor too little.

[0161] In the embodiments of the present application, firstly, the information of the current size is extracted from the optimal balancing current parameter to clearly understand the total amount of energy that needs to be transmitted; secondly, the control components in the energy transmission path are adjusted according to the current size, such as adjusting the resistance or power controller to change the energy transmission intensity in the path, and the intensity is adjusted to a corresponding level when the current is large or small; finally, the energy is stably transmitted between the starting area and the receiving area according to the set intensity, for example, in the outdoor energy storage base station, the balancing circuit connected with A and B analyzes that the current size is a certain level, and the transmission intensity of the path between A and B is adjusted to the corresponding level, so that the energy is transmitted from A to B according to the intensity, and the transmission amount meets the demand.

[0162] In step 145, during the energy transmission process, the SOC deviation value of each area is continuously monitored, and when the SOC deviation value is reduced to the preset range, the balancing circuit is controlled to stop energy transmission.

[0163] Among them, the current running power is the current power output or input state of each area (reflecting the working intensity), the preliminary deviation amount is the preliminary result obtained in b4, the dynamic scaling is to adjust the size of the preliminary deviation amount according to the running power, the deviation amount that needs to be made up is the final result obtained after scaling, which can fit the actual running state of the area, and finally form an adjustment amount that meets the actual working demand.

[0164] In the embodiments of the present application, first, after the energy transmission starts, the monitoring device continuously detects the SOC deviation value of the starting area and the receiving area, and tracks the change of the power difference in real time; second, the deviation value detected each time is compared with the preset allowed range of the system to determine whether the energy transfer achieves the expected effect; when the deviation value is reduced to the preset range, a stop instruction is immediately sent to the corresponding balancing circuit, and the control circuit terminates the energy transmission. For example, in an outdoor energy storage base station, when energy is transmitted from A to B, the power difference between the two is continuously monitored, and when the difference is reduced to the allowed range of the system, an instruction is sent to make the balancing circuit connected to A and B stop working, thereby avoiding excessive transmission.

[0165] The present application provides the following specific examples: in the energy storage system of an outdoor energy storage base station, there are three areas A, B and C. First, the current suitable for energy transfer is set to be sent to the energy transfer circuit of the corresponding area through the transmission path. These circuits interpret the received settings to determine where the energy flows out and where it flows in, such as determining that the energy flows from A to B and from C to A. Then, according to the positional relationship of these areas, the transmission line is selected, A and B are adjacent, so the line is directly connected, C and A are not adjacent, so the special cross-area line is used. Then, the energy transmission intensity on the line is adjusted according to the size of the current setting, so that the energy is transferred between the areas in the appropriate amount. During the transmission process, the power difference between the areas is continuously monitored, and when the difference is reduced to the allowed range of the system, the control circuit stops the energy transmission, so that the power state of each area is balanced.

[0166] By performing steps 141-145, the embodiments of the present application accurately send the optimal parameters to the corresponding circuit through step 141, providing correct instructions for energy transmission; step 142 determines the direction of energy transmission, ensuring that the energy is transferred in the expected direction; step 143 selects the appropriate path to reduce energy loss and improve transmission efficiency; step 144 adjusts the transmission intensity to make the energy transmission amount meet the demand, avoiding too much or too little; step 145 stops the transmission in time by continuous monitoring to prevent excessive transfer leading to new imbalance. The overall process ensures that the energy is efficiently, accurately and moderately transferred between areas, and finally the power state of each area is balanced and stable.

[0167] In one possible embodiment, step 143 selects the corresponding energy transmission path according to the positional distribution of the starting area and the receiving area, including:

[0168] c1, obtaining the positional information of the starting area and the receiving area, and determining the relative positional relationship based on the positional information.

[0169] The position information is specific position data of the starting area and the receiving area in the energy storage system, including their distribution coordinates or layout positions; the relative position relationship is a mutual orientation or distance relationship determined according to the position information, such as adjacent, separated by other areas or distance, and the like, and the spatial relationship of the two areas in the system can be determined by determining the relationship, thereby providing a basis for subsequent transmission path searching.

[0170] In the embodiments of the present application, first, the position information of the starting area and the receiving area is obtained by the position recording device in the system, and the specific distribution of the two areas in the system is determined; second, the position information is analyzed to determine whether the two areas are directly adjacent, separated by other areas or far away, thereby determining the relative position relationship. For example, in an outdoor energy storage base station, after obtaining the position information of the starting area A and the receiving area B, it is found that the two areas are directly adjacent, and the position information of the starting area C and the receiving area A shows that they are separated by the area B, thereby determining the corresponding relative position relationship.

[0171] c2, based on the relative position relationship, list all potential transmission paths that can connect the starting area and the receiving area.

[0172] The relative position relationship is the orientation or distance relationship of the starting area and the receiving area; the potential transmission path is all transmission lines that can connect the two areas according to the relationship, including direct connection lines and lines passing through other areas, and listing these paths can comprehensively cover all possible connection modes to provide sufficient alternative schemes for subsequent screening.

[0173] In the embodiments of the present application, first, the possible routes from the starting area to the receiving area are analyzed according to the relative position relationship determined in c1; second, all lines that can connect the two areas are listed according to the line layout of the system, if the two areas are adjacent, the direct connection line and the line that may pass through the surrounding area are listed; if the two areas are separated by other areas, all possible lines passing through the intermediate area are listed. For example, in an outdoor energy storage base station, the starting area A is adjacent to the receiving area B, and based on this relationship, all possible potential transmission paths such as the line from A directly to B and the line from A to B through C are listed.

[0174] c3, check the current use state of each potential transmission path, and screen out the standby transmission path in the available state.

[0175] The current use state is whether each potential transmission path is occupied by other tasks, which is divided into being used and not being used; the available state refers to the state of not being occupied and being immediately available for current transmission; the standby transmission path is the available path screened from the potential path, which can ensure that the current energy transmission proceeds smoothly and avoids path conflicts.

[0176] In the embodiments of the present application, firstly, the current use state of each potential transmission path is checked by the path monitoring device in the system to know which paths are being used and which are idle; secondly, the paths being used are excluded, and the paths in the unused state are screened out as backup transmission paths. For example, in an outdoor energy storage base station, after checking the potential paths listed from A to B, it is found that the line from A directly to B is being used by other transmission, while the line from A to B through C is idle, so the latter is screened as a backup transmission path.

[0177] c4. Priorities are configured for all backup transmission paths, and the first backup transmission path is selected as the energy transmission path according to the order from high to low of the priorities.

[0178] Among them, the priority is the priority order set for the backup transmission path, which is usually determined according to the transmission efficiency, length or stability of the path, and the path with high efficiency, short distance and high stability has higher priority; the first backup transmission path is the first path in the priority order, and selecting it as the energy transmission path can ensure more efficient and stable transmission.

[0179] In the embodiments of the present application, firstly, the priorities are configured for each backup path according to the transmission efficiency, length or stability of the backup transmission path, and the path with high efficiency, short distance and high stability is set to the highest priority; secondly, all backup paths are sorted in order from high to low according to the priorities; finally, the first path in the order is selected as the energy transmission path. For example, in an outdoor energy storage base station, the priorities are configured for the backup paths from A to B, the line from A to B through C is set to high priority because of its short distance and small loss, and the line from A through the edge line is set to low priority, so the former is selected as the transmission path.

[0180] The application provides the following specific examples: in the energy storage system of an outdoor energy storage base station, there are three regions A, B and C. First, the position conditions of the starting regions A and C and the receiving regions B and A are obtained, and after analysis, it is clear that A and B are directly adjacent, and C and A are separated by B. Next, according to such a position relationship, all possible lines that can connect them are listed: the lines from A to B include A directly to B, A through C to B, and A through the edge line to B; the lines from C to A include C directly to A, C through B to A, and C through the auxiliary line to A. Then, the use conditions of these lines are checked, and it is found that the lines from A directly to B and the lines from C through B to A are being used by other transmissions, and the remaining lines from A through C to B, A through the edge line to B, C directly to A, and C through the auxiliary line to A are all in idle state, and these idle lines are used as backup lines. Finally, according to the transmission efficiency, distance length and stability of the lines, the backup lines are set in priority order, and the lines from A through C to B are arranged in front because of the short distance and small loss, and the lines from C directly to A are arranged in front because of the high efficiency, so the two lines are selected as the energy transmission lines from A to B and from C to A.

[0181] By performing c1-c4, the embodiment of the application provides a spatial basis for path selection by obtaining position information and determining the relative position relationship; ensures the comprehensiveness of the alternative by listing all potential paths; avoids transmission conflicts by screening available backup paths; and improves transmission efficiency and stability by configuring priority and selecting the optimal path. The overall process enables the selection of energy transmission paths to be more scientific and reasonable, ensuring that energy can be transmitted through the most suitable path, reducing loss and ensuring smooth transmission.

[0182] Figure 2 A structural diagram of a SOC dynamic balancing control device for an energy storage system based on reinforcement learning provided by the embodiment of the application is shown in Figure 2 The system comprises:

[0183] The adjusting module 21 is configured to dynamically adjust the operating parameters of the battery modules in the target region of the energy storage system based on reinforcement learning.

[0184] The collecting module 22 is configured to collect the first temperature and humidity values and SOC deviation values of the target region and other regions after the adjustment of the operating parameters is completed.

[0185] The computing module 23 is configured to calculate the optimal balancing current parameters corresponding to each region based on the first temperature and humidity values and SOC deviation values of all regions by using an adaptive control algorithm, and the optimal balancing current parameters include the size and direction of the balancing current.

[0186] The optimization module 24 is configured to transmit the optimal equalization current parameter to the equalization circuit of the corresponding area, and the equalization circuit is configured to transfer energy between the battery modules in the target area and other areas to reduce the SOC deviation between areas.

[0187] Figure 2 The SOC dynamic equalization control device based on reinforcement learning can perform Figure 1 The SOC dynamic equalization control method based on reinforcement learning has the same implementation principles and technical effects as the above-mentioned embodiment, and will not be described in detail here. The specific operation of each module and unit in the SOC dynamic equalization control device based on reinforcement learning has been described in detail in the embodiment related to the method, and will not be described in detail here.

[0188] In one possible design, Figure 2 The SOC dynamic equalization control device based on reinforcement learning can be implemented as a computing device, such as a server. Figure 3 As shown in the figure, the computing device can include a storage component 31 and a processing component 32.

[0189] The storage component 31 stores one or more computer instructions, wherein the one or more computer instructions are called and executed by the processing component 32.

[0190] The processing component 32 is configured to control the flight height and heading angle of the detection unmanned aerial vehicle, so that the laser radar and the thermal infrared sensor deployed on the detection unmanned aerial vehicle respectively collect three-dimensional point cloud data of a vertical section of a mountain fire area and thermal radiation distribution data of the mountain fire area at multiple angles; combine the three-dimensional point cloud data and the thermal radiation distribution data to generate a target fire spread path; use an embedded real-time processing chip deployed in the detection unmanned aerial vehicle to perform real-time processing on the thermal radiation distribution data, identify temperature abnormal areas and remove interference areas, and generate fire point position and classification results; based on the target fire spread path, the fire point position and the classification results, send a dynamic scheduling instruction to a cluster of fire extinguishing unmanned aerial vehicles through a low-latency communication link to adjust the position, fire extinguishing agent amount and coverage range of each fire extinguishing unmanned aerial vehicle in the cluster of fire extinguishing unmanned aerial vehicles.

[0191] The processing component 32 can include one or more processors to execute computer instructions to complete all or part of the steps in the above method. Of course, the processing component can also be one or more Application-Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Process Device (DSPD), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components, for executing the above method.

[0192] The storage component 31 is configured to store various types of data to support the operation of the terminal. The storage component can be implemented by any type of volatile or non-volatile storage devices or a combination thereof, such as Random Access Memory (RAM), Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0193] Of course, the computing device can also include other components, such as an input / output interface, a display component, a communication component, etc.

[0194] The input / output interface provides an interface between the processing component and peripheral interface modules, which can be output devices, input devices, etc.

[0195] The communication component is configured to facilitate wired or wireless communication between the computing device and other devices, etc.

[0196] The computing device can be a physical device or an elastic computing host provided by a cloud computing platform, and the computing device can be a cloud server, and the processing component, the storage component, etc. can be basic server resources rented or purchased from the cloud computing platform.

[0197] The embodiment of the application further provides a computer storage medium, which stores a computer program, and the computer program can realize the above-mentioned method when being executed by a computer. Figure 1 The embodiment of the application further provides a computer storage medium, which stores a computer program, and the computer program can realize the above-mentioned method when being executed by a computer.

[0198] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-mentioned system, device and unit can refer to the corresponding processes in the foregoing method embodiments, and will not be described here.

[0199] The device embodiments described above are only schematic, and the units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, that is, they can be located in one place, or can be distributed on multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the embodiment scheme. Those skilled in the art can understand and implement without creative labor.

[0200] Through the foregoing description of the embodiments, those skilled in the art can clearly understand that each embodiment can be realized by means of software and the necessary general hardware platform, and of course, it can also be realized by hardware. Based on such understanding, the above technical solutions can be embodied in the form of a software product, which can be stored in a computer readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the method described in each embodiment or some parts of the embodiment.

[0201] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the application, and not to limit them; although the application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for some technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the application.

Claims

1. A dynamic equilibrium control method for the State of Charge (SOC) of an energy storage system based on reinforcement learning, characterized in that, include: The operating parameters of the battery modules in the target area of ​​the energy storage system are dynamically adjusted based on reinforcement learning. After the operating parameters are adjusted, the first temperature and humidity values ​​and SOC deviation values ​​of the target area and other areas are collected. Based on the first temperature and humidity values ​​and the SOC deviation values ​​of all regions, the optimal equalization current parameters corresponding to each region are calculated using an adaptive control algorithm. The optimal equalization current parameters include the magnitude and direction of the equalization current. The optimal equalization current parameters are transmitted to the equalization circuit of the corresponding region. The equalization circuit then transfers energy between the target region and the battery modules in other regions to reduce the SOC deviation between the regions. The method of dynamically adjusting the operating parameters of battery modules in the target area of ​​the energy storage system based on reinforcement learning includes: The system collects second temperature and humidity values ​​from multiple areas in the energy storage system and the SOC value of the battery modules in each area. The second temperature and humidity values ​​are collected earlier than the first temperature and humidity values. The SOC values ​​of battery modules in all regions are aggregated to form the SOC distribution of the energy storage system. Based on the second temperature and humidity value, the daytime temperature and humidity trend is obtained by combining historical meteorological data. Based on the daytime temperature and humidity trends and the SOC distribution of the energy storage system, a global equilibrium strategy is generated using reinforcement learning. Based on the global equilibrium strategy, target areas where the temperature and humidity values ​​exceed the set temperature and humidity thresholds and the SOC deviation exceeds the preset deviation threshold are identified. The operating parameters of the battery modules in the target area are dynamically adjusted by the controller in the corresponding area. The operating parameters include the charge / discharge rate and the operating parameters of the temperature and humidity control unit. Based on the daytime temperature and humidity trends and the SOC distribution of the energy storage system, a global balancing strategy is generated using reinforcement learning. Based on the global balancing strategy, target areas where the temperature and humidity values ​​exceed set temperature and humidity thresholds and the SOC deviation exceeds a preset deviation threshold are identified, including: The daytime temperature and humidity trend and the SOC distribution of the energy storage system are used as input features for reinforcement learning, and temperature and humidity feature weights and SOC feature weights are configured respectively. Based on the temperature and humidity characteristics and the SOC characteristic weights, the daytime temperature and humidity trends and the SOC distribution of the energy storage system are weighted and fused to generate initial strategy parameters. Based on the initial strategy parameters, and combined with real-time operational feedback data from all regions, the target strategy parameters, which include temperature, humidity and SOC correlation control coefficients, are obtained through multiple rounds of iterative adjustments using reinforcement learning algorithms. Based on the target strategy parameters, a global equilibrium strategy is constructed that includes the allowable fluctuation range of temperature and humidity and the allowable deviation range of SOC for all regions. The second temperature and humidity values ​​of all regions are compared with the allowable temperature and humidity range to screen out regions with abnormal temperature and humidity. The actual SOC values ​​of all regions are compared with the allowable SOC fluctuation range to screen out regions with unbalanced SOC. The overlapping area between the abnormal temperature and humidity area and the SOC imbalance area is defined as the target area.

2. The SOC dynamic equilibrium control method for energy storage systems based on reinforcement learning according to claim 1, characterized in that, Based on the first temperature and humidity values ​​and the SOC deviation values ​​of all regions, an adaptive control algorithm is used to calculate the optimal equalization current parameters for each region. The optimal equalization current parameters include the magnitude and direction of the equalization current, including: The first temperature and humidity values ​​and the SOC deviation values ​​of all regions are correlated to obtain the dynamic influence relationship between temperature and humidity values ​​and SOC deviation. Based on the dynamic influence relationship, corresponding adjustment coefficients are configured for all regions. Based on the adjustment coefficients, the SOC deviation values ​​of all regions are quantified and converted to obtain the amount of deviation that needs to be compensated for in each region. The magnitude of the equalization current is determined based on the value of the deviation, and the direction of the equalization current is determined based on the positive or negative attribute of the deviation. Preliminary current parameters are generated based on the magnitude and the direction. The initial current parameters are dynamically corrected using an adaptive control algorithm to obtain the optimal balanced current parameters for each region.

3. The SOC dynamic equilibrium control method for energy storage systems based on reinforcement learning according to claim 2, characterized in that, The step involves configuring corresponding adjustment coefficients for all regions based on the dynamic influence relationship, and then quantifying and converting the SOC deviation values ​​of all regions based on these adjustment coefficients to obtain the amount of deviation that needs to be compensated for in each region, including: The dynamic influence relationship is described by dividing the temperature and humidity value range into multiple continuous intervals and configuring a corresponding adjustment coefficient for each interval. Determine the relevant interval based on the first temperature and humidity values ​​of each region, and match the corresponding adjustment coefficient; The adjustment coefficient is corrected by considering the number of battery modules in each region and the SOC consistency between modules to obtain the module correlation coefficient for each region. Multiply the module correlation coefficient by the SOC deviation value of the corresponding region to obtain the preliminary deviation amount for each region; By combining the current operating power of each region, the initial deviation is dynamically scaled to obtain the deviation that needs to be compensated for in each region.

4. The SOC dynamic equilibrium control method for energy storage systems based on reinforcement learning according to claim 1, characterized in that, The step of transmitting the optimal equalization current parameters to the equalization circuit in the corresponding region, and using the equalization circuit to transfer energy between the target region and battery modules in other regions to reduce the SOC deviation between regions, includes: The optimal equalization current parameters are sent to the equalization circuit in the corresponding area through the signal transmission channel; The optimal equalization current parameters are analyzed using the equalization circuit to determine the starting and receiving regions of energy transfer. Based on the location distribution of the starting area and the receiving area, a corresponding energy transmission path is selected; Based on the current magnitude in the optimal balanced current parameters, the energy transmission intensity of the energy transmission path is adjusted so that energy is transmitted between the battery modules in the starting region and the receiving region according to the set intensity. During the energy transmission process, the SOC deviation value of each region is continuously monitored. When the SOC deviation value is reduced to a preset range, the equalization circuit is controlled to stop the energy transmission.

5. The SOC dynamic equalization control method for energy storage systems based on reinforcement learning according to claim 4, characterized in that, The step of selecting the corresponding energy transmission path based on the location distribution of the starting area and the receiving area includes: Obtain the position information of the starting region and the receiving region, and determine the relative positional relationship based on the position information; Based on the relative positional relationship, list all potential transmission paths that can connect the starting area and the receiving area; Check the current usage status of each potential transmission path and filter out the backup transmission paths that are currently in an available state. All backup transmission paths are assigned priorities, and the first backup transmission path is selected as the energy transmission path according to the priority from high to low.

6. A dynamic balancing control device for a storage system's state of charge (SOC) based on reinforcement learning, characterized in that, A method for implementing a reinforcement learning-based dynamic equilibrium control method for a storage system's state of charge (SOC) as described in any one of claims 1 to 5, comprising: The adjustment module is used to dynamically adjust the operating parameters of the battery modules in the target area of ​​the energy storage system based on reinforcement learning; The data acquisition module is used to acquire the first temperature and humidity values ​​and SOC deviation values ​​of the target area and other areas after the operating parameters have been adjusted. The calculation module is used to calculate the optimal equalization current parameters for each region based on the first temperature and humidity values ​​and the SOC deviation values ​​of all regions using an adaptive control algorithm. The optimal equalization current parameters include the magnitude and direction of the equalization current. An optimization module is used to transmit the optimal equalization current parameters to the equalization circuit of the corresponding region. The equalization circuit then performs energy transfer between the target region and the battery modules in other regions to reduce the SOC deviation between the regions.

7. A computing device, characterized in that, It includes a processing component and a storage component; the storage component stores one or more computer instructions; the one or more computer instructions are invoked and executed by the processing component to implement the reinforcement learning-based SOC dynamic balancing control method for energy storage systems as described in any one of claims 1 to 5.

8. A computer storage medium, characterized in that, The system contains a computer program that, when executed by a computer, implements a reinforcement learning-based dynamic equilibrium control method for an energy storage system's state of charge (SOC).

Images