Energy storage load distribution method, system and device and computer readable storage medium

Abstract

The invention discloses an energy storage load distribution method, system and device and a computer readable storage medium, and relates to the technical field of energy storage systems. Dividing the energy storage unit into energy storage areas based on the power grid data set; for each energy storage region, based on the power grid data set, constructing a local load distribution game model taking the energy storage unit as a game main body; based on the power grid data set, constructing a global load distribution game model taking the energy storage area as a game main body; calculating the local load distribution game model and the global load distribution game model through a reinforcement learning algorithm to generate a target configuration strategy; and performing load configuration on the energy storage unit according to the target configuration strategy. The dynamic real-time load distribution between the energy storage units and the energy storage areas is realized, the optimization cooperation inside the energy storage units and the power balance between the energy storage areas are also realized, finally, the accurate global load balance and the optimal resource distribution are realized, and the energy efficiency of the energy storage system is improved.

Description

Technical Field

[0001] This application relates to the field of energy storage system technology, and more specifically, to an energy storage load distribution method, system, device, and computer storage medium. Background Technology

[0002] Currently, load distribution in energy storage systems primarily relies on load scheduling algorithms, which typically allocate load to energy storage units based on static models or preset rules. However, with the continuous changes in grid demand and the widespread adoption of renewable energy, the interaction and collaboration among multiple energy storage units in complex energy storage systems are becoming increasingly intricate. Static load distribution models are slow to react to fluctuations in grid demand or rapid changes in the status of energy storage devices, resulting in energy storage systems being unable to flexibly respond to demand changes and potentially leading to problems such as uneven load distribution, decreased system efficiency, or localized overload.

[0003] In summary, how to accurately balance energy storage loads and improve system energy efficiency is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0004] The purpose of this application is to provide an energy storage load distribution method, which can, to some extent, solve the technical problem of how to accurately balance energy storage loads and improve system energy efficiency. This application also provides an energy storage load distribution system, electronic equipment, and a computer-readable storage medium.

[0005] To achieve the above objectives, this application provides the following technical solution:

[0006] A method for allocating energy storage loads, comprising:

[0007] Obtain the power grid dataset;

[0008] Based on the aforementioned power grid dataset, the energy storage units are divided into energy storage areas;

[0009] For each energy storage region, a local load allocation game model is constructed based on the power grid dataset, with the energy storage units as the main players in the game.

[0010] Based on the aforementioned power grid dataset, a global load allocation game model is constructed with energy storage areas as the main players.

[0011] The target configuration strategy is generated by performing calculations on the local load allocation game model and the global load allocation game model using reinforcement learning algorithms.

[0012] Configure the load of the energy storage unit according to the target configuration strategy.

[0013] Preferably, the acquisition of the power grid dataset includes:

[0014] Obtain real-time load demand data of the power grid, which includes the total load of the power grid during the target period and the load demand of each energy storage unit.

[0015] To obtain the power generation and real-time power consumption of the power grid;

[0016] Based on the power generation and the real-time power consumption, the power supply and demand status of the power grid is generated.

[0017] Obtain the operating status of the energy storage unit, including the power output, health status, and operating temperature of the energy storage unit;

[0018] The capacity status of the energy storage unit is generated based on its power output.

[0019] Obtain historical load allocation records for energy storage units;

[0020] The system aggregates real-time load demand data, power grid supply and demand status, operating status, capacity status, and historical load allocation records to generate a comprehensive dataset.

[0021] The comprehensive dataset is preprocessed to generate a power grid dataset. The preprocessing includes removing outlier data, filling in missing data, and data standardization.

[0022] Preferably, for each energy storage area, based on the power grid dataset, a local load allocation game model is constructed with energy storage units as the main players, including:

[0023] For each energy storage region, the load demand of the energy storage units within the region is accumulated to generate the local load demand of the energy storage region.

[0024] The utility function of the energy storage unit is generated based on its power output and capacity status.

[0025] For each energy storage region, a local load allocation game model is constructed with energy storage units as the main players, based on the number of energy storage units, the power output of the energy storage units, and the utility function of the energy storage units.

[0026] Preferably, the step of constructing a global load allocation game model based on the power grid dataset, with energy storage areas as the main players, includes:

[0027] The utility function of the energy storage area is generated based on the power output of the energy storage area, the local load demand of the energy storage area, and the capacity status of the energy storage area.

[0028] Based on the number of energy storage areas, the power output of the energy storage areas, and the utility function of the energy storage areas, a local load allocation game model is constructed with energy storage units as the main players.

[0029] Preferably, a target configuration strategy is generated by performing a reinforcement learning algorithm on a local load allocation game model, including:

[0030] The first environmental state is generated based on the load requirements of the energy storage unit, the capacity status of the energy storage unit, and the operating status of the energy storage unit.

[0031] Based on the power output strategy of the energy storage unit, the first action set is generated;

[0032] Based on the utility function of the energy storage unit, a first immediate return is generated, and based on the first immediate return, a first value function is generated.

[0033] Construct a first strategy function for allocating power to the energy storage unit;

[0034] For each energy storage region, with the goal of matching the total power output of the energy storage region with the local load demand, the first environmental state, the first action, the first value function and the first policy function are processed by reinforcement learning algorithm to generate the optimal power output strategy of the energy storage unit in the energy storage region.

[0035] The optimal power output strategy of the energy storage unit is taken as one of the target configuration strategies.

[0036] Preferably, a target configuration strategy is generated by performing a reinforcement learning algorithm on the global load allocation game model, including:

[0037] A second environmental state is generated based on the local load demand of the energy storage area, the capacity status of the energy storage area, and the operating status of the energy storage area.

[0038] Based on the power output strategy of the energy storage area, a second set of actions is generated;

[0039] Based on the utility function of the energy storage area, a second immediate return is generated, and based on the second immediate return, a second value function is generated;

[0040] Construct a second strategy function for allocating power to the energy storage region;

[0041] With the goal of matching the total power output of all energy storage areas with the global load of the power grid, the second environmental state, the second action, the second value function, and the second strategy function are processed by reinforcement learning algorithm to generate the optimal power output strategy for the energy storage areas.

[0042] The optimal power output strategy of the energy storage area is taken as one of the target configuration strategies.

[0043] Preferably, after configuring the energy storage unit according to the target configuration strategy, the method further includes:

[0044] Obtain historical load demand data of the power grid;

[0045] Historical load demand data is processed by a time series model to generate a predicted load demand for the power grid after a set time in the future. The time series model has adaptive weights.

[0046] Based on the predicted load demand, a power prediction output strategy for the energy storage configuration unit is generated after a set time in the future;

[0047] In response to the discrepancy between the actual power output strategy of the energy storage unit after a set future time and the predicted power output strategy, the parameter vectors of the first strategy function and the second strategy function are updated.

[0048] An energy storage load distribution system, comprising:

[0049] The dataset acquisition module is used to acquire power grid datasets.

[0050] An energy storage area division module is used to divide energy storage units into energy storage areas based on the power grid dataset;

[0051] The local game model generation module is used to construct a local load allocation game model with energy storage units as the main players in the game for each energy storage area, based on the power grid dataset.

[0052] The global game model generation module is used to construct a global load allocation game model with energy storage areas as the main game subjects based on the power grid dataset.

[0053] The configuration strategy generation module is used to generate target configuration strategies by performing calculations on the local load allocation game model and the global load allocation game model using reinforcement learning algorithms.

[0054] The load configuration module is used to configure the load of the energy storage unit according to the target configuration strategy.

[0055] An electronic device, comprising:

[0056] Memory, used to store computer programs;

[0057] A processor, used to implement the steps of any of the above-described energy storage load distribution methods when executing the computer program.

[0058] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of any of the above-described energy storage load distribution methods.

[0059] This application provides a method for energy storage load allocation, which involves: acquiring a power grid dataset; dividing energy storage units into energy storage regions based on the power grid dataset; for each energy storage region, constructing a local load allocation game model with energy storage units as the main players based on the power grid dataset; constructing a global load allocation game model with energy storage regions as the main players based on the power grid dataset; generating a target configuration strategy by performing calculations on the local and global load allocation game models using a reinforcement learning algorithm; and configuring the load of energy storage units according to the target configuration strategy. Based on a power grid dataset, this application constructs a local load allocation game model with energy storage units as the main body and a global load allocation game model with energy storage regions as the main body through regional division. A reinforcement learning algorithm is used to iteratively update the power allocation strategy, achieving not only dynamic real-time load allocation between energy storage units and energy storage regions, but also optimized collaboration within energy storage units and power balance between energy storage regions. Compared with static or pre-defined load scheduling methods, energy storage units can adaptively update their strategies and automatically optimize power output based on the real-time load demand of the power grid and their own operating status. This significantly improves the flexibility and response speed of the energy storage system and reduces the energy efficiency degradation caused by load imbalance. Furthermore, the multi-entity game mechanism ensures that each energy storage unit not only cooperates to meet the power grid's needs but also improves its own efficiency through game theory, thereby achieving accurate global load balance and optimal resource allocation, and improving the energy efficiency of the energy storage system. The energy storage load allocation system, electronic device, and computer-readable storage medium provided in this application also solve the corresponding technical problems. Attached Figure Description

[0060] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0061] Figure 1 A flowchart illustrating an energy storage load allocation method provided in this application embodiment;

[0062] Figure 2 A schematic diagram illustrating the cooperation and conflict mechanisms between energy storage units and energy storage areas;

[0063] Figure 3 The flowchart shows the energy storage load allocation method based on the negative feedback reinforcement game algorithm.

[0064] Figure 4 A schematic diagram of an energy storage load distribution system provided in this application embodiment;

[0065] Figure 5This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application;

[0066] Figure 6 This is another structural schematic diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0067] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0068] Currently, load distribution in energy storage systems primarily relies on load scheduling algorithms, which typically allocate load to energy storage units based on static models or preset rules. However, with the continuous changes in grid demand and the widespread adoption of renewable energy, the interaction and collaboration among multiple energy storage units in complex energy storage systems are becoming increasingly intricate. Static load distribution models are slow to react to fluctuations in grid demand or rapid changes in the status of energy storage devices, resulting in energy storage systems being unable to flexibly respond to demand changes and potentially leading to problems such as uneven load distribution, decreased system efficiency, or localized overload.

[0069] Coordinating the operation of multiple energy storage units in a multi-unit energy storage system presents a significant challenge. Load scheduling algorithms lack effective modeling of the interactions between these units, and optimization methods typically employ simple partitioning or independent operation strategies, failing to fully utilize the synergistic effects among units. This leads to uneven resource utilization and low overall energy efficiency. Furthermore, when facing competition and cooperation issues in multi-unit systems, load scheduling algorithms often adopt centralized control, lacking an autonomous decision-making mechanism that can easily result in single-point failures or decreased scheduling efficiency. Additionally, as grid load demand constantly changes, energy storage systems require adaptive scheduling capabilities. Load scheduling algorithms also have significant shortcomings in this regard. Their rigidity makes the system inflexible in responding to sudden power demands, unable to dynamically adjust load allocation strategies based on real-time grid changes. This may lead to over-reliance on pre-set schemes, hindering the achievement of balanced power supply and optimal scheduling of energy storage units.

[0070] In summary, the main drawbacks of existing load scheduling algorithms are as follows: first, they cannot dynamically adjust according to real-time changes in grid load demand, leading to low system efficiency; second, they lack an effective multi-energy storage unit coordination mechanism, failing to fully utilize the overall system performance; and third, they lack adaptive load allocation and scheduling capabilities, making it difficult to cope with the rapid changes and complex demands of modern power grids. The energy storage load allocation method provided in this application overcomes these shortcomings.

[0071] Please see Figure 1 , Figure 1 A flowchart of an energy storage load allocation method provided in an embodiment of this application.

[0072] An energy storage load allocation method provided in this application embodiment may include the following steps:

[0073] Step S101: Obtain the power grid dataset.

[0074] In practical applications, when allocating load to energy storage units in the power grid, it is necessary to refer to the power grid dataset. The data types in the power grid dataset can be flexibly determined according to the application scenario. For example, it may include the total load of the power grid, the load demand of the energy storage units, and the power output of the energy storage units.

[0075] In an exemplary embodiment, to facilitate subsequent data processing, real-time load demand data of the power grid can be acquired during the process of acquiring the power grid dataset. This real-time load demand data includes the total load L of the power grid during the target time period. t and the load requirements D of each energy storage unit i,t Where i represents the energy storage unit number and t represents the time point, the total load of the power grid during the time period T can be expressed as L(t) = {L1, L2, ..., L...} T}, where T represents the time length of the real-time load demand data, and the grid load demand can be obtained and recorded in time series through real-time communication with the power dispatch center; the power generation G of the grid is obtained. t and real-time power consumption C t Based on power generation and real-time power consumption, the power supply and demand status Δ of the power grid is generated. t Δ t =G t -C t This allows for the description of whether power supply and demand are balanced by leveraging the power supply and demand status of the power grid; and the acquisition of the operating status S of the energy storage unit. i (t), the operating state includes the power output P of the energy storage unit. i,t Health status H i Operating temperature T i P i,t This represents the power output of energy storage unit i at time t, in kilowatts (H). iThe health status of the energy storage unit is represented by a range of [0,1], where 1 indicates optimal condition and 0 indicates a fault. T i This indicates the current operating temperature of the energy storage unit, in degrees Celsius; the capacity state C of the energy storage unit is generated based on its power output. i , C i (0) represents the initial capacity of the energy storage unit; retrieve the historical load distribution record R of the energy storage unit. i (t), R i (t) represents the load distribution result of energy storage unit i at a past time point t; for real-time load demand data L(t), power supply and demand status Δ(t), and operating status S i (t), Capacity State C i (t) and historical load distribution records The data is then aggregated to generate a comprehensive dataset D(t). N represents the total number of energy storage units; the comprehensive dataset is preprocessed to generate a power grid dataset. The preprocessing includes removing outlier data, filling in missing data, and data standardization.

[0076] In specific application scenarios, during the preprocessing of a comprehensive dataset to generate a power grid dataset, a data integrity check can be performed on the comprehensive dataset D(t) to identify and fill in missing data, such as missing values ​​in time series data. The data is processed using historical data interpolation or mean interpolation based on adjacent time points; then, abnormal data is identified and removed, such as load data L(t) and energy storage unit operating status data S that exceed reasonable ranges. i (t), according to the set threshold range [L] min ,L max ] and [S min ,S max Filtering is performed, L min and L max S represents the preset minimum and preset maximum values ​​of the load data, respectively. min and S max These are the preset minimum and maximum values ​​for the energy storage unit's operating status data, respectively; finally, the data is normalized, and the grid load data L(t), power supply and demand status Δ(t), and energy storage unit operating status data S are normalized. i (t), Capacity State C i (t) is normalized to the interval [0,1] to generate a normalized input dataset D for optimizing load distribution. norm (t): Among them, L norm (t) represents the normalized grid load demand, Δ norm(t) represents the normalized electricity supply and demand situation, S i,norm (t) represents the normalized operating state of the energy storage unit, R i (t) represents the historical load allocation record of the energy storage unit, C i,norm (t) represents the normalized capacity state of the energy storage unit.

[0077] Step S102: Based on the power grid dataset, divide the energy storage unit into energy storage areas.

[0078] In practical applications, considering that energy storage units are located in different regions, they can be managed by region. That is, based on the power grid dataset, energy storage units are divided into energy storage regions. For example, the energy storage units in the energy storage system can be divided into multiple energy storage regions R according to geographical location, power demand, energy storage capacity, and network topology characteristics. j Each energy storage area R j Includes several energy storage units E j,k j represents the energy storage area number, and k represents the energy storage unit number within the energy storage area.

[0079] Step S103: For each energy storage area, based on the power grid dataset, construct a local load allocation game model with energy storage units as the main players.

[0080] In practical applications, the load demand of an energy storage area can be met by the energy storage units. Therefore, for each energy storage area, a local load allocation game model can be constructed based on the power grid dataset, with the energy storage units as the main players. This model can then be used to configure the load of the energy storage units so that the power output of the energy storage area matches the load demand.

[0081] In the exemplary embodiment, for each energy storage region, during the process of constructing a local load allocation game model with energy storage units as the main players based on the power grid dataset, for each energy storage region, the load demand D of the energy storage units within the energy storage region is... j,k (t) is accumulated to generate the local load demand L of the energy storage area. Rj (t), D j,k (t) represents the energy storage unit E j,k The load demand at time t can be obtained and recorded in real time through communication with the power dispatch center. j Indicates energy storage area R j The number of energy storage units contained within; based on the power output and capacity status of the energy storage units, the energy storage unit E is generated. j,k utility function U j,k (t), U j,k (t)=-(α j,k (P j,k(t)-D j,k (t)) 2 +β j,k C j,k (t)), α j,k and β j,k C is the weighting coefficient. j,k (t) represents the energy storage unit E j,k The capacity status; for each energy storage region, based on the number of energy storage units, the power output of the energy storage units, and the utility function of the energy storage units, a local load allocation game model G is constructed with the energy storage units as the main players. Rj , P j,k (t) represents the energy storage unit E j,k The power output at time t has a policy space of... , and Energy storage unit E j,k The minimum and maximum power output.

[0082] Step S04: Based on the power grid dataset, construct a global load allocation game model with energy storage areas as the main players.

[0083] In practical applications, the total load demand of the power grid can be met by energy storage areas. Therefore, based on the power grid dataset, a global load allocation game model with energy storage areas as the main players can be constructed. This model can then be used to configure the load of energy storage areas so that the power output of the power grid is matched with the load demand.

[0084] In the exemplary embodiment, during the process of constructing a global load allocation game model with energy storage areas as the main players based on the power grid dataset, the power output P of the energy storage areas can be used as a reference. Rj (t), Local load demand of the energy storage area L Rj (t) and the capacity state C of the energy storage area Rj (t), generating the utility function U of the energy storage region. Rj (t), U Rj (t)=-(α Rj P Rj (t)-L Rj (t)) 2 +β Rj C Rj (t)), L Rj (t) represents the energy storage region R j Local load requirements, C Rj (t) represents the capacity state of the energy storage region, α Rj and β Rj P is the weighting coefficient. Rj (t) represents the energy storage region Rj The power output at time t, the policy space is: , and Let M be the minimum and maximum power output of the energy storage region, respectively. Based on the number of energy storage regions M, the power output of the energy storage regions, and the utility function of the energy storage regions, a local load allocation game model G is constructed with the energy storage units as the main players. global , .

[0085] Step S105: Calculate the local load distribution game model and the global load distribution game model using a reinforcement learning algorithm to generate the target configuration strategy.

[0086] In practical applications, considering that the load configuration of energy storage units affects the power output of the energy storage area, and the load demand of the energy storage area in turn affects the load configuration of the energy storage units, there is a mutual influence between the local load allocation game model and the global load allocation game model, such as... Figure 2 As shown, solving this problem individually is cumbersome and difficult to pinpoint. Therefore, since Reinforcement Learning (RL) is a machine learning method, and its basic framework is the Markov Decision Process, which allows an agent to learn the optimal policy through trial and error in its interaction with the environment, this application uses reinforcement learning algorithms to compute local load allocation game models and global load allocation game models to generate target configuration policies.

[0087] In an exemplary embodiment, the local load distribution game model and the global load distribution game model can be solved together based on reinforcement learning, or they can be solved separately. For ease of understanding, the process will be explained by taking the separate solution as an example.

[0088] In specific application scenarios, during the process of generating target configuration strategies by performing calculations on the local load allocation game model using reinforcement learning algorithms, each energy storage region R... j Independently participate in the local load allocation game, within the already determined local load demand L. Rj Independent operation under (t) constraints, reinforcement learning algorithm for local load distribution game model G Rj Solving this problem essentially involves applying the mechanism of "energy storage units as the main players in the game and utility functions as the optimization objective" to feasible strategy iterations, where energy storage unit E... j,k Using the current state and historical strategy as input, at the power output boundary Internal selection action P j,k (t), optimize load distribution within the region, and use the utility function U j,kThe negative value of (t) is used as an immediate reward for reinforcement learning updates, and after multiple iterations, the total power output within the energy storage region is matched to L. Rj (t), and drive the local strategy to converge to Nash equilibrium. Based on this, the load demand D of the energy storage unit can be used as a basis. j,k (t), Capacity State C of Energy Storage Unit j,k (t) and the operating state S of the energy storage unit j,k (t), generating the first environment state s j,k (t); Power output strategy P based on energy storage unit j,k (t), generate the first action set a j,k (t); Utility function U based on energy storage unit j,k (t), generating the first instant reward R j,k (t), R j,k (t)=-U j,k (t), and generate the first value function based on the first immediate return. , γ is the discount factor for the energy storage unit, with a value range of (0,1]. a' represents "the alternative actions in the set of possible actions for the energy storage unit at the next moment", which is used to maximize the optimal action for the next state when updating the value function, corresponding to the alternative power output strategy P at the energy storage unit level. j,k (t); Construct the first strategy function π for allocating power to the energy storage unit. j,k (s j,k (t)); For each energy storage region, with the goal of matching the total power output of the energy storage region with the local load demand, a reinforcement learning algorithm is used to process the first environmental state, the first action, the first value function, and the first policy function to generate the optimal power output strategy for the energy storage units in the energy storage region. , The symbol "*" represents the optimal policy function, which is the policy that maximizes the value function Q in the current state. In this process, the local level utilizes the value function... Optimize the strategy and obtain When a deviation between the actual allocation and the expected allocation is detected, the policy function parameters are updated using a policy gradient approach. This process is then iteratively updated to optimize the energy storage unit E. j,k The strategy reaches Nash equilibrium, the local load allocation game converges, and the energy storage area R... j The load distribution within the system is optimized; the optimal power output strategy of the energy storage unit is taken as one of the target configuration strategies.

[0089] In specific application scenarios, within a high-level global load distribution game model, the energy storage region R... jAs a player in the game, participating in the overall game, the goal is to achieve global electricity demand L through cooperation and confrontation mechanisms among different regions. total (t) and inter-regional electricity supply and demand relationship Δ Rj (t)=G Rj (t)-L Rj (t) Optimize global load distribution strategy P global (t) represents the global power output, Δ Rj (t)=G Rj (t)-L Rj (t) is used to characterize the energy storage region R j The supply-demand deviation relative to its local load demand is determined by the regional power output P. Rj (t) and L Rj (t) is an intermediate quantity derived from Δ Rj (t) can be fed back to the global load distribution game model as a global state / monitoring quantity to guide the dynamic adjustment of regional power output strategies. Solving the global load distribution game model using reinforcement learning algorithms essentially translates the global strategy of "energy storage regions as the main players" into executable reinforcement learning updates, and uses convergence criteria to ensure the stability and usability of the strategy. Each energy storage region R... j P adjusts its own power output strategy based on the power output and load demand of other regions. Rj (t), and optimizes global load distribution through cooperation and adversarial mechanisms, so that the global load distribution strategy gradually converges to Nash equilibrium. n represents the number of iterations and expresses the convergence target. In practical applications, the stopping condition can be set as the change between two consecutive iterations being less than a preset threshold or reaching the maximum number of iterations. .

[0090] Based on this, in the process of generating the target configuration strategy by calculating the global load allocation game model using reinforcement learning algorithms, the local load demand L of the energy storage area can be considered. Rj (t), Capacity status of the energy storage area C Rj (t) and the operating status S of the energy storage area Rj (t), generating the second environment state s Rj (t); Power output strategy P based on energy storage region Rj (t), generate the second action set A Rj (t); Utility function U based on the energy storage region Rj (t), generating the second instant reward R Rj (t), R Rj (t)=-U Rj (t), and generate a second value function based on the second immediate return. , Γ is the discount factor for the energy storage region, with a value range of (0,1]. A' represents "the alternative actions in the set of available actions for the energy storage region at the next moment", which is used to maximize the optimal action for the next state when updating the value function, corresponding to the alternative power output strategy P at the energy storage region level. Rj (t); Construct a second strategy function Π for allocating power to the energy storage region. Rj (S Rj (t)); With the goal of matching the total power output of all energy storage areas with the global load of the power grid, a reinforcement learning algorithm is used to process the second environmental state, the second action, the second value function, and the second strategy function to generate the optimal power output strategy for the energy storage areas. , During this process, within each scheduling time t, the energy storage regions R j At the power output boundary Internal selection action P Rj (t), derived from global power output Regional local load demand L Rj (t) Calculate the instant return R of the region Rj (t)=-U Rj (t), then update the relation iteratively according to the second value function. And obtain the policy function Π Rj (S Rj The improvement of (t) enables each energy storage region to gradually adjust its own strategy while taking into account the power output and load demand of other regions; the optimal power output strategy of the energy storage region is taken as one of the target configuration strategies.

[0091] Step S106: Configure the load of the energy storage unit according to the target configuration strategy.

[0092] In practical applications, after generating the target configuration strategy, the energy storage units and energy storage areas can be configured with loads according to the target configuration strategy.

[0093] In specific application scenarios, the first and second policy functions in reinforcement learning are used to generate load configuration policies. The accuracy of the policy functions affects the load allocation of energy storage units. To ensure accurate load allocation, negative feedback can be used to adjust the policy functions. That is, after configuring the load of the energy storage units according to the target configuration policy, historical load demand data L from the power grid can also be obtained. real (t-nΔt); By processing historical load demand data using a time series model with adaptive weights, the predicted load demand of the power grid after a set future time is generated. , N p w is the number of sampling points for historical load demand data.n For adaptive weights, satisfying Based on the predicted load demand, generate a power prediction output strategy for the energy storage configuration unit after a set future time. , μ j,k For energy storage unit E j,k The adjustment coefficient is used to detect whether the actual power output strategy of the energy storage unit after a set future time is consistent with the predicted power output strategy. If the actual power output strategy of the energy storage unit after the set future time is inconsistent with the predicted power output strategy, the parameter vectors of the first policy function and the second policy function are updated, for example, by recalculating the power output strategy using the policy gradient method in deep reinforcement learning. , θ j,k and Θ Rj Energy storage unit E j,k and energy storage area R j The policy function's parameter vector, where α and β are the learning rates, J j,k (θ j,k ) and J Rj (Θ Rj These are the strategy performance metrics. The complete process is as follows: Figure 3 As shown, this application can combine time series analysis and adaptive weighting models to predict the future load demand of the power grid in real time, adjust the power output strategy of the energy storage unit in advance, and predict changes in power demand and dynamically optimize power output before load changes occur by combining historical load data and the operating status system of the energy storage unit, thus ensuring the balance of load supply and demand. Compared with load scheduling methods that rely solely on current load data for power scheduling, the predictive scheduling mechanism of this application can effectively avoid scheduling lag problems when the power grid load peaks and valleys occur, and improve the overall response efficiency and scheduling accuracy of the energy storage system.

[0094] This application provides a method for energy storage load allocation, which involves: acquiring a power grid dataset; dividing energy storage units into energy storage regions based on the power grid dataset; for each energy storage region, constructing a local load allocation game model with energy storage units as the main players based on the power grid dataset; constructing a global load allocation game model with energy storage regions as the main players based on the power grid dataset; generating a target configuration strategy by performing calculations on the local and global load allocation game models using a reinforcement learning algorithm; and configuring the load of energy storage units according to the target configuration strategy. Based on a power grid dataset, this application constructs a local load allocation game model with energy storage units as the main body and a global load allocation game model with energy storage regions as the main body through regional division. A reinforcement learning algorithm is used to iteratively update the power allocation strategy, achieving not only dynamic real-time load allocation between energy storage units and energy storage regions, but also optimized collaboration within energy storage units and power balance between energy storage regions. Compared with static or pre-defined load scheduling methods, energy storage units can adaptively update their strategies and automatically optimize power output based on the real-time load demand of the power grid and their own operating status. This significantly improves the flexibility and response speed of the energy storage system and reduces the energy efficiency degradation caused by load imbalance. Furthermore, the multi-entity game mechanism ensures that each energy storage unit not only collaborates to meet the power grid's needs but also improves its own efficiency through game theory, thereby achieving accurate global load balance and optimal resource allocation, and improving the energy efficiency of the energy storage system.

[0095] To verify the feasibility and effectiveness of this application in optimizing load distribution in energy storage systems, two actual power grid energy storage system datasets were used, from the energy storage management systems of power grids in regions A and B, respectively. The dataset from region A covered load distribution data from January to June 2021, while the dataset from region B covered electricity demand and energy storage system operation data from January to July 2021. To verify the adaptability of this application in complex dynamic load distribution environments, experimental analysis was conducted on these two datasets.

[0096] In the experiment, each dataset was divided into a training set and a test set in an 8:2 ratio to ensure that the training data could cover the fluctuation characteristics of the grid load, while the test data could reflect the actual effect of the load allocation strategy. In the process of optimizing the load allocation strategy, the power output of the energy storage unit and the energy storage area was dynamically adjusted in real time based on the reinforcement game algorithm.

[0097] For the status information of energy storage units, the real-time load demand of the power grid, the power supply and demand relationship, and the operating status of the energy storage units are used as input features. Specifically, the load demand information of the power grid dataset in region A includes hourly load changes, with a total of 4,320 sample points. The power grid dataset in region B contains more detailed minute-level load data, with a total of 10,080 samples. To verify the adaptability of this application, experiments were conducted on these two datasets respectively, and the reinforcement game algorithm of this application was used to optimize the load allocation of the energy storage system.

[0098] In model training, the Q-learning algorithm was used to update the strategy of the energy storage unit. Based on the historical load data and operating status information of the energy storage unit, the power output strategy of the energy storage unit was gradually adjusted. During the training process, the learning rate was set to 0.001, the batch size was 64, and the training was conducted for 200 epochs. To ensure the reliability of the experimental results, the Adam optimizer was used to optimize the model, and the grid load balancing degree and response time were used as the main evaluation indicators.

[0099] Experimental results on the power grid dataset in region A show that the reinforcement game algorithm proposed in this application has good adaptability under large load fluctuations. The energy storage unit can quickly adjust the power output according to the real-time load demand of the power grid. The response time of the energy storage system during the peak and valley periods of load demand is reduced by 50%, from an average of 60 seconds in the traditional method to 30 seconds. At the same time, the load distribution balance is improved by 15%, as shown in Table 1.

[0100] Table 1. Comparison of the proposed method and traditional methods on the power grid dataset of region A.

[0101]

[0102] Analysis of Table 1 shows that the method in this application significantly improves load response speed and peak load handling capacity, especially in terms of load distribution balance. Traditional methods often exhibit response lag when load fluctuations are large, while this application, through reinforcement learning algorithms, can predict load changes in advance and make rapid adjustments.

[0103] Experiments on the power grid dataset in region B further validated the applicability of this application. In this dataset, the load fluctuations of the power grid are more frequent, especially when the renewable energy generation changes drastically. The responsiveness of the energy storage system is crucial. The experimental results show that the method of this application still exhibits stable load distribution capability in this complex environment. The system can achieve real-time adjustment during the load distribution process, and the average response time is reduced from 50 seconds in the traditional method to 20 seconds. The system energy efficiency is improved by about 18%, as shown in Table 2.

[0104] Table 2 Comparison results of the proposed method and traditional methods on the power grid dataset of region B.

[0105]

[0106] Analysis of Table 2 shows that in the power grid dataset of region B, the load allocation optimization method proposed in this application not only improves the response speed, but also enhances the load handling capacity and overall system energy efficiency. In particular, when the renewable energy generation changes, the system can react quickly, ensuring the stable operation of the power grid.

[0107] In summary, the method presented in this experiment demonstrates excellent adaptability and robustness. Under conditions of drastic fluctuations in grid load demand, the energy storage system can adjust its power output in a timely manner according to the optimization strategy of the reinforced game algorithm, ensuring the stable operation of the grid. Compared with traditional methods, this application combines local and global game models, enabling each energy storage unit to autonomously optimize its strategy and ultimately achieve optimal allocation of global load.

[0108] Comparative experiments with traditional methods demonstrate that the proposed method exhibits significant advantages across multiple metrics. Regarding response time, traditional methods, relying on fixed rules or static models, struggle to cope with complex grid demand changes, leading to load lag in allocation. In contrast, the reinforcement learning algorithm in this application enables energy storage units to adaptively adjust power output through multiple policy updates, significantly shortening response time. Furthermore, the proposed method demonstrates superior performance in load balancing and system energy efficiency. Through a multi-agent game mechanism, energy storage units can not only optimize power within local areas but also ensure power supply and demand balance between different energy storage regions through global game theory, ultimately achieving overall system load balancing. This collaborative and adversarial mechanism allows for more efficient utilization of system resources, improving system energy efficiency by 15%-18% compared to traditional methods.

[0109] Experimental results demonstrate that the energy storage system load allocation optimization method based on reinforcement game theory can effectively solve the dynamic response and load balancing problems in power grid load allocation. Experimental results show that the method of this application exhibits excellent response and load handling capabilities in practical applications, and can significantly improve the overall energy efficiency of the system. In the future, with the continuous changes in power grid load demand, the method of this application has broad application prospects.

[0110] Please see Figure 4 , Figure 4 This is a schematic diagram of the structure of an energy storage load distribution system provided in an embodiment of this application.

[0111] An energy storage load distribution system provided in this application embodiment may include:

[0112] Data set acquisition module 101 is used to acquire power grid data sets;

[0113] The energy storage area division module 102 is used to divide the energy storage unit into energy storage areas based on the power grid dataset;

[0114] The local game model generation module 103 is used to construct a local load allocation game model with energy storage units as the main players in the game, based on the power grid dataset, for each energy storage area.

[0115] The global game model generation module 104 is used to construct a global load allocation game model with energy storage areas as the main game subjects based on the power grid dataset.

[0116] The configuration strategy generation module 105 is used to generate a target configuration strategy by performing calculations on the local load allocation game model and the global load allocation game model using a reinforcement learning algorithm.

[0117] The load configuration module 106 is used to configure the load of the energy storage unit according to the target configuration strategy.

[0118] This application provides an energy storage load allocation system. The dataset acquisition module can be specifically used to: acquire real-time load demand data of the power grid, including the total load of the power grid during a target time period and the load demand of each energy storage unit; acquire the power generation and real-time power consumption of the power grid; generate the power supply and demand status of the power grid based on the power generation and real-time power consumption; acquire the operating status of the energy storage units, including the power output, health status, and operating temperature of the energy storage units; generate the capacity status of the energy storage units based on their power output; acquire historical load allocation records of the energy storage units; summarize the real-time load demand data, power supply and demand status of the power grid, operating status, capacity status, and historical load allocation records to generate a comprehensive dataset; and preprocess the comprehensive dataset to generate a power grid dataset, the preprocessing including removing abnormal data, filling in missing data, and data standardization.

[0119] This application provides an energy storage load allocation system in which a local game model generation module can be specifically used to: accumulate the load demand of energy storage units within each energy storage area to generate the local load demand of the energy storage area; generate the utility function of the energy storage unit based on its power output and capacity status; and construct a local load allocation game model with the energy storage unit as the main player in each energy storage area, based on the number of energy storage units, their power output, and their utility function.

[0120] The energy storage load allocation system provided in this application embodiment includes a global game model generation module that can be specifically used to: generate a utility function for an energy storage area based on the power output of the energy storage area, the local load demand of the energy storage area, and the capacity status of the energy storage area; and construct a local load allocation game model with energy storage units as the main players based on the number of energy storage areas, the power output of the energy storage areas, and the utility function of the energy storage areas.

[0121] This application provides an energy storage load allocation system. The configuration strategy generation module can be specifically used for: generating a first environmental state based on the load demand, capacity status, and operating status of the energy storage units; generating a first action set based on the power output strategy of the energy storage units; generating a first immediate reward based on the utility function of the energy storage units, and generating a first value function based on the first immediate reward; constructing a first strategy function for allocating power to the energy storage units; for each energy storage region, with the goal of matching the total power output of the energy storage region with the local load demand, processing the first environmental state, first action, first value function, and first strategy function through a reinforcement learning algorithm to generate the optimal power output strategy for the energy storage units in the energy storage region; and using the optimal power output strategy of the energy storage units as one of the target configuration strategies.

[0122] This application provides an energy storage load allocation system. The configuration strategy generation module can be specifically used to: generate a second environmental state based on the local load demand of the energy storage area, the capacity status of the energy storage area, and the operating status of the energy storage area; generate a second action set based on the power output strategy of the energy storage area; generate a second immediate reward based on the utility function of the energy storage area, and generate a second value function based on the second immediate reward; construct a second strategy function for allocating power to the energy storage area; with the goal of matching the total power output of all energy storage areas with the global load of the power grid, process the second environmental state, second action, second value function, and second strategy function using a reinforcement learning algorithm to generate the optimal power output strategy for the energy storage area; and use the optimal power output strategy of the energy storage area as one of the target configuration strategies.

[0123] The energy storage load distribution system provided in this application embodiment may further include:

[0124] The historical data acquisition module is used to acquire historical load demand data of the power grid after the load configuration module configures the energy storage unit according to the target configuration strategy.

[0125] The demand forecasting module is used to process historical load demand data through a time series model to generate the predicted load demand of the power grid after a set time in the future. The time series model has adaptive weights.

[0126] The strategy prediction module is used to generate a power prediction output strategy for the energy storage configuration unit after a set time in the future, based on the predicted load demand.

[0127] The strategy adjustment module is used to update the parameter vectors of the first strategy function and the second strategy function in response to the inconsistency between the actual power output strategy and the power prediction output strategy of the energy storage unit after a set time in the future.

[0128] This application also provides an electronic device and a computer-readable storage medium, both of which have the corresponding effects of the energy storage load distribution method provided in the embodiments of this application. Please refer to... Figure 5 , Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0129] An electronic device provided in this application includes a memory 201 and a processor 202. The memory 201 stores a computer program, and the processor 202 executes the computer program to implement the steps of the energy storage load distribution method described in any of the above embodiments.

[0130] Please see Figure 6 Another electronic device provided in this application embodiment may further include: an input port 203 connected to the processor 202 for transmitting commands input from the outside to the processor 202; a display unit 204 connected to the processor 202 for displaying the processing results of the processor 202 to the outside; and a communication module 205 connected to the processor 202 for enabling communication between the electronic device and the outside. The display unit 204 may be a display panel, a laser scanning display, etc.; the communication method adopted by the communication module 205 includes, but is not limited to, Mobile High-Definition Link (MHL), Universal Serial Bus (USB), High-Definition Multimedia Interface (HDMI), wireless connection: Wireless Fidelity (WiFi), Bluetooth communication technology, Bluetooth Low Energy communication technology, and communication technology based on IEEE 802.11s.

[0131] This application provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the steps of the energy storage load distribution method described in any of the above embodiments.

[0132] The computer-readable storage media involved in this application include random access memory (RAM), memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disks, removable disks, CD-ROMs (compact disc read-only memory), or any other form of storage media known in the art.

[0133] This application provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of the energy storage load allocation method described in any of the above embodiments.

[0134] For descriptions of relevant parts of the energy storage load distribution system, electronic device, and computer-readable storage medium provided in the embodiments of this application, please refer to the detailed descriptions of the corresponding parts in the energy storage load distribution method provided in the embodiments of this application, which will not be repeated here. Furthermore, parts of the technical solutions provided in the embodiments of this application that are consistent with the implementation principles of corresponding technical solutions in the prior art have not been described in detail to avoid excessive elaboration.

[0135] It should also be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0136] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for allocating energy storage loads, characterized in that, include: Obtain the power grid dataset; Based on the aforementioned power grid dataset, the energy storage units are divided into energy storage areas; For each energy storage region, a local load allocation game model is constructed based on the power grid dataset, with the energy storage units as the main players in the game. Based on the aforementioned power grid dataset, a global load allocation game model is constructed with energy storage areas as the main players. The target configuration strategy is generated by performing calculations on the local load allocation game model and the global load allocation game model using reinforcement learning algorithms. Configure the load of the energy storage unit according to the target configuration strategy.

2. The method according to claim 1, characterized in that, The acquisition of the power grid dataset includes: Obtain real-time load demand data of the power grid, which includes the total load of the power grid during the target period and the load demand of each energy storage unit. Obtain the power generation and real-time power consumption of the power grid; Based on the power generation and the real-time power consumption, the power supply and demand status of the power grid is generated; Obtain the operating status of the energy storage unit, including the power output, health status, and operating temperature of the energy storage unit; The capacity status of the energy storage unit is generated based on its power output. Obtain historical load allocation records for energy storage units; The system aggregates real-time load demand data, power grid supply and demand status, operating status, capacity status, and historical load allocation records to generate a comprehensive dataset. The comprehensive dataset is preprocessed to generate a power grid dataset. The preprocessing includes removing outlier data, filling in missing data, and data standardization.

3. The method according to claim 2, characterized in that, For each energy storage region, based on the power grid dataset, a local load allocation game model is constructed with energy storage units as the main players, including: For each energy storage region, the load demand of the energy storage units within the region is accumulated to generate the local load demand of the energy storage region. The utility function of the energy storage unit is generated based on its power output and capacity status. For each energy storage region, a local load allocation game model is constructed with energy storage units as the main players, based on the number of energy storage units, the power output of the energy storage units, and the utility function of the energy storage units.

4. The method according to claim 3, characterized in that, The construction of a global load allocation game model based on the power grid dataset, with energy storage areas as the main players, includes: The utility function of the energy storage area is generated based on the power output of the energy storage area, the local load demand of the energy storage area, and the capacity status of the energy storage area. Based on the number of energy storage areas, the power output of the energy storage areas, and the utility function of the energy storage areas, a local load allocation game model is constructed with energy storage units as the main players.

5. The method according to claim 4, characterized in that, The local load allocation game model is processed using reinforcement learning algorithms to generate a target configuration strategy, including: The first environmental state is generated based on the load requirements of the energy storage unit, the capacity status of the energy storage unit, and the operating status of the energy storage unit. Based on the power output strategy of the energy storage unit, the first action set is generated; Based on the utility function of the energy storage unit, a first immediate return is generated, and based on the first immediate return, a first value function is generated. Construct a first strategy function for allocating power to the energy storage unit; For each energy storage region, with the goal of matching the total power output of the energy storage region with the local load demand, the first environmental state, the first action, the first value function and the first policy function are processed by reinforcement learning algorithm to generate the optimal power output strategy of the energy storage unit in the energy storage region. The optimal power output strategy of the energy storage unit is taken as one of the target configuration strategies.

6. The method according to claim 5, characterized in that, The global load distribution game model is computed using reinforcement learning algorithms to generate target configuration strategies, including: A second environmental state is generated based on the local load demand of the energy storage area, the capacity status of the energy storage area, and the operating status of the energy storage area. Based on the power output strategy of the energy storage area, a second set of actions is generated; Based on the utility function of the energy storage area, a second immediate return is generated, and based on the second immediate return, a second value function is generated; Construct a second strategy function for allocating power to the energy storage region; With the goal of matching the total power output of all energy storage areas with the global load of the power grid, the second environmental state, the second action, the second value function, and the second strategy function are processed by reinforcement learning algorithm to generate the optimal power output strategy for the energy storage areas. The optimal power output strategy of the energy storage area is taken as one of the target configuration strategies.

7. The method according to claim 6, characterized in that, After configuring the energy storage unit according to the target configuration strategy, the method further includes: Obtain historical load demand data of the power grid; Historical load demand data is processed by a time series model to generate a predicted load demand for the power grid after a set time in the future. The time series model has adaptive weights. Based on the predicted load demand, a power prediction output strategy for the energy storage configuration unit is generated after a set time in the future; In response to the discrepancy between the actual power output strategy of the energy storage unit after a set future time and the predicted power output strategy, the parameter vectors of the first strategy function and the second strategy function are updated.

8. An energy storage load distribution system, characterized in that, include: The dataset acquisition module is used to acquire power grid datasets. An energy storage area division module is used to divide energy storage units into energy storage areas based on the power grid dataset; The local game model generation module is used to construct a local load allocation game model with energy storage units as the main players in the game for each energy storage area, based on the power grid dataset. The global game model generation module is used to construct a global load allocation game model with energy storage areas as the main game subjects based on the power grid dataset. The configuration strategy generation module is used to generate target configuration strategies by performing calculations on the local load allocation game model and the global load allocation game model using reinforcement learning algorithms. The load configuration module is used to configure the load of the energy storage unit according to the target configuration strategy.

9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, configured to implement the steps of the energy storage load distribution method as described in any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the energy storage load distribution method as described in any one of claims 1 to 7.

Images