Power grid side energy storage energy management method combined with big data analysis

Abstract

The application discloses a power grid side energy storage energy management method combined with big data analysis and belongs to the technical field of new energy storage, wherein the method comprises the following steps: acquiring a historical operation big data set; constructing a battery health state dynamic degradation model based on historical working condition data; associating the battery health state dynamic degradation model with an initial battery health state, calculating a battery health loss cost corresponding to unit throughput energy, and calculating an expected frequency modulation income according to historical market data; constructing a reinforcement learning environment, training an agent in the reinforcement learning environment by using the historical operation big data set; and driving the agent to generate a charging and discharging action instruction according to a real-time power grid state, a real-time energy storage state and real-time market data, so as to control the operation of the power grid side energy storage system. The application solves the technical problem that the power grid side energy storage frequency modulation energy management in the prior art cannot dynamically balance the frequency modulation income and the battery health loss, thereby leading to a low net income in the whole life cycle.

Description

Technical Field

[0001] This invention relates to the field of new energy storage technology, specifically to a grid-side energy storage management method that combines big data analysis. Background Technology

[0002] With the advancement of dual-carbon goals, the penetration rate of new energy sources such as wind power and photovoltaics in the power system continues to increase. The randomness, volatility, and intermittency of their output pose a severe challenge to the frequency stability of the power grid, leading to an explosive growth in the demand for fast frequency regulation resources. Grid-side energy storage systems, with their advantages of fast response, high regulation accuracy, and flexible deployment, have become a core technological means to support power grid frequency regulation. However, the initial investment cost of energy storage systems is high, and battery life is highly sensitive to operating conditions, making the net return over the entire life cycle a core bottleneck restricting their large-scale commercial application. How to achieve a dynamic balance between maximizing frequency regulation revenue and minimizing battery health losses while meeting the requirements of power grid frequency regulation services is a key technical challenge that the industry urgently needs to overcome.

[0003] Existing grid-side energy storage frequency regulation energy management technologies generally have significant limitations. Most methods prioritize maximizing short-term frequency regulation benefits as the sole optimization objective, failing to accurately quantify the differentiated impacts of different charge / discharge rates, temperatures, and states of charge on battery aging. This leads to abnormally accelerated battery aging and a substantial reduction in overall lifecycle economic benefits. Some solutions incorporating battery health models often employ static models based on electrochemical mechanisms, resulting in complex parameter calibration and poor generalization capabilities, making them ill-suited for complex and variable frequency regulation conditions. Furthermore, traditional reinforcement learning decision-making frameworks suffer from limitations such as simplistic reward function design, high online training risks, and insufficient robustness in real-time decision-making, failing to meet the stringent requirements for engineering implementation. Summary of the Invention

[0004] This application provides a grid-side energy storage energy management method that combines big data analysis, aiming to solve the technical problem in the existing technology that grid-side energy storage frequency regulation energy management cannot dynamically balance frequency regulation revenue and battery health loss, resulting in low net revenue over the entire life cycle.

[0005] In view of the above problems, this application provides a grid-side energy storage management method combining big data analysis, including: acquiring a historical operating big data set of the grid-side energy storage system, wherein the historical operating big data set includes at least historical operating condition data and historical market data; based on the historical operating condition data, constructing a dynamic degradation model of battery health status with battery aging time-series characteristics as input and battery capacity decay increment as output; associating the dynamic degradation model of battery health status with the initial health status of the battery, calculating the battery health loss cost corresponding to unit throughput energy, and calculating the expected frequency regulation revenue based on the historical market data; constructing a reinforcement learning environment, wherein the reward function of the reinforcement learning environment is the difference between the expected frequency regulation revenue and the battery health loss cost, and training the agent in the reinforcement learning environment using the historical operating big data set; and driving the trained agent to generate charging and discharging action commands, executing the charging and discharging action commands, and controlling the operation of the grid-side energy storage system according to real-time grid status, real-time energy storage status, and real-time market data.

[0006] One or more technical solutions provided in this application have at least the following technical effects or advantages: This paper adopts a technical solution that utilizes a large dataset of historical operation data of grid-side energy storage, including historical operating condition data and historical market data, to construct a dynamic degradation model of battery health status. It quantifies the battery health loss cost and expected frequency regulation benefit corresponding to a unit of energy throughput, and constructs a reinforcement learning environment with the difference between the two as the reward function to train the agent. This solution drives the agent to generate and execute real-time charging and discharging action commands, which solves the technical problem of not being able to dynamically balance frequency regulation benefits and battery health loss in existing technologies. It achieves the technical effect of accurately controlling the battery aging rate while meeting the grid frequency regulation requirements, and significantly improving the net benefit of the grid-side energy storage system throughout its entire life cycle.

[0007] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the following are specific embodiments of this application. Attached Figure Description

[0008] Figure 1 A flowchart illustrating a grid-side energy storage management method incorporating big data analytics is provided for embodiments of this application.

[0009] Figure 2 This application provides a flowchart illustrating the dynamic degradation model of battery health status in a grid-side energy storage energy management method that incorporates big data analysis, as illustrated in the embodiments of this application. Detailed Implementation

[0010] To further illustrate the technical means and effects of the present invention in achieving its intended purpose, the following detailed description of the specific implementation methods, structures, features, and effects of the present invention, in conjunction with the accompanying drawings and preferred embodiments, is provided below.

[0011] The overall concept of the technical solution provided in this application is as follows: This application provides a grid-side energy storage management method that incorporates big data analytics. It builds a frequency regulation revenue evaluation model adapted to grid assessment rules based on historical data, constructs a reward function that deducts battery losses from revenue, establishes a reinforcement learning environment, and trains the agent. Adaptive constraints are set based on battery health status, and charging and discharging commands are output through real-time rolling optimization and correction of actions, thereby achieving intelligent frequency regulation control of the energy storage system.

[0012] After introducing the basic principles of this application, various non-limiting embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0013] Examples, such as Figure 1 As shown in the embodiments of this application, a grid-side energy storage energy management method combining big data analysis is provided. The method includes: Step S1: Obtain the historical operating data set of the grid-side energy storage system. The historical operating data set includes at least historical operating condition data and historical market data.

[0014] Specifically, grid-side energy storage systems refer to electrochemical energy storage systems that are directly connected to the public power grid and provide ancillary services such as frequency regulation, peak shaving, and backup. They typically consist of multiple battery clusters, converters, EMS (Electrical Management System), and monitoring systems. Historical operational big data refers to a standardized collection of multi-source time-series data generated by the energy storage system during continuous operation over a period of time, covering multiple dimensions such as equipment operation, grid interaction, and market transactions. The data time span is usually no less than one year. Historical operating condition data refers to historical data reflecting the operating status of the energy storage system's own equipment and is the core data source for describing battery aging stress. Historical market data refers to historical data reflecting the trading rules, price signals, and grid dispatch instructions of the power ancillary services market and is the basis for calculating frequency regulation revenue.

[0015] Specifically, data acquisition terminals are deployed at various key nodes of the grid-side energy storage system, including battery management systems, converters, grid metering devices, and market trading terminals. Raw operational data is collected in real time using industrial communication protocols such as Modbus and IEC61850. After preliminary protocol conversion and data compression via an edge gateway, the data is uploaded to a local data server. Standardized preprocessing operations are performed on the uploaded raw data, including removing outliers and duplicate data caused by sensor failures or communication interruptions, filling in missing data using linear interpolation or nearest-neighbor methods, aligning all data to the same timestamp granularity, and classifying and labeling data according to data type. The preprocessed data is stored in a time-series database and a relational database, respectively. The time-series database stores high-frequency sampled operating condition data, while the relational database stores low-frequency market trading data and equipment ledger data. A standardized dataset is constructed based on the stored historical data, divided into training, validation, and test sets according to time sequence, providing unified data input for subsequent battery health status modeling and revenue calculation.

[0016] This step, by comprehensively collecting and standardizing multi-source historical operating data from grid-side energy storage systems, provides a high-quality data foundation for subsequent construction of dynamic degradation models of battery health status, calculation of expected frequency regulation revenue, and establishment of reinforcement learning environments. The sufficiently long time span and sufficiently fine sampling granularity ensure that the data can fully reflect the aging patterns of batteries under different operating conditions and the price fluctuation characteristics of the electricity market, thereby improving the prediction accuracy of subsequent models and the robustness of control strategies. At the same time, the offline training mode based on historical data avoids the risks of equipment damage and economic losses caused by directly training models on real energy storage systems, reducing the implementation cost of the technical solution.

[0017] Step S2: Based on the historical operating data, construct a dynamic degradation model of battery health status with battery aging time sequence characteristics as input and battery capacity decay increment as output.

[0018] Furthermore, such as Figure 2 As shown, step S2 includes: performing correlation analysis on the historical operating condition data to mine multi-source time-series feature combinations that are strongly correlated with the battery aging rate. The multi-source time-series feature combinations include historical cumulative throughput energy, historical average charge-discharge rate, historical battery operating temperature, and historical state of charge variation range; obtaining the historical measured battery capacity decay corresponding to the multi-source time-series feature combinations in time from the battery performance test records; and training a machine learning model with the multi-source time-series feature combinations as input and the historical measured battery capacity decay as the expected output to obtain a dynamic degradation model of battery health status.

[0019] Specifically, battery aging time-series characteristics refer to a series of characteristic parameters that change continuously over time, reflecting the magnitude and duration of aging stress on the battery. Unlike static battery rated parameters, these values ​​change dynamically with variations in battery operating conditions. Battery capacity degradation increment refers to the reduction in the actual chargeable / dischargeable capacity of the battery within a specific time period compared to the previous moment. It is a core quantitative indicator for measuring the degree of battery aging during that period, typically measured in Ah or kWh. The battery health state dynamic degradation model is a machine learning model that can predict the battery capacity degradation increment within a specific future time period based on battery history and current operating conditions. It reflects the dynamic changes in battery aging rates under different operating stresses.

[0020] Specifically, a multi-dimensional correlation analysis was performed on the preprocessed historical operating condition data. A combination of Pearson correlation coefficient and mutual information methods was used to calculate the correlation between each candidate time-series feature and the battery aging rate. A correlation coefficient threshold of 0.7 was set to select multi-source time-series feature combinations strongly correlated with the battery aging rate. These included historical cumulative throughput energy reflecting long-term battery usage intensity, historical average charge / discharge rate reflecting charge / discharge current magnitude, historical battery operating temperature reflecting the battery's operating environment temperature, and historical state of charge variation range reflecting the battery's charge / discharge depth. Monthly full charge / discharge capacity calibration data was extracted from the battery performance test records regularly conducted by the energy storage power station. The battery capacity decay increment between two adjacent calibrations was calculated, and each time stamp was aligned to the data. Each capacity decay increment is matched one-to-one with a multi-source time-series feature combination within the corresponding time period to construct a labeled dataset for model training. The matched dataset is divided into training, validation, and test sets in an 8:1:1 ratio. LSTM is selected as the basic machine learning model, using multi-source time-series feature combinations as input and the corresponding historical measured battery capacity decay increment as the expected output. The model is set with 3 hidden layers, 64 hidden layer neurons, and a learning rate of 0.001. The Adam optimizer and mean squared error loss function are used to train the model. During training, the generalization ability of the model is monitored in real time using the validation set. Training stops when the validation set loss no longer decreases for 10 consecutive epochs. The performance of the trained model is evaluated using the test set. When the model's determination coefficient R... 2 When the value is greater than 0.9, a dynamic degradation model of battery health status can be obtained, which can be used to predict the incremental battery capacity decay under different operating conditions.

[0021] Preferably, the dynamic degradation model for battery health status adopts a multi-input temporal fusion LSTM-Attention architecture, specifically designed for the long-term temporal dependence and multi-stress coupling characteristics of battery aging. The model input layer receives a 30-day, 4-dimensional temporal feature matrix, corresponding to four core aging features: historical cumulative throughput energy, historical average charge / discharge rate, historical battery operating temperature, and historical state of charge variation range. A batch normalization layer follows the input layer to eliminate differences in the dimensions of different features and accelerate training convergence. The core layer consists of two stacked bidirectional LSTM networks, each containing 64 hidden units, capable of simultaneously capturing the forward evolution and inverse dependencies of battery aging, overcoming the limitation of traditional unidirectional LSTMs in utilizing future information. A multi-head self-attention layer follows the LSTM layer, automatically highlighting the impact of key aging events such as high-rate charge / discharge and high-temperature operation by calculating the attention weights of features at different time steps, improving the model's prediction accuracy for extreme conditions. Finally, two fully connected layers perform feature fusion and mapping, outputting the predicted incremental value of battery capacity degradation within the next month.

[0022] The model construction and training follow a time-series data flow. Based on preprocessed historical operating condition data, a sliding time window method is used to generate training samples, with a window size of 30 days and a step size of 1 day. The multi-source time-series features within each window are timestamped with the measured capacity decay increment at the end of the corresponding time period, addressing the time-scale mismatch between low-frequency capacity calibration data and high-frequency operating condition data. The dataset is divided chronologically into a training set, a validation set, a middle 10% and a test set, and a final 10%, strictly prohibiting random partitioning to avoid data leakage. During training, mean squared error is used as the loss function, and the Adam optimizer is used for parameter updates. The initial learning rate is set to 0.001 and dynamically adjusted using a cosine annealing strategy. An early stopping mechanism is also introduced: training stops when the validation set loss no longer decreases for 10 consecutive epochs to prevent overfitting. The coefficient of determination R0 is used to measure the loss. 2 The performance of a model is evaluated using three metrics: mean absolute error (MAE), root mean square error (RMSE), and average absolute error (RSE). 2 ≥0.92, MAE≤0.5% of rated capacity, to ensure that the model can accurately predict the dynamic degradation process of the battery under different operating stress combinations.

[0023] Furthermore, the output layer of the battery health status dynamic degradation model is designed to regress the battery capacity decay increment. It inherits the abstract features from the front-end feature extraction network, completing dimension mapping, result output, and value range constraints. The overall structure is a multi-level fully connected, activation-based, and value-range-constrained structure, adapting to the physical characteristics of battery aging values. The front-end LSTM and attention module output a high-dimensional temporal fusion feature vector. This vector is then compressed through a dimensionality-reducing fully connected layer, fusing global information. An activation function introduces non-linear expressive power, and finally, the terminal output node outputs a single-dimensional predicted value. Simultaneously, business rules are used to limit the output range, avoiding abnormal results that violate battery physics.

[0024] The intermediate transition fully connected layer, with one hidden fully connected layer and 32 nodes, receives the high-dimensional feature vectors from the upper layer, performs secondary feature selection and dimensionality compression, and uses ReLU as the activation function to strengthen the nonlinear correlation between features and adapt to the capacity decay caused by the coupling of multiple aging features. The main output fully connected layer, a single-node fully connected layer, serves as the final output node. It has no additional activation function and directly outputs the original predicted value of the battery capacity decay increment, with units consistent with the label. Post-processing value range constraints are added based on the battery's physical characteristics. Since battery capacity only decays and does not recover under normal operating conditions, a lower limit is truncated to force the predicted value to be ≥0. Simultaneously, an upper limit threshold is set based on energy storage system operation and maintenance experience. When the value exceeds the maximum reasonable decay value, it is corrected to the threshold to avoid abnormal model predictions and ensure that the output results conform to actual operating patterns.

[0025] The sole objective of the output layer is single-value regression output, corresponding to the battery capacity decay increment within a specified time period. It does not perform multi-class or multi-task outputs and strictly matches the label format used during model training. This design aligns with the application scenario of this solution, and the output results can be directly substituted into the subsequent health loss cost calculation formula, achieving seamless integration between model predictions and upper-layer business calculations.

[0026] The dynamic degradation model of battery health constructed in this step can accurately capture the dynamic impact of different combinations of operating stresses on the battery aging rate. Compared with traditional static empirical models, it improves the accuracy of capacity decay prediction and provides a solid physical basis for the subsequent quantification of health loss costs of charging and discharging operations.

[0027] Step S3: Associate the battery health status dynamic degradation model with the battery initial health status, calculate the battery health loss cost corresponding to unit throughput energy, and calculate the expected frequency regulation revenue based on the historical market data.

[0028] Furthermore, the battery health state dynamic degradation model is correlated with the battery's initial health state to calculate the battery health loss cost per unit throughput energy. This includes: collecting the real-time operating data stream of the current grid-side energy storage system and performing feature extraction on the real-time operating data stream to generate a real-time multi-source time-series feature combination characterizing the current operating stress; inputting the real-time multi-source time-series feature combination into the battery health state dynamic degradation model to predict the dynamic battery capacity decay increment caused by executing the charge-discharge action sequence to be evaluated; calculating the unit benchmark throughput energy cost based on the battery's initial health state and the battery's rated total throughput energy; multiplying the dynamic battery capacity decay increment by a dynamic aging acceleration factor, and then multiplying it by the unit benchmark throughput energy cost to obtain the dynamic battery health loss cost corresponding to the charge-discharge action sequence to be evaluated, wherein the dynamic aging acceleration factor is jointly determined by the battery's current health state and the stress level of the charge-discharge action sequence.

[0029] Specifically, the initial health state of a battery refers to its health state when a new battery leaves the factory, usually defined as 100%. This represents the baseline state of the battery's initial rated capacity and charge / discharge performance, and serves as the benchmark for calculating the battery's life-cycle degradation. The unit benchmark throughput energy cost refers to the average depreciation cost corresponding to each kilowatt-hour of electrical energy processed over the entire life-cycle under ideal standard operating conditions. The calculation formula is the total initial investment cost of the battery system divided by the battery's rated total throughput energy. The dynamic battery capacity decay increment refers to the reduction in the actual usable capacity of the battery after executing a specific charge / discharge sequence relative to before that sequence. Its value changes dynamically with operating conditions and the battery's current health state. The dynamic aging acceleration factor is a coefficient used to quantify the amplification or reduction effect of the battery's current health state and the stress level of the charge / discharge sequence on the battery aging rate. A coefficient greater than 1 indicates an accelerated aging rate relative to standard operating conditions, while a coefficient less than 1 indicates a slower aging rate. Operating stress refers to the various external conditions and operating parameters that the battery experiences during operation, which accelerate the aging of its internal electrochemical materials. These mainly include charge / discharge rate, battery temperature, state-of-charge operating range, and cycle count. Real-time multi-source time-series feature combination refers to a set of multiple features that are strongly correlated with the battery aging rate and change continuously over time, extracted from the real-time operation data stream of the grid-side energy storage system. Specifically, these features include real-time cumulative throughput energy, real-time average charge-discharge rate, real-time battery operating temperature, and real-time state of charge variation range.

[0030] Specifically, by deploying an edge controller locally at the energy storage power station, real-time operational data streams of the grid-side energy storage system are continuously collected. The raw data is cleaned, aligned, and feature extracted to generate a real-time multi-source time-series feature combination that accurately characterizes the current operating stress experienced by the battery. This real-time multi-source time-series feature combination is input into a pre-trained dynamic degradation model of battery health status. Based on historically learned aging patterns, the model predicts the dynamic battery capacity decay increment that will result from executing a certain charge-discharge sequence to be evaluated. Simultaneously, based on the battery's initial health status at the time of manufacture and the rated total throughput energy provided by the manufacturer, combined with the initial investment cost of the battery system, a unit benchmark throughput energy cost is calculated as the benchmark value for calculating health loss cost. A dynamic aging acceleration factor, jointly determined by the battery's current health status and the stress level of the charge-discharge sequence to be evaluated, is introduced. The predicted dynamic battery capacity decay increment is multiplied by this dynamic aging acceleration factor, and then multiplied by the unit benchmark throughput energy cost to obtain the dynamic battery health loss cost corresponding to executing the charge-discharge sequence to be evaluated. This achieves accurate quantification of battery health loss under different operating conditions.

[0031] Preferably, the extracted real-time multi-source temporal features representing the current operating stress of the battery are combined and aligned with the charging and discharging action sequence to be evaluated in terms of time dimension. The charging and discharging power value of each time step in the action sequence is converted into the corresponding charging and discharging rate. Based on the current state of charge of the battery and the charging and discharging power, the range of state of charge change at each time step is recursively calculated. At the same time, the predicted value of the battery operating temperature at each time step is obtained by combining the temperature prediction model of the battery thermal management system. A complete multi-source temporal feature matrix covering the current historical feature window and the future action feature window is generated. This feature matrix is ​​input into the trained battery health state dynamic degradation model. The long-cycle temporal dependence features of battery aging are extracted through a two-layer bidirectional LSTM network. The impact of high-stress time steps such as high-rate charging and discharging and extreme SOC operation on aging is calculated and weighted by a multi-head self-attention layer. The output layer predicts the battery capacity decay increment corresponding to each decision step step by step. The predicted capacity decay increments of all time steps are accumulated to obtain the total dynamic battery capacity decay increment caused by executing the entire charging and discharging action sequence to be evaluated.

[0032] The calculation of the multi-head self-attention layer first involves taking the temporal feature matrix X output by the bidirectional LSTM layer, which has a shape of [sequence length, 64], and then combining the 64-dimensional aging feature vector at each time step with three learnable linear transformation matrices W. q W k W vThe query matrix Q, key matrix K, and value matrix V are generated through multiplication. Then, Q, K, and V are uniformly split along the feature dimension into eight independent heads, each corresponding to a feature subspace of dimension 8. Within each head, the dot product of the transposes of Q and K is calculated and scaled by √8. The attention weight matrix for each head is obtained by normalization using the softmax function. The weight matrix is ​​then multiplied by the corresponding value matrix V to obtain the attention output for each head. Finally, the outputs of the eight heads are concatenated and fused through a linear transformation layer to obtain the weighted temporal features. The weights are determined entirely based on the physical characteristics of battery aging, and W is automatically optimized during model training through backpropagation. q W k W v The parameters allow high-stress time steps, such as charge / discharge rates exceeding 0.3C, SOC exceeding the optimal range of 20%-80%, and battery temperatures exceeding 35°C, to achieve higher scores in the QK dot product calculation. As a result, these time steps are assigned 3-5 times the attention weight of the mild operating condition time steps. In extreme cases, such as high-rate charge / discharge above 0.5C, or SOC below 10% or above 90%, the weight can be increased to more than 8 times, ensuring that the model can accurately capture key events that play a dominant role in battery aging.

[0033] Among them, W q The query matrix transforms the input time-series feature vector into a query vector, used to actively retrieve information from other time steps in the sequence that are relevant to the current time step. In this application, the equivalent model problem is: which time steps in the sequence are most relevant to the battery aging state at the current time step? For example, when processing a time series containing a 0.5C high-rate charge-discharge cycle, W... q The characteristics of this high-stress time step will be converted into a query vector with a unique identifier, used to find other similar conditions in the sequence that may accelerate aging. k The key-index matrix transforms the input temporal feature vector into a key vector for matching the query vector, essentially acting as an index label for each time step in the sequence. In this application, it functions as the model's answer index, where the runtime stress features at each time step, including charge / discharge rate, temperature, and SOC, are indexed by W. k This is converted into a unique key vector. The higher the dot product similarity between the query vector and the key vector, the stronger the correlation between the runtime stress at the two time steps and battery aging. v The index matrix transforms the input time-series feature vector into a value vector, containing the original feature information that needs to be weighted and fused. In this application, it corresponds to the model's answer content, i.e., the actual contribution feature to battery aging at each time step. When the query vector has a high matching degree with a certain key vector, the corresponding value vector is assigned a higher weight and ultimately integrated into the feature representation of the current time step.

[0034] Preferably, when calculating the battery health loss cost per unit throughput energy, a dynamic battery health state degradation model is used to predict the dynamic battery capacity decay increment caused by executing the charge / discharge sequence to be evaluated. This increment is a physical quantity characterizing battery health loss. Using the battery's initial health state as a benchmark, and combining it with its rated total throughput energy over its entire lifespan, a reference quantity is calculated, namely the benchmark health state degradation equivalent per unit throughput energy. A dynamic aging acceleration factor, jointly determined by the battery's current health state and the stress level of the action sequence, is introduced. The predicted dynamic capacity decay increment is multiplied by this acceleration factor and the benchmark degradation equivalent to obtain a normalized health loss equivalent value. This value quantifies the equivalent loss degree of the action sequence on the battery health state, thus allowing for comparison and trade-off with the expected frequency modulation benefits under a unified dimension.

[0035] This step, by associating the dynamic degradation model of battery health with the initial health state and introducing a dynamic aging acceleration factor, overcomes the technical bottlenecks of traditional static health loss cost calculation methods, which suffer from low accuracy and inability to reflect the impact of operating conditions and health status. It achieves dynamic and accurate quantification of battery health loss caused by any charging and discharging sequence, providing a reliable quantitative basis for the subsequent construction of reinforcement learning reward functions. This enables the agent to fully weigh frequency regulation benefits against battery health loss during the decision-making process, thereby effectively slowing down battery aging while ensuring the quality of frequency regulation services and significantly improving the net benefits of the energy storage system throughout its entire life cycle.

[0036] Furthermore, calculating the expected frequency regulation revenue based on the historical market data includes: constructing a revenue assessment model that considers the frequency regulation performance compensation coefficient based on the historical market data, wherein the revenue assessment model is associated with the call power and the accuracy coefficient of historical frequency regulation actions; inputting the charging and discharging action command to be executed and its historical performance evaluation indicators into the revenue assessment model, and outputting the expected frequency regulation revenue.

[0037] Specifically, historical market data refers to the set of historical data related to frequency regulation service settlement in the power ancillary services market. This mainly includes historical frequency regulation benchmark prices, compensation coefficients corresponding to different performance levels, historical frequency regulation call records of the power grid, historical settlement revenue and performance assessment data of each market participant, etc., forming the basis for constructing a revenue assessment model. The frequency regulation performance compensation coefficient is a quantitative indicator used in the power market to measure the quality of frequency regulation services. It is usually calculated comprehensively from three core dimensions: response speed, regulation accuracy, and regulation rate. A higher coefficient indicates better frequency regulation service quality and higher compensation revenue; its typical range is 0.8-1.5. Call power refers to the frequency regulation power command value issued by the power grid dispatching agency to the energy storage system, or the charging and discharging power value that the energy storage system intends to execute, measured in MW. It is a core parameter for calculating basic frequency regulation revenue. The accuracy coefficient of historical frequency regulation actions is a statistical indicator characterizing the accuracy of the energy storage system's execution of frequency regulation commands over a past period. It is calculated by weighting historical response time deviation, regulation power deviation, and regulation completion rate data, and is used to predict the frequency regulation performance of the current action. Benefit Assessment Model: A mathematical model built based on historical market data fitting, used to predict the frequency regulation benefits that can be obtained by performing the proposed charging and discharging action, based on the parameters of the proposed action and historical performance indicators. Historical performance evaluation indicators refer to the statistical values ​​of various performance parameters of the energy storage system in the past when performing frequency regulation tasks, including average response time, average regulation accuracy, average regulation rate, and average regulation deviation, etc., which are the basis for calculating the expected frequency regulation performance compensation coefficient.

[0038] Specifically, historical market data is collected from the power market trading platform and dispatch system. After cleaning, deduplication, and standardization, the quantitative relationship between frequency regulation revenue and dispatch power, execution time, and frequency regulation performance is analyzed. A revenue assessment model considering the frequency regulation performance compensation coefficient is constructed. This model decomposes the total frequency regulation revenue into two parts: basic revenue and performance compensation revenue. The basic revenue is proportional to the dispatch power and execution time, while the performance compensation revenue is the product of the basic revenue and the frequency regulation performance compensation coefficient. The model explicitly links the magnitude of the dispatch power and the accuracy coefficient of historical frequency regulation actions to the final revenue. The charging and discharging power values ​​and the execution time of each time step in the instruction to be executed for charging and discharging actions are extracted. At the same time, the historical performance evaluation indicators of the energy storage system in the most recent statistical period are retrieved, including average response time, average regulation accuracy, and average regulation rate. The expected frequency regulation performance compensation coefficient for this action is calculated through a preset performance scoring formula. The dispatch power, execution time, and expected frequency regulation performance compensation coefficient are input into the constructed revenue assessment model, and the expected frequency regulation revenue for each decision step is calculated step by step. The summation yields the total expected frequency regulation revenue for executing the entire charging and discharging action sequence.

[0039] Preferably, the revenue assessment model adopts a hierarchical, modular, multi-input, single-output regression structure, consisting of four parts: an input layer, a feature fusion layer, a dual-branch core computation layer, and an output layer. The input layer receives four types of core features: first, the time-step power values ​​and execution durations of the proposed charging / discharging sequence; second, the historical performance evaluation indicators of the energy storage system over the past 30 days, including average response time, regulation power deviation rate, and regulation completion rate; third, the real-time benchmark price and supply-demand coefficient of the frequency regulation market on that day; and fourth, the statistical values ​​of the accuracy coefficients of historical frequency regulation actions. The feature fusion layer standardizes the input features, and through weighted summation, merges multiple historical performance indicators into a single comprehensive performance score, while extracting time-dimensional features to distinguish revenue differences during peak and off-peak periods. The dual-branch core computation layer is the core of the model. The first branch calculates the basic frequency modulation (FM) revenue, with the formula: Basic Revenue = Σ(Call Power × Execution Duration × Real-Time Benchmark Price). The second branch calculates the performance compensation revenue, with the formula: Performance Compensation Revenue = Basic Revenue × FM Performance Compensation Coefficient. The FM performance compensation coefficient is calculated using a pre-fitted nonlinear mapping function, combining the comprehensive performance score and historical accuracy coefficient. A power range correction factor is introduced to provide additional weighting for performance compensation at high-rate adjustments above 0.5C. The output layer sums the results from both branches, outputting the total expected FM revenue from executing the entire action sequence.

[0040] The model construction and training employ a hybrid mode combining offline supervised training and online incremental updates. During the offline training phase, at least 12 months of historical market settlement data and corresponding energy storage system operation data are collected to construct a training dataset containing over 100,000 samples. Outlier removal, missing value imputation, and time alignment preprocessing are performed on the data, with the actual settlement revenue of each sample used as the label. Model initialization uses linear regression as the basic framework, superimposed with polynomial nonlinear terms to fit the nonlinear relationship between the performance compensation coefficient and revenue, while L2 regularization is introduced to prevent overfitting. The training process uses mean squared error as the loss function, employs gradient descent to optimize model parameters, and adjusts hyperparameters through 5-fold cross-validation to ensure the model's mean absolute percentage error on the test set is below 5%. In the online incremental update phase, the previous day's actual settlement data and action execution data are automatically acquired daily to form incremental training samples. Incremental training is performed on the model on the local edge controller, updating the mapping function parameters of the performance compensation coefficient. Full parameter fine-tuning is performed every 7 days, enabling the model to continuously adapt to dynamic adjustments in market rules, equipment performance degradation, and grid frequency regulation requirements, ensuring the long-term accuracy of revenue prediction.

[0041] This step, by constructing a revenue assessment model that aligns with the actual settlement rules of the current power ancillary services market, correlates the frequency regulation performance compensation coefficient with the historical frequency regulation accuracy coefficient of the energy storage system. This solves the technical problems in existing technologies where frequency regulation revenue prediction only considers the base power revenue, ignores performance compensation leading to low prediction accuracy, and fails to guide the agent to optimize frequency regulation quality. It can accurately quantify the expected frequency regulation revenue under different charging and discharging actions and different performance levels, providing a reliable quantitative basis for the construction of reinforcement learning reward functions. It can guide the agent to actively optimize the response speed and adjustment accuracy of charging and discharging actions during the decision-making process, achieving higher frequency regulation performance compensation while better meeting the frequency regulation needs of the power grid.

[0042] Step S4: Construct a reinforcement learning environment, wherein the reward function of the reinforcement learning environment is the difference between the expected frequency modulation revenue and the battery health deterioration cost, and train the agent in the reinforcement learning environment using the historical running large dataset.

[0043] Furthermore, constructing a reinforcement learning environment includes: defining the state space of the agent, which includes the grid frequency regulation demand signal, the state of charge of the energy storage system, and the average temperature of the battery pack; defining the action space of the agent, which is a set of charging and discharging power values ​​that the grid-side energy storage system is allowed to execute in the current state; and defining the reward function of the reinforcement learning environment, which, in each decision step, obtains an immediate reward value by the difference between the expected frequency regulation revenue calculated based on the historical market data and the battery health loss cost calculated based on the historical operating condition data.

[0044] Specifically, the reinforcement learning environment is a virtual simulation environment that simulates the dynamic interaction between the grid-side energy storage system and the grid, market, and batteries. It serves as the carrier for the agent to learn through trial and error and optimize its strategy, outputting the state and corresponding reward value for the next moment based on the agent's actions. The state space refers to the set of all environmental state variables that the agent can observe during decision-making. In this scheme, three variables that best reflect the core characteristics of system operation are selected to constitute all the input information for the agent's decision-making. The grid frequency regulation demand signal refers to the power regulation command issued by the grid dispatching agency to the energy storage system in real time. A positive value indicates that the energy storage system needs to discharge and supply power to the grid, while a negative value indicates that the energy storage system needs to charge and absorb power from the grid. The unit is MW. The action space refers to the set of all legal actions that the agent can choose to execute in the current state. In this scheme, it is specifically the set of charging and discharging power values ​​allowed to be output by the energy storage system. The reward function is the core mechanism of reinforcement learning, used to quantitatively evaluate the quality of the agent's execution of a certain action. It is the only signal guiding the agent to learn the optimal strategy. The immediate reward value refers to the reward feedback obtained by the agent immediately after executing an action at a certain decision step, used to update the agent's policy network parameters.

[0045] Specifically, the core objective of this step is to transform the multi-objective optimization problem of grid-side energy storage frequency regulation energy management into a standard Markov decision process. The state space of the agent is defined, and three core state variables are selected: grid frequency regulation demand signal, energy storage system state of charge (SOC), and battery pack average temperature. The grid frequency regulation demand signal comes from real-time commands from the grid dispatch system, while the SOC and battery pack average temperature come from real-time monitoring data of the energy storage system. These three variables together constitute the agent's complete observation of the current system operating state. The action space of the agent is then defined, based on the rated power range of the energy storage converter, the battery's charge / discharge safety limits, and the current SOC's potential for... Using energy, a set of charge and discharge power values ​​that are allowed to be executed at the current moment are determined, where positive values ​​represent discharge power and negative values ​​represent charging power. The action space will be dynamically adjusted according to changes in the system state. A reward function for the reinforcement learning environment is defined. Within each decision step, the benefit evaluation model constructed in step S3 is called to calculate the expected frequency modulation benefit that can be obtained by executing the currently selected action. At the same time, the battery health state dynamic degradation model is called to calculate the battery health loss cost caused by executing the action. The difference between the expected frequency modulation benefit and the battery health loss cost is used as the immediate reward value for that decision step. The environment will update the state at the next moment according to the action finally selected by the agent.

[0046] Preferably, the reinforcement learning environment adopts a high-fidelity modular simulation architecture, consisting of five core modules: a power grid state simulator, an energy storage system physical simulator, a market revenue calculator, a dynamic constraint manager, and a state transition engine. Its mathematical model can be represented as a Markov decision process. Where S is the state space, A is the action space, P is the state transition probability, and R is the reward function. The discount factor is used. The power grid state simulator trains a generative adversarial network (GAN) based on historical frequency regulation demand data, which can simulate the fluctuation patterns of power grid frequency regulation demand under different seasons and different renewable energy penetration rates. The energy storage system physical simulator integrates the aforementioned battery health state dynamic degradation model and battery thermodynamic model, which can accurately simulate the impact of charging and discharging actions on battery SOC, temperature, and health state. The market revenue calculator has a built-in revenue evaluation model to calculate the frequency regulation revenue corresponding to different actions in real time. The dynamic constraint manager dynamically updates the upper and lower bounds of the action space according to the current health state and operating parameters of the battery. The state transition engine is responsible for integrating the outputs of each module, calculating the state at the next moment after the action is executed, and returning the instant reward. When building the environment, the physical parameters of each module are calibrated based on a large dataset of at least one year of historical operation, so that the state deviation between the simulation environment and the actual system is less than 3%. Random noise is injected to simulate the uncertainty of the power grid and the market, thereby enhancing the robustness of the environment.

[0047] Preferably, the agent adopts an actor-critic architecture based on proximal policy optimization, consisting of a policy network, a value network, and a multi-step lookahead planner. The policy network is a 3-layer fully connected neural network. Its input is a 3-dimensional state vector, including grid frequency regulation demand, SOC, and battery temperature. The output is the Gaussian distribution mean and variance of the continuous charge and discharge power values. The Tanh activation function is used to restrict the output to the [-1,1] interval, and then it is mapped to the actual power range. The value network is also a 3-layer fully connected neural network. Its input is the same as that of the policy network, and its output is the value estimate of the current state, which is used to evaluate the merits of the policy. The multi-step lookahead planner, combined with a high-fidelity simulation environment, can generate candidate action sequences covering 12 decision steps in the future. The training employs a two-stage approach combining offline pre-training and online fine-tuning: In the first stage, offline pre-training is performed in a high-fidelity simulation environment using a large historical dataset. The network parameters are updated using a pruned PPO loss function. An experience replay buffer is introduced during training to store high-quality trajectories and improve sample utilization until the average cumulative reward of the policy on the validation set converges. In the second stage, the pre-trained agent is deployed to a semi-physical simulation platform for online fine-tuning. Through limited interaction with the actual system, the policy is further optimized to adapt to subtle differences in the real environment, ultimately achieving the goal of maximizing net profit over the entire lifecycle.

[0048] This step, by scientifically defining the state space, action space, and core reward function, unifies the two conflicting objectives of maximizing frequency regulation revenue and minimizing battery health loss into a single net revenue optimization objective. This solves the technical problem of dynamic trade-offs in multi-objective optimization in existing technologies. In particular, by directly using the precisely quantified expected frequency regulation revenue and battery health loss cost in step S3 as inputs to the reward function, the optimization objective of the intelligent agent is completely consistent with the economic benefit objective of the energy storage system throughout its entire life cycle. This enables the agent to automatically learn the optimal charging and discharging strategy under different grid demands, market prices, and battery health states.

[0049] Furthermore, the agent in the reinforcement learning environment is trained using the historical operational dataset, including: constructing a high-fidelity simulation environment containing the state evolution of the power grid, market, and equipment based on the historical operational dataset; initializing the policy network parameters of the agent in the high-fidelity simulation environment, and using the agent to generate candidate charge-discharge power plan sequences covering multiple future decision steps based on the current state observation data; inputting the candidate charge-discharge power plan sequences into the high-fidelity simulation environment for forward simulation, predicting the future state evolution path after executing the candidate charge-discharge power plan sequences, and calculating the corresponding cumulative reward value based on the future state evolution path; calculating the gradient update amount of the agent's policy network based on the cumulative reward value, and performing backpropagation to update the policy network parameters of the agent, iterating continuously until the policy performance of the agent meets the preset convergence condition, thus completing the training of the agent.

[0050] Specifically, the high-fidelity simulation environment is a virtual simulation platform built upon historical operational big data. It accurately reproduces fluctuations in grid frequency regulation demand, market price changes, energy storage device state evolution, and battery aging patterns. It comprises four core subsystems: a grid dispatch simulation module, a market price simulation module, an energy storage device physical model, and a battery health state dynamic degradation model. The policy network refers to the core decision-making network of the reinforcement learning agent, typically employing a deep neural network structure. Its input is the current state observation data, and its output is the probability distribution of candidate charge / discharge power plan sequences or the direct output of the optimal action sequence. Its parameters are continuously optimized through training. The candidate charge / discharge power plan sequence refers to a continuous set of charge / discharge power instructions generated by the agent based on the current state, covering multiple future decision steps. It is the object of forward simulation evaluation, typically containing 10-30 future decision steps, corresponding to actions over 20-60 seconds. Forward simulation refers to the process of simulating the execution of the candidate charge / discharge power plan sequence in the high-fidelity simulation environment, progressively calculating the state changes and reward values ​​at each time step, used to predict the long-term impact of the action sequence. The cumulative reward value refers to the weighted sum of the instantaneous reward values ​​of all decision steps during the execution of the entire candidate charge / discharge power plan sequence, reflecting the long-term comprehensive benefits of the action sequence. The gradient update amount refers to the direction and magnitude of the policy network parameter adjustment calculated based on the cumulative reward value, used to optimize the network parameters through the backpropagation algorithm, making the agent tend to choose action sequences with higher cumulative rewards.

[0051] Specifically, a high-fidelity simulation environment encompassing the evolution of the power grid, market, and equipment states is constructed based on a large historical operational dataset. This environment can accurately output the next-moment power grid state, energy storage SOC, battery temperature, and battery health status based on the input charge / discharge power command, and calculate the immediate reward value using a revenue assessment model and a health loss cost model. The agent's policy network parameters are initialized using the Xavier uniform initialization method. The policy network employs a three-layer fully connected neural network structure, with 4 nodes in the input layer corresponding to the four variables in the state space, 128 nodes in the hidden layer, and 20 nodes in the output layer, corresponding to a candidate charge / discharge power plan sequence covering the next 20 decision steps. In each training iteration cycle, the agent generates a sequence covering the next 20 decision steps based on the current state observation data returned by the high-fidelity simulation environment through the policy network. A candidate charge / discharge power plan sequence with a step size is generated. This candidate sequence is then input into a high-fidelity simulation environment for forward simulation. Each action in the sequence is simulated, and the states of the power grid, market, and equipment are updated step by step to obtain a complete future state evolution path. The instantaneous reward value of each decision step is accumulated, and the cumulative reward value corresponding to the candidate sequence is calculated. A near-end policy optimization algorithm is used to calculate the gradient update amount of the agent's policy network based on the cumulative reward value. The policy network parameters are updated through the backpropagation algorithm, while a pruning factor is introduced to limit the parameter update amplitude to ensure training stability. The process of generating candidate sequences, forward simulation, calculating cumulative rewards, and updating parameters is repeated until the average cumulative reward fluctuation of the agent within 100 consecutive training cycles is less than 5% and the net benefit index of the entire life cycle meets the preset requirements, thus completing the training of the agent.

[0052] The preferred method for constructing a high-fidelity simulation environment encompassing the state evolution of the power grid, market, and equipment based on historical operational datasets is as follows: The historical operational dataset is multi-dimensionally split and timestamped for labeling, extracting the following time-series datasets: power grid AGC frequency regulation demand, frequency regulation market price, energy storage device charging / discharging power and state response, and battery operating conditions and capacity degradation. These serve as dedicated training data for four core subsystems. A power grid dispatch simulation module is constructed, learning from the power grid AGC frequency regulation demand time-series dataset the fluctuation patterns, step change characteristics, and power regulation amplitude distribution under different seasons, time periods, and power grid load levels. A time-series generative adversarial network is used to train and generate a stochastic frequency regulation demand sequence capable of reproducing real statistical characteristics. Then, a market price simulation module is constructed, mining the coupling relationship between price and frequency regulation demand intensity, power grid supply-demand gap, and time period type from the frequency regulation market price time-series dataset. A conditional long short-term memory network is used to train and generate a market price that dynamically links with the power grid frequency regulation demand. The system first generates a lattice sequence; then, it constructs a physical model of the energy storage device, fits the nonlinear mapping relationship between charge / discharge power and state of charge, battery pack temperature, and individual cell voltage from the energy storage device's charge / discharge power and state response dataset, and establishes a semi-physical, semi-data-driven model that includes internal resistance characteristics, thermal conduction characteristics, and charge / discharge efficiency characteristics. This model can accurately output the SOC, temperature, and voltage status at the next moment based on the input charge / discharge power command and initial state. It then integrates a pre-trained dynamic degradation model of battery health status, enabling it to predict the dynamic degradation process of battery capacity based on the real-time operating stress parameters output by the energy storage device's physical model. Finally, it integrates the four core subsystems through a unified time synchronization engine, establishes a state interaction mechanism between the subsystems, and performs joint calibration using complete historical operating segments. It adjusts the parameter weights of each subsystem to ensure that the average absolute percentage error between the grid state, energy storage device state, and battery health status output by the simulated environment and the real historical data is less than 5%, ultimately forming a high-fidelity simulation environment that can accurately reproduce real operating scenarios.

[0053] The specific basis for adjusting the parameter weights of each subsystem is the error contribution of each subsystem output to the corresponding measured data in the complete historical operation segments, as well as the influence weight of each subsystem on the decision-making of the reinforcement learning agent. The basic weight of the grid frequency regulation demand and market price subsystem is set to 0.25, the basic weight of the energy storage equipment physical model is set to 0.3, and the basic weight of the battery health status dynamic degradation model is set to 0.2. Simultaneously, a time decay factor is introduced to ensure that the weight of historical data from the past three months reaches 60% to reflect the time-varying characteristics of equipment performance. The specific method involves first selecting 100 typical complete historical operation segments covering morning peak, evening peak, flat periods, low periods, and extreme high and low temperature conditions as the calibration benchmark set, and then running each of the four subsystems to obtain their respective simulations. The output sequence is used to calculate the mean absolute percentage error and root mean square error of each subsystem's output compared to the measured data. The variance decomposition method is used to quantify the contribution ratio of each subsystem's error to the total error of the overall environmental output. The basic weights are multiplied by the error contribution ratios to obtain the dynamic adjustment weights of each subsystem. Then, with minimizing the weighted total error of the overall environmental output as the objective function, the particle swarm optimization algorithm is used to globally optimize the internal parameters of each subsystem. The error contribution ratio is recalculated and the dynamic adjustment weights are updated every 10 iterations until the overall weighted total error is less than 5% and the individual errors of each subsystem meet the preset thresholds. The results are verified using 50 independent historical segments that were not calibrated to ensure that the adjusted parameter weights have good generalization ability under different operating conditions.

[0054] This step employs a training method based on multi-step forward simulation and cumulative rewards, which solves the problem that traditional single-step reinforcement learning training methods are prone to causing agents to be short-sighted and ignore the long-term cumulative losses of charging and discharging actions. This enables agents to make decisions from the perspective of maximizing net benefits throughout their entire lifecycle. At the same time, the high-fidelity simulation environment built based on historical operational big data ensures a high degree of consistency between the training scenario and the real operational scenario. The trained strategy can be directly deployed and applied without additional debugging, which greatly reduces the implementation cost and risk of the technical solution.

[0055] Step S5: Based on the real-time grid status, real-time energy storage status, and real-time market data, drive the trained agent to generate charging and discharging action commands, execute the charging and discharging action commands, and control the operation of the grid-side energy storage system.

[0056] Furthermore, based on real-time grid status, real-time energy storage status, and real-time market data, the trained agent is driven to generate charging and discharging action commands, including: fusing the real-time grid status, real-time energy storage status, and real-time market data with short-term forecast information to construct a complete state representation defined in the agent's state space; inputting the complete state representation into the trained agent, which then outputs a preliminarily optimized future multi-step charging and discharging power plan sequence; and performing real-time rolling optimization on the preliminarily optimized charging and discharging power plan sequence to generate the final executable charging and discharging action commands.

[0057] Specifically, real-time grid status refers to the core parameters of grid operation at the current moment, including grid AGC frequency regulation demand signals, grid frequency deviation, regional control deviation, and grid safety constraints. This serves as the direct basis for the energy storage system to provide frequency regulation services. Real-time energy storage status refers to the operating parameters of the energy storage system itself at the current moment, including the average state of charge of the battery pack, the highest / lowest voltage of individual battery cells, the average temperature of the battery pack, the current health status of the batteries, and the operating status of the converter. This data is collected in real-time by the battery management system. Real-time market data refers to the latest transaction data in the power ancillary services market at the current moment, including the latest frequency regulation service clearing price, market supply and demand, and updated information on grid frequency regulation performance assessment rules. The update frequency is consistent with the market clearing cycle. Complete state representation refers to a four-dimensional state vector that perfectly matches the state space definition of the reinforcement learning agent after fusing real-time data with short-term forecast information. This is the sole input for the agent's decision-making and includes four core variables: grid frequency regulation demand, energy storage SOC, battery temperature, and market price.

[0058] Specifically, by deploying an edge controller locally at the energy storage power station, real-time grid status, real-time energy storage status, and real-time market data are synchronously collected at a sampling frequency of 1 second. The real-time data and prediction information are timestamped and normalized to construct a complete state representation that perfectly matches the state space definition of the reinforcement learning agent. The current battery health status, as assessed by the battery health status dynamic degradation model, is acquired in real time. The upper and lower bounds of the agent's action space are adjusted proportionally according to the health status to generate an updated action space boundary as a hard constraint. The complete state representation is input into the trained reinforcement learning agent, which outputs a preliminary optimized charge and discharge power plan sequence covering the next 24 decision steps. This sequence then enters a real-time rolling optimization loop. At each decision-making moment, the system receives the latest operating condition data and ultra-short-term forecast data. Using the first action of the planned sequence generated by the agent as the initial value, it performs local re-optimization within a rolling optimization framework with a time-domain of four decision steps. Simultaneously, it applies real-time calculated battery internal resistance constraints, thermal limit constraints, and SOC safety range constraints to solve for candidate charging and discharging action commands that satisfy all the latest boundary conditions. The candidate commands are then finally verified by a hard constraint verification module. If the command satisfies all constraints, it is directly sent to the energy storage converter for execution. If it does not satisfy all constraints, it is automatically adjusted to the most recent valid command value to control the grid-side energy storage system to complete the corresponding charging and discharging action. After the action is completed, the system state is updated, and the system enters the next decision cycle.

[0059] Preferably, the method for obtaining short-term forecast information is as follows: For ultra-short-term power grid frequency regulation demand forecasting, a similarity-day matching weighted method is used. From the historical database, 3-5 historical days with the highest similarity to the current date in five dimensions—season type, weekday attribute, time period division, overall power grid load level, and proportion of renewable energy output—are selected. The 1-minute resolution frequency regulation demand sequence corresponding to these historical days for the next hour is extracted. A weighted coefficient is calculated based on the comprehensive similarity between each historical day and the current day; the higher the similarity, the greater the weight. The frequency regulation demand forecast sequence for the next hour is obtained through weighted averaging. For frequency regulation market price forecasting, a historical price index decaying moving average method is used to extract the 1-minute resolution moving average of the same time period over the past 14 days. The market clearing price data is calculated using a 5-minute resolution. Weights are assigned based on time proximity, with the most recent day having a weight of 0.3, the previous two days 0.2, the previous 3-7 days 0.06 each, and the previous 8-14 days 0.02 each. The weighted average is then used as the predicted market price for the next 15 minutes. For battery temperature change prediction, based on the battery thermal balance equation, and combined with the current measured average battery pack temperature, real-time charging and discharging power, ambient temperature, and the cooling / heating power parameters of the thermal management system, the difference between the battery's heat generation and dissipation per unit time is calculated. This is then used to calculate the battery temperature change trend for the next 10 minutes. These three prediction results are then integrated to form a complete short-term forecast.

[0060] The specific steps for calculating the weighted coefficient based on the comprehensive similarity between each historical day and the current day are as follows: Based on historical statistical analysis, determine the degree of influence of each matching feature dimension on the fluctuation of power grid frequency regulation demand. Assign differentiated basic weights to five core features: seasonal type, weekday attribute, time period division, overall power grid load level, and regional renewable energy output ratio, respectively, at 0.15, 0.10, 0.30, 0.25, and 0.20, ensuring that features with a greater impact on frequency regulation demand have higher weights. Calculate the normalized Euclidean distance between the current day and each candidate historical day in each feature dimension, and substitute the distance values ​​into the formula. Convert to single-feature similarity of the corresponding dimension ,in Let be the normalized distance of the j-th historical day in the i-th feature dimension. The value ranges from 0 to 1, with higher values ​​indicating higher similarity. The five single-feature similarities for each historical day are multiplied by the corresponding feature's base weight and summed to obtain the overall similarity between that historical day and the current day. ,in The base weight is set for the i-th feature; subsequently, the reciprocal of the comprehensive similarity of all 3-5 similar days selected is used to obtain the original weight value. Then, all the original weight values ​​are normalized, that is... , where n is the number of similar days, and finally the weighting coefficient corresponding to each similar day is obtained, and the sum of all weighting coefficients is 1, to ensure the rationality and accuracy of the weighted average result.

[0061] This step employs a hybrid control architecture that combines offline training with global optimization and online rolling local re-optimization. It balances the global optimization of offline training with the real-time adaptability of online optimization. At the same time, it introduces adaptive action space constraints based on health status and real-time physical safety constraints, which ensures both the economy of charging and discharging commands and the safety of equipment operation. It can effectively adapt to the rapid fluctuations in grid frequency regulation demand and the time-varying characteristics of energy storage equipment performance.

[0062] Furthermore, real-time rolling optimization is performed on the initially optimized charge and discharge power plan sequence, including: at each decision moment, receiving the latest operating condition data, including ultra-short-term grid status and market forecast data; based on the first action of the charge and discharge power plan sequence generated by the agent, combined with the latest operating condition data, performing local re-optimization within a time-shortened rolling optimization framework, and applying real-time calculated battery internal resistance and thermal limit constraints, and outputting an instantaneous charge and discharge action command that satisfies the latest boundary conditions.

[0063] Specifically, the time-domain rolling optimization framework refers to an optimization framework that significantly shortens the optimization time domain from the multi-step plan output by the agent to 1-4 decision steps, sacrificing some global optimality for real-time online computation. Local re-optimization refers to re-optimizing the actions for a limited number of future steps within the time-domain, using the initial plan generated by the agent as the initial value and incorporating the latest operating data. This only adjusts the actions for the current and a few future steps without changing the overall strategy direction. Battery internal resistance constraint refers to the maximum allowable charge / discharge power constraint calculated in real-time based on the battery's current health state and temperature. Battery internal resistance increases as SOH and temperature decrease; excessive charge / discharge power can lead to overcurrent and overheating. Thermal limit constraint refers to the maximum allowable charge / discharge power constraint set to prevent battery overheating damage. When the battery temperature approaches the safe upper limit, the charge / discharge power limit is automatically reduced.

[0064] Specifically, at each decision-making moment synchronized with the grid AGC command cycle, the edge controller receives the latest operating condition data in real time, including the current grid AGC frequency regulation demand, grid frequency deviation, ultra-short-term grid frequency regulation demand forecast for the next 16 seconds, the latest 15-minute frequency regulation market price and the price forecast for the next 15 minutes, the real-time average temperature of the battery pack and the temperature forecast for the next 10 minutes; the first action of the preliminary optimized charge and discharge power plan sequence generated by the agent at the previous decision-making moment is extracted as the initial value for this re-optimization, and a rolling optimization framework with a time domain shortened to 4 decision steps is constructed. The optimization objective is set to maximize the cumulative reward value within this shortened time domain, i.e., the sum of the expected frequency regulation revenue of the next 4 steps minus the battery health loss cost; during the optimization process, Dynamic constraints are applied, including battery internal resistance constraints calculated based on the current battery SOH and temperature, where the maximum charge / discharge power = allowable fluctuation range of battery terminal voltage / real-time battery internal resistance; thermal limit constraints calculated based on the current battery temperature and the cooling capacity of the thermal management system, where the maximum charge / discharge power decreases by 5% for every 1°C increase when the battery temperature is >35°C; and a SOC safety range constraint, maintaining the SOC between 0.2 and 0.8. A fast quadratic programming algorithm is used to solve this constrained optimization problem, obtaining the optimal action sequence in the shortened time domain. Only the first action of this optimal sequence is selected as the immediate charge / discharge action command for the current decision step, while the remaining three actions are retained as the initial reference for re-optimization at the next decision moment, entering the next decision cycle.

[0065] Preferably, during real-time rolling optimization, the time domain of the rolling optimization is determined to contain N consecutive decision steps, denoted as t, t+1, ..., t+N-1. The optimization objective is to maximize the cumulative reward within this time domain. First, the instantaneous reward for the k-th decision step is defined. ,in For the expected frequency modulation revenue at step k, Let the cost of battery health degradation at step k be the objective function; based on this, construct a maximization objective function: ; To reduce the total cumulative reward in the time domain, inequality constraints were subsequently added to form a complete optimization model. The constraints are as follows: the upper limit of single-step charge / discharge power calculated from the real-time battery internal resistance. The upper limit of power based on the thermal limit determined by the battery temperature. The minimum of the two values ​​is taken as the power hard constraint. At the same time, energy storage state of charge constraints are added. System rated power constraints , Let be the charging and discharging power to be solved in the k-th step; the entire objective function takes the initial power sequence output by the agent as the starting point of the iteration, seeks optimization within all constraint boundaries, and finally obtains the optimal power sequence in the shortened time domain.

[0066] For example, in a 10MW / 20MWh grid-side lithium iron phosphate energy storage power station, the first action of the preliminary optimized charge / discharge power plan sequence generated by the agent at the previous decision moment was a +6MW discharge, corresponding to a 4-step time-domain plan of [+6MW, +6MW, +5MW, +5MW]. At the current decision moment, the latest operating condition data received shows that the grid frequency regulation demand has suddenly increased from +5MW to +7MW and will remain at around +7MW for the next 16 seconds, while the real-time temperature of the battery pack has increased from 28℃ to 36℃. Based on the battery internal resistance constraint, the current maximum allowable discharge power is 8MW. According to the thermal limit constraint, the maximum allowable discharge power at a battery temperature of 36℃ is 95% of the rated power, i.e., 9.5MW. The smaller value of 8MW is taken as the power limit. With +6MW as the initial value, the optimization is carried out in 4 steps within the time domain. The optimization objective is to maximize the cumulative reward in the next 16 seconds. The optimal action sequence obtained is [+7MW, +7MW, +6MW, +6MW]. The first action +7MW is selected as the immediate charge and discharge action command and sent to the PCS for execution. Compared with the originally planned +6MW, it better responds to the sudden increase in grid frequency regulation demand, while satisfying all safety constraints.

[0067] This step, through a time-domain rolling optimization framework, effectively solves the problem that offline-trained reinforcement learning strategies cannot adapt to sudden changes in real-time operating conditions. While ensuring the optimality of the global strategy, it significantly improves the response capability to rapid fluctuations in grid frequency regulation demand. At the same time, the real-time applied dynamic constraints on battery internal resistance and thermal limits can automatically adjust the power limits according to the current health status and operating temperature of the battery, which not only avoids safety risks such as battery overcurrent and overheating, but also makes full use of the battery's available capacity.

[0068] Furthermore, the method also includes adaptive action space constraints based on health status: acquiring the current health status of the battery as evaluated by the battery health status dynamic degradation model in real time; dynamically adjusting the upper and lower bounds of the agent's action space according to the current health status of the battery to generate an updated action space boundary, wherein, when the health status declines, the charging and discharging power limits and the state of charge operating range are tightened proportionally; and using the updated action space boundary as a hard constraint to drive the agent to generate charging and discharging action commands.

[0069] Specifically, adaptive action space constraints are a mechanism that dynamically adjusts the range of actions an agent can perform based on the current health state of the battery. The core principle is to gradually reduce operational stress during battery aging to avoid accelerating battery degradation. The upper and lower bounds of the action space refer to the maximum discharge and charging power that the agent can output, as well as the upper and lower limits of the allowable state of charge range. Proportional tightening means that the adjustment magnitude of the action space boundaries maintains a linear proportional relationship with the decline in battery health, ensuring that the relative operational stress experienced by the battery remains essentially consistent throughout its entire lifespan. Hard constraints refer to the inviolable boundary conditions that the agent must strictly adhere to when generating actions; any actions exceeding these boundaries will be directly filtered or corrected to boundary values.

[0070] Specifically, a real-time data interface is established with the battery health status dynamic degradation model to obtain the current battery health status assessment value every 24 hours. This assessment value comprehensively considers factors such as battery capacity decay, internal resistance increase, and cycle count. Based on a preset linear adjustment rule, the action space boundary is dynamically calculated and updated according to the current battery health status. The calculation formula for the charge / discharge power limit is: Current power limit = Rated power × (Current SOH / 100%). The calculation formula for the SOC operating range is: Current SOC lower limit = Initial SOC lower limit + (0.1 × (100% - Current SOH) / 20%), Current SOC upper limit = Initial SOC upper limit - (0.1 × (100% - Current SOH) / 20%). The algorithm (OH) / 20%) achieves a linear decrease in the charge / discharge power limit from the rated value to 80% of the rated value when the battery health status decreases from 100% to 80%. The SOC operating range is also tightened proportionally from the initial 0.1-0.9 to 0.2-0.8. The updated action space boundary is embedded as a hard constraint into the agent's decision-making process. Before the agent generates charge / discharge action commands, all illegal actions exceeding the power limit and SOC range are filtered out. Simultaneously, the updated boundary is added as a constraint to the optimization model during real-time rolling optimization. This drives the agent to generate optimal charge / discharge action commands within the updated legal action space, ensuring that all output commands meet the safe operation requirements of the battery under its current health status.

[0071] This step introduces adaptive action space constraints based on health status, which achieves dynamic matching between battery operating stress and health status. This solves the problem that traditional fixed action space constraints still apply high operating stress after battery aging, leading to accelerated battery degradation, and can extend the actual service life of the battery.

[0072] Furthermore, the method is executed on an edge controller deployed locally on the grid-side energy storage system.

[0073] Specifically, at the grid-side energy storage power station site, an edge controller meeting industrial-grade protection standards is deployed. A real-time operating system and software environment adapted to the frequency regulation control requirements of energy storage are built. All developed core functional modules, including the real-time data acquisition module, state representation fusion module, trained reinforcement learning agent module, revenue evaluation module considering performance compensation, battery health loss calculation module, real-time rolling optimization module, adaptive action space constraint module, and charge / discharge command issuance module, are all ported and deployed to the edge controller. Local data communication interfaces are established between the edge controller and the battery management system, energy storage converter, grid dispatch AGC system, and power ancillary service market platform. Through industrial communication protocols, real-time grid status and real-time energy storage data are synchronously acquired at a sampling frequency of 1 second. The system collects energy status and real-time market data. The edge controller completes the entire process locally, including data preprocessing, state representation construction, agent decision generation, rolling optimization, and constraint verification. It generates final charging and discharging commands every 4 seconds and directly sends them to the PCS for execution, forming a local real-time closed-loop control. Simultaneously, the edge controller encrypts and uploads hourly operational statistics, battery health status data, and revenue data to the cloud server via 4G / 5G network. The cloud server uses this data to periodically perform incremental training and updates on the reinforcement learning agent model and battery degradation model, and sends the updated model parameters to the edge controller during low-load periods at night. The edge controller then performs hot updates of the model parameters without affecting real-time control, achieving continuous optimization through edge-edge collaboration.

[0074] This step, by deploying all core control logic on a local edge controller, achieves millisecond-level response for energy storage frequency regulation control, completely avoiding the network latency and network interruption risks associated with cloud deployment, significantly improving the real-time performance and reliability of control, while also ensuring the security and privacy of power plant operation data.

[0075] In summary, the grid-side energy storage management method combining big data analysis provided in this application has the following technical effects: 1. By using a large historical operating dataset to construct a dynamic degradation model of battery health status, quantifying the health loss cost per unit of energy throughput and the expected frequency regulation benefit, and constructing a reinforcement learning environment with the difference between the two as the reward function to train the agent and drive real-time decision-making, the problem of difficulty in dynamically balancing frequency regulation benefit and battery health loss is solved, and the net benefit of the grid-side energy storage system throughout its entire life cycle is maximized.

[0076] 2. By adopting a technique that inputs real-time multi-source time-series features into a dynamic degradation model to predict capacity decay increments and introduces a dynamic aging acceleration factor determined by battery health status and operating stress to calculate health loss costs, the problem that static cost calculation cannot reflect differences in operating conditions is solved, and dynamic and accurate quantification of health loss for any charging and discharging operation is achieved.

[0077] 3. By adopting the technique of receiving the latest ultra-short-term prediction data at each decision moment, performing time-domain local re-optimization based on the agent's first planned action, and applying real-time battery constraints, the problem of high computational complexity and poor real-time performance of global optimization is solved. This ensures both decision-making speed and safe battery operation under the latest boundary conditions.

[0078] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any modifications, equivalent changes, and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A grid-side energy storage management method combining big data analysis, characterized in that: The method includes: Obtain historical operating data of the grid-side energy storage system, wherein the historical operating data includes at least historical operating condition data and historical market data; Based on the historical operating data, a dynamic degradation model of battery health status is constructed, with battery aging time sequence characteristics as input and battery capacity decay increment as output. The battery health status dynamic degradation model is correlated with the battery initial health status to calculate the battery health loss cost per unit throughput energy, and the expected frequency regulation revenue is calculated based on the historical market data. A reinforcement learning environment is constructed, wherein the reward function of the reinforcement learning environment is the difference between the expected frequency modulation revenue and the battery health depletion cost, and the agent in the reinforcement learning environment is trained using the historical running large dataset. Based on real-time grid status, real-time energy storage status, and real-time market data, the trained agent generates charging and discharging action commands, executes the charging and discharging action commands, and controls the operation of the grid-side energy storage system.

2. The grid-side energy storage management method combining big data analysis as described in claim 1, characterized in that, Based on the historical operating data, a dynamic degradation model of battery health status is constructed, taking battery aging time-series characteristics as input and battery capacity decay increment as output, including: Correlation analysis was performed on the historical operating data to uncover a combination of multi-source time-series features that are strongly correlated with the battery aging rate. The combination of multi-source time-series features includes historical cumulative throughput energy, historical average charge-discharge rate, historical battery operating temperature, and historical state of charge variation range. From the battery performance test records, obtain the historical measured battery capacity decay amount that corresponds to the multi-source time-series feature combination in time. Using the multi-source time-series feature combination as input and the historical measured battery capacity decay as the desired output, a machine learning model is trained to obtain a dynamic degradation model of battery health status.

3. The grid-side energy storage management method combining big data analysis as described in claim 2, characterized in that, The battery health state dynamic degradation model is correlated with the battery's initial health state to calculate the battery health loss cost per unit throughput energy, including: The system collects real-time operating data streams of the current grid-side energy storage system and performs feature extraction operations on the real-time operating data streams to generate a real-time multi-source time-series feature combination characterizing the current operating stress. The real-time multi-source time-series features are combined and input into the battery health state dynamic degradation model to predict the dynamic battery capacity decay increment caused by executing the charging and discharging action sequence to be evaluated. Based on the battery's initial health state and the battery's rated total throughput energy, calculate the unit benchmark throughput energy cost; Multiply the dynamic battery capacity decay increment by the dynamic aging acceleration factor, and then multiply it by the unit benchmark throughput energy cost to obtain the dynamic battery health loss cost corresponding to the charge-discharge action sequence to be evaluated. The dynamic aging acceleration factor is jointly determined by the current health state of the battery and the stress level of the charge-discharge action sequence.

4. The grid-side energy storage management method combining big data analysis as described in claim 3, characterized in that, The expected frequency modulation revenue is calculated based on the aforementioned historical market data, including: Based on the historical market data, a revenue assessment model considering the frequency modulation performance compensation coefficient is constructed. The revenue assessment model is associated with the call power and the accuracy coefficient of historical frequency modulation actions. The intended charge / discharge command and its historical performance evaluation indicators are input into the revenue assessment model, and the expected frequency regulation revenue is output.

5. The grid-side energy storage management method combining big data analysis as described in claim 4, characterized in that, Building a reinforcement learning environment includes: Define the state space of the intelligent agent, which includes the grid frequency regulation demand signal, the state of charge of the energy storage system, and the average temperature of the battery pack. Define the action space of the intelligent agent, which is a set of charging and discharging power values ​​that the grid-side energy storage system is allowed to execute in the current state; Define a reward function for the reinforcement learning environment. In each decision step, the reward function will obtain an instant reward value by the difference between the expected frequency modulation revenue calculated based on the historical market data and the battery health depreciation cost calculated based on the historical operating condition data.

6. The grid-side energy storage management method combining big data analysis as described in claim 5, characterized in that, Training the agent in the reinforcement learning environment using the historical large dataset includes: A high-fidelity simulation environment containing the evolution of power grid, market, and equipment status is constructed based on the aforementioned historical operational big data set. In the high-fidelity simulation environment, the policy network parameters of the agent are initialized, and the agent is used to generate a candidate charge-discharge power plan sequence covering multiple future decision steps based on the current state observation data. The candidate charge-discharge power plan sequence is input into the high-fidelity simulation environment for forward simulation to predict the future state evolution path after executing the candidate charge-discharge power plan sequence, and the corresponding cumulative reward value is calculated based on the future state evolution path. Based on the accumulated reward value, the gradient update amount of the agent's policy network is calculated, and the policy network parameters of the agent are updated by backpropagation. This process is repeated until the policy performance of the agent meets the preset convergence condition, thus completing the training of the agent.

7. The grid-side energy storage management method combining big data analysis as described in claim 6, characterized in that, Based on real-time grid status, real-time energy storage status, and real-time market data, the trained agent is driven to generate charging and discharging action commands, including: By fusing the real-time grid status, real-time energy storage status, and real-time market data with short-term forecast information, a complete state representation defined by the agent's state space is constructed. The complete state representation is input into the trained agent, which then outputs a preliminarily optimized sequence of future multi-step charge and discharge power plans. The initially optimized charge and discharge power plan sequence is subjected to real-time rolling optimization to generate the final executable charge and discharge action instructions.

8. The grid-side energy storage management method combining big data analysis as described in claim 7, characterized in that, Perform real-time rolling optimization on the initially optimized charge / discharge power plan sequence, including: At every decision-making moment, receive the latest operating condition data, including ultra-short-term grid status and market forecast data; Based on the first action of the charge / discharge power plan sequence generated by the intelligent agent, and combined with the latest operating condition data, local re-optimization is performed within a time-shortened rolling optimization framework. Real-time calculated battery internal resistance and thermal limit constraints are applied, and an instantaneous charge / discharge action command that satisfies the latest boundary conditions is output.

9. The grid-side energy storage management method combining big data analysis as described in claim 1, characterized in that, The method also includes adaptive action space constraints based on health status: The current health status of the battery, as assessed by the dynamic degradation model of battery health status, is obtained in real time. Based on the current health status of the battery, the upper and lower bounds of the action space of the intelligent agent are dynamically adjusted to generate an updated action space boundary. When the health status decreases, the charging and discharging power limits and the state of charge operating range are tightened proportionally. The updated action space boundary is used as a hard constraint to drive the agent to generate charging and discharging action commands.

10. The grid-side energy storage management method combining big data analysis as described in claim 1, characterized in that, The method is executed on an edge controller deployed locally on the grid-side energy storage system.

Images