A new energy station charging and discharging strategy automatic optimization control system

Abstract

The present application relates to the technical field of new energy station operation control, in particular to a new energy station charging and discharging strategy automatic optimization control system, the system comprises: a data acquisition module acquires the multi-source heterogeneous operation data of photovoltaic, fan, energy storage and power grid in the station. The data preprocessing module aligns and cleans the data in time and space, and generates a standard time series data set. The strategy optimization module calls an improved deep deterministic policy gradient algorithm, which is designed according to the multi-time scale coupling characteristics of the station, performs rolling optimization analysis on the time series data set, and solves the optimal charging and discharging power instruction sequence of the energy storage battery pack in the future dispatching period. The instruction generation module performs safety check and smoothing processing on the sequence and the real-time dispatching instruction to generate the final executable time-of-use charging and discharging control strategy. The instruction issuing module issues the strategy to the energy storage converter for execution.

Description

Technical Field

[0001] This invention relates to the field of operation control technology for new energy power stations, and in particular to an automatic optimization control system for charging and discharging strategies of new energy power stations. Background Technology

[0002] In power systems with a high proportion of renewable energy integration, power station-level energy storage systems, through charging and discharging operations, smooth power fluctuations, participate in grid frequency regulation, and achieve peak-valley arbitrage, are key means to improve renewable energy consumption and grid stability. Maximizing the value of energy storage systems relies on efficient and precise charging and discharging strategy optimization control. Currently, technical solutions in this field mainly fall into two categories. One category is control methods based on fixed rules or empirical thresholds, such as setting charging and discharging trigger conditions based on the energy storage's state of charge or net power deviation. These methods are logically simple and have a fast response, but their strategies are rigid and difficult to adapt to the strong randomness of renewable energy output and complex electricity price signals, thus failing to achieve economic optimization. The other category uses traditional optimization algorithms, such as linear programming and model predictive control, aiming to minimize power station operating costs or maximize profits, solving the optimization problem based on predicted data. These methods consider economic efficiency to some extent, but their optimization models typically rely on accurate predictions of variables such as wind and solar power output and electricity prices, and make numerous linearization or simplification assumptions to the system model in exchange for solvability.

[0003] The operation of renewable energy power plants involves multiple time-scale control objectives, including power fluctuation mitigation at the second / minute level, energy transfer at the hour level, and plan tracking at the daily level. These objectives are coupled and mutually constraining. Existing methods are insufficient in handling this multi-time-scale coupling characteristic, either designing strategies only for a single time scale or merging multiple objectives through simple weighting, failing to deeply characterize and utilize the inherent relationships between dynamics at different time scales. Furthermore, wind and solar power output and the market environment are highly complex and uncertain. Traditional optimization algorithms have limited adaptability and self-learning evolution capabilities when facing decision-making problems with continuous, high-dimensional state and action spaces. Therefore, a control system is needed that can automatically learn and adapt to the multi-time-scale operational coupling characteristics of power plants, directly optimizing sequential decisions under uncertain environments to generate better and more robust charging and discharging strategies. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of existing technologies and propose an automatic optimization control system for charging and discharging strategies of new energy power stations.

[0005] To achieve the above objectives, the present invention adopts the following technical solution: an automatic optimization control system for charging and discharging strategies of new energy power stations, comprising:

[0006] The data acquisition module collects multi-source heterogeneous operation data of the new energy power station. The multi-source heterogeneous operation data includes the real-time power curve of the photovoltaic power generation unit, the output fluctuation sequence of the wind turbine generator, the charge history of the energy storage battery pack, and the real-time electricity price signal of the power station's grid connection point.

[0007] The data preprocessing module performs spatiotemporal alignment and data cleaning on the multi-source heterogeneous operating data to generate a standard format time series dataset of the station's operating status.

[0008] The strategy optimization module calls an improved deep deterministic strategy gradient algorithm to perform rolling optimization analysis on the time series dataset of the power station's operating status. The improved deep deterministic strategy gradient algorithm is optimized based on the multi-timescale coupling characteristics of the new energy power station. Using the improved deep deterministic strategy gradient algorithm, the optimal charging and discharging power command sequence of the energy storage battery pack in the next scheduling cycle is obtained.

[0009] The instruction generation module performs safety boundary verification and instruction smoothing processing based on the optimal charging and discharging power instruction sequence and the real-time acquired power grid dispatch instructions, and generates the final executable time-segmented charging and discharging control strategy.

[0010] The instruction sending module sends the time-segmented charging and discharging control strategy to the energy storage converter, controlling the energy storage battery pack to perform the corresponding charging or discharging operations.

[0011] As a further aspect of the present invention, the multi-source heterogeneous operational data is subjected to spatiotemporal alignment and data cleaning to generate a standard-format time-series dataset of station operational status, including:

[0012] Identify the data timestamps and sampling frequencies of the real-time power curve of the photovoltaic power generation unit, the output fluctuation sequence of the wind turbine generator set, the state of charge history of the energy storage battery pack, and the real-time electricity price signal of the power station grid connection point;

[0013] Using the data with the highest sampling frequency as a benchmark, cubic spline interpolation is used to unify the time scale of the data with the low sampling frequency, so that all data sequences have the same and equally spaced timestamps.

[0014] Outlier detection and processing are performed on the time-aligned data sequence. A local outlier factor algorithm is used to identify and remove transient noise points in the power data. Linear interpolation is used to fill the data gaps formed after the removal of the transient noise points.

[0015] The real-time power curve of the photovoltaic power generation unit and the output fluctuation sequence of the wind turbine generator are normalized to map the power value to the range of zero to one.

[0016] The normalized power data, the unnormalized state-of-charge history of the energy storage battery pack, and the real-time electricity price signal of the power station's grid connection point are integrated into a time-series dataset of the power station's operating status with a unified field format, arranged in timestamp order.

[0017] As a further aspect of the present invention, the improved deep deterministic strategy gradient algorithm is optimized based on the multi-timescale coupling characteristics of new energy power plants, and its working steps include:

[0018] A dual-network architecture is constructed, which includes an evaluation network and a policy network. The policy network is used to output continuous charging and discharging power actions based on the current operating status of the site, and the evaluation network is used to evaluate the long-term expected benefits of the state-action pair.

[0019] In the input of the evaluation network, features representing coupling relationships at different time scales are explicitly introduced, including the input of second-level photovoltaic fluctuation variance, minute-level load change trend and hour-level electricity price cycle pattern after feature splicing.

[0020] During the learning process of the policy network, a time scale consistency constraint is added. The time scale consistency constraint requires that when the numerical set of the system running state variables is the same but the dominant time scale characteristics are different, the output action of the policy network should satisfy a smooth transition relationship.

[0021] In the environment interaction stage of algorithm training, a multi-timescale hybrid reward function is designed. The multi-timescale hybrid reward function simultaneously includes fluctuation smoothing reward calculated based on second-level data, tracking and scheduling instruction reward calculated based on minute-level data, and economic benefit reward calculated based on hour-level data.

[0022] A hierarchical experience replay mechanism is adopted to store training data. The data is classified and stored into different experience pools according to the dominant time scale characteristics in the interaction data. During training, samples are taken from each experience pool proportionally to balance the learning of the operating modes at different time scales.

[0023] As a further aspect of the present invention, the explicit introduction of features characterizing coupling relationships at different time scales into the input of the evaluation network specifically includes:

[0024] From the time-series dataset of the station's operating status, the real-time power curve of the photovoltaic power generation unit within a preset time window is extracted, and the variance of the power within the time window is calculated as a second-level fluctuation feature.

[0025] From the time series dataset of the station's operating status, extract the output fluctuation sequence of the wind turbine generator within a longer preset time window, and fit the slope of its linear trend line as the minute-level change trend feature.

[0026] From the real-time electricity price signal of the grid connection point of the power station, identify the time and magnitude of the price peak and trough within a complete electricity price cycle in the past, and encode them into a cyclic feature vector as an hourly electricity price cycle pattern feature.

[0027] The second-level fluctuation characteristics, the minute-level change trend characteristics, and the hour-level electricity price cycle pattern characteristics are concatenated to form a multi-dimensional coupled feature vector;

[0028] The multidimensional coupled feature vector is merged with the current basic state vector of the station and used as the input of the evaluation network.

[0029] As a further aspect of the present invention, the improved deep deterministic policy gradient algorithm is used to solve for the optimal charging and discharging power command sequence of the energy storage battery pack in a future scheduling cycle, including:

[0030] The latest time period data in the time series dataset of the station's operating status is used as the current state input of the algorithm;

[0031] The policy network in the improved deep deterministic policy gradient algorithm is deployed in online inference mode, and the policy network directly outputs a suggested charge / discharge power action value based on the current state.

[0032] The current state and the suggested charge / discharge power action values ​​are input into the evaluation network of the improved deep deterministic policy gradient algorithm to obtain the long-term expected return estimate of the current state-action pair.

[0033] Using a model predictive control framework, starting from the current moment, the state transition and action selection process is executed forward in the virtual environment, simulating the state transition and action selection at each step in a future scheduling cycle, and accumulating to calculate the total virtual long-term expected revenue.

[0034] The action output of the policy network at each step in the rolling optimization is adjusted until the virtual total long-term expected benefit reaches its maximum. At this point, the sequence of charging and discharging power actions in the next scheduling cycle starting from the current moment is recorded, which is the optimal charging and discharging power instruction sequence.

[0035] As a further aspect of the present invention, based on the optimal charging and discharging power command sequence and the real-time acquired grid dispatching commands, safety boundary verification and command smoothing processing are performed to generate a final executable time-segmented charging and discharging control strategy, including:

[0036] Obtain the real-time safe operating boundary parameters of the energy storage battery pack, including the current maximum allowable charging power, maximum discharging power, upper limit of state of charge, and lower limit of state of charge.

[0037] Each power command value in the optimal charge and discharge power command sequence is compared with the real-time safe operation boundary parameter. If the power command value exceeds the maximum charging power or maximum discharging power limit, it is clamped to the corresponding limit. If the predicted state of charge will exceed the upper or lower limit after executing the generated power command, the power command is adjusted in reverse to bring the predicted state of charge back to the safe range.

[0038] Receive real-time dispatch instructions issued by the automatic generation control of the power grid, and superimpose the power instruction sequence after safety boundary verification with the real-time dispatch instructions to form a preliminary joint control instruction sequence;

[0039] The preliminary joint control command sequence is subjected to low-pass filtering to remove high-frequency step components generated by optimization calculations, ensuring that the command change rate is within the physical response capability range of the energy storage converter.

[0040] The smoothed instruction sequence is divided into multiple consecutive control periods according to the time resolution of the scheduling cycle, and each control period is assigned a unique time period identifier and power setting value to form the structured time-segmented charging and discharging control strategy.

[0041] As a further aspect of the present invention, the preliminary joint control command sequence is subjected to low-pass filtering processing, including:

[0042] Determine the maximum allowable power change rate of the energy storage converter when switching between charging and discharging states, and set it as a constraint condition for the filtering process;

[0043] Design a finite impulse response digital low-pass filter whose cutoff frequency is calculated based on the maximum allowable power change rate and the update cycle of the control command;

[0044] The initial joint control command sequence is used as the input signal and convolved through the finite impulse response digital low-pass filter to obtain the filtered command sequence.

[0045] Check the difference sequence of the filtered instruction sequence to ensure that the rate of change between any adjacent instructions is less than or equal to the maximum allowable rate of change of power.

[0046] For local segments that do not meet the rate of change constraint, linear interpolation is used for correction to generate a smooth command sequence that finally meets the physical response constraint.

[0047] As a further aspect of the present invention, the step of using the data with the highest sampling frequency as a benchmark and employing cubic spline interpolation to perform time-scale unification processing on the data with lower sampling frequencies, so that all data sequences have the same and equally spaced timestamps, includes:

[0048] From the real-time power curve of the photovoltaic power generation unit, the output fluctuation sequence of the wind turbine generator set, the state of charge history of the energy storage battery pack, and the real-time electricity price signal of the grid connection point of the power station, the data sequence with the highest sampling frequency is identified, and the set of timestamps of the data sequence is determined as the target unified time reference.

[0049] For each data sequence with a sampling frequency lower than the highest sampling frequency, a piecewise cubic polynomial function is constructed to represent the continuous change relationship of the data sequence based on its original non-equal interval timestamps and corresponding data values.

[0050] Substitute each equally spaced time point in the target unified time base into the corresponding piecewise cubic polynomial function for calculation to obtain the interpolation estimate of the data sequence at each target time point;

[0051] The calculated interpolated estimate is compared with the original value of the data sequence at the target time point. For time points where the original value exists, the interpolation accuracy is evaluated by calculating the residual between the interpolated estimate and the original value.

[0052] After interpolating all low-sampling-frequency data sequences, all data sequences are rearranged according to the target unified time base to form multiple aligned data sequences with the same and equally spaced timestamps, providing a foundation for subsequent outlier detection and data integration.

[0053] As a further aspect of the present invention, the model predictive control framework is adopted, starting from the current moment, to perform the state transition and action selection process forward in the virtual environment, simulating the state transition and action selection at each step in a future scheduling cycle, and accumulating and calculating the total virtual long-term expected revenue, including:

[0054] In the model predictive control framework, the current system state is used as the initial state for rolling optimization, and the system state includes the latest station operating state extracted from the station operating state time series dataset.

[0055] Initialize an empty virtual action sequence and an empty virtual state sequence, and store the initial state into the virtual state sequence;

[0056] In the virtual environment, starting from the current moment, the simulation progresses forward with a fixed control step size. For each future moment in the process, the following simulation steps are executed:

[0057] The current online policy network is loaded from the improved deep deterministic policy gradient algorithm, and the last state in the virtual state sequence is input into the online policy network to obtain a suggested virtual charge / discharge power action.

[0058] The current simulated future time, the last state in the virtual state sequence, and the obtained suggested virtual charge / discharge power action are input into a preset station state transition model to predict the system state at the next moment after the suggested virtual charge / discharge power action is executed.

[0059] The predicted system state at the next moment is stored in the virtual state sequence, and the suggested virtual charging and discharging power actions are stored in the virtual action sequence.

[0060] The online evaluation network is loaded from the improved deep deterministic policy gradient algorithm to calculate the single-step long-term expected return of the state-action pair at the current simulation time, and the long-term expected return is accumulated into the virtual total return.

[0061] Repeat the simulation steps until a complete future scheduling cycle is covered;

[0062] Ultimately, we obtain a virtual state evolution trajectory from the current moment to the end of a future scheduling cycle, a corresponding virtual action sequence, and a cumulatively calculated virtual total long-term expected return.

[0063] As a further aspect of the present invention, the system further includes:

[0064] The strategy online self-updating module is used to continuously collect the actual operating data of the energy storage battery pack after executing the time-sharing charge and discharge control strategy, including the actual charge and discharge power curve, the actual state of charge trajectory and the actual net power curve of the site.

[0065] The actual operating data is compared with the predicted data or expected state used in the strategy generation stage to calculate the strategy execution deviation, including power tracking deviation, state of charge prediction deviation and economic benefit deviation.

[0066] When the statistical mean of the strategy execution deviation exceeds the preset update threshold, the online learning process of the strategy is triggered;

[0067] In the online learning process of the strategy, the recently generated actual operation data and the corresponding historical status data of the site are combined to form a new training sample, and the evaluation network and policy network in the improved deep deterministic policy gradient algorithm are incrementally trained.

[0068] By updating the online inference model using the network parameters after incremental training, the online adaptive adjustment of the charging and discharging strategy optimization control system can be achieved.

[0069] Compared with the prior art, the advantages and positive effects of the present invention are as follows:

[0070] The improved deep deterministic policy gradient algorithm is a deep reinforcement learning method for continuous control problems. Its core improvement lies in the specialized design of the algorithm structure and reward function for the specific physical scenario and operational constraints of source-storage coordination in renewable energy power plants. This algorithm does not rely on precise modeling of complex system physics; instead, it learns the mapping strategy from state to optimal action through interactive trial and error with an environment containing multi-source heterogeneous operational data. It takes a time-series dataset of renewable energy power plant operating states as input, and its neural network architecture can effectively extract deep features from high-dimensional data. The algorithm output is a series of continuous charging and discharging power commands within future scheduling cycles, which is an optimal search problem in a continuous action space. This method can directly process random sequences of wind and solar power output and real-time electricity price signals, integrating the fluctuation patterns, cyclical regularities, and market patterns implicit in historical data into the strategy generation process. Through end-to-end optimization, the strategy can dynamically adjust during execution to cope with deviations between actual operation and prediction, overcoming the shortcomings of traditional model predictive control that heavily relies on prediction accuracy. The strategy itself has the potential for online learning and continuous optimization. As operational data accumulates, its decision-making performance can be continuously improved, enabling it to adapt more effectively to complex operating environments.

[0071] Optimization based on the multi-timescale coupling characteristics of new energy power plants is the core principle of this strategy optimization module design. This means that during algorithm training and decision-making, the interactions and constraints between control objectives at different timescales, such as second / minute-level fluctuation smoothing, hourly-level energy arbitrage, and daily-level plan tracking, are explicitly considered and utilized. In algorithm implementation, this is typically achieved by designing a network structure with multi-scale feature extraction capabilities or constructing a composite reward function that integrates multiple timescale objectives. For example, the reward function can simultaneously include an immediate reward reflecting power smoothness, a phased reward reflecting arbitrage gains, and a final reward reflecting medium- to long-term SOC balance. Through this design, the optimization algorithm can "foresee" the impact of its single-step optimal action on subsequent longer-term operational objectives when searching for the optimal action. The resulting charging and discharging strategy is no longer a simple superposition or switching of multiple single-timescale strategies, but a unified decision sequence coordinated and optimized across the global time span. This allows the energy storage system to smooth out instantaneous power fluctuations while reserving the necessary capacity space for its participation in the subsequent frequency regulation market or for peak-valley price arbitrage, avoiding conflicts between control objectives. On the basis of ensuring power grid quality and equipment safety, it significantly improves the overall economic efficiency of the station operation and the completion of the operation plan. Attached Figure Description

[0072] Figure 1 This is a timing diagram of the automatic optimization control system for charging and discharging strategies of new energy power stations as described in this invention.

[0073] Figure 2A flowchart illustrating the working steps for multi-timescale optimization of the improved deep deterministic strategy gradient algorithm;

[0074] Figure 3 A flowchart illustrating the process of improving the DDPG algorithm to solve for the optimal charge / discharge power command sequence. Detailed Implementation

[0075] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0076] See Figure 1 This invention provides an automatic optimization control system for charging and discharging strategies in new energy power stations, the overall implementation of which is as follows:

[0077] The data acquisition module collects real-time power curves of photovoltaic power generation units, output fluctuation sequences of wind turbine generators, historical state of charge of energy storage battery packs, and real-time electricity price signals at the grid connection point of the power station. This data constitutes multi-source heterogeneous operational data for the new energy power station. The data preprocessing module receives this multi-source heterogeneous operational data, performs spatiotemporal alignment and data cleaning, and generates a standard-format time-series dataset of the power station's operational status. The strategy optimization module calls an improved deep deterministic strategy gradient algorithm, which optimizes based on the multi-timescale coupling characteristics of the new energy power station and performs rolling optimization analysis on the time-series dataset of the power station's operational status to obtain the optimal charging and discharging power command sequence for the energy storage battery pack within the next scheduling cycle. The command generation module performs safety boundary checks and command smoothing based on the optimal charging and discharging power command sequence and the real-time grid dispatch commands, generating the final executable time-segmented charging and discharging control strategy. The command issuance module issues the time-segmented charging and discharging control strategy to the energy storage converter, controlling the energy storage battery pack to perform the corresponding charging or discharging operations.

[0078] In one embodiment of the present invention, spatiotemporal alignment and data cleaning are performed on multi-source heterogeneous operational data to generate a standard-format time-series dataset of power plant operation status. The data timestamps and sampling frequencies of the real-time power curves of photovoltaic power generation units, the output fluctuation sequences of wind turbine generators, the state-of-charge history of energy storage battery packs, and the real-time electricity price signals of the power plant grid connection point are identified. Using the data with the highest sampling frequency as a benchmark, cubic spline interpolation is used to unify the time scale of the data with lower sampling frequencies, ensuring that all data sequences have the same and equally spaced timestamps. During the time-scale unification process using cubic spline interpolation, the data sequence with the highest sampling frequency is identified from the real-time power curves of photovoltaic power generation units, the output fluctuation sequences of wind turbine generators, the state-of-charge history of energy storage battery packs, and the real-time electricity price signals of the power plant grid connection point, and the set of timestamps of the data sequences is determined as the target unified time benchmark. For each data sequence with a sampling frequency lower than the highest sampling frequency, a piecewise cubic polynomial function is constructed based on its original non-equally spaced timestamps and corresponding data values ​​to represent the continuous change relationship of the data sequence. Substituting each equally spaced time point in the target unified time base into the corresponding piecewise cubic polynomial function, the interpolated estimate of the data sequence at each target time point is obtained. The calculated interpolated estimate is compared with the original value of the data sequence at the target time point. For time points where the original value exists, the interpolation accuracy is evaluated by calculating the residual between the interpolated estimate and the original value. After interpolating all low-sampling frequency data sequences, all data sequences are rearranged according to the target unified time base, forming multiple aligned data sequences with the same and equally spaced timestamps, providing a foundation for subsequent outlier detection and data integration. Outlier detection and processing are performed on the time-aligned data sequences. A local outlier factor-based algorithm is used to identify and remove transient noise points in the power data, and linear interpolation is used to fill the data gaps formed after the removal of transient noise points. The real-time power curves of the photovoltaic power generation units and the output fluctuation sequences of the wind turbine generators are normalized, mapping the power values ​​to the interval between zero and one. The normalized power data, the unnormalized state-of-charge history of the energy storage battery packs, and the real-time electricity price signal of the power station's grid connection point are integrated into a time-series dataset of the power station's operating status with a unified field format, according to the timestamp order.

[0079] In practical implementation, the data preprocessing module of the automatic optimization control system for charging and discharging strategies in new energy power plants performs spatiotemporal alignment and data cleaning of multi-source heterogeneous operating data. This can be illustrated through a specific scenario of an operating day. Assume that in a certain new energy power plant, the data acquisition module simultaneously collects four types of operating data during the same morning period: the real-time power curve of the photovoltaic power generation unit (sampling frequency: 1 data point per second); the output fluctuation sequence of the wind turbine generator set (sampling frequency: 1 data point per 10 seconds); the state-of-charge history of the energy storage battery pack (sampling frequency: 1 data point per 30 seconds); and the real-time electricity price signal at the power plant's grid connection point (sampling frequency: 1 data point per 5 minutes). In practical implementation, the data preprocessing module identifies the timestamps and sampling frequencies of these four types of data and determines that the real-time power curve of the photovoltaic power generation unit has the highest sampling frequency of 1 time per second. The data preprocessing module determines the set of timestamps of the photovoltaic power generation unit data, i.e., a continuous time point sequence starting from 08:00:00 with 1-second intervals, as the target unified time reference.

[0080] In practice, the data with the highest sampling frequency is used as the benchmark, and cubic spline interpolation is employed to unify the time scale of the data with lower sampling frequencies. For the wind turbine output fluctuation sequence with a sampling frequency of one data point every 10 seconds, the data preprocessing module constructs a piecewise cubic polynomial function to represent the continuous change relationship of the data sequence based on its original timestamps (e.g., 08:00:00, 08:00:10, 08:00:20...) and the corresponding power values. In practice, each equally spaced time point (e.g., 08:00:01, 08:00:02, 08:00:03...) in the target unified time benchmark is substituted into the piecewise cubic polynomial function constructed from the wind turbine output sequence for calculation, obtaining the interpolated estimate of the wind turbine output at the target time point per second. For the energy storage battery pack state of charge history with a sampling frequency of 1 data point every 30 seconds and the real-time electricity price signal with a sampling frequency of 1 data point every 5 minutes, the data preprocessing module uses the same cubic spline interpolation method to process them and calculate their interpolation estimates at the target time point per second.

[0081] In practice, the data preprocessing module compares the calculated interpolated estimates with the original values ​​of the data sequence at the target time point. For example, for the output data of wind turbine generators, there are original sampled values ​​at the target time reference point of 08:00:10. The data preprocessing module calculates the residual between the interpolated estimate obtained by cubic spline interpolation at this time point and the original sampled value, and evaluates the accuracy of the interpolation process by analyzing the residuals at all such time points. In practice, after completing the interpolation processing of all low-sampling frequency data sequences, the data preprocessing module rearranges all the original power data per second of photovoltaic power generation units, the interpolated power data per second of wind turbine generators, the interpolated state-of-charge data per second of energy storage battery groups, and the interpolated electricity price data per second of grid connection points according to the target unified time reference with equal intervals per second starting from 08:00:00, forming four aligned data sequences with the same and equally spaced timestamps.

[0082] In some embodiments, after time-scale unification, the data preprocessing module performs outlier detection and processing on the time-aligned data sequence. The data preprocessing module uses a local outlier factor algorithm to scan the aligned real-time power curves of photovoltaic power generation units and the power output fluctuation sequences of wind turbine generators, identifying and removing transient noise points in the power data. For example, at 08:05:23, the original photovoltaic power value is 850 kW, but the data for the seconds before and after it fluctuates around 500 kW. The local outlier factor algorithm identifies this 850 kW value as a transient noise point and removes it. In a specific implementation, the data preprocessing module uses linear interpolation to fill the data gaps formed after removing transient noise points, and performs linear calculations using the values ​​of adjacent normal data points before and after the outlier to obtain a replacement value for that time point.

[0083] In practical implementation, the data preprocessing module normalizes the real-time power curves of the processed photovoltaic power generation units and the output fluctuation sequences of the wind turbine generator sets. The data preprocessing module first obtains the rated capacity of the photovoltaic and wind power generation units from historical data, and divides the actual power value at each time point by the corresponding rated capacity, thereby mapping the power value to a range of zero to one. Optionally, the normalization calculation can be expressed as:

[0084]

[0085] in: This represents the normalized power value at timestamp t. This represents the original power value at timestamp t. This indicates the rated power capacity of the power generation unit. In some embodiments, the data preprocessing module integrates normalized photovoltaic and wind power data, historical data on the state of charge of unnormalized energy storage battery packs, and real-time electricity price signals at the power station's grid connection point, according to a unified timestamp order. It can be understood that the data preprocessing module creates a record for each timestamp, and each record contains five fields: "timestamp," "normalized photovoltaic power," "normalized wind power," "energy storage state of charge," and "real-time electricity price," thereby generating a standard-format time-series dataset of the power station's operating status for subsequent use by the strategy optimization module. It can be understood that through the above specific steps, multi-source heterogeneous operating data from different sources with different sampling frequencies are transformed into a dataset with strictly synchronized time series and a unified and standardized format.

[0086] In one embodiment of the present invention, the improved deep deterministic strategy gradient algorithm is optimized based on the multi-timescale coupling characteristics of new energy power plants, see reference. Figure 2Its working principle involves constructing a dual-network architecture comprising an evaluation network and a policy network. The policy network outputs continuous charging and discharging power actions based on the current operating state of the power plant, while the evaluation network assesses the long-term expected returns of the state-action pairs. The input to the evaluation network explicitly incorporates features representing coupling relationships at different time scales, including the concatenation of second-level photovoltaic fluctuation variance, minute-level load change trends, and hourly electricity price cycle patterns. During the learning process of the policy network, a time scale consistency constraint is added, requiring the policy network to output actions that satisfy a smooth transition relationship when the numerical sets of system operating state variables are the same but the dominant time scale characteristics differ. In the environment interaction phase of algorithm training, a multi-time scale hybrid reward function is designed, which simultaneously includes fluctuation mitigation rewards calculated based on second-level data, tracking and scheduling command rewards calculated based on minute-level data, and economic benefit rewards calculated based on hourly data. A hierarchical experience replay mechanism is used to store training data, classifying and storing it into different experience pools according to the dominant time scale characteristics in the interaction data. During training, samples are taken proportionally from each experience pool to balance the learning of operating patterns at different time scales. The evaluation network explicitly incorporates features representing coupling relationships at different time scales into its input. Specifically, this includes extracting real-time power curves of photovoltaic power generation units within a preset time window from the power plant's operational time-series dataset, and calculating the power variance within that time window as a second-level fluctuation feature. It also involves extracting the output fluctuation sequence of wind turbine generators within a longer preset time window from the same dataset, fitting the slope of its linear trend line as a minute-level trend feature. Furthermore, it identifies the price peaks, troughs, and their magnitudes within a past complete price cycle from the real-time electricity price signal at the power plant's grid connection point, encoding these as cyclic feature vectors as hourly price cycle pattern features. The second-level fluctuation features, minute-level trend features, and hourly price cycle pattern features are then concatenated to form a multi-dimensional coupling feature vector. This multi-dimensional coupling feature vector is then merged with the power plant's current basic state vector and used as input to the evaluation network.

[0087] In its implementation, the improved deep deterministic policy gradient algorithm is optimized based on the multi-timescale coupling characteristics of new energy power plants. Its working principle can be illustrated through a specific online inference and training scenario. Assume that at a certain scheduling moment, the policy optimization module receives a standard-format time-series dataset of power plant operating states from the data preprocessing module. This dataset includes the normalized power of photovoltaic power generation units, the normalized power of wind turbine generators, the state of charge of energy storage battery packs, and real-time electricity price data for the current and historical moments. The improved deep deterministic policy gradient algorithm constructs a dual-network architecture comprising an evaluation network and a policy network. In its implementation, the policy network is designed as a deep neural network. Its input layer receives the current power plant operating state vector, and its output layer generates a continuous action value representing the charging and discharging power of the energy storage battery packs. Positive values ​​are defined as discharging, and negative values ​​are defined as charging. The evaluation network is also designed as a deep neural network. Its input layer receives a state-action pair formed by concatenating the current state vector and the action value output by the policy network, and its output layer outputs a scalar value used to evaluate the long-term expected benefit of this state-action pair.

[0088] In practical implementation, features representing coupling relationships at different time scales are explicitly introduced into the input of the evaluation network. The improved deep deterministic policy gradient algorithm extracts the real-time power curves of photovoltaic power generation units within a preset time window from the time-series dataset of the power plant's operating status. For example, it extracts data from the most recent 300 sampling points (i.e., the most recent 300 seconds when the sampling frequency is 1Hz), calculates the variance of these 300 power values, and uses this variance value as a second-level fluctuation feature. The improved deep deterministic policy gradient algorithm also extracts the output fluctuation sequence of wind turbine generators within a longer preset time window from the same dataset. For example, it extracts data from the most recent 1800 sampling points (i.e., the most recent 30 minutes), performs linear regression on this sequence, fits the slope of its linear trend line, and uses this slope value as a minute-level trend feature. The improved deep deterministic policy gradient algorithm identifies the peak and trough times, peak and trough amplitudes within a complete past price cycle (e.g., 24 hours) from real-time electricity price signals at the power plant's grid connection point. It encodes the time-of-day information into a cyclic feature vector using sine and cosine functions, and combines this with price amplitude information to construct an hourly electricity price cycle pattern feature. In implementation, the calculated second-level fluctuation features, minute-level trend features, and hourly electricity price cycle pattern features are concatenated to form a multi-dimensional coupled feature vector. The improved deep deterministic policy gradient algorithm merges this multi-dimensional coupled feature vector with the power plant's current basic state vector (e.g., current photovoltaic power, wind power, state of charge, and electricity price) as input to the evaluation network. This allows the evaluation network to explicitly perceive and weigh system dynamics at different time scales when assessing the value of actions.

[0089] In some embodiments, during the learning process of the policy network, the improved deep deterministic policy gradient algorithm incorporates a time-scale consistency constraint. This constraint is implemented by designing a specific loss function term. This constraint requires that the policy network's output actions should satisfy a smooth transition relationship when the numerical sets of system operating state variables are the same but the dominant time-scale characteristics differ. For example, consider two different system states where their instantaneous values ​​of photovoltaic power, wind power, state of charge, and electricity price are identical. However, one state corresponds to a large historical second-level photovoltaic fluctuation variance (second-level disturbance dominates), while the other state corresponds to a small historical second-level photovoltaic fluctuation variance but a clear minute-level downward trend in wind power (minute-level trend dominates). It can be understood that the time-scale consistency constraint will prompt the policy network to output charging and discharging power action values ​​for these two states without drastic jumps, but rather to exhibit consistency, thereby ensuring stable control behavior of the policy when facing disturbances dominated by different time scales.

[0090] In practical implementation, during the environment interaction phase of algorithm training, the improved deep deterministic policy gradient algorithm designs a multi-timescale hybrid reward function. This multi-timescale hybrid reward function simultaneously includes fluctuation mitigation rewards calculated based on second-level data, tracking and scheduling instruction rewards calculated based on minute-level data, and economic benefit rewards calculated based on hour-level data. Optionally, the multi-timescale hybrid reward function can be expressed as:

[0091]

[0092] in: Represents the total reward for a single step. This represents the smoothing reward calculated based on the second-level photovoltaic power fluctuation variance. This represents the tracking reward calculated based on the deviation between the net power of the power station and the dispatch instructions at the minute-by-minute level. Representing the economic reward calculated based on hourly electricity prices and charging / discharging behavior, α, β, and γ are non-negative coefficients used to balance the reward weights at different time scales. In some embodiments, the improved deep deterministic policy gradient algorithm employs a hierarchical experience replay mechanism to store training data. In specific implementations, the algorithm categorizes and stores the interaction data into different experience pools based on the dominant time scale characteristics. For example, state-action-reward data related to high volatility and rapid response are stored in a "second-level experience pool," data related to power plan tracking are stored in a "minute-level experience pool," and data related to electricity price arbitrage are stored in an "hour-level experience pool." During training, the improved deep deterministic policy gradient algorithm samples data from each experience pool in a preset ratio (e.g., 4:3:3) to form a training batch, thereby learning operating patterns at different time scales in a balanced manner and avoiding excessive bias towards a certain time scale objective during policy optimization while ignoring others.

[0093] In one embodiment of the present invention, an improved deep deterministic policy gradient algorithm is used to solve for the optimal charging and discharging power command sequence of the energy storage battery pack in a future scheduling cycle. (See reference...) Figure 3 This process involves using the latest time-series data from the facility's operational status dataset as the current state input for the algorithm. The policy network in the improved deep deterministic policy gradient algorithm is deployed in online inference mode, directly outputting a suggested charging / discharging power action value based on the current state. The current state and the suggested charging / discharging power action value are then input into the evaluation network of the improved deep deterministic policy gradient algorithm to obtain an estimate of the long-term expected return of the current state-action pair. A model predictive control framework is used, starting from the current moment, to perform the state transition and action selection process in a virtual environment, simulating each step of the state transition and action selection within a future scheduling cycle, and accumulating the total virtual long-term expected return. The actions output by the policy network at each step in the rolling optimization are adjusted until the virtual total long-term expected return reaches its maximum. At this point, the charging / discharging power action sequence for the next scheduling cycle starting from the current moment is recorded, which is the optimal charging / discharging power command sequence. In the model predictive control framework, the current system state is used as the initial state for rolling optimization, and the system state includes the latest facility operational status extracted from the facility's operational status dataset. Initialize an empty virtual action sequence and an empty virtual state sequence, storing the initial state in the virtual state sequence. In the virtual environment, starting from the current time step, advance with a fixed control step size. For each future time step during the advancement process, execute the following simulation steps: Load the current online policy network from the improved deep deterministic policy gradient algorithm, input the last state in the virtual state sequence into the online policy network, and obtain a proposed virtual charging / discharging power action. Input the currently simulated future time step, the last state in the virtual state sequence, and the obtained proposed virtual charging / discharging power action into a pre-defined station state transition model to predict the system state at the next time step after executing the proposed virtual charging / discharging power action. Store the predicted system state at the next time step into the virtual state sequence, and store the proposed virtual charging / discharging power action into the virtual action sequence. Load the online evaluation network from the improved deep deterministic policy gradient algorithm, calculate the single-step long-term expected reward corresponding to the state-action pair at the current simulation time, and accumulate the long-term expected reward into the virtual total reward. Repeat the simulation steps until a complete future scheduling cycle is covered. Ultimately, we obtain a virtual state evolution trajectory from the current moment to the end of a future scheduling cycle, a corresponding virtual action sequence, and a cumulatively calculated virtual total long-term expected return.

[0094] In practical implementation, the improved deep deterministic policy gradient algorithm is used to solve for the optimal charging and discharging power command sequence of the energy storage battery pack within a future scheduling cycle. This can be illustrated through a specific online optimization calculation example. The policy optimization module receives the time-series dataset of the power station's operating status at time T0 (e.g., 14:00:00). The latest time-series data in the dataset includes the normalized power of the photovoltaic power generation unit (0.65), the normalized power of the wind turbine generator (0.42), the state of charge of the energy storage battery pack (0.60), the real-time electricity price (0.48 yuan / kWh), and second-level photovoltaic fluctuation variance (0.002), minute-level wind power trend slope (-0.05), and hourly electricity price cycle feature vector [0.8, -0.3] extracted from historical data. The policy optimization module integrates this data into a multi-dimensional state vector and uses it as the current state input for the improved deep deterministic policy gradient algorithm. In practical implementation, the policy network in the improved deep deterministic policy gradient algorithm is deployed in online inference mode. The policy network receives the current state vector, performs multi-layer forward propagation calculations within the network, and directly outputs a continuous suggested action value representing the charging and discharging power of the energy storage battery pack. For example, the policy network outputs a value of -0.25, indicating that it suggests the battery should be charged at a power of 0.25 megawatts in the current state. The policy optimization module combines the current state vector with the suggested charging and discharging power action value output by the policy network into a complete state-action pair, and inputs this state-action pair into the evaluation network in the improved deep deterministic policy gradient algorithm. The evaluation network evaluates the state-action pair and outputs a scalar value, such as 12.7, which represents the estimated long-term expected return that the evaluation network can obtain after executing the suggested action in the current state.

[0095] The strategy optimization module employs a model predictive control framework (MMC). Starting from the current time T0, it performs state transitions and action selections in a rolling manner within the virtual environment, simulating each step of the state transition and action selection within a future scheduling cycle. The MMC uses the current system state as the initial state for rolling optimization. The system state includes the latest station operating state extracted from the station operating state time-series dataset, i.e., the aforementioned multi-dimensional state vector. The strategy optimization module initializes an empty virtual action sequence and an empty virtual state sequence, storing the initial state vector in the virtual state sequence. In the virtual environment, starting from the current time T0, the strategy optimization module advances with a fixed control step size (e.g., 5 minutes). For each future time in the advancement process, it executes a series of simulation steps. In specific implementation, for the first future time T1 (i.e., T0 + 5 minutes), the strategy optimization module loads the current online policy network from the improved deep deterministic policy gradient algorithm, inputs the last state in the virtual state sequence (i.e., the initial state vector) into the online policy network, and the online policy network outputs a suggested virtual charging / discharging power action for time T1, for example, -0.20 MW. The strategy optimization module inputs the current simulated future time T1, the last state in the virtual state sequence, and the obtained suggested virtual charge / discharge power actions into a preset power station state transition model. This model is a prediction model based on physical laws and data-driven approaches. Based on the input state, action, and time information, it predicts the system state at the next time T2 after executing the suggested virtual charge / discharge power actions. For example, the power station state transition model predicts the following states at time T2: normalized power of photovoltaic power generation units 0.68, normalized power of wind turbine generators 0.40, state of charge of energy storage batteries 0.61, real-time electricity price 0.50 yuan / kWh, and updated multi-timescale characteristics. The strategy optimization module stores the predicted system state at time T2 into the virtual state sequence and the suggested virtual charge / discharge power actions at time T1 into the virtual action sequence.

[0096] In practice, the strategy optimization module loads an online evaluation network from an improved deep deterministic policy gradient algorithm to calculate the single-step long-term expected return corresponding to the state-action pair at the current simulation time T1. The online evaluation network receives the state and action at time T1 and outputs a single-step return estimate, for example, 10.5. The strategy optimization module adds this value to the virtual total return. The strategy optimization module repeats the simulation steps. For the next future time T2, using the latest stored T2 state in the virtual state sequence as input, the online policy network outputs a new virtual charging / discharging power action (e.g., 0.10 MW). The station state transition model predicts the system state at time T3, and the online evaluation network calculates and adds the return estimate at time T2. The strategy optimization module continues to repeat this process until the simulation covers a complete future scheduling cycle (e.g., the next 4 hours, corresponding to 48 5-minute control steps). The strategy optimization module obtains a virtual state evolution trajectory from the current time T0 to the end of the future scheduling cycle, a corresponding virtual action sequence, and a cumulatively calculated virtual total long-term expected return. Refer to Table 1, which shows data on a virtual state evolution trajectory and a virtual action sequence.

[0097] Table 1: Virtual State and Action Sequences Generated by Rolling Optimization of the Model Predictive Control Framework

[0098]

[0099] In some embodiments, optionally, the policy optimization module adjusts the actions output by the policy network at each step of the rolling optimization until the virtual total long-term expected return reaches its maximum. The adjustment process is implemented through iterative optimization. After obtaining the virtual action sequence and virtual total long-term expected return in each simulation, the policy optimization module calculates the gradient of the virtual total long-term expected return relative to the policy network parameters and updates the internal parameters of the online policy network using the gradient ascent method. It is understood that the updated online policy network will generate different virtual action sequences in the next simulation, potentially leading to different virtual total long-term expected returns. The policy optimization module repeats the simulation and parameter update cycle. The adjustment process terminates when the increment of the virtual total long-term expected return is less than a preset threshold or the maximum number of iterations is reached. At this point, the charging and discharging power action sequence generated by the online policy network with the final parameters in the last simulation is recorded for the next scheduling cycle starting from the current time T0. This sequence is the optimal charging and discharging power command sequence. In some embodiments, the cumulative calculation of the virtual total long-term expected return can be expressed as:

[0100]

[0101] in: Represents the virtual total long-term expected return. This represents the total number of simulation steps from the current moment to the end of the scheduling cycle. Indicates the sequence number of the simulation step (starting from 0). This represents the discount rate for future rewards (0 < γ < 1). This represents the estimated long-term expected return for a single step, calculated by the evaluation network in the i-th simulation step. It can be understood that through multiple rolling simulations of the model predictive control framework and online adjustment of the policy network parameters, the policy optimization module can explore different action sequences in the virtual environment and ultimately select the sequence that maximizes the cumulative discount return. The maximized sequence of actions is output as the optimal instruction.

[0102] In one embodiment of the present invention, based on the optimal charge / discharge power command sequence and the real-time acquired grid dispatch command, safety boundary verification and command smoothing are performed to generate a final executable time-sharing charge / discharge control strategy. This includes acquiring the real-time safe operating boundary parameters of the energy storage battery pack, which include the currently allowed maximum charging power, maximum discharging power, upper limit of state of charge, and lower limit of state of charge. Each power command value in the optimal charge / discharge power command sequence is compared with the real-time safe operating boundary parameters. If the power command value exceeds the maximum charging power or maximum discharging power limit, it is clamped to the corresponding limit. If the predicted state of charge after executing the generated power command will exceed the upper or lower limit, the power command is adjusted in reverse to bring the predicted state of charge back to the safe range. Real-time dispatch command issued by the grid automatic generation control is received, and the power command sequence after safety boundary verification is superimposed with the real-time dispatch command to form a preliminary joint control command sequence. The preliminary joint control command sequence is low-pass filtered to remove high-frequency step components generated by optimization calculations, ensuring that the command change rate is within the physical response capability range of the energy storage converter. The smoothed command sequence is divided into multiple consecutive control periods according to the time resolution of the scheduling cycle. Each control period is assigned a unique time period identifier and power setpoint, forming a structured time-segmented charging and discharging control strategy. The initial joint control command sequence undergoes low-pass filtering, including determining the maximum allowable power change rate of the energy storage converter during charging and discharging state switching and setting it as a constraint for the filtering process. A finite impulse response (FIR) digital low-pass filter is designed, with its cutoff frequency calculated based on the maximum allowable power change rate and the update cycle of the control commands. The initial joint control command sequence is used as the input signal and convolved through the FIR digital low-pass filter to obtain the filtered command sequence. The difference sequence of the filtered command sequence is checked to ensure that the change rate between any adjacent commands is less than or equal to the maximum allowable power change rate. For local segments that do not meet the change rate constraint, linear interpolation is used for correction, generating a final smooth command sequence that satisfies the physical response constraint.

[0103] In practical implementation, the instruction generation module performs safety boundary verification and instruction smoothing based on the optimal charge / discharge power instruction sequence and the real-time acquired grid dispatch instructions, generating a final executable time-segmented charge / discharge control strategy. Assuming that at the start of a dispatch cycle, the strategy optimization module sends the optimal charge / discharge power instruction sequence for the next two hours to the instruction generation module, this sequence contains eight consecutive power instruction values ​​at 15-minute intervals. The instruction generation module obtains the real-time safe operating boundary parameters of the energy storage battery pack from the energy storage battery management system. These real-time safe operating boundary parameters include the currently allowed maximum charging power of 1.5 MW, maximum discharging power of 2.0 MW, upper limit of state of charge of 0.95, and lower limit of state of charge of 0.15. The instruction generation module then compares each power instruction value in the optimal charge / discharge power instruction sequence with the actual... The system compares the safe operating boundary parameters. For example, the third instruction in the sequence is -2.2 MW, which represents the charging power requirement. However, its absolute value exceeds the maximum charging power limit of 1.5 MW. The instruction generation module clamps this instruction value to -1.5 MW. In practice, based on the clamped power instruction sequence and the initial state of charge (SOC) of the energy storage battery pack (0.60), the instruction generation module predicts the SOC of the battery after executing each power instruction. The prediction calculation uses the ampere-hour integration method. If the prediction finds that the SOC will drop to 0.12 after executing the sixth instruction, which is lower than the SOC limit of 0.15, the instruction generation module initiates a reverse adjustment. Starting from the sixth instruction point, it traces back and appropriately reduces the discharge power or increases the charging power at previous times, so that the predicted SOC after the sixth instruction point rises back to the safe range.

[0104] After completing the dual safety boundary verification of power and state of charge, the instruction generation module receives real-time automatic generation control instructions from the power grid dispatch center, such as an instruction requiring an increase of 0.8 MW of output in the next time period. The instruction generation module algebraically superimposes the power instruction sequence after safety boundary verification with this real-time dispatch instruction to form a preliminary joint control instruction sequence. The superposition is based on the principle of time matching. In specific implementation, assuming that the real-time dispatch instruction acts on the third instruction period, the instruction generation module adds the power instruction value of -0.5 MW after verification in the third period to the positive 0.8 MW required by the dispatch instruction to obtain the preliminary joint control instruction of +0.3 MW for that period. Refer to Table 2 to show the processing process from the optimal instruction to the preliminary joint instruction.

[0105] Table 2: Preliminary Joint Control Command Table Generated by Security Boundary Verification and Command Overlay

[0106]

[0107] In practical implementation, the instruction generation module performs low-pass filtering on the initial joint control instruction sequence. The module first determines the maximum allowable power change rate of the energy storage converter during charging / discharging state switching. This parameter is determined by the physical characteristics of the equipment, for example, set to 0.5 megawatts per second. The instruction generation module sets this maximum allowable power change rate as a rigid constraint condition for the filtering process. The module designs a finite impulse response digital low-pass filter to filter out high-frequency step components in the instructions. In some embodiments, optionally, the filter's cutoff frequency... The design is based on system response requirements and sampling period, and the calculation formula is as follows:

[0108]

[0109] in: Indicates the cutoff frequency of the filter. This represents the desired system response time constant. The selection must ensure that it is within the instruction update cycle. Within this range, the maximum change in power command shall not exceed the maximum permissible rate of change in power. and The product of these two signals can be understood as ensuring the smoothness of the instructions and meeting the responsiveness of the physical devices. The instruction generation module takes the initial joint control instruction sequence as the input signal and performs convolution calculation through the designed finite impulse response digital low-pass filter to obtain the filtered instruction sequence.

[0110] The instruction generation module examines the difference sequence of the filtered instruction sequence and calculates the power change rate between adjacent instruction points. For example, if the first instruction value after filtering is 0.5 MW and the second instruction value is -0.5 MW, with a time interval of 900 seconds, the change rate is -0.0011 MW / s. The instruction generation module verifies that the absolute value of the change rate between all adjacent instructions is less than or equal to the maximum allowable power change rate of 0.5 MW / s. For minor fluctuations that may not meet the change rate constraint due to filtering, the instruction generation module uses linear interpolation for final correction. In specific implementation, assuming that the calculated change rate for an adjacent instruction point is 0.52 MW / s, slightly exceeding the constraint, the instruction generation module interpolates between these two instruction points. A new command point is input, and the power value at that point is calculated through linear interpolation to ensure that the rate of change between adjacent points strictly meets the constraints, thus generating a smooth command sequence that finally meets the physical response constraints. The command generation module divides the smoothed command sequence into multiple consecutive control periods according to the time resolution of the scheduling cycle, and assigns a unique time period identifier and power setpoint to each control period. For example, a structured time-segmented charging and discharging control strategy is formed, such as "Time period 1: 14:00-14:15, power setpoint +0.48MW; Time period 2: 14:15-14:30, power setpoint -0.52MW". It can be understood that this strategy clarifies the specific power setpoint that the energy storage converter should execute in each time interval.

[0111] In one embodiment of the present invention, the online strategy self-update module is used to continuously collect actual operating data of the energy storage battery pack after implementing the time-sharing charge-discharge control strategy, including the actual charge-discharge power curve, the actual state-of-charge trajectory, and the actual net power curve of the site. The actual operating data is compared with the predicted data or expected state used in the strategy generation stage to calculate the strategy execution deviation, including power tracking deviation, state-of-charge prediction deviation, and economic benefit deviation. When the statistical mean of the strategy execution deviation exceeds a preset update threshold, the online strategy learning process is triggered. In the online strategy learning process, recently generated actual operating data and corresponding historical site state data are combined to form new training samples, and incremental training is performed on the evaluation network and strategy network in the improved deep deterministic strategy gradient algorithm. The network parameters after incremental training are used to update the online inference model, realizing the online adaptive adjustment of the charge-discharge strategy optimization control system.

[0112] In practical implementation, the online strategy self-updating module continuously collects actual operating data of the energy storage battery pack after implementing the time-sharing charge-discharge control strategy. Assuming an operating cycle from 14:00 to 15:00, the command issuing module controls the energy storage battery pack to execute the time-sharing charge-discharge control strategy issued by the command generation module. The online strategy self-updating module collects the actual charge-discharge power curve of the energy storage battery pack during this period from the energy storage battery management system, the actual state-of-charge trajectory from the battery monitoring unit, and the actual net power curve of the site from the site energy management system. In practical implementation, the online strategy self-updating module compares the actual operating data with the predicted data or expected state used in the strategy generation stage to calculate the strategy execution deviation. The online strategy self-updating module compares the actual charge-discharge power curve with the optimal charge-discharge power command sequence output by the strategy optimization module, calculates the difference between the two at each sampling point, and calculates the root mean square error of the difference over the entire cycle. This root mean square error is used as the power tracking deviation. The online strategy update module compares the actual state-of-charge (SOC) trajectory with the SOC trajectory predicted by the battery model based on the optimal charge / discharge power command sequence, and calculates the absolute difference between the two at the end of the cycle. This absolute difference is taken as the SOC prediction deviation. Based on the actual net power curve and real-time electricity price signal, the online strategy update module calculates the actual economic benefits gained by the power station through energy storage operations within the cycle, and compares this with the economic benefits predicted by the strategy optimization module when generating commands. The relative difference between the two is taken as the economic benefit deviation.

[0113] In practical implementation, when the statistical mean of power tracking deviation, state of charge prediction deviation, and economic benefit deviation exceeds a preset update threshold, the online policy self-update module triggers the online policy learning process. The preset update threshold can be set as power tracking deviation greater than 0.1 MW, state of charge prediction deviation greater than 0.05, and economic benefit deviation greater than 5%. Assuming that the calculated power tracking deviation within a statistical window is 0.15 MW, the state of charge prediction deviation is 0.08, and the economic benefit deviation is -7%, the online policy self-update module determines that the statistical mean of these three deviations has exceeded their respective thresholds and then triggers the online learning process. Optionally, the online policy self-update module can use a comprehensive indicator to determine whether to trigger an update: total policy execution deviation. It can be calculated as:

[0114]

[0115] in: Indicates the overall strategy execution deviation. This represents the normalized power tracking error. This indicates the normalized state of charge prediction bias. This indicates the deviation in normalized economic returns. , , These are preset weighting coefficients, which are used when the overall strategy execution deviation... The online learning process is triggered when the preset threshold is exceeded.

[0116] In the online learning process of the strategy, the online self-updating module combines recently generated actual operating data with corresponding historical site status data to form new training samples. The module extracts historical site status data from the database used by the strategy optimization module for each decision during the 14:00-15:00 operating cycle. This includes real-time power curves of photovoltaic power generation units, output fluctuation sequences of wind turbine generators, historical state of charge of energy storage battery packs, and real-time electricity price signals at the site's grid connection point. The module combines the historical status data at each decision moment, the actual action commands issued by the strategy at that moment (mapped from the actual charge / discharge power curve), and the reward value calculated based on subsequent actual operating results (e.g., actual revenue calculated by combining actual net power and electricity price) into a new training sample. In practice, the online self-updating module uses this new batch of training samples to incrementally train the evaluation network and policy network in the improved deep deterministic policy gradient algorithm. The incremental training process does not reinitialize the network parameters; instead, it calculates the gradient of the loss function based on the existing evaluation network and policy network parameters using the new training samples and updates the network parameters with a small learning rate. It is understandable that incremental training enables the evaluation network and policy network to learn the latest dynamic characteristics of the system and the effectiveness of policy execution based on feedback from actual operation.

[0117] In some embodiments, after incremental training is completed, the online self-updating module updates the online inference model using the parameters of the incrementally trained evaluation network and policy network. The online self-updating module then deploys the updated network parameter file to the policy optimization module, replacing the original online policy network and online evaluation network parameters in the policy optimization module. The update process is typically completed silently during periods of low system load or before the start of the next scheduling cycle, enabling online adaptive adjustment of the charging / discharging policy optimization control system. This allows subsequent policy generation to be based on a model that more closely reflects the actual system characteristics. It can be understood that by continuously collecting actual operating data, calculating execution deviations, and triggering incremental training and model updates based on actual data when the deviation exceeds a threshold, the online self-updating module enables the entire control system to adapt to changes in operating conditions and continuously improve policy performance.

[0118] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention in any other way. Any person skilled in the art may make changes or modifications to the above-disclosed technical content to create equivalent embodiments that can be applied to other fields. However, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the protection scope of the present invention.

Claims

1. An automatic optimization control system for charging and discharging strategies in new energy power stations, characterized in that, The system includes: The data acquisition module collects multi-source heterogeneous operation data of the new energy power station. The multi-source heterogeneous operation data includes the real-time power curve of the photovoltaic power generation unit, the output fluctuation sequence of the wind turbine generator, the historical state of charge of the energy storage battery pack, and the real-time electricity price signal of the power station's grid connection point. The data preprocessing module performs spatiotemporal alignment and data cleaning on the multi-source heterogeneous operating data to generate a standard format time series dataset of the station's operating status. The strategy optimization module calls an improved deep deterministic strategy gradient algorithm to perform rolling optimization analysis on the time series dataset of the power station's operating status. The improved deep deterministic strategy gradient algorithm is optimized based on the multi-timescale coupling characteristics of the new energy power station. Using the improved deep deterministic strategy gradient algorithm, the optimal charging and discharging power command sequence of the energy storage battery pack in the next scheduling cycle is obtained. The instruction generation module performs safety boundary verification and instruction smoothing processing based on the optimal charging and discharging power instruction sequence and the real-time acquired power grid dispatch instructions, and generates the final executable time-segmented charging and discharging control strategy. The instruction sending module sends the time-sharing charging and discharging control strategy to the energy storage converter, controlling the energy storage battery pack to perform the corresponding charging or discharging operations.

2. The automatic optimization control system for charging and discharging strategies of new energy power stations according to claim 1, characterized in that, The multi-source heterogeneous operational data is spatiotemporally aligned and cleaned to generate a standard-format time-series dataset of station operational status, including: Identify the data timestamps and sampling frequencies of the real-time power curve of the photovoltaic power generation unit, the output fluctuation sequence of the wind turbine generator set, the state of charge history of the energy storage battery pack, and the real-time electricity price signal of the power station grid connection point; Using the data with the highest sampling frequency as a benchmark, cubic spline interpolation is used to unify the time scale of the data with the low sampling frequency, so that all data sequences have the same and equally spaced timestamps. Outlier detection and processing are performed on the time-aligned data sequence. A local outlier factor algorithm is used to identify and remove transient noise points in the power data. Linear interpolation is used to fill the data gaps formed after the removal of the transient noise points. The real-time power curve of the photovoltaic power generation unit and the output fluctuation sequence of the wind turbine generator are normalized to map the power value to the range of zero to one. The normalized power data, the unnormalized state-of-charge history of the energy storage battery pack, and the real-time electricity price signal of the power station's grid connection point are integrated into a time-series dataset of the power station's operating status with a unified field format, arranged in timestamp order.

3. The automatic optimization control system for charging and discharging strategies of new energy power stations according to claim 1, characterized in that, The improved deep deterministic strategy gradient algorithm is optimized based on the multi-timescale coupling characteristics of new energy power plants, and its working steps include: A dual-network architecture consisting of an evaluation network and a policy network is constructed. The policy network is used to output continuous charging and discharging power actions based on the current operating state of the site, while the evaluation network is used to evaluate the long-term expected benefits of the state-action pair. In the input of the evaluation network, features representing coupling relationships at different time scales are explicitly introduced, including the input of second-level photovoltaic fluctuation variance, minute-level load change trend and hour-level electricity price cycle pattern after feature splicing. During the learning process of the policy network, a time scale consistency constraint is added. The time scale consistency constraint requires that when the numerical set of the system running state variables is the same but the dominant time scale characteristics are different, the output action of the policy network should satisfy a smooth transition relationship. In the environment interaction stage of algorithm training, a multi-timescale hybrid reward function is designed. The multi-timescale hybrid reward function simultaneously includes fluctuation smoothing reward calculated based on second-level data, tracking and scheduling instruction reward calculated based on minute-level data, and economic benefit reward calculated based on hour-level data. A hierarchical experience replay mechanism is adopted to store training data. The data is classified and stored into different experience pools according to the dominant time scale characteristics in the interaction data. During training, samples are taken from each experience pool proportionally to balance the learning of the operating modes at different time scales.

4. The automatic optimization control system for charging and discharging strategies of new energy power stations according to claim 3, characterized in that, The input to the evaluation network explicitly incorporates features characterizing coupling relationships at different time scales, specifically including: From the time-series dataset of the station's operating status, the real-time power curve of the photovoltaic power generation unit within a preset time window is extracted, and the variance of the power within the time window is calculated as a second-level fluctuation feature. From the time series dataset of the station's operating status, extract the output fluctuation sequence of the wind turbine generator within a longer preset time window, and fit the slope of its linear trend line as the minute-level change trend feature. From the real-time electricity price signal of the grid connection point of the power station, identify the time and magnitude of the price peak and trough within a complete electricity price cycle in the past, and encode them into a cyclic feature vector as an hourly electricity price cycle pattern feature. The second-level fluctuation characteristics, the minute-level change trend characteristics, and the hour-level electricity price cycle pattern characteristics are concatenated to form a multi-dimensional coupled feature vector; The multidimensional coupled feature vector is merged with the current basic state vector of the station and used as the input of the evaluation network.

5. The automatic optimization control system for charging and discharging strategies of new energy power stations according to claim 1, characterized in that, Using the improved deep deterministic policy gradient algorithm, the optimal charge / discharge power command sequence for the energy storage battery pack in the next scheduling cycle is obtained, including: The latest time period data in the time series dataset of the station's operating status is used as the current state input of the algorithm; The policy network in the improved deep deterministic policy gradient algorithm is deployed in online inference mode, and the policy network directly outputs a suggested charge / discharge power action value based on the current state. The current state and the suggested charge / discharge power action values ​​are input into the evaluation network of the improved deep deterministic policy gradient algorithm to obtain the long-term expected return estimate of the current state-action pair. Using a model predictive control framework, starting from the current moment, the state transition and action selection process is executed forward in the virtual environment, simulating the state transition and action selection at each step in a future scheduling cycle, and accumulating to calculate the total virtual long-term expected revenue. The action output of the policy network at each step in the rolling optimization is adjusted until the virtual total long-term expected benefit reaches its maximum. At this point, the sequence of charging and discharging power actions in the next scheduling cycle starting from the current moment is recorded, which is the optimal charging and discharging power instruction sequence.

6. The automatic optimization control system for charging and discharging strategies of new energy power stations according to claim 5, characterized in that, Based on the optimal charging and discharging power command sequence and the real-time acquired grid dispatch commands, safety boundary verification and command smoothing are performed to generate a final executable time-segmented charging and discharging control strategy, including: Obtain the real-time safe operating boundary parameters of the energy storage battery pack, including the current maximum allowable charging power, maximum discharging power, upper limit of state of charge, and lower limit of state of charge. Each power command value in the optimal charge and discharge power command sequence is compared with the real-time safe operation boundary parameter. If the power command value exceeds the maximum charging power or maximum discharging power limit, it is clamped to the corresponding limit. If the predicted state of charge will exceed the upper or lower limit after executing the generated power command, the power command is adjusted in reverse to bring the predicted state of charge back to the safe range. Receive real-time dispatch instructions issued by the automatic generation control of the power grid, and superimpose the power instruction sequence after safety boundary verification with the real-time dispatch instructions to form a preliminary joint control instruction sequence; The preliminary joint control command sequence is subjected to low-pass filtering to remove high-frequency step components generated by optimization calculations, ensuring that the command change rate is within the physical response capability range of the energy storage converter. The smoothed instruction sequence is divided into multiple consecutive control periods according to the time resolution of the scheduling cycle, and each control period is assigned a unique time period identifier and power setting value to form the structured time-segmented charging and discharging control strategy.

7. The automatic optimization control system for charging and discharging strategies of new energy power stations according to claim 6, characterized in that, The preliminary joint control command sequence is subjected to low-pass filtering, including: Determine the maximum allowable power change rate of the energy storage converter when switching between charging and discharging states, and set it as a constraint condition for the filtering process; Design a finite impulse response digital low-pass filter whose cutoff frequency is calculated based on the maximum allowable power change rate and the update cycle of the control command; The initial joint control command sequence is used as the input signal and convolved through the finite impulse response digital low-pass filter to obtain the filtered command sequence. Check the difference sequence of the filtered instruction sequence to ensure that the rate of change between any adjacent instructions is less than or equal to the maximum allowable rate of change of power. For local segments that do not meet the rate of change constraint, linear interpolation is used for correction to generate a smooth command sequence that finally meets the physical response constraint.

8. The automatic optimization control system for charging and discharging strategies of new energy power stations according to claim 2, characterized in that, The process of using the data with the highest sampling frequency as a benchmark and employing cubic spline interpolation to unify the time scale of the data with lower sampling frequencies ensures that all data sequences have the same and equally spaced timestamps, including: From the real-time power curve of the photovoltaic power generation unit, the output fluctuation sequence of the wind turbine generator set, the state of charge history of the energy storage battery pack, and the real-time electricity price signal of the grid connection point of the power station, the data sequence with the highest sampling frequency is identified, and the set of timestamps of the data sequence is determined as the target unified time reference. For each data sequence with a sampling frequency lower than the highest sampling frequency, a piecewise cubic polynomial function is constructed to represent the continuous change relationship of the data sequence based on its original non-equal interval timestamps and corresponding data values. Substitute each equally spaced time point in the target unified time base into the corresponding piecewise cubic polynomial function for calculation to obtain the interpolation estimate of the data sequence at each target time point; The calculated interpolated estimate is compared with the original value of the data sequence at the target time point. For time points where the original value exists, the interpolation accuracy is evaluated by calculating the residual between the interpolated estimate and the original value. After interpolating all low-sampling-frequency data sequences, all data sequences are rearranged according to the target unified time base to form multiple aligned data sequences with the same and equally spaced timestamps, providing a foundation for subsequent outlier detection and data integration.

9. The automatic optimization control system for charging and discharging strategies of new energy power stations according to claim 5, characterized in that, The aforementioned model predictive control framework, starting from the current moment, performs state transitions and action selection processes forward in the virtual environment, simulating the state transitions and action selections at each step within a future scheduling cycle, and cumulatively calculating the total virtual long-term expected revenue, including: In the model predictive control framework, the current system state is used as the initial state for rolling optimization, and the system state includes the latest station operating state extracted from the station operating state time series dataset. Initialize an empty virtual action sequence and an empty virtual state sequence, and store the initial state into the virtual state sequence; In the virtual environment, starting from the current moment, the simulation progresses forward with a fixed control step size. For each future moment in the process, the following simulation steps are executed: The current online policy network is loaded from the improved deep deterministic policy gradient algorithm, and the last state in the virtual state sequence is input into the online policy network to obtain a suggested virtual charge / discharge power action. The current simulated future time, the last state in the virtual state sequence, and the obtained suggested virtual charge / discharge power action are input into a preset station state transition model to predict the system state at the next moment after the suggested virtual charge / discharge power action is executed. The predicted system state at the next moment is stored in the virtual state sequence, and the suggested virtual charging and discharging power actions are stored in the virtual action sequence. The online evaluation network is loaded from the improved deep deterministic policy gradient algorithm to calculate the single-step long-term expected return of the state-action pair at the current simulation time, and the long-term expected return is accumulated into the virtual total return. Repeat the simulation steps until a complete future scheduling cycle is covered; Ultimately, we obtain a virtual state evolution trajectory from the current moment to the end of a future scheduling cycle, a corresponding virtual action sequence, and a cumulatively calculated virtual total long-term expected return.

10. The automatic optimization control system for charging and discharging strategies of new energy power stations according to claim 1, characterized in that, The system also includes: The strategy online self-updating module is used to continuously collect the actual operating data of the energy storage battery pack after executing the time-sharing charge and discharge control strategy, including the actual charge and discharge power curve, the actual state of charge trajectory and the actual net power curve of the site. The actual operating data is compared with the predicted data or expected state used in the strategy generation stage to calculate the strategy execution deviation, including power tracking deviation, state of charge prediction deviation and economic benefit deviation. When the statistical mean of the strategy execution deviation exceeds the preset update threshold, the online learning process of the strategy is triggered; In the online learning process of the strategy, the recently generated actual operation data and the corresponding historical status data of the site are combined to form a new training sample, and the evaluation network and policy network in the improved deep deterministic policy gradient algorithm are incrementally trained. By updating the online inference model using the network parameters after incremental training, the online adaptive adjustment of the charging and discharging strategy optimization control system can be achieved.

Images