Optical storage system economic dispatching method fusing uncertainty information

Abstract

The invention belongs to the field of optical storage system scheduling, particularly relates to an optical storage system economic scheduling method fusing uncertain information, and aims to improve the scheduling intelligence level of a photovoltaic energy storage system in an uncertain environment. The method comprises the following steps: acquiring photovoltaic array generation power and meteorological data through a data acquisition module, and dividing the photovoltaic array generation power and meteorological data into a training set and a test set after preprocessing; and training a time sequence convolutional network by using the training set, performing photovoltaic power day-ahead point prediction through the time sequence convolutional network, and generating a multi-confidence interval prediction result based on prediction error distribution. And a compressed sensing and sparse coding method is adopted, an interval prediction result is converted into a sparse connection weight matrix, and after reservoir network initialization is completed and spectral radius normalization is carried out, a policy network and a value network embedded with a near-end policy optimization algorithm are constructed based on the initialized reservoir network. And iteratively training the strategy network and the value network in a preset photovoltaic power station simulation environment, and outputting a day-ahead scheduling strategy meeting the economic optimum.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of optical storage system scheduling, in particular to an optical storage system economic scheduling method fusing uncertainty information. BACKGROUND

[0002] With the continuous improvement of the penetration rate of photovoltaic power generation in the power system, the intermittency and randomness of its output power have become a key factor restricting the safe and stable operation of the power grid. As an important means of adjusting new energy fluctuations, achieving energy time shift and peak load shifting, the reasonable configuration and efficient scheduling of energy storage systems have core values in improving new energy consumption capacity and ensuring system economy and security.

[0003] In the scenario of large-scale photovoltaic energy storage collaborative scheduling, the scheduling model needs to consider multi-dimensional state variables and continuous decision variables at the same time, resulting in an exponential growth of the system state action space, greatly increasing the computational complexity and optimization difficulty. Traditional reinforcement learning algorithms face challenges such as slow convergence speed of strategies and weak generalization ability in such problems, and due to the natural uncertainty of photovoltaic power prediction, the future state has strong fuzziness and unknowability, and the traditional algorithm model often fails to implement the scheduling strategy. SUMMARY

[0004] The purpose of the present application is to provide an optical storage system economic scheduling method fusing uncertainty information, aiming to improve the intelligent level of photovoltaic energy storage system scheduling in an uncertain environment.

[0005] To achieve the above purpose, the technical scheme adopted by the present application is as follows: The present application provides an optical storage system economic scheduling method fusing uncertainty information, comprising: S1: obtaining photovoltaic array power generation power and meteorological data through a data acquisition module, and dividing them into a training set and a test set after preprocessing; wherein the data acquisition module refers to a combination of hardware devices for real-time acquisition of photovoltaic array power generation power and meteorological data, and preprocessing includes data cleaning and normalization processing; S2: training a time series convolution network using the training set, performing photovoltaic power day-ahead point prediction through the time series convolution network, and generating multi-confidence interval prediction results based on the prediction error distribution; S3: using compressed sensing and sparse coding methods, converting the interval prediction results into a sparse connection weight matrix, completing the initialization of the reservoir network and performing spectral radius normalization, and then based on the initialized reservoir network, constructing a strategy network and a value network embedded with a proximal policy optimization algorithm; S4: in a preset photovoltaic power station simulation environment, iteratively training the strategy network and the value network, and outputting a day-ahead scheduling strategy that meets the economic optimization; wherein the photovoltaic power station simulation environment refers to a simulation system containing various revenue models, cost models and constraint conditions.

[0006] In step S1, data is collected by voltage sensors, current sensors, temperature sensors, irradiance sensors and wind speed sensors; the Z-score standardization method is used for preprocessing to map the data measured by the sensors to the [0, 1] interval.

[0007] In step S2, the upper and lower bounds of the power prediction interval under different confidence levels are constructed by a probability statistical method based on the inverse function of the cumulative distribution function of the prediction error.

[0008] In step S3, the construction of the sparse connection weight matrix includes: based on the compressed sensing theory, an optimization model containing a robust loss function and a regularization constraint is constructed, and a sparse reconstruction matrix is solved by an iterative shrinkage threshold algorithm.

[0009] The robust loss function adopts a robust function based on a scaling factor, and the regularization constraint includes an L1 norm constraint on the sparse reconstruction matrix and a Frobenius norm constraint on the noise-induced matrix.

[0010] In step S3, the spectral radius normalization refers to scaling the sparse connection weight matrix obtained by initialization according to the modulus of its maximum eigenvalue, so that the echo state characteristic of the reservoir network is satisfied.

[0011] In step S3, the policy network outputs the action probability distribution under the current state, and the value network outputs the expected return estimate of the current state; the policy network and the value network share the reservoir network structure.

[0012] The photovoltaic power station simulation environment in step S4 includes: a VRB energy storage system operation income model, a photovoltaic power station power grid operation income model, a VRB energy storage system investment cost model and a photovoltaic grid fluctuation overrun compensation model. The VRB energy storage system operation income model is as follows: wherein, is the operation income of the VRB energy storage system, is the discharge unit price of the VRB energy storage system at time t, is the charging unit price of the VRB energy storage system at time t, is the discharge power of the VRB energy storage system at time t, is the charging power of the VRB energy storage system at time t; The VRB energy storage system investment cost model is as follows: wherein, is the average installation cost of the VRB energy storage system at each time point. The average operation and maintenance cost per point in time for a VRB energy storage system. The unit power cost of VRB energy storage systems This refers to the rated power of the VRB energy storage system. The unit capacity cost of VRB energy storage systems, The rated capacity of the VRB energy storage system, For the discount rate, For the lifecycle of VRB energy storage systems, This is the annual operation and maintenance cost coefficient; The photovoltaic grid connection fluctuation over-limit compensation model is as follows: in, This is the compensation cost coefficient for excessive fluctuations in photovoltaic grid connection. The upper limit of volatility is determined based on the installed capacity and the target period. Let t be the actual fluctuation of the grid-connected power of the photovoltaic power station. For time t, the portion of the actual fluctuation rate of the grid-connected power of the photovoltaic power plant that exceeds the upper limit allowed by the grid. The constraints of the photovoltaic power plant simulation environment include: charging and discharging power constraints of the VRB energy storage system, capacity upper and lower limit constraints, and photovoltaic grid-connected power fluctuation constraints. The charging and discharging power constraints of the VRB energy storage system are as follows: in, It is a binary set, indicating that ESS is in a charging or discharging state; The upper and lower limits of capacity are constrained as follows: in, and These are the minimum and maximum values ​​of the capacity, respectively. The photovoltaic grid-connected power fluctuation constraint is as follows: in, This refers to the rated power value of the photovoltaic power station. and Representing the target period Time and The grid-connected power value of the photovoltaic power station at any given time.

[0013] The optimization objectives of the strategy training include: maximizing operating revenue, minimizing the investment cost of the VRB energy storage system, and minimizing the compensation cost for excessive fluctuations in photovoltaic grid connection.

[0014] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. The method provided by the embodiment of the present application is a photovoltaic energy storage system economic scheduling method fusing uncertainty information, which generates a multi-confidence interval prediction result through a time sequence convolution network, initializes a reserve pool network by combining a compressed sensing method and a sparse modeling method, enhances the perception and adaptability of the system to photovoltaic power fluctuations, avoids scheduling strategy failure caused by prediction errors, and improves the response capability of the photovoltaic energy storage system to uncertainty. In addition, the spectral radius normalization processing of the reserve pool network ensures its dynamic stability, and the sparse connection structure reduces the computational complexity, and the reserve pool network can still converge efficiently in a multi-dimensional state variable and continuous decision space, thereby greatly improving the intelligent level of the photovoltaic energy storage system in an uncertain environment and obtaining a matched scheduling strategy more efficiently and reliably.

[0015] 2. The reinforcement learning framework constructed based on the proximal policy optimization algorithm realizes multi-objective balance of maximum operation income, minimum energy storage cost, and minimum fluctuation overrun compensation cost, takes into account short-term income and long-term cost control, and optimizes the economy and stability of the scheduling strategy. BRIEF DESCRIPTION OF DRAWINGS

[0016] Figure 1 is a flowchart of the photovoltaic energy storage system economic scheduling method fusing uncertainty information provided by the embodiment of the present application; Figure 2 is a whole framework diagram of the photovoltaic energy storage system economic scheduling method fusing uncertainty information provided by the embodiment of the present application; Figure 3 is an initial weight matrix construction framework diagram of the reserve pool network provided by the embodiment of the present application; Figure 4 is a photovoltaic energy storage economic scheduling framework diagram based on the PPO algorithm provided by the embodiment of the present application. DETAILED DESCRIPTION

[0017] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the protection scope of the present application.

[0018] With reference to Figure 1 and Figure 2 , the present application provides a photovoltaic energy storage system economic scheduling method fusing uncertainty information, which comprises: S1: Obtain the photovoltaic array power generation power and meteorological data through the data acquisition module, and divide them into training set and test set after preprocessing. Among them, the data acquisition module refers to the combination of hardware devices for real-time acquisition of photovoltaic array power generation power and meteorological data, and preprocessing includes data cleaning and normalization processing.

[0019] In step S1, data is collected by voltage sensor, current sensor, temperature sensor, irradiance sensor and wind speed sensor, and after the sensor collects data, the data collected by the sensor is transmitted to the data collector sensing network through RS485 / 422 communication protocol, the data is monitored in real time, and the time information is recorded synchronously.

[0020] In preprocessing, data cleaning is mainly used to remove noise and outliers in original collected data, such as jump data and missing values caused by sensor failure, to avoid interference of false data on model training and ensure accuracy and reliability of input samples, providing high-quality basic data for subsequent prediction and scheduling model. Normalization processing maps photovoltaic power, meteorological data and the like of different orders of magnitude to a unified numerical interval. For example, the normalization processing uses Z-score standardization method to map the data measured by the sensor to the interval [0, 1].

[0021] In some embodiments, the preprocessed data is divided into training set and test set in the ratio of 8:2. According to the training set, a training data matrix is constructed, each row of which contains time stamp, photovoltaic power generation power, meteorological data and the like. When dividing, ensure the time sequence of data, and the test set does not overlap.

[0022] S2: Train the time series convolution network using the training set, make day-ahead point prediction of photovoltaic power through the time series convolution network, and generate multi-confidence interval prediction results based on prediction error distribution.

[0023] Step S2 aims to realize photovoltaic power prediction and quantify uncertainty. Training the time series convolution network using the training set and generating multi-confidence interval prediction results can improve the accuracy and reliability of photovoltaic power prediction. The time series convolution network captures the long and short term dependence relationship of time series data through dilated convolution, which helps to reduce single point prediction error. At the same time, the multi-confidence interval prediction generated based on error distribution breaks through the limitation of traditional point prediction which only provides a single value, and quantifies the fluctuation range of future power, providing data support for coping with uncertainty. By quantifying uncertainty, the reserve pool network constructed subsequently can more accurately capture the dynamic characteristics of the system, reduce the interference of extreme fluctuations on scheduling optimization, lay a data foundation for realizing economic optimal scheduling, and improve the stable operation ability of the photovoltaic storage system under complex working conditions.

[0024] Temporal Convolutional Networks (TCNs) are used to predict photovoltaic power day-ahead, which can obtain predicted values. The error is obtained by comparing the predicted values ​​with the actual values.

[0025] In step S2, the upper and lower bounds of the power prediction interval under different confidence levels are constructed using probabilistic statistical methods based on the inverse function of the cumulative distribution function of the prediction error.

[0026] S3: Using compressed sensing and sparse coding methods, the interval prediction results are transformed into a sparse connection weight matrix. After initializing the reserve pool network and normalizing the spectral radius, a policy network and a value network with embedded near-end policy optimization algorithm are constructed based on the initialized reserve pool network.

[0027] The process of using compressed sensing and sparse coding to process interval prediction results and construct the network enhances the network's ability to perceive uncertain information. Through compressed sensing and sparse coding, the photovoltaic power fluctuation range and probabilistic characteristics inherent in the interval prediction results are transformed into a sparse connection weight matrix. This embeds uncertain information into the storage pool network during initialization, enhancing its dynamic capture capability of photovoltaic output fluctuations and avoiding information loss caused by traditional random initialization. Spectral radius normalization ensures that the storage pool network satisfies echo state characteristics, guaranteeing the stability of its dynamic response and preventing divergence during training. Furthermore, the sparse connection structure significantly reduces the network parameter size and computational complexity, enabling the model to operate efficiently even when dealing with multidimensional scheduling variables.

[0028] For example, refer to Figure 3 In step S3, the construction of the sparse connection weight matrix includes: constructing an optimization model containing a robust loss function and regularization constraints based on compressed sensing theory, and solving the sparse reconstruction matrix through an iterative shrinkage threshold algorithm.

[0029] The robust loss function adopts a robust function based on the scaling factor, and the regularization constraints include the L1 norm constraint on the sparse reconstruction matrix and the Frobenius norm constraint on the noise-induced matrix.

[0030] In step S3, spectral radius normalization refers to scaling the sparse connection weight matrix obtained from initialization according to the modulus of its largest eigenvalue, so that the reservoir network satisfies the echo state characteristic.

[0031] In step S3, the policy network outputs the action probability distribution in the current state, and the value network outputs the expected return estimate in the current state; the policy network and the value network share the reserve pool network structure.

[0032] S4: In the preset photovoltaic power station simulation environment, the strategy network and the value network are iteratively trained, and an economic optimal day-ahead scheduling strategy is output. The photovoltaic power station simulation environment refers to a simulation system including various income models, cost models and constraint conditions.

[0033] The photovoltaic power station simulation environment in step S4 includes a VRB energy storage system operation income model, a photovoltaic power station power grid connection operation income model, a VRB energy storage system investment cost model and a photovoltaic grid fluctuation overrun compensation model.

[0034] For example, for constructing the VRB energy storage system operation income model, the VRB energy storage system operation income is generally related to the charging and discharging power, and can be specifically represented as: (1) wherein, represents the discharging unit price of the VRB energy storage system at time t, represents the charging unit price of the VRB energy storage system at time t, represents the discharging power of the VRB energy storage system at time t, represents the charging power of the VRB energy storage system at time t. For constructing the photovoltaic power station power grid connection operation income model, the planned operation income of the photovoltaic power station is an important indicator for measuring its economic benefits, mainly derived from the electricity sales revenue of the photovoltaic grid-connected power. In the economic benefits of the grid-connected photovoltaic power at time t can be represented by the following formula: (2) wherein, represents the electricity sales price to the grid, represents the photovoltaic power generation power.

[0035] For constructing the VRB energy storage system investment cost model, the investment cost of the VRB energy storage system mainly includes installation and operation and maintenance costs. In the day-ahead scheduling plan, the investment cost is originally calculated as an average investment cost of each time point according to the service life of the VRB energy storage system, in order to unify the time scale of configuration and operation and adapt to the optimization demand of each time point. Specifically, it can be represented by the following formula: (3) (4) (5) wherein, is the average installation cost of the VRB energy storage system at each time point. is the average operation and maintenance cost of the VRB energy storage system at each time point, The unit power cost of the VRB energy storage system, The rated power of the VRB energy storage system, The unit capacity cost of the VRB energy storage system, The rated capacity of the VRB energy storage system, The discount rate, The life cycle of the VRB energy storage system, The annual operation and maintenance cost coefficient. Since the VRB energy storage system relies on ion exchange to realize charging and discharging, there is no obvious aging mechanism in the battery. Therefore, the life cycle of the VRB energy storage system is a constant.

[0036] For the construction of the photovoltaic grid-connected fluctuation overrun compensation model, if the VRB energy storage system cannot suppress the photovoltaic grid-connected fluctuation, resulting in the photovoltaic grid-connected fluctuation exceeding the upper limit, then other units with flexible adjustment capacity must be used to adjust the excess fluctuation. Therefore, the operation cost of the system will inevitably increase, and the calculation formula of the cost is as follows: (6) Wherein, represents the photovoltaic grid-connected fluctuation overrun compensation cost coefficient, is the fluctuation rate upper limit specified according to the installed capacity and the target period, represents the actual fluctuation degree of the photovoltaic power station grid-connected power at the dispatching time step t, that is, at time t, represents the part of the actual fluctuation rate of the photovoltaic power station grid-connected power at the dispatching time step t that exceeds the grid-allowed upper limit.

[0037] The constraint conditions of the photovoltaic power station simulation environment include the charge and discharge power constraints of the VRB energy storage system, the capacity upper and lower limit constraints, and the photovoltaic grid-connected power fluctuation rate constraints.

[0038] In the charge and discharge power constraints of the VRB energy storage system, the VRB energy storage system has limited charging power and discharging power , wherein the charging power is negative and the discharging power is positive. and respectively represent the maximum power that can be charged from the system and discharged from the system.

[0039] (7) Wherein, is a binary set, indicating that the ESS is in a charging or discharging state. When represents the charging mode, and the capacity change of the VRB energy storage system can be represented as: (8) When represents the discharge mode. The capacity variation of the VRB energy storage system is shown as follows: (9) wherein, and are the energy storage values of the energy storage unit at and moments, represents the power loss rate, indicating the energy lost by the VRB energy storage system in the charging and discharging process.

[0040] For the upper and lower capacity constraints, since the VRB energy storage system has a limited capacity, in order to prevent overcharging or overdischarging of the VRB energy storage system, the specific constraints are as follows: (10) wherein, and are the minimum and maximum values of the defined capacity.

[0041] For the photovoltaic grid-connected power fluctuation rate constraint, according to the relevant technical regulations of the state on photovoltaic grid connection, the grid-connected power fluctuation should meet the fluctuation index requirements on the premise of not affecting the normal operation of the power grid. The fluctuation rate constraint of photovoltaic power in the target time period t can be given by the following expression: (11) (12) wherein, is the rated power value of the photovoltaic power station, and represent the grid-connected power values of the photovoltaic power station at moment and moment in the target period, respectively.

[0042] The optimization objectives of the strategy training include: maximizing the operating income, minimizing the investment cost of the VRB energy storage system, and minimizing the compensation fee for exceeding the limit of photovoltaic grid-connected fluctuation.

[0043] As a possible implementation manner, the specific expression of the optimization objective is as follows: (13) (14) wherein, represents the operating income of each time step, represents the limited dispatching range, are calculated by formulas (1) to (6), respectively.

[0044] The state vector and action vector of the optical storage system are defined, wherein the state vector reflects the current state of the photovoltaic power station, including photovoltaic power generation power, photovoltaic grid-connected power, energy storage system capacity, grid electricity selling price and grid fluctuation rate, which can be represented by the following formula: (15) A significant feature of the grid-connected operation of the photovoltaic power station is that the VRB energy storage system is the only decision variable, and its charging and discharging behavior directly affects the operating state of the system. Therefore, the operating mode and charging and discharging power of the VRB energy storage system are defined as the action vector, which is specifically represented as: (16) Based on the training set and test set divided in step S1, the TCN network is used for photovoltaic power day-ahead prediction, and the specific formula is as follows: (17) Wherein, in the formula, N is the historical feature matrix, is the TCN neural network, is the obtained photovoltaic power day-ahead prediction value.

[0045] The prediction interval is calculated by analyzing the difference between the prediction value of TCN and the actual observation value, i.e. the prediction error. The calculation formula of the prediction error is as follows: (18) The prediction error is -quantile which is obtained by estimating the inverse function of the cumulative distribution function, and the specific formula is as follows: (19) Wherein, is the estimated CDF of the prediction error, is the inverse function thereof.

[0046] In step S2, based on the obtained prediction error distribution, the photovoltaic power interval prediction results under different confidence levels are constructed by probability statistical method, and the confidence interval is defined using the quantile of the error distribution: (20) Wherein, ranges from 85% to 95% with a step of 5%. For each time point , the prediction interval of the actual value under different confidence is calculated by the following formula: (21) Wherein, and respectively represent the confidence of time ​ The upper and lower bounds of the prediction interval.

[0047] To characterize the underlying structural relationships in observed data and guide neural networks to form information-selective connection patterns, the method provided in this application introduces a sparse learning mechanism based on compressed sensing theory. In compressed sensing theory, sparse representation models assume that data can be reconstructed from a set of coefficients with a small number of non-zero elements. This idea can be formalized as the following optimization objective: (twenty two) in, Represents the interval prediction data matrix. For sparse reconstruction matrix, The noise-induced matrix; function and Used for constraints and This invention employs robust functions. ,in This is a scaling factor to enhance the model's robustness to noise.

[0048] In the learning process of the reconstructed matrix, the design of the robust loss function is crucial. To ensure the sparsity of the learned reconstructed matrix, this application focuses on sparse reconstructed matrices. Apply regularization (L1 norm) constraints, and simultaneously, apply them to the noise-induced matrix. Imposing the F-norm (Frobenius norm) constraint primarily aims to control the global energy of noise, thereby improving the robustness and stability of the model. The above optimization problem can be equivalently transformed into: (twenty three) in, This represents the scaling factor of the robust loss function. Describing the F-norm, Represents the L1 norm, parameters This objective function balances the sparsity of the matrix representation with the robustness of the model. It possesses both robustness and sparsity control capabilities, making it suitable for information compression modeling under uncertain environments.

[0049] To solve the aforementioned non-convex objective function, this invention employs the ISTA (Iterative Shrinkage-Thresholding Algorithm), which approximates the sparse solution by alternately performing gradient descent and soft thresholding operations. The final sparse reconstruction matrix is ​​then obtained. And use it as the initial weight matrix of the reserve pool network. .

[0050] For step S3, to ensure that the reservoir network satisfies the echo state property (ESP), the spectral radius of the reservoir weight matrix is usually normalized. For example, the target spectral radius is set to , and usually satisfies . By scaling the initial reservoir matrix according to its spectral radius, the normalized reservoir connection weight matrix is obtained, and the specific form is: (24) wherein represents the spectral radius of the matrix , that is, the modulus of the largest eigenvalue. The normalization operation can effectively ensure the stability of the network dynamics and enhance the response ability of the network to the input sequence.

[0051] For example, referring to Figure 4 , in step S3, the policy network and the value network of the embedding proximal policy optimization algorithm are constructed based on the initialized reservoir network, and the policy network receives the current environment state as input. The input is injected into the reservoir network through the input weight matrix . The state of the neurons inside the reservoir is updated according to the following formula: (25) wherein is a hyperbolic tangent activation function applied element by element, which provides nonlinear transformation capability. is the reservoir state at the previous time, which provides necessary time sequence memory. The output of the policy network, that is, the action under the given state is obtained by the following formula: (26) wherein is the only trainable parameter of the policy network, which is denoted as . To solve the optimal policy parameter , the PPO algorithm is used in the embodiments of the present application. The core of the algorithm is to construct a target function with a clipping mechanism, which can realize the improvement of the policy under the premise of ensuring the stable update of the policy. The target function with the clipping mechanism can be defined as: (27) (28) wherein is the policy network parameter before updating, and is the parameter state after the last iteration or before the current updating step; Step size representing policy network parameter update, used to control parameter iteration speed; denotes the gradient of the policy parameter; is the loss function of the policy network. Probability ratio denotes the probability ratio of the current policy and the old policy under the same state-action pair, as follows: (29) where, denotes the clipping of the probability ratio to ensure that the magnitude of policy update is controlled in the interval , so as to prevent the policy from changing drastically and improve the stability of the training process. Parameter is the clipping coefficient, which is set to 0.1 in the embodiments of the present application. denotes the estimated value of the advantage function, which is used to measure the pros and cons of selecting action under state relative to the average policy behavior.

[0052] For example, to construct a value network, the value network is also constructed on the reservoir computing framework, and its goal is to parameterize the approximation of the state value function that follows the policy and provide a benchmark for policy optimization. The reservoir state evolution value network receives the same environment state as input. The input is injected into the reservoir network through an independent input weight matrix . The update mechanism of its reservoir state is the same as that of the policy network, satisfying the following formula: (30) where, is the reservoir weight matrix that is exactly the same as the policy network, is the state of the value network reservoir at the previous moment. The output of the value network is the value scalar estimate of state , which is obtained by linear transformation of the high-dimensional dynamic features : (31) where, is the only trainable parameter set of the value network, denoted as . It maps the reservoir state to the real-valued state value estimate. The learning goal of the parameter is to minimize the mean square error between the value network prediction value and the target return , and its objective function is defined as follows: (32) (33) wherein, denotes the parameter state of the value network before the current iteration update step starts, is the loss function of the value network, E(·) is the mathematical expectation, is the target return, usually defined as the discount and future cumulative reward, to measure the long-term benefits of the action policy under the current state. By minimizing the above loss function, the estimation ability of the value network for the state value can be effectively improved, thereby providing accurate reference for policy optimization.

[0053] The reward function is used to measure the immediate feedback signal obtained by the agent from the environment after taking a specific action, as a key guiding signal in the learning process, by reflecting the positive or negative influence of the current behavior on the task goal, prompting the agent to adjust the policy to make optimal decisions. Its mathematical form is defined as follows (34) wherein, denotes the reward received by the agent at time step t, is the discount factor, which determines the degree of influence of future rewards on current returns.

[0054] Through the policy network and value network constructed in the iteration step S4, the network parameters are optimized through the loss function and the reward function, and finally the policy that satisfies the economic optimization of the day-ahead scheduling is obtained.

[0055] The present application is committed to improving the intelligent scheduling level of the photovoltaic energy storage system in an uncertain environment, and breaking through the limitations of traditional methods in state representation and optimization efficiency. The embodiment of the present application proposes an economic scheduling method of photovoltaic energy storage system integrating uncertainty information. First, a photovoltaic power prediction module based on time series convolution network (TCN) is built, and interval prediction results are generated by analyzing the prediction error distribution to quantify the fluctuation range of future power; then the principle of compressed sensing is introduced, a sparse connection weight matrix is constructed, and the reserve pool network is initialized with it, forming a dynamic state expression structure with uncertainty perception ability. At the same time, combined with the spectral radius normalization processing, it ensures that the reserve pool network meets the echo state characteristics, and improves the response performance to time series input. Further, embed this structure into the deep reinforcement learning framework based on proximal policy optimization (PPO), construct the policy network and value network, and realize the economic optimal scheduling of photovoltaic energy storage system through iterative optimization.

[0056] In the description of the present specification, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0057] The above merely illustrates the specific embodiments of the present application, but the protection scope of the present application is not limited thereto, any person skilled in the art can easily think of the changes or replacements within the technical range disclosed by the present application, which should be covered in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. An economic scheduling method for a photovoltaic-storage system that integrates uncertain information, characterized in that, include: S1: Acquire photovoltaic array power generation and meteorological data through the data acquisition module, and divide them into training set and test set after preprocessing; S2: Train a temporal convolutional network using the training set, perform day-ahead prediction of photovoltaic power using the temporal convolutional network, and generate multi-confidence interval prediction results based on the prediction error distribution; S3: Use compressed sensing and sparse coding methods to transform the interval prediction results into a sparse connection weight matrix, complete the initialization of the reserve pool network and normalize the spectral radius, and construct a policy network and a value network embedded with a near-end policy optimization algorithm based on the initialized reserve pool network; S4: In a preset photovoltaic power plant simulation environment, iteratively train the policy network and the value network, and output the day-ahead scheduling policy that satisfies economic optimality; wherein, the photovoltaic power plant simulation environment refers to a simulation system that includes various revenue models, cost models and constraints.

2. The method for economic scheduling of a photovoltaic-storage system incorporating uncertain information according to claim 1, characterized in that: In step S1, data is collected by voltage sensor, current sensor, temperature sensor, irradiance sensor and wind speed sensor; preprocessing uses Z-score normalization to map the sensor measured data to the [0,1] interval.

3. The economic scheduling method for a photovoltaic-storage system incorporating uncertain information according to claim 1, characterized in that: In step S2, the upper and lower bounds of the power prediction interval under different confidence levels are constructed using probabilistic statistical methods based on the inverse function of the cumulative distribution function of the prediction error.

4. The economic scheduling method for a photovoltaic-storage system incorporating uncertain information according to claim 1, characterized in that, In step S3, the construction of the sparse connection weight matrix includes: constructing an optimization model containing a robust loss function and regularization constraints based on compressed sensing theory, and solving the sparse reconstruction matrix through an iterative shrinkage threshold algorithm.

5. The economic scheduling method for a photovoltaic-storage system incorporating uncertain information according to claim 4, characterized in that: The robust loss function adopts a robust function based on the scaling factor, and the regularization constraints include the L1 norm constraint on the sparse reconstruction matrix and the Frobenius norm constraint on the noise-induced matrix.

6. The economic scheduling method for a photovoltaic-storage system incorporating uncertain information according to claim 1, characterized in that: In step S3, spectral radius normalization refers to scaling the sparse connection weight matrix obtained from initialization according to the modulus of its largest eigenvalue, so that the reservoir network satisfies the echo state characteristic.

7. The economic scheduling method for a photovoltaic-storage system incorporating uncertain information according to claim 1, characterized in that: In step S3, the policy network outputs the action probability distribution in the current state, and the value network outputs the expected return estimate in the current state; the policy network and the value network share the reserve pool network structure.

8. The economic scheduling method for a photovoltaic-storage system incorporating uncertain information according to claim 1, characterized in that, The photovoltaic power plant simulation environment in step S4 includes: VRB energy storage system operation revenue model, photovoltaic power plant grid-connected operation revenue model, VRB energy storage system investment cost model, and photovoltaic grid-connected fluctuation over-limit compensation model; the VRB energy storage system operation revenue model is as follows: in, Revenue from the use of VRB energy storage systems Let t be the discharge unit price of the VRB energy storage system. Let t be the charging unit price of the VRB energy storage system. Let be the discharge power of the VRB energy storage system at time t. Let t be the charging power of the VRB energy storage system at time t; The investment cost model for VRB energy storage systems is as follows: in, The average installation cost of a VRB energy storage system at each point in time; The average operation and maintenance cost per point in time for a VRB energy storage system. The unit power cost of VRB energy storage systems This refers to the rated power of the VRB energy storage system. The unit capacity cost of VRB energy storage systems, The rated capacity of the VRB energy storage system, For the discount rate, For the lifecycle of VRB energy storage systems, This is the annual operation and maintenance cost coefficient; The photovoltaic grid connection fluctuation over-limit compensation model is as follows: in, This is the compensation cost coefficient for excessive fluctuations in photovoltaic grid connection. The upper limit of volatility is determined based on the installed capacity and the target period. Let t be the actual fluctuation of the grid-connected power of the photovoltaic power station. For time t, the portion of the actual fluctuation rate of the grid-connected power of the photovoltaic power plant that exceeds the upper limit allowed by the grid.

9. The economic scheduling method for a photovoltaic-storage system incorporating uncertain information according to claim 8, characterized in that, The constraints of the photovoltaic power plant simulation environment include: charging and discharging power constraints of the VRB energy storage system, upper and lower capacity limits, and photovoltaic grid-connected power fluctuation constraints. The charging and discharging power constraints of the VRB energy storage system are as follows: in, It is a binary set, indicating that ESS is in a charging or discharging state; The upper and lower limits of capacity are constrained as follows: in, and These are the minimum and maximum values ​​of the capacity, respectively. The photovoltaic grid-connected power fluctuation constraint is as follows: in, This refers to the rated power value of the photovoltaic power station. and Representing the target period Time and The grid-connected power value of the photovoltaic power station at any given time.

10. The economic scheduling method for a photovoltaic-storage system incorporating uncertain information according to claim 1, characterized in that, The optimization objectives of the strategy training include: maximizing operating revenue, minimizing the investment cost of the VRB energy storage system, and minimizing the compensation cost for excessive fluctuations in photovoltaic grid connection.

Images