Photovoltaic energy storage joint optimization scheduling method and system based on deep reinforcement learning

Abstract

The application provides a photovoltaic energy storage combined optimization scheduling method and system based on deep reinforcement learning, relates to the technical field of energy management, and comprises the following steps: collecting multi-source heterogeneous time sequence data, extracting energy storage health state features through a double encoder-decoder network and a recursive graph convolution network, dividing energy storage power intervals, and constructing a feasible region; based on the power difference value sequence, extracting fluctuation features through segmentation, and constructing a risk evaluation index; generating multiple search paths in the feasible region, obtaining energy storage charging and discharging power time sequences through iterative search, and generating scheduling instructions. The method realizes photovoltaic energy storage combined optimization scheduling under the safety constraint of energy storage, improves the photovoltaic consumption rate, and improves the economic efficiency of the system.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of energy management, and in particular to a photovoltaic and energy storage joint optimization scheduling method and system based on deep reinforcement learning. BACKGROUND

[0002] With the increasing proportion of renewable energy in power systems, photovoltaic power generation is widely used due to its clean and environmentally friendly characteristics. However, the intermittency and volatility of photovoltaic power generation pose challenges to the safe and stable operation of the power grid. Energy storage systems, as an important regulating means, can effectively smooth the fluctuation of photovoltaic output and improve the stability and reliability of the power grid. The coordinated operation of photovoltaic and energy storage has become a key technology in new power systems, and reasonable photovoltaic and energy storage joint scheduling is of great significance to improve the economy and reliability of the system.

[0003] In the field of photovoltaic and energy storage joint optimization scheduling, traditional methods mainly rely on deterministic mathematical models and simple rule strategies. These methods are usually based on historical data and empirical rules, and use optimization algorithms such as linear programming and dynamic programming for solution. With the increasing complexity of power systems and the increasing uncertainty factors, in recent years, scheduling methods based on machine learning, especially deep reinforcement learning, have gradually attracted attention. Such methods can learn optimal strategies through interaction with the environment, and have strong adaptability and generalization ability.

[0004] However, existing photovoltaic and energy storage joint scheduling methods still have deficiencies. Existing methods generally ignore the health state evaluation of energy storage systems and fail to fully consider the aging characteristics and life decay factors of energy storage batteries, resulting in scheduling strategies that may accelerate the wear and tear of energy storage devices and reduce the service life of the energy storage system. Traditional scheduling algorithms are difficult to effectively process multi-source heterogeneous time series data sets, and the description of photovoltaic power fluctuation is not accurate enough. The risk assessment mechanism is lacking, and it is not flexible enough to respond to different power fluctuation scenarios, reducing the system's ability to adapt to extreme situations. Existing optimization algorithms are prone to local optimization in the search process and have low computational efficiency, making it difficult to meet the real-time scheduling needs of large-scale systems. The global optimization ability under complex constraints is limited, affecting the economy and feasibility of the scheduling scheme. SUMMARY

[0005] The embodiment of the present application provides a photovoltaic and energy storage joint optimization scheduling method and system based on deep reinforcement learning, which can solve the problems in the prior art.

[0006] In a first aspect, the embodiment of the present application provides a photovoltaic and energy storage joint optimization scheduling method based on deep reinforcement learning, comprising:

[0007] Collecting photovoltaic power generation power data, energy storage operating state data and power grid load data to form a multi-source heterogeneous time series data set;

[0008] Based on the multi-source heterogeneous time series data set, the energy storage features are extracted by a double encoder-decoder network structure and a recursive graph convolution network of dynamic time series graph structure to generate an energy storage health state feature vector; the comprehensive power feature value and the capacity threshold are calculated according to the energy storage health state feature vector, and the energy storage safe operation range is determined through a two-dimensional coordinate system;

[0009] According to the energy storage safe operation range, the energy storage power interval is divided, and the energy storage charging and discharging power feasible region is constructed in combination with the energy storage physical constraints;

[0010] The difference sequence of photovoltaic power generation power data and grid load data in the multi-source heterogeneous time series data set is calculated; the power fluctuation feature vector is extracted by segmenting the difference sequence, and the power fluctuation risk evaluation index is constructed based on the power fluctuation feature vector and the modified probability distribution parameters;

[0011] A plurality of parallel search paths are generated in the energy storage charging and discharging power feasible region, the power fluctuation risk evaluation index is used to divide the dominant search path and the inferior search path, and the energy storage charging and discharging power time sequence is obtained by iterative search and local optimal solution identification;

[0012] The energy storage dispatching instruction sequence is generated according to the energy storage charging and discharging power time sequence.

[0013] In an optional embodiment, based on the multi-source heterogeneous time series data set, the energy storage features are extracted by a double encoder-decoder network structure and a recursive graph convolution network of dynamic time series graph structure to generate an energy storage health state feature vector, which includes:

[0014] The multi-source heterogeneous time series data set is input into the double encoder-decoder network structure, and the initial feature matrix is obtained by time series feature extraction and state feature fusion, which contains the time series information and state information of the energy storage operation;

[0015] The dynamic time series graph structure is constructed based on the initial feature matrix, the nodes in the dynamic time series graph structure correspond to the energy storage operation states at different times, and the weights between the nodes in the dynamic time series graph structure are calculated to obtain the time series correlation weight matrix through the adaptive attention mechanism of thermodynamic constraints;

[0016] The recursive graph convolution network is constructed according to the time series correlation weight matrix, the convolution kernel parameters in the recursive graph convolution network are updated by the recursive unit of thermodynamic constraints, the initial hidden layer state of the recursive unit is set according to the thermodynamic constraints, and the thermodynamic time series features are obtained by performing graph convolution operation; the thermodynamic time series features and the initial feature matrix are fused through residual connection to obtain the fusion enhanced features;

[0017] The fusion enhanced features are multi-scale decomposed to obtain a multi-scale feature representation including global operation features, local fluctuation features and trend features; the multi-scale feature representation is input into a feature mapping network, and is reduced dimensionally to a low-dimensional manifold space through nonlinear projection transformation to obtain an energy storage health state feature vector.

[0018] In an optional embodiment, a comprehensive power feature value and a capacity threshold value are calculated according to the energy storage health state feature vector, and a safe operation range of the energy storage is determined through a two-dimensional coordinate system, including:

[0019] The energy storage health state feature vector is decomposed into a power feature component and a capacity feature component through nonlinear mapping, and the power feature component includes a charging power component and a discharging power component;

[0020] The charging power component and the discharging power component are multiplied by coupling coefficients of internal resistance and temperature of the energy storage to determine maximum charging power and maximum discharging power, and a comprehensive power feature value is calculated;

[0021] The capacity feature component is multiplied by weighting coefficients of cycle number and cycle depth and is accumulated to obtain a capacity threshold value;

[0022] A two-dimensional coordinate system is established with the comprehensive power feature value as the vertical coordinate and the capacity threshold value as the horizontal coordinate, a plurality of feature points are selected as region center points in the two-dimensional coordinate system, a region division diagram is constructed based on the region center points, distances from any point in the two-dimensional coordinate system to each region center point are calculated, and the region corresponding to the region center point closest to the corresponding point is taken as the region to which the corresponding point belongs; a classifier is constructed using a kernel function to determine the classification boundary of different regions; the comprehensive power feature value and the capacity threshold value are mapped to different regions of the two-dimensional coordinate system based on membership calculation, and the classification boundary of the different regions determines the safe operation range of the energy storage.

[0023] In an optional embodiment, a difference sequence of photovoltaic power generation power data and grid load data in a multi-source heterogeneous time series data set is calculated; the difference sequence is segmented and a power fluctuation feature vector is extracted, and a risk evaluation index is constructed based on the power fluctuation feature vector to correct the probability distribution parameters, including:

[0024] The difference sequence of the photovoltaic power generation power data and the grid load data is calculated;

[0025] The power fluctuation trend feature of the difference value sequence is calculated, a fluctuation feature vector is constructed according to a power rising rate, a falling rate and a duration, a double-threshold comparison method is used to identify a fluctuation conversion point, the difference value sequence is divided into a plurality of feature subsequences with the fluctuation conversion point as a boundary, a first-order difference and a second-order difference of each feature subsequence are calculated to construct a difference feature matrix, singular value decomposition is performed on the difference feature matrix to obtain a feature vector group, a power fluctuation feature vector is obtained by performing dynamic weight allocation on the feature vector group based on the energy storage health state feature vector, and a power fluctuation risk evaluation index is generated according to the direction and modulus value of the power fluctuation feature vector.

[0026] The difference value data in the energy storage power interval is counted, and a distribution feature parameter is calculated. The power fluctuation feature vector is used as a correction factor to adjust the distribution feature parameter to obtain a corrected probability distribution function.

[0027] The statistical feature quantity of the corrected probability distribution function is calculated, a risk evaluation threshold is determined in combination with the energy storage charge and discharge response characteristics, and a power fluctuation risk evaluation index is generated.

[0028] The power fluctuation feature vector is updated according to real-time operation data, and the power fluctuation risk evaluation index is dynamically adjusted.

[0029] In an optional embodiment, the double-threshold comparison method is used to identify the fluctuation conversion point, which includes:

[0030] Based on the power time sequence data, an instantaneous power change rate is determined.

[0031] The instantaneous power change rate is accumulated and summed in a sliding time window to obtain a power change accumulation, and a fluctuation persistence index is recorded by recording the duration that the sign of the instantaneous power change rate remains the same.

[0032] Based on the statistical distribution characteristics of the power time sequence data and the charge and discharge power limit of the energy storage device, upper and lower thresholds of the power change rate are determined, and based on the capacity constraint of the energy storage device and the length of the sliding time window, a threshold of the power change accumulation is determined.

[0033] It is judged whether the instantaneous power change rate exceeds the upper and lower thresholds of the power change rate, and whether the power change accumulation exceeds the threshold of the power change accumulation. In the time period meeting the double-threshold condition, the extreme point of the instantaneous power change rate is found, and is recorded as a candidate conversion point.

[0034] The time interval and the power change amplitude of adjacent candidate conversion points are calculated, a time threshold is set based on the fluctuation persistence index, an amplitude threshold is set based on the power change rate threshold, and the candidate conversion points with a time interval less than the time threshold and a power change amplitude less than the amplitude threshold are deleted.

[0035] The candidate conversion point determined after the screening is determined as a power fluctuation conversion point, and the power time series data is segmented based on the power fluctuation conversion point.

[0036] In an optional embodiment, a plurality of parallel search paths are generated within the energy storage charging and discharging power feasible region, the dominant search path and the inferior search path are divided based on the power fluctuation risk evaluation index, and the energy storage charging and discharging power time series is obtained by iterative search and local optimal solution identification, including:

[0037] An initial search point set is constructed within the energy storage charging and discharging power feasible region, and a plurality of parallel search paths are generated based on the initial search point set;

[0038] The power fluctuation risk evaluation index and the change trend of each parallel search path are calculated, and the dominant search path and the inferior search path are determined based on the dynamic weighted mean;

[0039] Each parallel search path is iteratively searched, the search step and the search direction of the dominant search path are extracted as feature parameters, and the search step and the search direction of the inferior search path are updated;

[0040] The evaluation value change amount of the parallel search path within a continuous preset number of iterations is recorded, and when the evaluation value change amount is less than a preset floating threshold, the current search point is recorded as a local optimal solution, a new search starting point is generated by random disturbance in the neighborhood of the current search point, and the set of local optimal solutions that meet the energy storage charging and discharging power feasible region constraint is selected as candidate solutions based on the new search starting point;

[0041] The solution with the optimal evaluation value in the candidate solution is selected to generate the energy storage charging and discharging power time series, and the energy storage charging and discharging power time series is adjusted online according to real-time operation data.

[0042] In an optional embodiment, the power fluctuation risk evaluation index and the change trend of each parallel search path are calculated, and the dominant search path and the inferior search path are determined based on the dynamic weighted mean, including:

[0043] The power fluctuation risk evaluation index of the plurality of parallel search paths is obtained as the evaluation value;

[0044] The difference of the evaluation value is calculated to obtain the change rate, and the difference of the change rate is calculated to obtain the change acceleration;

[0045] The evaluation value and the change rate are weighted and summed, and multiplied by the change acceleration to obtain the path score;

[0046] The dynamic weighted mean of the path score is calculated according to the weight coefficients of the real-time state adjustment of the evaluation value and the change rate;

[0047] Determine the parallel search path with a path score less than or equal to the dynamic weighted mean as a disadvantaged search path, and determine the parallel search path with a path score greater than the dynamic weighted mean as an advantageous search path.

[0048] In a second aspect of the embodiment of the present application, a photovoltaic energy storage joint optimization scheduling system based on deep reinforcement learning is provided, comprising:

[0049] A first unit is configured to collect photovoltaic power generation data, energy storage operation state data and power grid load data to form a multi-source heterogeneous time series data set;

[0050] A second unit is configured to extract energy storage features through a double encoder-decoder network structure and a recursive graph convolution network of a dynamic time series graph structure based on the multi-source heterogeneous time series data set to generate an energy storage health state feature vector; calculate a comprehensive power feature value and a capacity threshold value according to the energy storage health state feature vector, and determine an energy storage safe operation range through a two-dimensional coordinate system;

[0051] A third unit is configured to divide an energy storage power interval according to the energy storage safe operation range, and construct an energy storage charging and discharging power feasible region in combination with energy storage physical constraints;

[0052] A fourth unit is configured to calculate a difference sequence of the photovoltaic power generation data and the power grid load data in the multi-source heterogeneous time series data set; segment the difference sequence and extract a power fluctuation feature vector, and construct a power fluctuation risk evaluation index based on the power fluctuation feature vector to correct probability distribution parameters;

[0053] A fifth unit is configured to generate a plurality of parallel search paths in the energy storage charging and discharging power feasible region, divide an advantageous search path and a disadvantaged search path based on the power fluctuation risk evaluation index, and obtain an energy storage charging and discharging power time sequence through iterative search and local optimal solution identification;

[0054] A sixth unit is configured to generate an energy storage scheduling instruction sequence according to the energy storage charging and discharging power time sequence.

[0055] In a third aspect of the embodiment of the present application, an electronic device is provided, comprising:

[0056] A processor;

[0057] A memory for storing processor-executable instructions;

[0058] The processor is configured to invoke the instructions stored in the memory to execute the method described above.

[0059] In a fourth aspect of the embodiment of the present application, a computer readable storage medium is provided, which stores computer program instructions, and the computer program instructions are executed by a processor to implement the method described above.

[0060] In the embodiment of the present application, by collecting multi-source heterogeneous time series data sets and combining a dual codec network and a recursive graph convolution network of dynamic time series graph structure, the energy storage health state features can be accurately extracted, the energy storage safe operation range can be effectively determined, and the service life and operation safety of the energy storage system are improved; The power fluctuation risk evaluation index is innovatively constructed, the fluctuation risk of the system can be more accurately evaluated by segmenting the difference sequence and extracting the power fluctuation feature vector, and a reliable basis is provided for the energy storage scheduling decision; The optimization strategy of multiple parallel search paths is adopted, combined with the division mechanism of the advantage path and the disadvantage path, the search efficiency and result quality of the scheduling scheme are significantly improved, the efficient cooperative operation of the photovoltaic energy storage system is realized, and the system operation cost is reduced. BRIEF DESCRIPTION OF DRAWINGS

[0061] Figure 1 The flowchart of the photovoltaic energy storage joint optimization scheduling method based on deep reinforcement learning of the embodiment of the present application is shown in the figure.

[0062] Figure 2 The adaptive iterative optimization flowchart of the energy storage health state feature extraction is shown in the figure.

[0063] Figure 3 The data flowchart of the photovoltaic energy storage power fluctuation risk evaluation is shown in the figure. DETAILED DESCRIPTION

[0064] In order to make the purpose, technical scheme and advantages of the embodiment of the present application clearer, the technical scheme in the embodiment of the present application will be described clearly and completely in combination with the drawings in the embodiment of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor belong to the scope of protection of the present application.

[0065] The technical scheme of the present application will be described in detail in specific embodiments. The following several specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in some embodiments.

[0066] Figure 1 The flowchart of the photovoltaic energy storage joint optimization scheduling method based on deep reinforcement learning of the embodiment of the present application is shown in the figure. Figure 1 As shown in the figure, the method comprises:

[0067] Collecting photovoltaic power generation power data, energy storage operation state data and power grid load data to form a multi-source heterogeneous time series data set;

[0068] Based on the multi-source heterogeneous time series data set, the energy storage features are extracted through a double encoder-decoder network structure and a recursive graph convolution network of dynamic time series graph structure to generate an energy storage health state feature vector; the comprehensive power feature value and the capacity threshold are calculated according to the energy storage health state feature vector, and the energy storage safe operation range is determined through a two-dimensional coordinate system;

[0069] According to the energy storage safe operation range, the energy storage power interval is divided, and the energy storage charge and discharge power feasible region is constructed in combination with the energy storage physical constraints;

[0070] The difference sequence of photovoltaic power generation power data and grid load data in the multi-source heterogeneous time series data set is calculated; the difference sequence is segmented and the power fluctuation feature vector is extracted, and the power fluctuation risk evaluation index is constructed based on the power fluctuation feature vector and the probability distribution parameter correction;

[0071] A plurality of parallel search paths are generated in the energy storage charge and discharge power feasible region, the power fluctuation risk evaluation index is used to divide the dominant search path and the inferior search path, and the energy storage charge and discharge power time sequence is obtained through iterative search and local optimal solution identification.

[0072] According to the energy storage charge and discharge power time sequence, the energy storage scheduling instruction sequence is generated.

[0073] In an optional implementation, based on the multi-source heterogeneous time series data set, the energy storage features are extracted through a double encoder-decoder network structure and a recursive graph convolution network of dynamic time series graph structure to generate an energy storage health state feature vector, which includes:

[0074] The multi-source heterogeneous time series data set is input into the double encoder-decoder network structure, and the initial feature matrix is obtained through time series feature extraction and state feature fusion, which contains time series information and state information of energy storage operation;

[0075] Based on the initial feature matrix, a dynamic time series graph structure is constructed, the nodes in the dynamic time series graph structure correspond to the energy storage operation states at different times, and the weights between the nodes in the dynamic time series graph structure are calculated to obtain a time series correlation weight matrix through an adaptive attention mechanism of thermodynamic constraints;

[0076] According to the time series correlation weight matrix, a recursive graph convolution network is constructed, the convolution kernel parameters in the recursive graph convolution network are updated by a recursive unit of thermodynamic constraints, the initial hidden layer state of the recursive unit is set according to the thermodynamic constraints, and the thermodynamic time series features are obtained by performing graph convolution operation; the thermodynamic time series features and the initial feature matrix are fused through residual connection to obtain fusion enhanced features;

[0077] The fusion enhanced features are multi-scale decomposed to obtain a multi-scale feature representation containing global operation features, local fluctuation features and trend features; the multi-scale feature representation is input into a feature mapping network, and is reduced in dimension through nonlinear projection transformation to a low-dimensional manifold space to obtain an energy storage health state feature vector.

[0078] In a specific embodiment, the multi-source heterogeneous time series data is first input into a dual codec network structure for processing. The network receives heterogeneous data including energy storage system operation parameters, photovoltaic output power, grid load demand, etc., and performs preliminary processing on these data through a time series feature extraction module. The time series feature extraction module adopts a combination of convolution layers and recurrent layers, the convolution kernel size of the convolution layer is set to 3x3 and the step is 1, which is used to capture local time dependence; the recurrent layer uses a gated recurrent unit, and the hidden state dimension is set to 128, which is used to learn long time series patterns. The state feature fusion module adopts a multi-head attention mechanism, the number of heads is set to 8, and the attention dimension is 64, which adaptively weights and fuses features from different sources to obtain an initial feature matrix with a dimension of 256x32, where 256 represents the time step and 32 represents the feature dimension.

[0079] When constructing a dynamic time series graph structure based on the initial feature matrix, the energy storage operation state at each time is taken as a node in the graph, and the node feature dimension is 32. The connection weight between nodes is calculated through an adaptive attention mechanism with thermodynamic constraints. The mechanism first calculates the similarity between the node features of any two time points, using cosine similarity as the metric, and the similarity value ranges from -1 to 1. Then a thermodynamic constraint factor is introduced, which is set according to the thermodynamic characteristics of the energy storage system, considering parameters such as energy storage temperature change rate and entropy increase rate, and the constraint factor value ranges from 0 to 2. Multiply the similarity and the thermodynamic constraint factor, and then normalize it through the softmax function to obtain the final time series correlation weight matrix, with a dimension of 256x256, representing the correlation strength between 256 time steps.

[0080] The construction of the recurrent graph convolution network is based on the time correlation weight matrix. The network contains 3 layers of graph convolution layers, and the output feature dimensions of each layer are 64, 128 and 64 respectively. The graph convolution kernel parameters are updated by the thermodynamic constrained recurrent unit, and the recurrent unit adopts the long short-term memory network structure with a memory cell size of 128. The initial hidden layer state of the recurrent unit is set according to the thermodynamic constraint, and the initial temperature is set to 25 degrees Celsius and the initial entropy value is set to 85% of the standard value. When performing graph convolution operation, the features of each node are aggregated in a weighted sum manner, and the weights come from the time correlation weight matrix. After the graph convolution operation is completed, the thermodynamic time series features with a dimension of 256x64 are obtained. The thermodynamic time series features and the initial feature matrix are fused through a residual connection, which is specifically implemented by projecting the thermodynamic time series features to the same dimension 256x32 as the initial feature matrix, then performing element-level addition operation, and then processing through an activation function. The activation function selected is leakyReLU with a negative slope of 0.2, and finally the fusion enhanced features with a dimension of 256x32 are obtained.

[0081] The fusion enhanced features are decomposed into multi-scale, using the discrete wavelet transform method, and the db4 wavelet basis is selected with 3 decomposition layers. After decomposition, the global running features, local fluctuation features and trend features are obtained. The global running features reflect the overall running condition of the energy storage system with a dimension of 256x16; the local fluctuation features capture short-term fluctuations with a dimension of 256x8; and the trend features represent long-term trends with a dimension of 256x8. These three types of features are combined to form a multi-scale feature representation with a total dimension of 256x32.

[0082] The multi-scale feature representation is input into the feature mapping network for nonlinear projection transformation. The feature mapping network consists of three fully connected layers with 128, 64 and 32 neurons respectively, and the activation function is the tanh function. Through this nonlinear transformation, the high-dimensional features are reduced to a low-dimensional manifold space to obtain an energy storage health state feature vector with a dimension of 256x16.

[0083] In practical applications, taking a certain photovoltaic energy storage station as an example, 30 consecutive days of operation data are collected, including sampling points every 5 minutes, a total of 8640 time points. The data include battery pack voltage (range 360-420 volts), current (range -100 to 100 amperes), temperature (range 15-45 degrees Celsius), state of charge (range 10%-90%), photovoltaic output power (range 0-1000 kilowatts), etc. After inputting these data into the method, the initial feature matrix extracted can reflect the performance changes of the battery under different charge and discharge cycles. The dynamic time series diagram constructed clearly shows the phenomenon of increased state transition probability under high temperature conditions (above 38 degrees Celsius) and rapid charge and discharge (current change rate exceeds 20 amperes / minute). The recurrent graph convolution network captures the abnormal pattern of a 12% increase in battery internal resistance after 3 consecutive days of heavy load (discharge depth exceeds 60%). Multi-scale decomposition shows that under unstable cloudy days, the local fluctuation eigenvalue of the energy storage system increases by 35%, and the system regulation frequency increases. The final health status feature vector shows that the health status of the energy storage system is 85%, and the remaining service life is about 7 years. It is recommended to charge and discharge when the peak-valley price difference is greater than 0.4 yuan / kilowatt hour to prolong the battery life and improve the economic benefit.

[0084] As shown in Figure 2 , an adaptive iterative optimization flowchart for energy storage health status feature extraction is shown.

[0085] In an optional implementation, a comprehensive power eigenvalue and a capacity threshold value are calculated according to the energy storage health status feature vector, and a safe operation range of the energy storage is determined through a two-dimensional coordinate system, including:

[0086] The energy storage health status feature vector is decomposed into a power feature component and a capacity feature component through nonlinear mapping, and the power feature component includes a charging power component and a discharging power component;

[0087] The charging power component and the discharging power component are multiplied by the coupling coefficients of the internal resistance and the temperature of the energy storage to determine the maximum charging power and the maximum discharging power, and a comprehensive power eigenvalue is calculated;

[0088] The capacity feature component is multiplied by the weighted coefficients of the cycle number and the cycle depth and accumulated to obtain a capacity threshold value;

[0089] A two-dimensional coordinate system is established with the comprehensive power characteristic value as the ordinate and the capacity threshold value as the abscissa. A plurality of feature points are selected as region center points in the two-dimensional coordinate system, and a region division diagram is constructed based on the region center points. The distance from any point in the two-dimensional coordinate system to each region center point is calculated, and the region corresponding to the region center point closest to the distance is taken as the region to which the corresponding point belongs. A kernel function is used to construct a classifier to determine the classification boundary of different regions. The comprehensive power characteristic value and the capacity threshold value are mapped to different regions of the two-dimensional coordinate system based on membership calculation, and the classification boundary of the different regions determines the safe operation range of the energy storage.

[0090] In a specific embodiment, the health state feature vector of the energy storage device can be obtained by monitoring, which contains multiple parameters such as voltage, current, temperature, internal resistance, etc. Nonlinear mapping processing is performed on the feature vector, which can be decomposed into power characteristic components and capacity characteristic components by singular value decomposition method. For example, for a health state feature vector containing 25 parameters, the power characteristic components obtained after nonlinear mapping can include charging power component and discharging power component. Assuming that the charging power component is 3.8 units and the discharging power component is 4.2 units, these values reflect the charging and discharging capacity of the energy storage device in the current state.

[0091] When determining the maximum charging power and the maximum discharging power, it is crucial to consider the influence of the internal resistance and temperature of the energy storage device. The internal resistance coupling coefficient of the energy storage can be set to 0.85, and the temperature coupling coefficient can be set to 0.92 in a 25°C environment. Multiplying the charging power component of 3.8 units by these coupling coefficients, the maximum charging power is calculated to be 3.8x0.85x0.92=2.97 units. Similarly, the discharging power component of 4.2 units is calculated to be 3.30 units after the same calculation. The comprehensive power characteristic value can be obtained by taking the average of the two, i.e. (2.97+3.30) / 2=3.14 units. This value represents the comprehensive power capacity index of the energy storage device in the current state.

[0092] The processing of the capacity characteristic component needs to be combined with the cycle number and cycle depth for calculation. Assuming that the capacity characteristic component is 85 units, the cycle number weight coefficient is set to 0.008, and the cycle depth weight coefficient is 0.015. When the energy storage device has experienced 200 cycles with an average cycle depth of 0.6, the capacity threshold value is calculated to be 85x(1-0.008x200-0.015x0.6)=69.89 units. This value represents the capacity state threshold of the energy storage device under the current usage conditions, which is an important indicator for evaluating the life state of the energy storage.

[0093] A two-dimensional coordinate system is established, with the capacity threshold as the horizontal coordinate and the comprehensive power eigenvalue as the vertical coordinate. In this coordinate system, a plurality of representative feature points are selected as the region center points, such as point (70, 3.2), point (60, 2.8), point (50, 2.5), point (40, 2.0), and point (30, 1.5), etc. These points can be determined through historical data analysis or expert experience, and represent the characteristic performance of the energy storage device at different health states.

[0094] The region division adopts a distance calculation method. For any point (x, y) in the two-dimensional coordinate system, the Euclidean distance to each region center point is calculated. For example, the distance from point (65, 3.0) to the center point (70, 3.2) is [(65-70) 2 +(3.0-3.2) 2 ] 1 / 2 =5.04 units, and the distances to other center points can be calculated in the same way. The point belongs to the region where the nearest center point is located.

[0095] When constructing the kernel function classifier, a radial basis function can be selected as the kernel function. Assuming that a radial basis function with a parameter γ=0.1 is selected, for a boundary point (x, y), the kernel function value of the point to the region center point (xi, yi) can be calculated by exp(-0.1×[(x-xi) 2 +(y-yi) 2 ]). By comparing the kernel function values of different regions, the classification boundary of point (x, y) can be determined. For example, the kernel function values of point (55, 2.7) to the center point (60, 2.8) and the center point (50, 2.5) are calculated, and the values are 0.72 and 0.65, respectively, so the point belongs to the region corresponding to the first center point.

[0096] In the boundary determination process, the concept of membership degree is introduced to enhance the rationality of classification. For any point (x, y), the membership degree of the point to a region can be obtained through normalization processing. Assuming that the distances from point (62, 2.9) to the center points of the regions are 8.06, 2.24, 12.37, 22.47, and 32.57 units, respectively, and the normalized membership degrees are 0.13, 0.57, 0.10, 0.06, and 0.04, respectively, so the point belongs to the second region, and the membership degree is 0.57.

[0097] Through the above analysis, the classification boundary of different regions can be determined in the two-dimensional coordinate system, so as to delimit the safe operation range of the energy storage. For example, the region with membership greater than 0.5 can be defined as a safe operation region, the region between 0.3 and 0.5 as a warning region, and the region less than 0.3 as a dangerous region. For an energy storage device with a comprehensive power characteristic value of 3.14 units and a capacity threshold of 69.89 units, the membership is calculated to be 0.68, which is located in the safe operation region, indicating that the energy storage device has the ability to continue safe operation.

[0098] In the present embodiment, the energy storage health state feature vector can be monitored in real time, the comprehensive power characteristic value and the capacity threshold are obtained by the above calculation method, and the current state point is located in the two-dimensional coordinate system. According to the region where it is located, the safe operation state of the energy storage device is judged, which provides a basis for operation and maintenance decision. When the state point approaches the boundary of the warning region or the dangerous region, a warning information is sent to remind the management personnel to take corresponding measures to ensure the continuous and safe operation of the energy storage system.

[0099] In an alternative embodiment, a difference sequence of photovoltaic power generation power data and grid load data in a multi-source heterogeneous time series data set is calculated; the difference sequence is segmented and a power fluctuation feature vector is extracted, and a risk evaluation index is constructed based on the power fluctuation feature vector and the probability distribution parameter is corrected.

[0100] The difference sequence of photovoltaic power generation power data and grid load data is calculated;

[0101] The power fluctuation trend feature of the difference sequence is calculated, a fluctuation feature vector is constructed according to the power rise rate, the fall rate and the duration, a double-threshold comparison method is used to identify the fluctuation conversion point, the difference sequence is divided into a plurality of feature subsequences with the fluctuation conversion point as the boundary, the first order difference and the second order difference of each feature subsequence are calculated to construct a difference feature matrix, the singular value decomposition of the difference feature matrix is performed to obtain a feature vector group, the power fluctuation feature vector is obtained by dynamically weighting the feature vector group based on the energy storage health state feature vector, and the energy storage power interval is divided according to the direction and modulus value of the power fluctuation feature vector.

[0102] The difference data in the energy storage power interval is counted, the data distribution feature parameter is calculated, the power fluctuation feature vector is taken as a correction factor, the distribution feature parameter is adjusted, and a corrected probability distribution function is obtained;

[0103] The statistical characteristic quantity of the corrected probability distribution function is calculated, the risk evaluation threshold is determined in combination with the energy storage charge and discharge response characteristics, and a power fluctuation risk evaluation index is generated;

[0104] The power fluctuation feature vector is updated according to the real-time operation data, and the power fluctuation risk evaluation index is dynamically adjusted.

[0105] In a specific embodiment, in the implementation of the photovoltaic energy storage joint optimization scheduling method based on deep reinforcement learning, the difference sequence calculation of photovoltaic power generation data and power grid load data is the basic work. By setting the sampling time interval to 5 minutes, the real-time power generation data of the photovoltaic power station and the regional power grid load data are obtained. Taking a day as an example, the photovoltaic power station generates 2.35 MW of power at 10:00, while the regional power grid load is 3.75 MW at the same period, and the calculated difference is -1.40 MW, indicating that the power grid needs an additional 1.40 MW of power support at this time. By calculating the data of the whole day in this way, a difference sequence containing 288 sampling points can be obtained.

[0106] When calculating the power fluctuation trend characteristics of the difference sequence, it is necessary to identify the transition point of the fluctuation. The transition point of the fluctuation refers to the position where the slope in the difference sequence changes significantly, which is identified by the double-threshold comparison method. In specific implementation, the rising slope threshold is set to 0.15 MW / min, the falling slope threshold is set to -0.12 MW / min, and the duration threshold is set to 15 minutes. When it is detected that the slope of three consecutive sampling points is greater than the rising slope threshold or less than the falling slope threshold, and the duration exceeds the duration threshold, the last sampling point that meets the condition is marked as a transition point of the fluctuation. For example, during the period from 12:30 to 12:45, the slopes of the three consecutive sampling points are detected to be 0.16, 0.18, and 0.17 MW / min, respectively, and the duration is 15 minutes. Therefore, 12:45 is marked as a transition point of the fluctuation. In this way, 24 transition points of the fluctuation are identified in the whole-day difference sequence.

[0107] The difference sequence is divided into 25 feature subsequences with the transition points of the fluctuation as boundaries. The first-order difference and the second-order difference of each feature subsequence are calculated to construct a difference feature matrix. Taking the third feature subsequence as an example, the subsequence contains 22 sampling points, the difference between each adjacent two points is calculated to obtain a first-order difference sequence with a length of 21, and the difference between adjacent elements in the first-order difference sequence is calculated to obtain a second-order difference sequence with a length of 20. The first-order difference sequence and the second-order difference sequence are combined into a difference feature matrix with a size of 2x20.

[0108] The singular value decomposition is performed on the difference feature matrix to obtain a feature vector group. Taking the third feature subsequence as an example, two main feature vectors are obtained after singular value decomposition, which are [0.85, 0.53] and [0.53, -0.85], and the corresponding singular values are 12.7 and 3.2. According to the health state feature vector of the energy storage system, dynamic weight distribution is performed on the feature vector group. The health state feature vector of the energy storage system is calculated by monitoring the real-time state of charge, cycle number and temperature of the energy storage system, etc. For example, the current health state feature vector of the energy storage system is [0.92, 0.78, 0.85], indicating that the energy storage system is in good working condition. The health state feature vector and the feature vector group are obtained by inner product operation, and the weight coefficients are 0.73 and 0.27, respectively. According to the weight coefficients, the feature vectors are weighted and synthesized to obtain the power fluctuation feature vector [0.76, 0.21]. The direction and modulus of the vector represent the dominant direction and intensity of the power fluctuation, and the modulus calculation result is 0.79.

[0109] Based on the power fluctuation feature vector, the energy storage power interval is divided into five intervals: maximum discharge interval (-2.5MW, -1.5MW), medium discharge interval (-1.5MW, -0.5MW), balance interval (-0.5MW, 0.5MW), medium charge interval (0.5MW, 1.5MW) and maximum charge interval (1.5MW, 2.5MW). The interval division considers the direction and modulus of the power fluctuation feature vector. The vector direction determines the offset direction of the interval, and the modulus determines the width adjustment coefficient of the interval.

[0110] The difference value data in each energy storage power interval is counted to calculate the data distribution characteristic parameters. Taking the medium charge interval as an example, there are 53 sampling points in this interval, and the calculated mean value is 0.95MW, the standard deviation is 0.26MW, the skewness is 0.12, and the kurtosis is 2.86. The power fluctuation feature vector is used as a correction factor to adjust the distribution characteristic parameters. The specific adjustment method is as follows: the modulus of the power fluctuation feature vector is used as the correction coefficient of the standard deviation, and the vector direction angle is used as the correction basis of the skewness. The corrected standard deviation is 0.26x0.79=0.21MW, and the skewness is adjusted to 0.12x(1+0.21 / 0.76)=0.15. Based on the corrected characteristic parameters, the kernel density estimation method is used to construct the corrected probability distribution function.

[0111] The statistical characteristic quantity of the modified probability distribution function is calculated, including the expected value, variance, quintile and ninety-fifth percentile. The statistical characteristic quantity calculation result of the modified probability distribution function of the medium charging interval is: the expected value is 0.93 MW, the variance is 0.044, the quintile is 0.59 MW, and the ninety-fifth percentile is 1.27 MW. In combination with the charging and discharging response characteristics of the energy storage system, the risk evaluation threshold is determined. The charging response rate of the energy storage system is 0.3 MW / minute, the discharging response rate is 0.35 MW / minute, the maximum continuous charging time is 120 minutes, and the maximum continuous discharging time is 90 minutes. Based on these parameters, the upper threshold of the risk evaluation of the medium charging interval is set to 1.3 MW, and the lower threshold is set to 0.55 MW. When the real-time difference exceeds the threshold range, the risk warning is triggered.

[0112] According to the risk evaluation threshold and the modified probability distribution function, a power fluctuation risk evaluation index is generated. The index includes two dimensions of fluctuation risk probability and fluctuation impact intensity. The fluctuation risk probability is calculated as the probability of the difference sequence exceeding the threshold range, and the fluctuation risk probability of the medium charging interval is 0.13, indicating that there is a 13% possibility of power fluctuation exceeding the threshold. The fluctuation impact intensity is calculated as the average deviation when exceeding the threshold, and the fluctuation impact intensity of the medium charging interval is 0.22 MW, indicating that the average deviation when exceeding the threshold is 0.22 MW.

[0113] According to the real-time operation data, the power fluctuation feature vector is updated to realize dynamic adjustment of the power fluctuation risk evaluation index. The real-time data is processed by a sliding window method, and the window length is set to 60 minutes and the sliding step is 5 minutes. After each window sliding, the power fluctuation feature vector in the current window is recalculated, and an exponential weighted average is performed with the historical feature vector, and the updated weight is 0.3. The updated power fluctuation feature vector is used to adjust the risk evaluation threshold and the modified probability distribution function, to realize dynamic adjustment of the risk evaluation index. For example, when it is detected that the power fluctuation feature vector modulus of three consecutive windows increases by more than 20%, the sensitivity of the risk evaluation threshold is adjusted accordingly, the trigger threshold is reduced, and the risk warning is performed in advance.

[0114] As shown in FIG. 1, a photovoltaic energy storage power fluctuation risk evaluation data flow diagram is shown. Figure 3

[0115] In an optional implementation, the double-threshold comparison method is used to identify the fluctuation transition point, including:

[0116] Based on the power time series data, the instantaneous power change rate is determined;

[0117] The instantaneous power change rate is accumulated and summed in a sliding time window to obtain a power change accumulation, and the duration of the same sign of the instantaneous power change rate is recorded to obtain a fluctuation persistence index; ​

[0118] determining power variation rate upper and lower threshold values based on statistical distribution characteristics of the power time series data and the charge and discharge power limits of the energy storage device, and determining a power variation accumulation threshold value based on the capacity constraint of the energy storage device and the length of the sliding time window;

[0119] determining whether the instantaneous power variation rate exceeds the power variation rate upper and lower threshold values and whether the power variation accumulation exceeds the power variation accumulation threshold value, finding extreme points of the instantaneous power variation rate in a time period that meets the double threshold conditions, and recording the extreme points as candidate conversion points;

[0120] calculating the time interval and power variation amplitude of adjacent candidate conversion points, setting a time threshold value based on the fluctuation persistence indicator, setting an amplitude threshold value based on the power variation rate threshold value, and deleting candidate conversion points with a time interval less than the time threshold value and a power variation amplitude less than the amplitude threshold value;

[0121] determining the candidate conversion points that pass the screening as power fluctuation conversion points, and segmenting the power time series data based on the power fluctuation conversion points as boundaries.

[0122] In one specific embodiment, power time series data is collected. Taking a certain photovoltaic power station as an example, the sampling period is 1 second, and 24 hours of power data are continuously collected, totaling 86400 data points. The collected power data is in kilowatts (kW) and has a value range of 0 to 5000 kW. The data shows that power fluctuation is more obvious during the daytime peak generation period.

[0123] Based on the collected power time series data, the instantaneous power variation rate is calculated. The instantaneous power variation rate is defined as the difference between the power values of adjacent time points divided by the time interval. For example, if the power at time t is 4200 kW and the power at time t+1 is 4250 kW, and the sampling interval is 1 second, then the instantaneous power variation rate at time t is (4250-4200) / 1=50 kW / s. This calculation is performed for all 86400 data points, resulting in 86399 instantaneous power variation rate values.

[0124] In order to evaluate the power variation trend over a period of time, a sliding time window is set to accumulate and sum the instantaneous power variation rate. In this embodiment, the sliding window length is set to 30 seconds. For any time t, the sum of the instantaneous power variation rates from time t to t+29 is calculated to obtain the power variation accumulation in the window. For example, if the instantaneous power variation rates from time t to t+29 are 50, 45, 30, -10, 15... kW / s, the sum is 2000 kW, and the power variation accumulation at time t is 2000 kW. At the same time, the duration for which the sign of the instantaneous power variation rate remains the same is recorded. For example, if the instantaneous power variation rates of 10 consecutive sampling points are all positive, then the fluctuation persistence indicator is 10 seconds.

[0125] The power change rate threshold is determined according to the statistical characteristics of the power time series data. By analyzing the change rate distribution of the 24-hour power data, it is found that 95% of the instantaneous power change rate falls within the range of -100 kW / s to 100 kW / s. Considering that the charge and discharge power limit of the energy storage device is 1000 kW, the upper threshold of the power change rate is set to 100 kW / s, and the lower threshold is set to -100 kW / s. Based on the energy storage device capacity constraint (such as 2000 kWh) and the sliding window length (30 seconds), the power change accumulation threshold is set to 1500 kW. This means that if the power cumulative change exceeds 1500 kW within a 30-second window, the energy storage device will be considered for charge and discharge adjustment.

[0126] The double-threshold judgment link is entered, and the power time series data is checked point by point. For each time t, it is judged whether the instantaneous power change rate exceeds the set upper and lower threshold (-100 kW / s to 100 kW / s), and whether the power change accumulation within the sliding window starting from that time exceeds 1500 kW. When both conditions are met, the extreme point of the instantaneous power change rate is found within that time period. For example, if the double-threshold condition is met within the time period from t to t+40, and the instantaneous power change rate reaches a local maximum of 130 kW / s at t+15, then t+15 is recorded as a candidate transition point.

[0127] The preliminary determined candidate transition points are screened. The time interval and power change amplitude between adjacent candidate transition points are calculated. According to the statistical fluctuation persistence index, the time threshold is set to 20 seconds, that is, the effective transition points of power fluctuation should be at least 20 seconds apart. At the same time, based on the power change rate threshold, the amplitude threshold is set to 500 kW, that is, the power change between adjacent transition points should at least reach 500 kW to be considered effective. For example, if the candidate transition points A and B are only 10 seconds apart, and the power difference is 300 kW, then the one with smaller power change rate is deleted.

[0128] The screened candidate transition points are determined as the final power fluctuation transition points. In actual cases, from the original 86400 data points, a total of 127 power fluctuation transition points are identified by the double-threshold comparison method. These transition points clearly mark the alternating positions of the power rising and falling segments, and each transition point records the timestamp and the corresponding power value.

[0129] The power time series data is segmented with the determined power fluctuation conversion points as boundaries. For example, if two adjacent conversion points are located at t1=3600 seconds and t2=3750 seconds, then the 150 data points between t1 and t2 are divided into a power segment. After the segmentation of all data is completed, 128 power segments are obtained (more than the number of conversion points by 1). The data in each power segment has a relatively consistent change trend, which can be used for subsequent energy storage scheduling decisions or renewable energy power prediction.

[0130] By this double-threshold comparison method, the key conversion points in power fluctuation can be effectively identified, avoiding the misjudgment problem caused by simply relying on the power change rate or cumulative amount, improving the accuracy of power fluctuation characteristic analysis, and providing a reliable basis for the optimization control of the energy storage system.

[0131] In an optional embodiment, a plurality of parallel search paths are generated in the energy storage charge-discharge power feasible region, the dominant search paths and the inferior search paths are divided based on the power fluctuation risk evaluation index, and the energy storage charge-discharge power time sequence is obtained by iterative search and local optimal solution identification.

[0132] An initial search point set is constructed in the energy storage charge-discharge power feasible region, and a plurality of parallel search paths are generated based on the initial search point set;

[0133] The power fluctuation risk evaluation index and the change trend of each parallel search path are calculated, and the dominant search paths and the inferior search paths are determined based on the dynamic weighted mean;

[0134] Each parallel search path is subjected to iterative search, the search step and the search direction of the dominant search path are extracted as feature parameters, and the search step and the search direction of the inferior search path are updated;

[0135] The evaluation value change amount of the parallel search path within a continuous preset number of iterations is recorded, when the evaluation value change amount is less than a preset floating threshold, the current search point is recorded as a local optimal solution, a new search starting point is generated by random disturbance in the neighborhood of the current search point, and the set of local optimal solutions that meet the energy storage charge-discharge power feasible region constraint is selected as candidate solutions based on the new search starting point;

[0136] The solution with the optimal evaluation value in the candidate solutions is selected to generate the energy storage charge-discharge power time sequence, and the energy storage charge-discharge power time sequence is adjusted online according to real-time operation data.

[0137] Within the feasible region of energy storage charging and discharging power, a uniformly distributed initial search point set is constructed. Assuming that the feasible region is composed of the upper and lower limits of the power and energy of the energy storage system, for example, the power range is [-5MW, 5MW] and the energy range is [0, 10MWh]. To construct the initial search point set, the feasible region is divided according to the preset interval. For day-ahead scheduling of 96 time points, 20 initial search points can be generated within the feasible region, and each search point is represented as a 96-dimensional vector representing the charging and discharging power of 96 time periods throughout the day. These initial points can be obtained by the Latin hypercube sampling method to ensure the uniformity of the points.

[0138] Based on the initial search point set, multiple parallel search paths are generated. Each initial search point serves as the starting point of a search path, and the search process will explore the feasible region in different directions from these starting points. For example, for 20 initial search points, 20 parallel search paths will be generated. Each path evolves independently, but can share and exchange information during the search process.

[0139] The power fluctuation risk evaluation index is calculated for each parallel search path. This index considers the smoothness of the energy storage charging and discharging power, the effect of suppressing power fluctuations in the grid, and the efficiency of using the energy storage. Specifically, the evaluation index can be represented as the weighted combination of the change rate of the energy storage charging and discharging power in consecutive periods, the standard deviation of the grid power fluctuation, and the efficiency of using the energy storage. For example, the time series of charging and discharging power of a certain path at the initial point is P = [1.2, 1.5, -0.8, -1.2,...] MW, and the power fluctuation risk evaluation index value is 0.85.

[0140] According to the size and trend of the power fluctuation risk evaluation index value, the dynamic weighted mean method is used to determine the dominant search path and the inferior search path. Paths with evaluation index values lower than the average of all path evaluation index values are defined as dominant paths, and those higher than the average are defined as inferior paths. For example, among the 20 paths, the evaluation index values are [0.85, 0.92, 0.79, 1.05,...], and the average is 0.88. Paths with index values of 0.85 and 0.79 are marked as dominant paths, while 1.05 is marked as an inferior path.

[0141] Iterative search is performed on all parallel search paths. In each iteration, the search step and search direction of the dominant search path are extracted as characteristic parameters. For example, the current search step of a certain dominant path is 0.2MW, and the search direction vector is D = [0.1, -0.05, 0.08,...]. These characteristic parameters are used to update the search behavior of the inferior search path, causing the inferior path to move towards the dominant region. The specific update method is: new step of inferior path = original step of inferior path x 0.5 + average step of dominant path x 0.5, new direction of inferior path = original direction of inferior path x 0.7 + average direction of dominant path x 0.3.

[0142] In the iteration process, the change amount of the evaluation value of each parallel search path in the continuous preset number of iterations is recorded. When the change amount of the evaluation value of a path in the continuous 10 iterations is less than the preset floating threshold 0.01, the current search point is recorded as a local optimal solution. For example, the evaluation values of a path in the 50th-60th iteration are [0.782, 0.781, 0.780, 0.779, 0.779, 0.778, 0.778, 0.778, 0.777, 0.777], the maximum change amount is 0.005, which is less than the threshold 0.01, and therefore the search point in the 60th iteration is recorded as a local optimal solution.

[0143] For the path that finds a local optimal solution, a new search starting point is generated by random disturbance in the neighborhood of the current search point. The disturbance amplitude is a random value in the range of ±5% of the current search point value. For example, the current local optimal solution corresponds to a charge and discharge power of 2.5 MW, and the new search starting point can be 2.5+2.5*5%*random(-1,1) = 2.6 MW. The search is restarted based on the new search starting point to avoid being trapped in a local optimum.

[0144] After multiple iterations and multiple re-searches, all local optimal solutions that meet the constraints of the energy storage charge and discharge power feasible region are taken as a candidate solution set. The constraint conditions include upper and lower limits of energy storage power, upper and lower limits of energy, state of charge range, etc. For example, the power constraint is [-5 MW, 5 MW], the energy constraint is [0, 10 MWh], and the state of charge range is [20%, 90%].

[0145] From the candidate solution set, the solution with the optimal evaluation value is selected as the final energy storage charge and discharge power time sequence. For example, there are 5 local optimal solutions in the candidate solution set, and their evaluation values are [0.77, 0.82, 0.79, 0.85, 0.80], respectively. The solution with the evaluation value of 0.77 is selected as the final solution, and the corresponding charge and discharge power time sequence is P_final = [1.2, 1.5, -0.8, -1.2, …] MW.

[0146] In real-time operation, the energy storage charge and discharge power time sequence is adjusted online according to real-time operation data. When the actual operation deviation exceeds the preset threshold (such as 5%), the charge and discharge power time sequence of the future period is recalculated based on the rolling optimization method, with the system state at the current time as the initial condition, and the above multi-path search method is used. For example, the actual measured power is 1.3 MW, while the planned power is 1.2 MW, the deviation is 8.3%, which exceeds the threshold of 5%, and therefore the online adjustment is triggered, and the updated power time sequence is P_adjusted = [1.3, 1.6, -0.7, -1.1, …] MW.

[0147] By the above method, the energy storage charge and discharge power time sequence considering the power fluctuation risk can be effectively generated, and the energy storage system operation efficiency and power grid stability are improved.

[0148] In an optional embodiment, the power fluctuation risk evaluation index and the change trend of each parallel search path are calculated, and the dominant search path and the inferior search path are determined based on the dynamic weighted mean, which includes:

[0149] The power fluctuation risk evaluation index of the plurality of parallel search paths is obtained as the evaluation value;

[0150] The difference of the evaluation value is calculated to obtain the change rate, and the difference of the change rate is calculated to obtain the change acceleration;

[0151] The evaluation value and the change rate are weighted and summed, and multiplied by the change acceleration to obtain the path score;

[0152] The weight coefficients of the evaluation value and the change rate are adjusted according to the real-time state, and the dynamic weighted mean of the path score is calculated;

[0153] The parallel search path with a path score less than or equal to the dynamic weighted mean is determined as the inferior search path, and the parallel search path with a path score greater than the dynamic weighted mean is determined as the dominant search path.

[0154] When calculating the power fluctuation risk evaluation index of the plurality of parallel search paths, the power fluctuation risk evaluation index of each parallel search path is obtained as the evaluation value. These evaluation indexes usually include parameters such as the amplitude, frequency, and waveform characteristics of power fluctuation. For example, for three different parallel search paths, power fluctuation data within a period of time can be collected to obtain the evaluation value of path 1 as 0.85, the evaluation value of path 2 as 0.72, and the evaluation value of path 3 as 0.91. These evaluation values reflect the performance of each path in stability, and the higher the value, the lower the power fluctuation risk and the more stable the system operation.

[0155] The change of the evaluation value is calculated in a preset time window. Assuming that the time window is set to 10 minutes and data is collected every minute, the evaluation value sequence of path 1 at time points t1 to t10 is [0.85, 0.83, 0.86, 0.89, 0.87, 0.84, 0.82, 0.85, 0.88, 0.89]. The evaluation value difference of adjacent time points is calculated to obtain the change rate sequence [-0.02, 0.03, 0.03, -0.02, -0.03, -0.02, 0.03, 0.03, 0.01]. The difference of the change rate is further calculated to obtain the change acceleration sequence [0.05, 0, -0.05, -0.01, 0.01, 0.05, 0, -0.02]. These change rate and change acceleration data reflect the dynamic change characteristics of the path evaluation value.

[0156] According to the calculated evaluation value and change rate, the comprehensive performance of the path is calculated by weighted summation. The initial weight coefficient of the evaluation value is set to 0.6, and the initial weight coefficient of the change rate is set to 0.4. Taking time t10 as an example, the evaluation value of path 1 is 0.89, the change rate is 0.01, and the change acceleration is -0.02. The path score of path 1 is calculated as follows: (0.89 x 0.6 + 0.01 x 0.4) x (-0.02) = -0.0108. The path score of path 2 is calculated as 0.0135 by the same method, and the path score of path 3 is calculated as 0.0089. The path score reflects the comprehensive performance of each search path, considering the current evaluation value, change trend, and acceleration characteristics of the change.

[0157] According to the real-time running state, the weight coefficients are dynamically adjusted. When in a stable state, more attention is paid to the current evaluation value, and the weight of the evaluation value can be increased to 0.75, and the weight of the change rate can be reduced to 0.25. When the system is in a state of large fluctuation, more attention is paid to the change trend, and the weight of the evaluation value can be reduced to 0.45, and the weight of the change rate can be increased to 0.55. Assuming that the current system detects large fluctuations, the path score of path 1 is recalculated after adjusting the weights: (0.89 x 0.45 + 0.01 x 0.55) x (-0.02) = -0.0089. The path score of path 2 is calculated as 0.0142 by the same method, and the path score of path 3 is calculated as 0.0075.

[0158] In order to comprehensively consider the path scores of multiple time points, a dynamic weighted mean of the path scores is calculated. In the time window, different weights are given to the path scores of each time point, and the closer to the current time, the greater the weight. The weights of t1 to t10 are set as [0.05, 0.05, 0.06, 0.07, 0.08, 0.10, 0.12, 0.15, 0.16, 0.16] in turn. The dynamic weighted mean of path 1 is 0.0056, the dynamic weighted mean of path 2 is 0.0102, and the dynamic weighted mean of path 3 is 0.0083. The dynamic weighted mean reflects the comprehensive performance of the search path in the entire time window.

[0159] According to the comparison result of the path score and the dynamic weighted mean, the dominant search path and the inferior search path are determined. The average of the dynamic weighted means of the three paths is calculated to obtain the threshold value 0.0080. The score of path 1 is 0.0056, which is less than the threshold value, and it is determined as the inferior search path; the score of path 2 is 0.0142, which is greater than the threshold value, and it is determined as the dominant search path; the score of path 3 is 0.0075, which is less than the threshold value, and it is determined as the inferior search path. The resource allocation of the dominant search path 2 will be continuously strengthened, and the resource investment of the inferior search paths 1 and 3 will be reduced, so as to improve the overall search efficiency.

[0160] The determination results of the superior and inferior paths are updated regularly. After a predetermined update period, for example, 30 minutes, the above calculation process is re-executed to obtain the latest path score and dynamic weighted mean, and the determination results of the superior and inferior paths are updated. Assuming that after an update period, the dynamic performance of path 1 improves, and the new path score is 0.0124, which is higher than the updated threshold value 0.0095, then path 1 changes from the inferior search path to the dominant search path. This dynamic updating mechanism ensures that the changes in the performance of each search path can be responded to in time, and the resource allocation strategy is optimized.

[0161] Through the calculation and analysis of the above power fluctuation risk evaluation indexes, the dominant search path with stability and development potential can be effectively identified, and the inferior search path with poor performance can be excluded, so as to realize the reasonable allocation of parallel search resources and improve the utilization efficiency.

[0162] The photovoltaic energy storage joint optimization scheduling system based on deep reinforcement learning in the embodiment of the application comprises:

[0163] The first unit is used for collecting photovoltaic power data, energy storage operation state data and power grid load data to form a multi-source heterogeneous time sequence data set;

[0164] The second unit is configured to extract energy storage features by a double-decoder network structure and a recursive graph convolution network of a dynamic timing diagram structure based on the multi-source heterogeneous timing data set, and generate an energy storage health state feature vector; a comprehensive power feature value and a capacity threshold value are calculated according to the energy storage health state feature vector, and a two-dimensional coordinate system is used to determine an energy storage safe operation range;

[0165] The third unit is configured to divide an energy storage power interval according to the energy storage safe operation range, and construct an energy storage charging and discharging power feasible region in combination with energy storage physical constraints.

[0166] The fourth unit is configured to calculate a difference sequence of photovoltaic power generation power data and grid load data in the multi-source heterogeneous timing data set; the difference sequence is segmented and a power fluctuation feature vector is extracted, and a power fluctuation risk evaluation index is constructed based on a probability distribution parameter corrected based on the power fluctuation feature vector.

[0167] The fifth unit is configured to generate a plurality of parallel search paths in the energy storage charging and discharging power feasible region, divide dominant search paths and inferior search paths based on the power fluctuation risk evaluation index, and obtain an energy storage charging and discharging power timing through iterative search and local optimal solution identification.

[0168] The sixth unit is configured to generate an energy storage scheduling instruction sequence according to the energy storage charging and discharging power timing.

[0169] In a third aspect, an electronic device is provided, including:

[0170] a processor;

[0171] a memory for storing processor-executable instructions;

[0172] The processor is configured to invoke the instructions stored in the memory to execute the method described above.

[0173] In a fourth aspect, a computer-readable storage medium is provided, which stores computer program instructions, and the computer program instructions are executed by a processor to implement the method described above.

[0174] The present application can be a method, device, system and / or computer program product. The computer program product can include a computer readable storage medium having computer readable program instructions loaded thereon for performing various aspects of the present application.

[0175] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, and are not intended to limit the present application; although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that the technical solutions recorded in the above embodiments can be modified, or some or all of the technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A photovoltaic energy storage joint optimization scheduling method based on deep reinforcement learning, characterized in that, The method comprises the following steps: Collect photovoltaic power generation data, energy storage operation state data and power grid load data to form a multi-source heterogeneous time series data set; Based on the multi-source heterogeneous time series data set, the energy storage features are extracted through a double encoder-decoder network structure and a recursive graph convolution network of dynamic time series graph structure to generate an energy storage health state feature vector; according to The comprehensive power feature value and the capacity threshold value are calculated based on the energy storage health state feature vector, and the safe operation range of the energy storage is determined through a two-dimensional coordinate system; According to the safe operation range of the energy storage, the energy storage power interval is divided, and the energy storage charging and discharging power feasible region is constructed in combination with the physical constraints of the energy storage; The difference sequence of the photovoltaic power generation data and the power grid load data in the multi-source heterogeneous time series data set is calculated; the difference sequence is segmented and the power fluctuation feature vector is extracted, and the power fluctuation risk evaluation index is constructed based on the power fluctuation feature vector; A plurality of parallel search paths are generated in the energy storage charging and discharging power feasible region, the power fluctuation risk evaluation index is used to divide the dominant search path and the inferior search path, and the energy storage charging and discharging power time sequence is obtained through iterative search and local optimal solution identification; The energy storage scheduling instruction sequence is generated according to the energy storage charging and discharging power time sequence.

2. The method of claim 1, wherein, Based on the multi-source heterogeneous time series data set, the energy storage features are extracted through a double encoder-decoder network structure and a recursive graph convolution network of dynamic time series graph structure to generate an energy storage health state feature vector, which comprises: The multi-source heterogeneous time series data set is input into the double encoder-decoder network structure, and the initial feature matrix is obtained through time series feature extraction and state feature fusion, which contains the time series information and state information of the energy storage operation; A dynamic time series graph structure is constructed based on the initial feature matrix, the nodes in the dynamic time series graph structure correspond to the energy storage operation state at different time, and the weight between the nodes in the dynamic time series graph structure is calculated through the time series correlation weight matrix by the adaptive attention mechanism of thermodynamic constraints; A recursive graph convolution network is constructed according to the time series correlation weight matrix, the convolution kernel parameters in the recursive graph convolution network are updated by the recursive unit of thermodynamic constraints, the initial hidden layer state of the recursive unit is set according to the thermodynamic constraints, and the thermodynamic time series features are obtained by performing graph convolution operation; the thermodynamic time series features and the initial feature matrix are fused through residual connection to obtain fusion enhanced features; The fusion enhanced features are subjected to multi-scale decomposition to obtain multi-scale feature representation containing global operation features, local fluctuation features and trend features; the multi-scale feature representation is input into a feature mapping network, and is reduced to a low-dimensional manifold space through nonlinear projection transformation to obtain an energy storage health state feature vector.

3. The method of claim 1, wherein, According to the energy storage health state feature vector, the comprehensive power feature value and the capacity threshold value are calculated, and the safe operation range of the energy storage is determined through a two-dimensional coordinate system, which comprises: The energy storage health state feature vector is decomposed into power feature components and capacity feature components through nonlinear mapping, and the power feature components include charging power components and discharging power components; The maximum charging power and the maximum discharging power are determined by multiplying the charging power components and the discharging power components with the coupling coefficients of the internal resistance and the temperature of the energy storage, respectively, to calculate the comprehensive power feature value. The capacity threshold value is obtained by multiplying the capacity characteristic component by the weighted coefficients of the cycle number and cycle depth and accumulating them; A two-dimensional coordinate system is established with the comprehensive power characteristic value as the ordinate and the capacity threshold value as the abscissa, a plurality of feature points are selected as regional center points in the two-dimensional coordinate system, a region division graph is constructed based on the regional center points, the distance from any point in the two-dimensional coordinate system to each regional center point is calculated, and the region corresponding to the regional center point with the shortest distance is taken as the region to which the corresponding point belongs; a classifier is constructed using a kernel function to determine the classification boundary of different regions; and the comprehensive power characteristic value and the capacity threshold value are mapped to different regions of the two-dimensional coordinate system based on membership calculation, and the classification boundary of the different regions determines the safe operation range of energy storage.

4. The method of claim 1, wherein, A difference sequence of photovoltaic power generation data and grid load data in a multi-source heterogeneous time series data set is calculated; the difference sequence is segmented and a power fluctuation feature vector is extracted, and a risk evaluation index is constructed based on the power fluctuation feature vector and the modified probability distribution parameters including: A difference sequence of photovoltaic power generation data and grid load data is calculated; A power fluctuation trend feature of the difference sequence is calculated, a fluctuation feature vector is constructed according to the power rise rate, fall rate and duration, a double-threshold comparison method is used to identify a fluctuation transition point, the difference sequence is divided into a plurality of feature subsequences with the fluctuation transition point as the boundary, a first-order difference and a second-order difference of each feature subsequence are calculated to construct a difference feature matrix, a singular value decomposition is performed on the difference feature matrix to obtain a feature vector group, a power fluctuation feature vector is obtained by dynamically weighting the feature vector group based on the energy storage health state feature vector, and the power fluctuation feature vector is used to divide the energy storage power interval according to the direction and modulus of the power fluctuation feature vector. The difference data in the energy storage power interval is counted, the distribution characteristic parameters are calculated, the power fluctuation feature vector is used as a correction factor to adjust the distribution characteristic parameters, and a modified probability distribution function is obtained; The statistical characteristic quantity of the modified probability distribution function is calculated, a risk evaluation threshold is determined in combination with the energy storage charge and discharge response characteristics, and a power fluctuation risk evaluation index is generated; The power fluctuation risk evaluation index is dynamically adjusted according to real-time operation data.

5. The method of claim 4, wherein, The double-threshold comparison method includes: Based on power time series data, the instantaneous power change rate is determined; In a sliding time window, the instantaneous power change rate is accumulated and summed to obtain a power change accumulation, and the duration for which the sign of the instantaneous power change rate remains the same is recorded to obtain a fluctuation persistence indicator; Based on the statistical distribution characteristics of the power time series data and the charge and discharge power limits of the energy storage device, the upper and lower thresholds of the power change rate are determined, and based on the capacity constraint of the energy storage device and the length of the sliding time window, the threshold of the power change accumulation is determined; It is judged whether the instantaneous power change rate exceeds the upper and lower thresholds of the power change rate and whether the power change accumulation exceeds the threshold of the power change accumulation, and in the time period that meets the double-threshold condition, the extreme point of the instantaneous power change rate is found and recorded as a candidate transition point; Calculate time intervals and power variation amplitudes of adjacent candidate transition points, set a time threshold based on the fluctuation persistence index, set an amplitude threshold based on the power variation rate threshold, and delete candidate transition points with time intervals less than the time threshold and power variation amplitudes less than the amplitude threshold; Determine the candidate transition points that pass the screening as power fluctuation transition points, and segment the power time series data based on the power fluctuation transition points.

6. The method of claim 1, wherein, Generate multiple parallel search paths within the energy storage charging and discharging power feasible region, divide dominant search paths and inferior search paths based on power fluctuation risk evaluation indexes, and obtain the energy storage charging and discharging power time series through iterative search and local optimal solution identification, including: Construct an initial search point set uniformly distributed within the energy storage charging and discharging power feasible region, and generate multiple parallel search paths based on the initial search point set; Calculate the power fluctuation risk evaluation index and change trend of each parallel search path, and determine the dominant search path and the inferior search path based on the dynamic weighted mean; Iteratively search each parallel search path, extract the search step and search direction of the dominant search path as feature parameters, and update the search step and search direction of the inferior search path; Record the evaluation value change amount of the parallel search path within a continuous preset number of iterations, and when the evaluation value change amount is less than a preset floating threshold, record the current search point as a local optimal solution, generate a new search starting point by randomly disturbing the neighborhood of the current search point, and search again based on the new search starting point. The set of local optimal solutions that meet the energy storage charging and discharging power feasible region constraints is used as candidate solutions; Select the solution with the optimal evaluation value from the candidate solutions to generate the energy storage charging and discharging power time series, and adjust the energy storage charging and discharging power time series online according to real-time operation data.

7. The method of claim 6, wherein, Calculate the power fluctuation risk evaluation index and change trend of each parallel search path, and determine the dominant search path and the inferior search path based on the dynamic weighted mean, including: Obtain the power fluctuation risk evaluation index of multiple parallel search paths as the evaluation value; Calculate the difference of the evaluation value within a preset time window to obtain the change rate, and calculate the difference of the change rate to obtain the change acceleration; Weighted sum of evaluation value and change rate, and multiply by change acceleration to get path score; Adjust the weight coefficients of the evaluation value and the change rate according to the real-time state, and calculate the dynamic weighted mean of the path score; Determine the parallel search path with a path score less than or equal to the dynamic weighted mean as the inferior search path, and determine the parallel search path with a path score greater than the dynamic weighted mean as the dominant search path.

8. A photovoltaic energy storage combined optimization and dispatch system based on deep reinforcement learning, for implementing the method of any one of the preceding claims 1-7, characterized in that, Including: The first unit is used to collect photovoltaic power generation data, energy storage operation state data and grid load data to form a multi-source heterogeneous time series data set; The second unit is used to extract energy storage features based on the multi-source heterogeneous time series data set through a double encoder-decoder network structure and a recursive graph convolution network with a dynamic time series graph structure to generate an energy storage health state feature vector; Calculate the comprehensive power feature value and the capacity threshold based on the energy storage health state feature vector, and determine the energy storage safe operation range through a two-dimensional coordinate system; The third unit is configured to divide the energy storage power interval according to the energy storage safe operation range, and construct an energy storage charging and discharging power feasible region in combination with energy storage physical constraints; The fourth unit is configured to calculate a difference sequence of photovoltaic power generation power data and grid load data in the multi-source heterogeneous time sequence data set, segment the difference sequence, extract a power fluctuation feature vector, correct a probability distribution parameter based on the power fluctuation feature vector, and construct a power fluctuation risk evaluation index; The fifth unit is configured to generate a plurality of parallel search paths in the energy storage charging and discharging power feasible region, divide dominant search paths and inferior search paths based on the power fluctuation risk evaluation index, and obtain an energy storage charging and discharging power time sequence through iterative search and local optimal solution identification. The sixth unit is configured to generate an energy storage scheduling instruction sequence according to the energy storage charging and discharging power time sequence.

9. An electronic device, comprising: It comprises: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to invoke the instructions stored in the memory to execute the method of any one of claims 1 to 7.

10. A computer-readable storage medium having stored thereon computer program instructions, wherein, The computer program instructions are executed by the processor to implement the method of any one of claims 1 to 7.

Images