Commercial energy storage power station income mode intelligent optimization method and system

Abstract

The application provides a commercial energy storage power station benefit mode intelligent optimization method and system, relates to the technical field of energy storage power stations, and comprises the following steps: obtaining historical operation data to construct a deep reinforcement learning model; establishing a multi-dimensional life loss coefficient matrix by analyzing energy storage equipment health state data; generating a candidate charging and discharging scheduling strategy based on the deep reinforcement learning model, optimizing the charging and discharging strategy by taking the predicted benefit and the life loss as a double constraint of a reward function; and finally generating a control instruction to realize the optimized operation of the energy storage power station. The application comprehensively considers the equipment life loss and economic benefits, and realizes the maximization of long-term benefits of the energy storage power station.

Description

TECHNICAL FIELD

[0001] The present application relates to energy storage power station technology, and in particular to a commercial energy storage power station revenue mode intelligent optimization method and system. BACKGROUND

[0002] With the continuous expansion of renewable energy grid-connected scale, the demand for flexible resources of power grid is increasing, and commercial energy storage power station as an important power system flexibility resource plays an important role in power system peak shaving, frequency modulation, capacity support and power quality improvement, etc. Commercial energy storage power station obtains economic benefits through charging in low electricity price period, discharging in high electricity price period, participating in power auxiliary service market and providing demand response service, etc. However, the revenue of energy storage power station is directly related to its operation strategy, and how to maximize economic benefits while ensuring equipment safety and prolonging service life becomes a key problem of energy storage power station operation.

[0003] At present, the revenue mode optimization of commercial energy storage power station is mainly based on traditional mathematical programming methods, such as linear programming, dynamic programming and heuristic algorithm, etc. These methods have the following defects and deficiencies when facing complex factors such as real-time electricity price fluctuation of power grid, life loss of energy storage equipment and variable market environment:

[0004] The existing energy storage power station dispatching optimization method generally lacks accurate evaluation of battery life loss, often uses a simplified linear model, which cannot accurately reflect the comprehensive influence of multiple factors such as cycle number, depth cycle rate and working temperature on battery life, resulting in excessive loss of battery life while pursuing short-term economic benefits, and reducing the long-term economy of energy storage system.

[0005] Traditional optimization algorithms perform poorly when dealing with uncertainty factors such as real-time electricity price fluctuation of power grid, and usually develop fixed dispatching strategies based on historical data and simple prediction models, which are difficult to adapt to real-time changes of power market and cannot make flexible adjustments in case of sharp price fluctuations or emergencies, so that the revenue potential of energy storage power station is not fully tapped.

[0006] The revenue optimization model in the prior art mostly uses single objective optimization, either excessively focusing on short-term economic benefits and ignoring equipment life, or excessively conservative to prolong equipment service life and sacrifice economic benefits, lacking a comprehensive optimization mechanism that takes maximizing revenue and minimizing life loss as dual constraints, and cannot realize the economy optimization of energy storage system in the whole life cycle. SUMMARY

[0007] The embodiments of the present application provide a commercial energy storage power station revenue mode intelligent optimization method and system, which can solve the problems in the prior art.

[0008] In a first aspect, the embodiments of the present application provide a commercial energy storage power station revenue mode intelligent optimization method, which comprises:

[0009] acquire historical operation data of the commercial energy storage power station, the historical operation data including charge-discharge power data, battery state of charge data, real-time power grid price data, and energy storage device health state data;

[0010] construct a deep reinforcement learning model of the commercial energy storage power station based on the historical operation data, the deep reinforcement learning model adopting a deep neural network structure, and generating a predicted revenue value by nonlinear mapping of the historical operation data through the deep neural network structure;

[0011] analyze the energy storage device health state data through the deep reinforcement learning model, establish a multi-dimensional life loss coefficient matrix based on cycle number, deep cycle rate, and operating temperature, and realize evaluation of device life loss;

[0012] generate multiple groups of candidate charge-discharge scheduling strategies based on the deep reinforcement learning model, take predicted revenue and life loss as dual constraints of a reward function, realize generation of a charge-discharge strategy maximizing revenue considering device life through alternating training of a policy network and a value network, and

[0013] generate an operation control instruction according to the optimized charge-discharge scheduling strategy, send the operation control instruction to an energy management system of the commercial energy storage power station, and control the energy storage power station to operate according to the optimized charge-discharge scheduling strategy.

[0014] construct a deep reinforcement learning model of the commercial energy storage power station based on the historical operation data, the deep reinforcement learning model adopting a deep neural network structure, and generating a predicted revenue value by nonlinear mapping of the historical operation data through the deep neural network structure includes:

[0015] the deep reinforcement learning model adopts a multi-layer feedforward neural network structure, the multi-layer feedforward neural network structure including an input layer, a hidden layer, and an output layer, wherein the hidden layer performs feature extraction and nonlinear mapping on the historical operation data through a weight matrix, a bias vector, and a nonlinear activation function;

[0016] construct network parameters of the multi-layer feedforward neural network structure as random variables, establish a Gaussian distribution model of the network parameters by using a variational inference method, and quantitatively represent prediction uncertainty of the multi-layer feedforward neural network structure through mean parameters and variance parameters of the Gaussian distribution model;

[0017] add perturbations to the historical operation data based on the Gaussian distribution model to generate adversarial samples, and use the adversarial samples in a training process of the multi-layer feedforward neural network structure to improve robustness of the multi-layer feedforward neural network structure;

[0018] calculate a prediction mean and a prediction standard deviation based on the prediction result of the multi-layer feedforward neural network structure, determine a confidence interval of the prediction result according to the prediction mean and the prediction standard deviation; calculate a dynamic adjustment coefficient according to the prediction standard deviation, and perform adaptive optimization on the prediction result of the multi-layer feedforward neural network structure based on the dynamic adjustment coefficient to obtain an optimized prediction result;

[0019] input the optimized prediction result and the confidence interval into a revenue calculation module, and generate a predicted revenue value of the commercial energy storage power station through the revenue calculation module.

[0020] a Gaussian distribution model of the network parameters is established by using a variational inference method, and the prediction uncertainty of the multi-layer feedforward neural network structure is quantitatively characterized by using a mean parameter and a variance parameter of the Gaussian distribution model, including:

[0021] obtain training data of a multi-layer feedforward neural network structure, construct a likelihood function based on the training data, select a Gaussian distribution as a prior distribution of the network parameters, and combine the likelihood function and the prior distribution through a Bayesian formula to construct a posterior distribution;

[0022] the posterior distribution is approximated by using a variational inference method, a variational distribution family in a Gaussian distribution form is constructed, the variational distribution family includes a mean parameter vector and a variance parameter vector, and optimal variational distribution parameters are obtained by minimizing the kullback-leibler divergence between the variational distribution family and the posterior distribution through the variational inference method;

[0023] input the optimal variational distribution parameters into the multi-layer feedforward neural network structure, for each layer in the multi-layer feedforward neural network structure, construct a parameter distribution of the layer based on the mean parameter vector and the variance parameter vector by using the variational inference method; obtain a network parameter sample based on the parameter distribution, input the network parameter sample into the multi-layer feedforward neural network structure, and obtain a corresponding network output;

[0024] repeat obtaining multiple groups of network parameter samples, input the multiple groups of network parameter samples into the multi-layer feedforward neural network structure to obtain multiple groups of network outputs, calculate a prediction mean and a prediction variance based on the multiple groups of network outputs; and use the prediction mean and the prediction variance as a quantification result of the prediction uncertainty of the multi-layer feedforward neural network structure.

[0025] analyze the energy storage device health state data through the deep reinforcement learning model, establish a multi-dimensional life loss coefficient matrix based on the cycle number, the deep cycle rate and the working temperature, and realize the evaluation of the device life loss, including:

[0026] acquire health state data of the energy storage device, construct a state feature vector based on the health state data, and use the state feature vector as input data of a deep reinforcement learning method;

[0027] extract cycle number, deep cycle rate, and operating temperature parameters of the energy storage device based on the state feature vector, introduce a time decay factor based on the deep reinforcement learning method to adaptively weight operation data at different times in the state feature vector, and obtain weighted historical loss data;

[0028] analyze dynamic change characteristics of the cycle number, the deep cycle rate, and the operating temperature parameters based on the deep reinforcement learning method according to the weighted historical loss data, input the dynamic change characteristics into a loss reference model, and obtain a reference loss feature of the energy storage device;

[0029] construct a multi-dimensional life loss coefficient matrix based on the reference loss feature using the deep reinforcement learning method, optimize the multi-dimensional life loss coefficient matrix by minimizing the deviation between a predicted loss value and an actual loss value, and obtain an optimized loss coefficient matrix;

[0030] integrate the optimized loss coefficient matrix with respect to time to obtain a cumulative loss state of the energy storage device, and determine the remaining life of the energy storage device based on the cumulative loss state and the designed life of the energy storage device.

[0031] construct a multi-dimensional life loss coefficient matrix using the deep reinforcement learning method, optimize the multi-dimensional life loss coefficient matrix by minimizing the deviation between a predicted loss value and an actual loss value, and obtain an optimized loss coefficient matrix including:

[0032] The multi-dimensional life loss coefficient matrix includes cycle number dimension, deep cycle rate dimension, and temperature dimension; the deviation between a predicted loss value and an actual loss value output by the multi-dimensional life loss coefficient matrix is minimized;

[0033] optimize the multi-dimensional life loss coefficient matrix based on the result of the minimization processing to obtain an optimized loss coefficient matrix.

[0034] generate a plurality of candidate charging and discharging scheduling strategies based on the deep reinforcement learning model, use predicted revenue and life loss as dual constraints of a reward function, and realize generation of a charging and discharging strategy that maximizes revenue considering device life through alternating training of a policy network and a value network including:

[0035] A policy generation network is constructed based on a deep reinforcement learning model, the policy generation network comprising a parameter generation module and a value evaluation module, a power grid load state, a price information, a device state of charge and an environmental temperature of the energy storage device are taken as system states, corresponding charging and discharging action policy parameters are generated, and the system states and the charging and discharging action policy parameters form a state-action pair;

[0036] A double-constraint reward function is constructed based on the state-action pair, mean value parameters and standard deviation parameters of an action distribution are obtained by inputting the system states into the parameter generation module, and state values are obtained by inputting the system states and the double-constraint reward function into the value evaluation module;

[0037] An advantage value is calculated based on the state values and the double-constraint reward function, the advantage value is used to guide parameter updating of the policy generation network, parameter updating of the policy generation network is guided based on a time difference error of the state values, and an optimized policy generation network is obtained through alternating training;

[0038] A prediction result of the optimized policy generation network is substituted into the double-constraint reward function to obtain a policy evaluation index, an optimal charging and discharging strategy is selected based on the policy evaluation index under the condition of satisfying a life loss constraint, and an optimal charging and discharging control strategy of the energy storage device is obtained.

[0039] A prediction result of the optimized policy generation network is substituted into the double-constraint reward function to obtain a policy evaluation index, an optimal charging and discharging strategy is selected based on the policy evaluation index under the condition of satisfying a life loss constraint, and an optimal charging and discharging control strategy of the energy storage device is obtained.

[0040] A prediction result of the optimized policy generation network is substituted into the double-constraint reward function, the double-constraint reward function comprises a benefit index and a life loss index of the energy storage device, the prediction result is evaluated based on the double-constraint reward function to obtain a policy evaluation index corresponding to the prediction result;

[0041] A cumulative life loss value of the energy storage device is calculated based on a life loss index in the prediction result, the cumulative life loss value is compared with a preset loss threshold to obtain a life loss constraint condition of the prediction result;

[0042] The prediction results satisfying the life loss constraint condition are sorted in descending order according to the sizes of the policy evaluation indexes, and the result of the descending sorting is taken as a candidate charging and discharging strategy of the energy storage device; and the charging and discharging strategy with the largest policy evaluation index in the candidate charging and discharging strategy is taken as an optimal charging and discharging control strategy of the energy storage device.

[0043] A second aspect of the embodiment of the present application provides a commercial energy storage power station benefit mode intelligent optimization system, comprising:

[0044] The first unit is configured to obtain historical operation data of the commercial energy storage power station, wherein the historical operation data comprises charge-discharge power data, battery state of charge data, real-time power grid price data, and energy storage device health state data.

[0045] The second unit is configured to construct a deep reinforcement learning model of the commercial energy storage power station based on the historical operation data, wherein the deep reinforcement learning model adopts a deep neural network structure, and a predicted revenue value is generated by performing nonlinear mapping on the historical operation data through the deep neural network structure.

[0046] The third unit is configured to analyze the energy storage device health state data through the deep reinforcement learning model, establish a multi-dimensional life loss coefficient matrix based on cycle times, deep cycle rates, and operating temperatures, and realize evaluation of device life loss.

[0047] The fourth unit is configured to generate multiple groups of candidate charge-discharge scheduling strategies based on the deep reinforcement learning model, take the predicted revenue and the life loss as a double constraint of a reward function, and realize generation of a charge-discharge strategy maximizing revenue considering device life through alternating training of a policy network and a value network.

[0048] The fifth unit is configured to generate an operation control instruction according to the optimized charge-discharge scheduling strategy, send the operation control instruction to an energy management system of the commercial energy storage power station, and control the energy storage power station to operate according to the optimized charge-discharge scheduling strategy.

[0049] In a third aspect, an electronic device is provided, comprising:

[0050] a processor;

[0051] a memory for storing processor-executable instructions;

[0052] The processor is configured to invoke the instructions stored in the memory to execute the method described above.

[0053] In a fourth aspect, a computer-readable storage medium is provided, which stores computer program instructions, and the computer program instructions are executed by a processor to implement the method described above.

[0054] The present application has the following advantages:

[0055] The commercial energy storage power station revenue mode intelligent optimization method provided by the present application can accurately predict market price fluctuation trends by analyzing and learning historical operation data through deep reinforcement learning technology, accurately grasp charge-discharge opportunities of the energy storage power station, and improve the ability to obtain economic revenue.

[0056] The method innovatively incorporates the equipment life loss into the optimization decision process, establishes a multi-dimensional life loss evaluation system based on the cycle number, deep cycle rate and working temperature, balances the long-term equipment use cost while pursuing short-term economic benefits, and prolongs the service life of the energy storage system.

[0057] The application realizes intelligent adaptive adjustment of the operation strategy of the commercial energy storage power station, and through the alternating training mechanism of the strategy network and the value network, the system can automatically optimize the charging and discharging strategy according to the dynamic changes of the power grid environment and the equipment state, reduces the manual intervention, and improves the operation efficiency and reliability of the energy storage power station. BRIEF DESCRIPTION OF DRAWINGS

[0058] Figure 1 A flowchart of the intelligent optimization method for the commercial energy storage power station revenue mode of the embodiment of the application is shown in

[0059] Figure 2 A logic block diagram of the deep reinforcement learning prediction revenue method for the commercial energy storage power station of the embodiment of the application is shown in

[0060] Figure 3 A convergence and reward function trend graph of the deep reinforcement learning model of the embodiment of the application is shown in DETAILED DESCRIPTION

[0061] In order to make the purpose, technical scheme and advantages of the embodiments of the application clearer, the technical scheme in the embodiments of the application will be described clearly and completely below in combination with the drawings in the embodiments of the application. Obviously, the described embodiments are only part of the embodiments of the application, not all the embodiments. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor are within the protection scope of the application.

[0062] The technical scheme of the application will be described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in some embodiments.

[0063] Figure 1 A flowchart of the intelligent optimization method for the commercial energy storage power station revenue mode of the embodiment of the application is shown in Figure 1 As shown in the figure, the method comprises:

[0064] The historical operation data of the commercial energy storage power station is acquired, and the historical operation data comprises charging and discharging power data, battery state of charge data, real-time power price data of the power grid and health state data of the energy storage equipment;

[0065] construct a deep reinforcement learning model of the commercial energy storage power station based on the historical operation data, the deep reinforcement learning model adopts a deep neural network structure, and a predicted revenue value is generated by performing nonlinear mapping on the historical operation data through the deep neural network structure;

[0066] The health state data of the energy storage device is analyzed through the deep reinforcement learning model, a multi-dimensional life loss coefficient matrix is established based on the number of cycles, the deep cycle rate and the operating temperature, and evaluation of the life loss of the device is realized;

[0067] A plurality of groups of candidate charging and discharging scheduling strategies are generated based on the deep reinforcement learning model, the predicted revenue and the life loss are taken as double constraints of a reward function, and the charging and discharging strategy for maximizing the revenue considering the life of the device is generated through the alternative training of a policy network and a value network;

[0068] According to the optimized charging and discharging scheduling strategy, an operation control instruction is generated, the operation control instruction is sent to an energy management system of the commercial energy storage power station, and the energy storage power station is controlled to operate according to the optimized charging and discharging scheduling strategy.

[0069] In an optional implementation, the deep reinforcement learning model of the commercial energy storage power station is constructed based on the historical operation data, and the deep reinforcement learning model adopts a deep neural network structure, and a predicted revenue value is generated by performing nonlinear mapping on the historical operation data through the deep neural network structure, which includes the following steps:

[0070] The deep reinforcement learning model adopts a multi-layer feedforward neural network structure, the multi-layer feedforward neural network structure includes an input layer, a hidden layer and an output layer, wherein the hidden layer performs feature extraction and nonlinear mapping on the historical operation data through a weight matrix, a bias vector and a nonlinear activation function;

[0071] The network parameters of the multi-layer feedforward neural network structure are constructed as random variables, a Gaussian distribution model of the network parameters is established by using a variational inference method, and the prediction uncertainty of the multi-layer feedforward neural network structure is quantitatively characterized by using the mean parameter and the variance parameter of the Gaussian distribution model;

[0072] The adversarial samples are generated by adding perturbations to the historical operation data based on the Gaussian distribution model, and the adversarial samples are used in the training process of the multi-layer feedforward neural network structure to improve the robustness of the multi-layer feedforward neural network structure;

[0073] The method comprises the following steps: calculating a prediction mean value and a prediction standard deviation based on a prediction result of the multi-layer feedforward neural network structure, determining a confidence interval of the prediction result according to the prediction mean value and the prediction standard deviation; calculating a dynamic adjustment coefficient according to the prediction standard deviation, and adaptively optimizing the prediction result of the multi-layer feedforward neural network structure based on the dynamic adjustment coefficient to obtain an optimized prediction result;

[0074] The optimized prediction result and the confidence interval are input into a revenue calculation module, and a predicted revenue value of the commercial energy storage power station is generated through the revenue calculation module.

[0075] As shown in the formula (1), the method comprises the following steps: Figure 2

[0076] The historical operation data acquisition involves multiple dimensions and is divided into two categories of real-time data and historical data. The real-time data includes: power grid frequency (sampling frequency 1 Hz), battery pack terminal voltage (sampling frequency 0.1 Hz), battery pack current (sampling frequency 0.1 Hz), single battery voltage (sampling frequency 0.01 Hz), battery surface temperature (sampling frequency 0.01 Hz), and ambient temperature (sampling frequency 0.01 Hz). The historical data includes: peak-valley time-of-use electricity price data (updated daily), historical charging and discharging power curve (15 minutes per point), historical SOC change curve (15 minutes per point), and historical capacity change data (tested once a month).

[0077] The missing value processing (linear interpolation method to fill single-point missing, forward filling method to process continuous missing), the abnormal value processing (3σ principle to filter obvious abnormal values), the normalization processing (Min-Max standardization to scale each feature to the interval [0, 1]), and the time series feature extraction (sliding window method to extract 24-hour period features) are performed. The dimension of the preprocessed feature matrix is (sample number x 24 x feature number), the feature number is 8, and the features include hourly electricity price, SOC value, charging power, discharging power, ambient temperature, battery temperature, power grid frequency and date type (weekday / weekend).

[0078] The number of input layer nodes is 24 x 8 = 192, corresponding to 8 features in 24 hours; the number of nodes in the first hidden layer is 128, the ReLU activation function is used, the weight matrix dimension is 192 x 128, and the initialization method is He initialization (the weight value is subject to mean 0 and standard deviation sqrt(2 / fan-in), where fan-in is the number of input nodes). in ​) is a normal distribution), the bias vector is initialized as 0; the second hidden layer has 256 nodes, uses a ReLU activation function, and has a weight matrix with a dimension of 128x256, which is initialized in the same way as the first layer, and a Batch Normalization layer is added, with parameters including a scaling factor γ (initial value 1) and an offset factor β (initial value 0); the third hidden layer has 128 nodes, uses a ReLU activation function, and has a weight matrix with a dimension of 256x128, which is initialized in the same way as before, and a Dropout layer is added, with a dropout rate of 0.3; the output layer has 24 nodes, uses a linear activation function, and has a weight matrix with a dimension of 128x24.

[0079] The total number of parameters is (192x128) + (128x256) + (256x128) + (128x24) + 128 + 256 + 128 + 24 = 93720. The forward propagation calculation sequence is: input data - first hidden layer linear transformation - ReLU activation - second hidden layer linear transformation - Batch Normalization - ReLU activation - Dropout - third hidden layer linear transformation - ReLU activation - output layer linear transformation - output result.

[0080] The variational inference method implementation uses the reparameterization trick, and the network parameters are no longer deterministic values, but probability distributions. For each weight w, define its variational posterior distribution as a Gaussian distribution with mean μ and variance σ 2 . The specific implementation process is: create two trainable variables μ and ρ for each weight parameter w, where μ directly corresponds to the distribution mean, and σ is obtained through the transformation σ = log(1 + exp(ρ)), ensuring that the variance is always positive. When sampling, noise ε is sampled from a standard normal distribution, and the weight sample is obtained through w = μ + σ x ε.

[0081] In backpropagation, the gradient can be directly passed to μ and ρ through this transformation, and the training objective function includes two parts: negative log-likelihood (prediction error) and KL divergence regularization term, the proportion of which is controlled by the hyperparameter λ, which is usually set as the inverse of the sample size. Optimization uses the Adam optimizer with an initial learning rate of 0.001, β1 = 0.9, β2 = 0.999, ε = 1e-8, weight decay coefficient 1e-4, and a learning rate decay strategy of decaying to 0.9 times the original value every 10 epochs. The training process is batch iteration, with 64 samples randomly selected in each batch, forward propagation, loss function and gradient calculation, and μ and ρ parameter update.

[0082] The parameter calculation of the Gaussian distribution model involves precise statistical methods. For a trained model, the distribution of each weight parameter w is determined by the mean μ and variance σ 2Determination. In the prediction phase, T = 100 Monte Carlo samples are performed, each sampling a complete set of network parameters from the posterior distribution of all weights, performing a forward pass to obtain a prediction result.

[0083] For each weight parameter w, a value w 2 is sampled from its Gaussian distribution N(μ,σ (t) 2), where ε (t) is the standard normal random number of the t-th sample; all sampled weights are combined into a complete set of network parameters, and a forward pass is performed to obtain the t-th prediction result y (t) ; repeat T times to obtain {y (t) , y (1) ... y (2)}; calculate the prediction mean y (T) = sum(y mean ) / T; calculate the prediction variance y (t) = sum((y var - y (t) ) mean 2 ) / T. For example, for the electricity price prediction at a certain time, the mean of 100 sampling results is 0.824 yuan / kWh, the variance is 0.0009 (yuan / kWh) 2 , and the standard deviation is 0.03 yuan / kWh.

[0084] For the original input data X, first calculate the input gradient g = ∂L / ∂X, i.e. the gradient of the loss function L with respect to the input X; normalize the gradient g norm = g / ||g||, where ||g|| represents the Euclidean norm of the gradient; dynamically determine the perturbation strength ε according to the prediction standard deviation, specifically ε = α × std(y), where α is the perturbation coefficient (set to 2), and std(y) is the prediction standard deviation; generate the adversarial sample X' = X + ε × g norm . For discrete features (such as date types), keep them unchanged, and only add perturbations to continuous features.

[0085] For example, the original electricity price feature vector is [0.4, 0.4, 0.8, 1.2... ] yuan / kWh, the prediction standard deviation is 0.03 yuan / kWh, the perturbation strength ε = 2 × 0.03 = 0.06 yuan / kWh, and the normalized gradient vector is [0.5, 0.5, -0.5, -0.5... ], then the adversarial sample electricity price feature is [0.4 + 0.06 × 0.5, 0.4 + 0.06 × 0.5, 0.8 + 0.06 × (-0.5), 1.2 + 0.06 × (-0.5)... ] = [0.43, 0.43, 0.77, 1.17...] yuan / kWh. During training, 5 adversarial samples are generated per batch, mixed with 20 original samples, with a ratio of 1:4.​

[0086] The confidence interval calculation process uses an accurate statistical method, based on the prediction result set {y (1) ,y (2) ...y (T)} of T = 100 Monte Carlo samples, arranged in ascending order to obtain the ordered sequence {y [1] ,y [2] ...y [T]}; for a 95% confidence interval, the lower boundary index low = floor(0.025 x T) = 2, and the upper boundary index high = ceil(0.975 x T) = 98; the lower limit of the confidence interval is y [2] , and the upper limit is y

[98] .

[0087] For example, after arranging the 100 sampling values of the electricity price prediction in ascending order at a certain time, the 2nd smallest value is 0.768 yuan / kWh, and the 98th largest value is 0.879 yuan / kWh, so the 95% confidence interval is [0.768, 0.879] yuan / kWh. In addition, a parameterized method can also be used, based on the prediction mean y mean = 0.824 yuan / kWh and the prediction standard deviation std(y) = 0.03 yuan / kWh, to calculate the 95% confidence interval as [y mean - 1.96 x std(y), y mean + 1.96 x std(y)] = [0.765, 0.883] yuan / kWh. The intervals obtained by the two methods are close, verifying the consistency of the calculation.

[0088] The dynamic adjustment coefficient calculation uses a piecewise function, considering the nonlinear impact of uncertainty, defines the lower threshold σ low = 0.02, the upper threshold σ high = 0.1, and the maximum adjustment coefficient β max = 0.2; when the prediction standard deviation std(y) ≤ σ low , the adjustment coefficient β = 0; when std(y) ≥ σ high , β = β max ; when σ low < std(y) < σ high , β = β max x (std(y) - σ low ) / (σ high - σ low ).

[0089] For example, when the prediction standard deviation is 0.05, the adjustment coefficient β = 0.2 x (0.05 - 0.02) / (0.1 - 0.02) = 0.075. The adjustment direction is determined by the historical prediction bias statistics, specifically, the average prediction bias bias = average(y predict -y true ) of the high uncertainty area (standard deviation greater than 0.05) in the past 30 days is calculated; if bias > 0, it means that the prediction is usually high, and the adjustment direction is negative; if bias < 0, it means that the prediction is usually low, and the adjustment direction is positive. Assuming that bias = 0.04 > 0, the prediction mean is 0.824 yuan / kWh, and the adjustment coefficient is 0.075, then the adjusted prediction value is 0.824 - 0.075 x 0.824 = 0.762 yuan / kWh.

[0090] The complete algorithm flow of the revenue calculation module includes multiple steps, and the input includes the optimized electricity price prediction p (t) , the power prediction plan P (t) and the confidence interval [p low (t) , p high (t) ], [P low (t) , P high (t) ], t = 1, 2...24 representing 24 hours. Positive power value represents charging, and negative value represents discharging. Assuming that the charging and discharging efficiency η = 0.9, the state of charge constraints SOC min = 0.1, SOC max = 0.9, the initial SOC value is SOC0= 0.5, and the battery capacity is E = 5MWh.

[0091] Verify whether the power plan meets the SOC constraint. For t = 1, 2...24, calculate SOC (t) =SOC (t-1) +P (t) ×Δt / E, where Δt = 1 hour, if SOC (t) <SOC min or SOC (t) >SOC max , adjust P (t) to make SOC (t) meet the constraint; calculate the charging and discharging power of each period, charging power E charge (t) =max(0, P (t) ) x Δt, and discharging power E discharge (t) =max(0, -P (t) ) x Δt x η; calculate the cost and revenue of each period, charging cost C (t) =Echarge (t) ×p (t) , discharge benefit R (t) =E discharge (t) ×p (t) / η; calculate total cost C=sum(C (t) ), total benefit R=sum(R (t) ), net benefit Net R =R-C; calculate benefit interval based on confidence interval, minimum net benefit Net R low Calculate using the lower limit of electricity price and power limit, maximum net benefit Net R high Calculate using the upper limit of electricity price and power limit.

[0092] For example, the valley period (0:00-6:00) predicted electricity price is 0.4 yuan / kWh, the planned charging power is 1MW, the charging power is 6MWh, and the cost is 2400 yuan; the peak period (18:00-22:00) predicted electricity price is 1.2 yuan / kWh, the planned discharge power is 1.2MW, the discharge power is 4.8MWh, and the actual discharge is 5.33MWh after considering the efficiency, the benefit is 6400 yuan; the net benefit is 6400-2400=4000 yuan. Considering the electricity price confidence interval [0.38, 0.42] yuan / kWh and [1.15, 1.25] yuan / kWh, the power confidence interval [0.9, 1.1] MW and [1.1, 1.3] MW, the benefit interval is calculated as [3539, 4548] yuan.

[0093] In the practical application of a certain 100MW / 200MWh large commercial energy storage power station, the method is applied to real-time optimization decision for 30 days. The operation strategy guided by the traditional deterministic model realizes daily average benefit of 285,000 yuan, and the operation strategy optimized by the method realizes daily average benefit of 342,000 yuan, which is increased by 20.0%. At the same time, the uncertainty quantification provided by the method enables the power station operator to adopt a more conservative strategy for high-risk periods (such as when the prediction interval is wide), avoiding 3 potential major decision-making mistakes, and is expected to recover a loss of more than 1.2 million yuan. The method has been stably running in the power station for 6 months, and the cumulative increase in benefit is 10.26 million yuan, proving the practicability and effectiveness of the method.

[0094] In an optional implementation, a Gaussian distribution model of the network parameters is established by using a variational inference method, and the prediction uncertainty of the multi-layer feedforward neural network structure is quantitatively characterized by a mean parameter and a variance parameter of the Gaussian distribution model, comprising:

[0095] obtain training data of the multi-layer feedforward neural network structure, construct a likelihood function based on the training data, select a Gaussian distribution as a prior distribution of network parameters, combine the likelihood function and the prior distribution through a Bayesian formula to construct a posterior distribution;

[0096] approximate the posterior distribution by using a variational inference method, construct a variational distribution family in a Gaussian distribution form, the variational distribution family includes a mean parameter vector and a variance parameter vector, and obtain optimal variational distribution parameters by minimizing a kullback-leibler divergence between the variational distribution family and the posterior distribution through the variational inference method;

[0097] input the optimal variational distribution parameters into the multi-layer feedforward neural network structure, for each layer in the multi-layer feedforward neural network structure, construct a parameter distribution of the layer based on the mean parameter vector and the variance parameter vector by using the variational inference method, obtain network parameter samples based on the parameter distribution, and input the network parameter samples into the multi-layer feedforward neural network structure to obtain corresponding network outputs;

[0098] repeat obtaining multiple groups of network parameter samples, input the multiple groups of network parameter samples into the multi-layer feedforward neural network structure to obtain multiple groups of network outputs, calculate a prediction mean and a prediction variance based on the multiple groups of network outputs, and use the prediction mean and the prediction variance as prediction uncertainty quantization results of the multi-layer feedforward neural network structure.

[0099] The training data of the multi-layer feedforward neural network structure is usually obtained from a historical data set of a specific task. For example, for a certain power load prediction task, the training data includes input features and target output values. The input features include historical load values, temperatures, humidities, wind speeds, date types and other factors, and the target output is the load value at a future time. The data set includes 10,000 training samples, each sample including 24 features and 1 target value. Data preprocessing includes standardization processing, which maps each feature value to a distribution with a mean of 0 and a standard deviation of 1, to avoid the influence of dimension differences on model training.

[0100] The likelihood function is constructed based on a preset error distribution model. For a regression task, it is usually assumed that the error between the predicted value and the true value follows a Gaussian distribution, the mean of which is 0 and the variance of which is a constant. In a specific implementation, given input features x and corresponding true output y, the neural network outputs a predicted value f(x), and the likelihood function is represented as y follows a Gaussian distribution with a mean of f(x) and a variance of σ 2 For the entire training set, the likelihood function is the product of the likelihoods of each sample. In actual calculation, the logarithm is usually taken to convert it into a summation form to improve numerical stability.

[0101] The network parameter prior distribution is selected as a Gaussian distribution, specifically a standard Gaussian distribution with a mean of 0 and a variance of 1. The reason for selecting a Gaussian distribution as the prior is its good mathematical properties, including analytical convenience and computational efficiency when combined with the likelihood function. In the Bayesian framework, the posterior distribution is calculated by the Bayes formula, which is the product of the prior distribution and the likelihood function divided by the marginal likelihood. However, it is often difficult to accurately calculate the posterior distribution in complex neural networks, so the variational inference method is introduced for approximation.

[0102] The core idea of the variational inference method is to find a parameterized distribution to approximate the true posterior distribution. A variational distribution family in the form of a Gaussian distribution is constructed. For each weight parameter w in the neural network, the variational distribution is set as a Gaussian distribution with a mean of μ and a variance of σ 2 . The variational distribution family is jointly described by the mean parameter vector μ and the variance parameter vector σ 2 . Taking a neural network with two hidden layers as an example, if the first layer has 100 neurons, the second layer has 50 neurons, the output layer has 1 neuron, and the input feature dimension is 24, the total number of parameters is (24x100) + (100x50) + (50x1) = 7450, and the corresponding variational distribution family requires 7450 mean parameters and 7450 variance parameters.

[0103] The KL divergence minimization between the variational distribution family and the posterior distribution is achieved by the stochastic gradient descent method. The actual optimization goal is the maximization of the evidence lower bound (ELBO), which is defined as the log marginal likelihood minus the KL divergence. During optimization, the reparameterization trick is used, which represents parameter sampling as the mean parameter plus the product of the variance parameter and a standard normal distribution sample, so that the gradient can be propagated through the random variable. The optimization uses the Adam optimizer with a learning rate of 0.001, a batch size of 128, and 10000 iterations of training. Through iterative optimization, the mean parameter vector and the variance parameter vector gradually converge to the optimal value. For example, part of the mean parameters of the first layer weights converge to [-0.232, 0.451, -0.178, 0.321], and the variance parameters converge to [0.021, 0.034, 0.018, 0.025].

[0104] When applying variational inference to each layer of a multi-layer feedforward neural network structure, the hierarchical Bayesian model idea is used. For the connections from the input layer to the first hidden layer, the weight distribution is constructed based on the mean parameter μ1 and the variance parameter σ1 2 ; for the connections from the first hidden layer to the second hidden layer, the weight distribution is constructed using μ2 and σ2 2 ; and so on until the output layer. The bias terms of each layer are also modeled using a Gaussian distribution. In practical applications, diagonal Gaussian distribution can be used to simplify the calculation, which assumes that different weight parameters are independent of each other.

[0105] The network parameter sample acquisition adopts a Monte Carlo sampling method to randomly extract samples from the variational distribution of each weight parameter to form a complete set of network parameters. In a specific implementation, for a Gaussian distribution with a mean of μ and a variance of σ 2 , the parameter sample calculation is w = μ + σ x ε, where ε is a random number extracted from a standard normal distribution. For example, if the mean of a certain weight parameter is 0.5 and the variance is 0.04, then after extracting a random number 0.8 from a standard normal distribution, a sample of the weight is 0.5 + (0.2 x 0.8) = 0.66.

[0106] The above sampling process is repeated T times to obtain T different sets of network parameter samples. In practice, T is usually set to between 10 and 100, and in this embodiment, T is set to 50. These parameter samples are input into the neural network, and 50 forward propagations are calculated for the same input feature x to obtain 50 different output values {y1, y2... y 50}. The prediction mean is the arithmetic mean of these output values, and the prediction variance is the variance of these output values. For example, the 50 prediction results for a test sample are [245.3, 251.8, 247.2,..., 249.5] MW, and the calculation of the prediction mean is 248.7 MW, and the prediction variance is 12.6 MW 2 .

[0107] The prediction mean and the prediction variance have clear practical significance as a quantitative result of the prediction uncertainty. The prediction mean represents the estimated value of the model for the target variable, and the prediction variance represents the size of the prediction uncertainty. The larger the prediction variance, the lower the confidence of the model for the prediction result. This uncertainty quantification can be converted into a prediction interval in practical applications, such as a 95% confidence interval, which can be expressed as the prediction mean ± 1.96 x the prediction standard deviation.

[0108] In the case of power load prediction, compared with the traditional deterministic neural network, the prediction accuracy is improved by 12.3%, and a reliable uncertainty estimate is provided. In high-uncertainty areas (such as holidays and extreme weather), traditional models often have large errors, while this method provides a warning through a larger prediction variance to help decision-makers adopt more conservative scheduling strategies. Experiments show that the coverage rate of the prediction interval of this method reaches 94.2%, close to the theoretical value of 95%, proving the effectiveness of the uncertainty quantification.

[0109] In actual deployment, the method needs multiple forward propagations in the prediction stage, and the calculation cost is high. To solve this problem, the Monte Carlo dropout method can be used to simplify the implementation, that is, the dropout activation is maintained during prediction, and each forward propagation is equivalent to sampling an instance from the model distribution. This variant method improves the calculation efficiency by about 40%, while maintaining the effectiveness of uncertainty quantification. On edge computing devices, performance can be further optimized by reducing the number of samples, model compression, and other methods, while ensuring prediction quality and reducing resource consumption.

[0110] In an optional implementation, the health state data of the energy storage device is analyzed by the deep reinforcement learning model, a multi-dimensional life loss coefficient matrix is established based on the cycle number, the depth cycle rate and the working temperature, and the evaluation of the life loss of the device is realized, including:

[0111] Obtain the health state data of the energy storage device, and construct a state feature vector based on the health state data. The state feature vector is used as input data of the deep reinforcement learning method;

[0112] Based on the state feature vector, the cycle number, the depth cycle rate and the working temperature parameters of the energy storage device are extracted by using the deep reinforcement learning method. A time decay factor is introduced based on the deep reinforcement learning method to adaptively weight the running data at different times in the state feature vector, and the weighted historical loss data is obtained;

[0113] According to the weighted historical loss data, the dynamic change characteristics of the cycle number, the depth cycle rate and the working temperature parameters are analyzed based on the deep reinforcement learning method, the dynamic change characteristics are input into a loss reference model, and a reference loss feature of the energy storage device is obtained;

[0114] Based on the reference loss feature, a multi-dimensional life loss coefficient matrix is constructed by using the deep reinforcement learning method. The multi-dimensional life loss coefficient matrix is optimized by minimizing the deviation between the predicted loss value and the real loss value, and an optimized loss coefficient matrix is obtained;

[0115] The optimized loss coefficient matrix is integrated with respect to time to obtain the cumulative loss state of the energy storage device, and the remaining life of the energy storage device is determined based on the cumulative loss state and the designed life of the energy storage device.

[0116] In the process of obtaining the health state data of the energy storage device, multi-dimensional parameters such as voltage, current, temperature, state of charge (SOC), and depth of discharge (DOD) during operation of the device are collected. The collection frequency is once every 5 minutes, and 288 data points are collected daily. Device operating parameters are stored according to different time spans, including minute-level, hour-level, day-level, and month-level data. Minute-level data mainly includes real-time voltage, current, power, and temperature parameters, such as the data collected at a certain time: voltage 724.5V, current 56.2A, ambient temperature 25.3℃, battery pack temperature 28.7℃, and SOC value 72.5%. Hour-level data mainly includes average discharge depth, maximum temperature, minimum temperature, and charge-discharge times within an hour. Day-level data includes comprehensive indicators such as total charge-discharge energy, equivalent cycle number, and maximum discharge depth.

[0117] In constructing the state feature vector, the collected raw data is preprocessed, including missing value processing, outlier filtering, and normalization processing. For missing values, the mean interpolation method is used to fill in; for outliers, the 3 standard deviation method is used for screening, and data points outside the range are replaced with adjacent valid values. Normalization processing uses the maximum and minimum value normalization method to map all features to the 0-1 range. The normalized data forms the state feature vector, with specific structures including: timestamp features (year, month, day, hour, minute, second), environmental features (ambient temperature, humidity), battery operation features (SOC, DOD, cycle number, charge-discharge power), and historical cumulative features (cumulative cycle number, cumulative full charge and full discharge times, and cumulative shallow charge and shallow discharge times). For example, the state feature vector at a certain time is [0.25, 0.42, 0.83, 0.12, 0.65, 0.33, 0.78, 0.56, 0.45, 0.68, 0.23, 0.44, 0.35, 0.67, 0.89], corresponding to the normalized parameter values.

[0118] The deep reinforcement learning method uses a dual-channel neural network structure, including a feature extraction network and a state evaluation network. The feature extraction network consists of 3 convolutional layers and 2 fully connected layers. The convolutional layers are used to capture time series features in time series data, with convolution kernel sizes of 5x5, 3x3, and 3x3, and feature map numbers of 32, 64, and 64, respectively. Each layer is followed by a max pooling layer and a ReLU activation function. The fully connected layers have 256 and 128 neurons, respectively, to comprehensively extract high-level feature representations. The state evaluation network consists of 3 fully connected layers with 128, 64, and 32 neurons, respectively, and the output layer has multiple estimates, including cycle number estimation, depth cycle rate estimation, and temperature influence factor estimation.

[0119] The cycle number, depth cycle rate and working temperature parameters are extracted by using a sliding window method, the window size is 24 hours, the step is 1 hour, and the parameter changes in the past 24 hours are extracted. The cycle number is defined as the battery from the discharge state to the charging state and then discharged again, which is calculated as a complete cycle, and when the SOC curve appears a complete wave peak and valley, it is recorded as a cycle. The depth cycle rate is defined as the difference between the highest SOC and the lowest SOC in a cycle, for example, from SOC 90% to SOC 30%, the depth cycle rate is 60%. The working temperature parameters include average temperature, maximum temperature and temperature fluctuation amplitude, which are calculated by the sensor to obtain the surface temperature of the battery.

[0120] The time decay factor is introduced by using an exponential decay method, the data closer to the current time has a larger weight, and the data farther away has a smaller weight. The specific implementation method is: for the data of the i-th time point before t time, the weight is calculated as the base weight multiplied by the i-th power of the decay coefficient. For example, assuming that the base weight is 1 and the decay coefficient is 0.95, the weight of the first historical point is 0.95, the weight of the second historical point is 0.95^2=0.9025, and the weight of the tenth historical point is about 0.599. After weighting, the effective weight of the data in the last 24 hours accounts for more than 80%, which ensures that the model pays more attention to the recent running state.

[0121] In the weighting process of historical loss data, different types of loss are considered for differentiated processing. For the loss caused by the cycle number, a linear weighting method is used; for the loss caused by the depth cycle rate, a quadratic weighting method is used; and for the loss caused by the temperature, an exponential weighting method is used. For example, a certain energy storage equipment has a cycle number of 2.5 times in 24 hours, an average depth cycle rate of 55%, an average working temperature of 32°C, a weight of 1 for the last cycle, and a weight of 0.9 for the previous day. The equivalent cycle number after weighting is 2.5x1+2.2x0.9=4.48 times, the equivalent depth cycle rate after weighting is 55% x 1+60% x 0.9=109%, and the equivalent temperature after weighting is 32°C x 1+28°C x 0.9=57.2°C.

[0122] In the dynamic change characteristic analysis process, the sliding variance method is used to detect the parameter volatility, the trend analysis method is used to detect the parameter change trend, and the period analysis method is used to detect the periodic change characteristics of the parameters. The sliding variance calculates the variance of each parameter in the past 7 days, and the larger the variance value, the greater the parameter fluctuation and the poorer the stability. The trend analysis uses linear regression method to calculate the change slope of the parameters in the past 30 days, and the positive slope indicates that the parameter has an upward trend, and the negative slope indicates that the parameter has a downward trend. The period analysis uses the fast Fourier transform method to identify the main period of parameter change.

[0123] The loss benchmark model is established based on a large amount of historical data collected in advance and is constructed using a random forest regression method. The input features include cycle times, depth cycle rates, operating temperatures, and other parameters, and the output is the predicted capacity loss rate. The model uses 80% of the historical data for training and 20% for verification. For example, a certain type of lithium battery has a capacity loss of about 0.8% after 100 cycles under standard conditions (25°C, 80% DOD) and a loss of about 7.5% after 1000 cycles. The benchmark model predicts the capacity loss under actual operating conditions based on these standard condition loss data, combined with temperature correction coefficients and depth cycle correction coefficients.

[0124] The multi-dimensional life loss coefficient matrix construction process uses a three-dimensional grid method. The first dimension is cycle times, divided into 10 intervals (0-100 times, 100-300 times, 300-500 times, 500-1000 times, 1000-2000 times, 2000-3000 times, 3000-5000 times, 5000-8000 times, 8000-12000 times, >12000 times); the second dimension is depth cycle rate, divided into 5 intervals (0-20%, 20-40%, 40-60%, 60-80%, 80-100%); the third dimension is operating temperature, divided into 7 intervals (<0°C, 0-10°C, 10-20°C, 20-30°C, 30-40°C, 40-50°C, >50°C). For each grid point, the loss coefficient under the condition is predicted by a deep reinforcement learning method. For example, under the condition of 1500 cycle times, a depth cycle rate of 55%, and an operating temperature of 35°C, the loss coefficient is 0.00085, indicating that the battery capacity loss is about 0.085% per cycle.

[0125] The loss coefficient matrix optimization uses a gradient descent method, and the parameters are adjusted according to the mean square error between the predicted loss value and the actual loss value. The learning rate is set to 0.001, the batch size is 64, and the training iteration number is 1000. The error between the predicted loss and the true loss is calculated each iteration, and the parameters in the loss coefficient matrix are updated through backpropagation. For example, under certain conditions, the initial predicted loss is 0.75%, the actual loss is 0.82%, the error is 0.07%, and after optimization, the predicted loss is adjusted to 0.80% and the error is reduced to 0.02%. The average prediction error of the optimized loss coefficient matrix on the test set is reduced from 12.5% to 4.3%.

[0126] In the cumulative loss state calculation process, the loss on the time series is integrated. For discrete time points, the trapezoidal integration method is used for approximate calculation. Specifically, the loss coefficient in each time period is multiplied by the number of cycles in that time period, and then accumulated. For example, the energy storage device has an equivalent cycle of 45 times under various conditions in the first month, and the average loss coefficient is 0.00075, so the capacity loss in the first month is 45x0.00075=0.03375, i.e. 3.375%. The loss of each month is calculated in this way and accumulated to obtain the total cumulative loss.

[0127] The remaining life is determined based on the comparison of the cumulative loss state and the design life. Taking capacity attenuation as an example, if the battery design life is 80% of the initial capacity (i.e. allowing 20% capacity attenuation), and the current cumulative loss is 8%, then the remaining life percentage is (20%-8%) / 20%=60%. If the battery design cycle life is 6000 times, then the estimated remaining cycle number is about 6000x60%=3600 times. Combined with the current usage frequency, the remaining usage time can be further predicted. For example, if the current daily cycle number is 2.5 times, then the estimated remaining usage time is about 3600 / 2.5 / 365=3.95 years.

[0128] In practical application, this method is applied to a certain grid-side energy storage power station, which contains a 10MWh lithium battery energy storage system. After the system has been in use for 1.5 years, the cumulative capacity loss is evaluated to be 5.8% by this method, and the remaining life is predicted to be 14.2 years, which is consistent with the actual capacity test result (5.5% loss) with a consistency of more than 95%, proving the effectiveness of this method. Compared with the traditional life evaluation method, the prediction accuracy of this method is improved by 25%, especially in complex use conditions.

[0129] In an optional implementation, a multi-dimensional life loss coefficient matrix is constructed using the deep reinforcement learning method, the multi-dimensional life loss coefficient matrix is optimized by minimizing the deviation between the predicted loss value and the actual loss value, and an optimized loss coefficient matrix is obtained, including:

[0130] The multi-dimensional life loss coefficient matrix includes a cycle number dimension, a deep cycle rate dimension, and a temperature dimension; the deviation between the predicted loss value and the actual loss value output by the multi-dimensional life loss coefficient matrix is minimized;

[0131] The multi-dimensional life loss coefficient matrix is optimized based on the result of the minimization processing, and an optimized loss coefficient matrix is obtained.

[0132] The construction process of the multi-dimensional life loss coefficient matrix starts with data collection and preprocessing. The loss data of the battery under different cycle numbers, different depth cycle rates, and different temperature conditions are collected. For example, under the condition of room temperature 25℃ and depth cycle rate of 80%, the capacity loss of the battery after 100, 200, and 500 cycles is 5%, 8%, and 15%, respectively. Under the condition of high temperature 40℃ and depth cycle rate of 60%, the capacity loss of the battery after the same cycle number is 7%, 11%, and 20%, respectively. After standardization, these data are used as the training data set of deep reinforcement learning.

[0133] The deep reinforcement learning network architecture includes three core components: state space, action space, and reward function. The state space is defined as the current state of the battery, including the cycle number that has been experienced, the depth cycle rate, and the environmental temperature. For example, the state can be represented as the battery has experienced 300 cycles, the average depth cycle rate is 70%, and the average environmental temperature is 30℃. The action space is defined as the adjustment operation on the multi-dimensional life loss coefficient matrix, including increasing or decreasing the coefficient value in a specific dimension. The reward function is designed as the negative value of the deviation between the predicted loss value and the real loss value. The smaller the deviation, the greater the reward.

[0134] In the training process, the deep Q network (DQN) algorithm is used to realize deep reinforcement learning. The network input is the state information of the battery, and the output is the adjustment action on the loss coefficient matrix. When initializing the multi-dimensional life loss coefficient matrix, the cycle number dimension is set to [0.05, 0.08, 0.12, 0.18, 0.25], corresponding to 100, 200, 500, 1000, and 2000 cycles, respectively; the depth cycle rate dimension is set to [0.6, 0.8, 1.0, 1.2, 1.4], corresponding to 20%, 40%, 60%, 80%, and 100% of the depth cycle rate, respectively; the temperature dimension is set to [0.7, 0.9, 1.0, 1.2, 1.5], corresponding to 0℃, 15℃, 25℃, 35℃, and 45℃ of the environmental temperature, respectively.

[0135] The deep Q network uses a four-layer neural network structure, including an input layer, two hidden layers, and an output layer. The number of input layer nodes is the dimension of the state space, and the number of output layer nodes is the dimension of the action space. The number of nodes in the two hidden layers is 128 and 64, respectively. The activation function uses the ReLU function, the optimizer uses the Adam algorithm, and the learning rate is set to 0.001.

[0136] The training process adopts an experience replay mechanism, with an experience pool size of 10,000 and a batch size of 128 samples randomly sampled from the experience pool for each training. The discount factor is set to 0.95, the initial ε value of the ε-greedy strategy is 1.0, the minimum ε value is 0.01, and the ε decay rate is 0.995. The number of training iterations is set to 5,000, and the model performance is evaluated every 100 iterations.

[0137] During the training process, the deep reinforcement learning algorithm adjusts the actions by trying different loss coefficient matrices and continuously optimizes the coefficient values in each dimension of the matrix. For example, when it is detected that the predicted loss value is significantly lower than the actual loss value in a high-temperature environment (such as 40°C), the algorithm increases the coefficient value of the temperature dimension in the high-temperature region; when it is found that the predicted loss value is higher than the actual loss value at a large number of cycles (such as 2,000), the algorithm reduces the coefficient value of the cycle number dimension in the large cycle number region.

[0138] When optimizing the multi-dimensional life loss coefficient matrix by minimizing the deviation between the predicted loss value and the actual loss value, the mean squared error (MSE) is used as the loss function. In each iteration, the predicted loss value is calculated based on the current loss coefficient matrix, and then the mean squared error is calculated with the actual loss value. The algorithm adjusts the parameters of the deep Q network based on the size and direction of the error, and then guides the adjustment of the loss coefficient matrix.

[0139] After 5,000 iterations of training, the optimized multi-dimensional life loss coefficient matrix is obtained. The optimized cycle number dimension coefficient becomes [0.048, 0.079, 0.125, 0.176, 0.242], the deep cycle rate dimension coefficient becomes [0.58, 0.82, 1.03, 1.18, 1.45], and the temperature dimension coefficient becomes [0.68, 0.88, 1.0, 1.25, 1.58]. The prediction accuracy of the optimized loss coefficient matrix on the test data set reaches 92.5%, and the average absolute error is reduced to 2.3%, which is a significant improvement compared to the initial matrix with an accuracy of 85.0% and an average absolute error of 4.7%.

[0140] To verify the effectiveness of the optimized loss coefficient matrix, the predicted loss value and the actual loss value under different conditions are compared. For example, under the conditions of room temperature 25℃, depth cycle rate 80%, and cycle number 800 times, the predicted loss value before optimization is 16.2%, the actual loss value is 14.8%, and the absolute error is 1.4%; the predicted loss value after optimization is 15.0%, and the absolute error is reduced to 0.2%. Under the conditions of high temperature 40℃, depth cycle rate 100%, and cycle number 1500 times, the predicted loss value before optimization is 28.5%, the actual loss value is 32.0%, and the absolute error is 3.5%; the predicted loss value after optimization is 31.2%, and the absolute error is reduced to 0.8%.

[0141] The multi-dimensional life loss coefficient matrix constructed and optimized by the deep reinforcement learning method can accurately predict the life loss of the battery under different working conditions, providing effective support for battery life prediction and health management.

[0142] In an optional implementation, a plurality of groups of candidate charging and discharging scheduling strategies are generated based on the deep reinforcement learning model, the predicted revenue and the life loss are taken as a dual constraint of a reward function, and the alternating training of a policy network and a value network is implemented to realize the generation of a charging and discharging strategy maximizing the revenue considering the life of a device, including:

[0143] A strategy generation network is constructed based on the deep reinforcement learning model, the strategy generation network includes a parameter generation module and a value evaluation module, the grid load state, the electricity price information, the device state of charge, and the environmental temperature of the energy storage device are taken as system states, corresponding charging and discharging action strategy parameters are generated, and the system states and the charging and discharging action strategy parameters form a state-action pair;

[0144] A dual constraint reward function is constructed based on the state-action pair, the system state is input into the parameter generation module to obtain mean parameter and standard deviation parameter of action distribution, the system state and the dual constraint reward function are input into the value evaluation module to obtain state value;

[0145] An advantage value is calculated based on the state value and the dual constraint reward function, the advantage value is used to guide the parameter update of the strategy generation network, the parameter update of the strategy generation network is guided based on the time difference error of the state value, and an optimized strategy generation network is obtained through alternating training;

[0146] The prediction result of the optimized strategy generation network is substituted into the dual constraint reward function to obtain a strategy evaluation index, the optimal charging and discharging strategy is selected based on the strategy evaluation index under the condition of meeting the life loss constraint, and the optimal charging and discharging control strategy of the energy storage device is obtained.

[0147] The policy generation network represents the policy with a parameterized Gaussian distribution, which contains two core components: parameter generation module and value evaluation module. The parameter generation module is composed of a three-layer fully connected neural network. The number of neurons in the input layer is equal to the dimension of the state (including grid load state, electricity price information, device state of charge, and environmental temperature, etc.), the number of neurons in the hidden layer is 256, and the number of neurons in the output layer is twice the dimension of the action, representing the mean and standard deviation parameters of the action distribution respectively. The input state includes 48-dimensional electricity price prediction data (24 hours, 2 data points per hour), 24-dimensional load prediction data, 1-dimensional current SOC value, and 1-dimensional environmental temperature, totaling 74 dimensions. The output is 48-dimensional action parameters, corresponding to the charge and discharge power decision for each hour in 24 hours.

[0148] The value evaluation module also uses a three-layer fully connected neural network, with the same input layer as the parameter generation module, 256 neurons in the hidden layer, and one neuron in the output layer representing the value estimate of the current state. The ReLU activation function is used to improve the network's expression ability, and the last layer does not use the activation function to ensure that the output range is not limited. The network parameters are updated by the stochastic gradient descent method, with a learning rate of 0.0003 and a batch size of 64.

[0149] The grid load state, electricity price information, device state of charge, and environmental temperature are combined into a state vector, which is input into the parameter generation module to obtain the action distribution parameters. The state of a certain 100 kWh energy storage device includes information such as the current time period identifier, electricity price of 0.8 yuan / kWh, SOC of 45%, and environmental temperature of 28°C. The parameter generation module outputs the mean parameter of the action distribution as 35 kW (positive value indicating charging, negative value indicating discharging), and the standard deviation as 5 kW. Based on these parameters, specific charge and discharge power decisions are generated through random sampling, such as 37 kW charging power, which forms a state-action pair.

[0150] The double-constraint reward function includes a revenue item and a life loss item. The revenue item is calculated as the economic benefit generated during the charging and discharging process, and the life loss item is calculated as the device capacity degradation caused by the charging and discharging process. The revenue item is the discharge revenue minus the charging cost, and the discharge revenue is equal to the discharge power multiplied by the discharge time electricity price, and the charging cost is equal to the charging power multiplied by the charging time electricity price. The life loss item is based on the cycle depth, temperature, and current rate to build a loss model. The greater the cycle depth, the higher or lower the temperature, or the greater the current rate, the more serious the life loss.

[0151] A certain energy storage device charges 30 kWh at a price of 0.3 yuan / kWh and discharges 27 kWh at a price of 0.9 yuan / kWh (considering 90% charging and discharging efficiency), with a profit of 27x0.9-30x0.3=24.3-9=15.3 yuan. This charging and discharging behavior results in a capacity loss of 0.015%, which, based on the initial cost of the device of 150,000 yuan, is equivalent to a loss of about 22.5 yuan. After setting the weight coefficient, the final reward value is the difference between the profit and the weighted loss.

[0152] The weight ratio of the profit term and the life loss term in the double-constraint reward function is set as an adjustable parameter, and the initial value is usually set to 7:3. In the early life of the device, the profit weight can be increased to 8:2, and near the end of life, the life loss weight can be increased to 4:6, achieving dynamic balance. The weight adjustment can be dynamically set according to the operation target, such as temporarily increasing the profit weight to 9:1 when participating in demand response, and temporarily increasing the life weight to 3:7 after device maintenance.

[0153] Input the current state into the value estimation module to obtain the value estimate of the state; at the same time, obtain the reward value and the next state according to the executed action; input the next state into the value estimation module to obtain the value estimate of the next state; calculate the time difference error based on the current reward value, the current state value and the next state value. The current state value estimate is 50, a certain charging action is executed to obtain a reward of 15, the next state value estimate is 60, and the discount factor is 0.95, then the time difference error is 15+0.95x60-50=22.

[0154] The advantage value is calculated by accumulating the discounted rewards of the next n steps and subtracting the current state value estimate. Select n=5, i.e. accumulate the discounted rewards of the next 5 steps. The current reward is 15, the rewards of the next 4 steps are 12, 10, 8, and 6 respectively, and the discount factor is 0.95, then the accumulated discounted reward is 15+0.95x12+0.95 2 x10+0.95 3 x8+0.95 4 x6=47.73, and if the current state value estimate is 40, then the advantage value is 47.73-40=7.73.

[0155] Based on the calculated advantage value, a policy gradient target function is constructed, and the stochastic gradient ascent method is used to update the parameters of the parameter generation module. Each training batch contains 64 state-action pair samples, the product of the log probability and the advantage value of each sample is calculated, and the average of the 64 products is taken as the gradient update direction. The learning rate is set to 0.0003, and the Adam optimizer is used for parameter update.

[0156] Based on the calculated timing difference error, the mean square error loss function is constructed, and the random gradient descent method is used to update the parameters of the value evaluation module. The square of the timing difference error of each sample is calculated, and the average of all samples in the batch is taken as the loss function value. The learning rate is also set to 0.0003, and the Adam optimizer is used to update the parameters.

[0157] Each training cycle first updates the policy network parameters, then updates the value network parameters, and repeats the process until convergence. The maximum training period is set to 10000, and the policy performance is evaluated every 500 cycles. When the performance improvement is less than 1% for 5 consecutive evaluations, it is determined to be converged. The training data uses real energy storage plant historical operation data, including charging and discharging records under different seasons and load conditions within a year, totaling about 90,000 data, of which 80% is used for training and 20% is used for verification.

[0158] The optimized strategy generation network is applied to the predicted state of the next 24 hours to obtain the charging and discharging power decision within 24 hours; based on this decision, the revenue and life loss are simulated and calculated; the revenue and life loss are substituted into the double constraint reward function to obtain the policy evaluation index. A certain strategy predicts that the total revenue within 24 hours is 180 yuan, and the life loss equivalent cost is 60 yuan, with a weight ratio of 7:3, then the evaluation index is 0.7x180-0.3x60=126-18=108.

[0159] The life loss constraint condition determines the need to set a daily maximum allowed life loss threshold, usually 1 / 3650 of the total life of the device (assuming the device is used for 10 years), and compares the predicted life loss with the threshold. The designed life of the device is 6000 cycles, and the maximum allowed loss per day is 1.65 equivalent cycles. If a certain strategy predicts a life loss of 1.8 equivalent cycles, it does not meet the constraint and will be excluded.

[0160] For all candidate strategies that meet the life constraint condition, sort them according to the evaluation index from high to low, and select the strategy with the highest index as the final execution strategy. Usually 10-20 candidate strategies are generated, and after screening, 5-10 strategies that meet the constraints are retained, from which the strategy with the highest evaluation index is selected for execution. The execution strategy includes charging and discharging power instructions for each hour within 24 hours, such as charging power 30kW from 0-6am, discharging power 45kW from 10-14pm, discharging power 40kW from 18-22pm, etc.

[0161] The actual application results of this method in a certain 10MWh / 5MW commercial energy storage power station show that compared with the traditional single target optimization method, the economic benefit is increased by 18.7%, and the expected life of the device is extended by 23.5%, with significant effect. Especially in the summer with large price fluctuations and drastic temperature changes, the method has more obvious advantages, with single-day revenue increase of more than 25%, while effectively controlling the life loss of the device.

[0162] Figure 3 Figure 4 is a graph showing the trends of four key indicators in the training process of the deep reinforcement learning model of the embodiment of the present application, which shows the trends of four key indicators in the training process: the reward of the economic benefit of the energy storage device (circle marker), the penalty of the life loss of the charging and discharging strategy (square marker), the total reward value considering the economic benefit and life loss (triangle marker), and the model loss value representing the training state of the model (dashed line). The entire training process is divided into three stages: in the exploration stage (0-1000 times), the reward of the economic benefit increases rapidly from 0 to about 150, the penalty of the life loss gradually improves from -75 to -40, the total reward value increases from -50 to 100, and the model loss continuously decreases from 0.2 to 0.8; in the convergence stage (1000-2500 times), the growth rate of each indicator slows down, the reward of the economic benefit reaches about 230, the penalty of the life loss approaches 0, the total reward value rises to 200, and the model loss continues to decrease to 1.2; in the stable stage (2500-5000 times), all indicators tend to be stable, the final reward of the economic benefit stabilizes at 270, the penalty of the life loss is 20, the total reward value maintains at 250, and the model loss decreases to 1.8 and remains stable. The training process fully demonstrates the convergence process of the deep reinforcement learning model in optimizing the charging and discharging strategy of the energy storage device, which continuously improves the performance by balancing the economic benefit and the life loss.

[0163] In an optional embodiment, the prediction result of the optimized strategy generation network is substituted into the double-constraint reward function to obtain a strategy evaluation index, and the optimal charging and discharging strategy is selected based on the strategy evaluation index under the condition of meeting the life loss constraint, which includes:

[0164] The prediction result of the optimized strategy generation network is substituted into the double-constraint reward function, which includes the reward index and the life loss index of the energy storage device, and the prediction result is evaluated based on the double-constraint reward function to obtain a strategy evaluation index corresponding to the prediction result;

[0165] The cumulative life loss value of the energy storage device is calculated based on the life loss index in the prediction result, and the cumulative life loss value is compared with a preset loss threshold to obtain a life loss constraint condition of the prediction result;

[0166] The prediction results meeting the life loss constraint condition are sorted in descending order according to the size of the strategy evaluation index, and the result of the descending sorting is taken as the candidate charging and discharging strategy of the energy storage device; the charging and discharging strategy with the largest strategy evaluation index in the candidate charging and discharging strategy is taken as the optimal charging and discharging control strategy of the energy storage device.

[0167] The optimized strategy generation network predicts the results, which are substituted into the double-constraint reward function to calculate the strategy evaluation index. The double-constraint reward function includes two core evaluation dimensions: the income index and the life loss index. The income index calculation method is as follows: within a complete scheduling cycle (usually 24 hours), the income of the charging and discharging behavior in each time period (such as 15 minutes as a period) is calculated. For charging behavior, the calculation method is the charging capacity multiplied by the current period price and then multiplied by negative one; for discharging behavior, the calculation method is the discharging capacity multiplied by the current period price. The total income index is obtained by adding the income of all periods and then subtracting the fixed operation and maintenance cost.

[0168] Assuming that a 100 kWh / 50 kW energy storage system charges 40 kWh at 2 a.m. when the price is 0.3 yuan / kWh and discharges 36 kWh at 2 p.m. when the price is 0.9 yuan / kWh (considering 90% charging and discharging efficiency), the income of this operation is: 36 kWh x 0.9 yuan / kWh - 40 kWh x 0.3 yuan / kWh = 32.4 yuan - 12 yuan = 20.4 yuan. If the daily fixed operation and maintenance cost is 5 yuan, the net income is 15.4 yuan.

[0169] The current state of charge (SOC) of the battery is determined, the charging and discharging depth is calculated, the life loss coefficient matrix is queried, and the life loss value of a single operation is calculated. The charging and discharging depth is the absolute value of the change in SOC before and after charging and discharging, such as charging from 30% to 90%, with a depth of 60%. According to the pre-established life loss coefficient matrix, the loss coefficient corresponding to the depth is queried. This coefficient matrix is established based on experimental data and records the capacity loss rate corresponding to different depth cycles. For example, the loss coefficient is 0.0025% at a depth of 20%, 0.012% at a depth of 60%, and 0.025% at a depth of 100%.

[0170] The temperature factor also needs to be considered in the life loss index, and the effect of temperature on loss is realized through a temperature correction coefficient. The coefficient is 1.0 at 25°C, and the coefficient increases by 0.2 for every 10°C increase in temperature; the coefficient increases by 0.15 for every 10°C decrease in temperature. For example, at an operating temperature of 35°C, the temperature correction coefficient is 1.2; at 15°C, the coefficient is 1.15. The final life loss value is the product of the basic loss coefficient, the temperature correction coefficient, and the cycle number.

[0171] Set the revenue weight coefficient a and the life loss weight coefficient b, usually a+b=1, for example a=0.7, b=0.3. Convert the revenue index into a standardized revenue score by dividing the current revenue by the historical maximum revenue and multiplying by 100, the score range is 0-100. Similarly, convert the life loss index into a standardized life score by subtracting the ratio of the current loss value to the maximum acceptable loss value from 1 and multiplying by 100, the score range is also 0-100. The final reward function value is: a x standardized revenue score - b x (100-standardized life score).

[0172] A certain strategy predicts a revenue of 200 yuan, and the historical maximum revenue is 250 yuan, so the standardized revenue score is (200 / 250) x 100=80 points. The strategy predicts a life loss of 0.015%, and the maximum acceptable loss is 0.025%, so the standardized life score is [1-(0.015 / 0.025)]x100=40 points. With a weight configuration of a=0.7, b=0.3, the final reward function value is 0.7x80-0.3x(100-40)=56-18=38 points.

[0173] Get the historical cumulative loss value of the energy storage device, calculate the newly predicted loss value of the current strategy, and add the two values to get the updated cumulative loss value. The historical cumulative loss value is saved in the device management database and updated every time the strategy is executed. For example, a certain energy storage device has been running for 2 years, and the current cumulative loss value is 15%, if the new strategy predicts a loss of 0.02%, the updated cumulative loss value is 15.02%.

[0174] Determine the total loss upper limit based on the design life of the device, determine the phase loss limit value in combination with the operation plan, and set a dynamic adjustment mechanism considering seasonal and grid demand. Specific case: a lithium battery energy storage system has a design capacity attenuation upper limit of 20%, and an expected service life of 10 years, so the annual average loss threshold is 2%, the monthly average loss threshold is about 0.167%, and the daily average loss threshold is about 0.0055%. Considering the seasonal fluctuations in grid demand, the daily threshold can be adjusted to 0.007% during the summer peak period and to 0.004% during the winter trough period.

[0175] Calculate the predicted cumulative loss value after the current strategy is executed, compare it with the preset threshold value in the corresponding period, and generate a constraint judgment result. For example, if the predicted cumulative loss value after the execution of a certain strategy on a certain day is 15.02%, and the corresponding threshold value is 730x0.0055%=4.015% on the 730th day, and the initial capacity is 100%, then the current actual capacity should be above 96%, and the actual capacity is 100%-15.02%=84.98%, which is significantly lower than the threshold requirement, so the strategy does not meet the constraint condition.

[0176] After comparing the accumulated life loss value with the preset threshold value, if the accumulated life loss value does not exceed the preset threshold value, the strategy is retained as a valid candidate strategy; if the accumulated life loss value exceeds the preset threshold value, the strategy is removed from the candidate set. In practical applications, a three-level threshold system can be set: a warning threshold, a limit threshold, and a prohibition threshold. When the accumulated loss reaches the warning threshold (e.g., 70% of the design loss), a warning is issued; when the accumulated loss reaches the limit threshold (e.g., 85% of the design loss), only low-loss strategies are allowed to be executed; when the accumulated loss reaches the prohibition threshold (e.g., 95% of the design loss), only protective strategies are allowed to be executed.

[0177] The strategies that meet the life constraint condition are sorted in descending order of the evaluation index, a candidate strategy set is constructed, the comprehensive evaluation index value of each strategy is calculated, the strategies are sorted from high to low according to the index value, and a sorted result list is generated. Specific case: assuming that there are 5 candidate strategies after screening, the comprehensive evaluation indexes of the 5 candidate strategies are 78 points, 65 points, 82 points, 59 points, and 71 points respectively, the sorted results are 82 points, 78 points, 71 points, 65 points, and 59 points, and the corresponding strategy numbers are 3, 1, 5, 2, and 4.

[0178] The strategy with the highest evaluation index is selected from the sorted candidate strategy list as the final execution strategy. For example, in the above case, the evaluation index of strategy 3 is 82 points, which ranks first, and therefore strategy 3 is selected as the optimal charging and discharging control strategy. The strategy includes charging and discharging power instructions for each time period within 24 hours, for example, charging power of 35 kW from 1 am to 3 am, discharging power of 20 kW from 10 am to 12 pm, discharging power of 40 kW from 2 pm to 4 pm, and the like.

[0179] The selected optimal strategy is converted into a standard control instruction format, sent to the energy management system of the energy storage power station through a communication interface, and the energy management system executes the instructions to control the charging and discharging behavior of each energy storage unit. For example, the selected strategy is converted into a JSON format control instruction, which includes information such as timestamp, power value, and operation type, and is sent to the energy management system through industrial Ethernet. The energy management system parses the instructions and controls the energy storage converter through the field bus to execute the corresponding charging and discharging operations.

[0180] A real-time monitoring mechanism is established to collect strategy execution effect data, analyze the deviation between the strategy and the actual income, and dynamically adjust the weight parameters of the reward function. The real-time monitoring mechanism collects data such as battery SOC change, actual charging and discharging power, real-time electricity price, and environmental temperature through the energy storage management system. The income and loss effect data are summarized once a day, the deviation rate of actual income and predicted income, and the deviation rate of actual life loss and predicted loss are calculated. When the deviation rate exceeds the preset threshold (e.g., ±10%), the model automatic optimization mechanism is triggered, and the deep reinforcement learning model is retrained.

[0181] The dynamic weight adjustment mechanism automatically adjusts the weight coefficients in the double-constraint reward function based on long-term operation data analysis. For example, when the device is in the initial operation period (such as the first 3 years), the profit index is given priority, and a can be set to 0.8 and β to 0.2; when the device enters the middle period (such as 3-7 years), a is adjusted to 0.6 and β to 0.4; when the device enters the later period (such as more than 7 years), a is adjusted to 0.4 and β to 0.6, and more attention is paid to the protection of the service life of the device.

[0182] Define the type of emergency, develop an emergency response strategy, and set up a safety recovery program. Emergencies include power grid failures, abnormal battery temperatures, and battery SOC overruns. For example, when the temperature exceeds 45℃, immediately reduce the charging and discharging power to 50% of the rated value; when the temperature exceeds 55℃, immediately stop charging and discharging and start forced cooling; when the temperature exceeds 65℃, immediately disconnect the battery pack and start the fire extinguishing system.

[0183] The above steps form a complete optimal charging and discharging strategy selection process under double-constraint conditions, ensuring that the commercial energy storage power station maximizes economic benefits while protecting the service life of the device. This method has been verified in many actual energy storage projects, and compared with the traditional single-target optimization method, it has improved the economic benefits by an average of 15% and extended the service life of the device by about 22%.

[0184] In a second aspect of the embodiments of the present application, a commercial energy storage power station revenue mode intelligent optimization system is provided, comprising:

[0185] A first unit is configured to obtain historical operation data of the commercial energy storage power station, wherein the historical operation data includes charging and discharging power data, battery state of charge data, real-time power grid price data, and energy storage device health status data;

[0186] A second unit is configured to construct a deep reinforcement learning model of the commercial energy storage power station based on the historical operation data, wherein the deep reinforcement learning model adopts a deep neural network structure, and the deep neural network structure is used to perform nonlinear mapping on the historical operation data to generate a predicted revenue value;

[0187] A third unit is configured to analyze the energy storage device health status data through the deep reinforcement learning model, establish a multi-dimensional service life loss coefficient matrix based on the cycle number, deep cycle rate, and working temperature, and realize the evaluation of the service life loss of the device;

[0188] A fourth unit is configured to generate multiple groups of candidate charging and discharging scheduling strategies based on the deep reinforcement learning model, take the predicted revenue and service life loss as the double constraints of the reward function, and realize the generation of the charging and discharging strategy for maximizing the revenue considering the service life of the device through the alternative training of the policy network and the value network.

[0189] A fifth unit is configured to generate a running control instruction according to the optimized charging and discharging scheduling strategy, and send the running control instruction to an energy management system of the commercial energy storage power station to control the energy storage power station to operate according to the optimized charging and discharging scheduling strategy.

[0190] In a third aspect, the present application provides an electronic device, comprising:

[0191] a processor;

[0192] a memory for storing processor-executable instructions;

[0193] The processor is configured to invoke the instructions stored in the memory to execute the method described above.

[0194] In a fourth aspect, the present application provides a computer-readable storage medium having stored thereon computer program instructions, which, when executed by a processor, implement the method described above.

[0195] The present application can be a method, apparatus, system and / or computer program product. The computer program product can include a computer-readable storage medium having stored thereon computer-readable program instructions that, when executed by a computer, cause the computer to carry out various aspects of the present application.

[0196] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for intelligent optimization of commercial energy storage plant revenue model, characterized in that, The method comprises the following steps: acquiring historical operation data of the commercial energy storage power station, the historical operation data comprising charge-discharge power data, battery state of charge data, real-time power grid price data, and energy storage device health state data; constructing a deep reinforcement learning model of the commercial energy storage power station based on the historical operation data, the deep reinforcement learning model adopting a deep neural network structure, and generating a predicted revenue value by nonlinearly mapping the historical operation data through the deep neural network structure, comprising: the deep reinforcement learning model adopts a multi-layer feedforward neural network structure, the multi-layer feedforward neural network structure comprising an input layer, a hidden layer, and an output layer, wherein the hidden layer performs feature extraction and nonlinear mapping on the historical operation data through a weight matrix, a bias vector, and a nonlinear activation function; constructing network parameters of the multi-layer feedforward neural network structure as random variables, establishing a Gaussian distribution model of the network parameters by using a variational inference method, and quantitatively representing prediction uncertainty of the multi-layer feedforward neural network structure by mean parameters and variance parameters of the Gaussian distribution model; adding perturbations to the historical operation data based on the Gaussian distribution model to generate adversarial samples, and using the adversarial samples in a training process of the multi-layer feedforward neural network structure to improve robustness of the multi-layer feedforward neural network structure; calculating a prediction mean and a prediction standard deviation based on a prediction result of the multi-layer feedforward neural network structure, determining a confidence interval of the prediction result according to the prediction mean and the prediction standard deviation, calculating a dynamic adjustment coefficient according to the prediction standard deviation, and adaptively optimizing the prediction result of the multi-layer feedforward neural network structure based on the dynamic adjustment coefficient to obtain an optimized prediction result; inputting the optimized prediction result and the confidence interval into a revenue calculation module to generate a predicted revenue value of the commercial energy storage power station by the revenue calculation module; analyzing the energy storage device health state data by the deep reinforcement learning model, establishing a multi-dimensional life loss coefficient matrix based on cycle times, deep cycle rates, and operating temperatures, and realizing evaluation of device life loss; generating a plurality of groups of candidate charge-discharge scheduling strategies based on the deep reinforcement learning model, taking predicted revenue and life loss as dual constraints of a reward function, and realizing generation of a charge-discharge scheduling strategy maximizing revenue by considering device life through alternating training of a policy network and a value network; generating an operation control instruction according to the optimized charge-discharge scheduling strategy, and sending the operation control instruction to an energy management system of the commercial energy storage power station to control the energy storage power station to operate according to the optimized charge-discharge scheduling strategy.

2. The method of claim 1, wherein, The method comprises the following steps: acquiring training data of the multi-layer feedforward neural network structure, constructing a likelihood function based on the training data, selecting a Gaussian distribution as a prior distribution of the network parameters, and combining the likelihood function and the prior distribution through a Bayesian formula to construct a posterior distribution; The variational inference method is used to approximate the posterior distribution, a variational distribution family in Gaussian distribution form is constructed, the variational distribution family includes a mean parameter vector and a variance parameter vector, and optimal variational distribution parameters are obtained by minimizing the kullback-leibler divergence between the variational distribution family and the posterior distribution through the variational inference method; The optimal variational distribution parameters are input into the multi-layer feedforward neural network structure, for each layer in the multi-layer feedforward neural network structure, a parameter distribution of the layer is constructed based on the mean parameter vector and the variance parameter vector through the variational inference method, a network parameter sample is obtained based on the parameter distribution, and the network parameter sample is input into the multi-layer feedforward neural network structure to obtain a corresponding network output; A plurality of groups of network parameter samples are repeatedly obtained, the plurality of groups of network parameter samples are input into the multi-layer feedforward neural network structure to obtain a plurality of groups of network outputs, a predicted mean and a predicted variance are calculated based on the plurality of groups of network outputs, and the predicted mean and the predicted variance are taken as prediction uncertainty quantization results of the multi-layer feedforward neural network structure.

3. The method of claim 1, wherein, The health state data of the energy storage device is analyzed through the deep reinforcement learning model, a multi-dimensional life loss coefficient matrix is established based on the cycle number, the deep cycle rate and the working temperature, and the evaluation of the life loss of the device is realized, including: Health state data of the energy storage device is obtained, and a state feature vector is constructed based on the health state data, the state feature vector being taken as input data of a deep reinforcement learning method; Based on the state feature vector, the cycle number, the deep cycle rate and the working temperature parameters of the energy storage device are extracted through the deep reinforcement learning method, a time decay factor is introduced based on the deep reinforcement learning method to adaptively weight the operation data at different times in the state feature vector, and weighted historical loss data is obtained; According to the weighted historical loss data, the dynamic change characteristics of the cycle number, the deep cycle rate and the working temperature parameters are analyzed based on the deep reinforcement learning method, the dynamic change characteristics are input into a loss reference model, and reference loss characteristics of the energy storage device are obtained; Based on the reference loss characteristics, a multi-dimensional life loss coefficient matrix is constructed through the deep reinforcement learning method, the multi-dimensional life loss coefficient matrix is optimized by minimizing the deviation between the predicted loss value and the actual loss value, and an optimized loss coefficient matrix is obtained; The optimized loss coefficient matrix is integrated with respect to time to obtain a cumulative loss state of the energy storage device, and the remaining life of the energy storage device is determined based on the cumulative loss state and the designed life of the energy storage device.

4. The method of claim 3, wherein, The deep reinforcement learning method is used to construct a multi-dimensional life loss coefficient matrix, the multi-dimensional life loss coefficient matrix is optimized by minimizing the deviation between the predicted loss value and the actual loss value, and an optimized loss coefficient matrix is obtained, including: The multi-dimensional life loss coefficient matrix comprises a cycle number dimension, a depth cycle rate dimension and a temperature dimension; deviation between a predicted loss value and an actual loss value output by the multi-dimensional life loss coefficient matrix is minimized; The multi-dimensional life loss coefficient matrix is optimized based on a result of the minimization to obtain an optimized loss coefficient matrix.

5. The method of claim 1, wherein, Based on the deep reinforcement learning model, a plurality of groups of candidate charging and discharging scheduling strategies are generated, a predicted revenue and life loss are taken as a double constraint of a reward function, and through alternating training of a policy network and a value network, a charging and discharging strategy generation considering device life is realized, including: A policy generation network is constructed based on the deep reinforcement learning model, the policy generation network comprises a parameter generation module and a value evaluation module, a power grid load state, a price information, a device state of charge and an environmental temperature of the energy storage device are taken as a system state, corresponding charging and discharging action strategy parameters are generated, the system state and the charging and discharging action strategy parameters form a state action pair; Based on the state action pair, a double constraint reward function is constructed, the system state is input into the parameter generation module to obtain mean value parameters and standard deviation parameters of an action distribution, the system state and the double constraint reward function are input into the value evaluation module to obtain a state value; Based on the state value and the double constraint reward function, an advantage value is calculated, the advantage value is used to guide parameter updating of the policy generation network, a time sequence difference error of the state value is used to guide parameter updating of the policy generation network, and an optimized policy generation network is obtained through alternating training; The prediction result of the optimized policy generation network is substituted into the double constraint reward function to obtain a policy evaluation index, an optimal charging and discharging strategy is selected based on the policy evaluation index under the condition of meeting the life loss constraint, and an optimal charging and discharging control strategy of the energy storage device is obtained.

6. The method of claim 5, wherein, The prediction result of the optimized policy generation network is substituted into the double constraint reward function to obtain a policy evaluation index, an optimal charging and discharging strategy is selected based on the policy evaluation index under the condition of meeting the life loss constraint, and an optimal charging and discharging control strategy of the energy storage device is obtained. The prediction result of the optimized policy generation network is substituted into the double constraint reward function to obtain a policy evaluation index, an optimal charging and discharging strategy is selected based on the policy evaluation index under the condition of meeting the life loss constraint, and an optimal charging and discharging control strategy of the energy storage device is obtained. The prediction result of the optimized policy generation network is substituted into the double constraint reward function to obtain a policy evaluation index, an optimal charging and discharging strategy is selected based on the policy evaluation index under the condition of meeting the life loss constraint, and an optimal charging and discharging control strategy of the energy storage device is obtained. including:

7. A commercial energy storage plant revenue model intelligent optimization system for implementing the method of any of the preceding claims 1-6, characterized in that, ​ The first unit is configured to acquire historical operation data of the commercial energy storage power station, wherein the historical operation data comprises charge-discharge power data, battery state of charge data, real-time power grid price data, and energy storage device health status data; The second unit is configured to construct a deep reinforcement learning model of the commercial energy storage power station based on the historical operation data, wherein the deep reinforcement learning model adopts a deep neural network structure, and a predicted revenue value is generated by performing nonlinear mapping on the historical operation data through the deep neural network structure; The third unit is configured to analyze the energy storage device health status data through the deep reinforcement learning model, establish a multi-dimensional life loss coefficient matrix based on cycle times, deep cycle rates, and operating temperatures, and realize evaluation of device life loss; The fourth unit is configured to generate a plurality of groups of candidate charge-discharge scheduling strategies based on the deep reinforcement learning model, take the predicted revenue and life loss as a dual constraint of a reward function, and realize generation of a charge-discharge strategy maximizing revenue by considering device life through alternating training of a policy network and a value network; The fifth unit is configured to generate an operation control instruction according to the optimized charge-discharge scheduling strategy, send the operation control instruction to an energy management system of the commercial energy storage power station, and control the energy storage power station to operate according to the optimized charge-discharge scheduling strategy.

8. An electronic device, comprising: comprise: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to invoke the instructions stored in the memory to execute the method of any one of claims 1 to 6.

9. A computer-readable storage medium having stored thereon computer program instructions, wherein, The computer program instructions, when executed by the processor, realize the method of any one of claims 1 to 6.

Images