Micro-grid energy optimization method and device based on distributed deep reinforcement learning

Abstract

The application provides a micro-grid energy optimization method and device based on distributed deep reinforcement learning, relates to the technical field of micro-grid energy optimization and intelligent scheduling, and is based on historical power consumption, meteorological data and corresponding power supply to construct a feature sequence in time sequence; then, two types of sequences are used to train an LSTM model to obtain a power consumption and power supply prediction model, and a prediction error is obtained through a sliding window; then, future next period meteorological data is obtained, a power supply prediction value is obtained in combination with a historical sequence, and based on historical meteorological data and error distribution, the power supply prediction value is calculated to belong to each interval probability as the first type of evidence; the similarity of future meteorological data and working conditions is used as a weight to distribute the second type of evidence, an abnormal fan is identified by using a dynamic time warping difference method, the power supply prediction value is corrected, the third type of evidence is generated, and finally, D-S fusion is performed to output the power supply prediction value of the next period in the future and make a power supply mode decision.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of micro-grid energy optimization and intelligent scheduling, in particular to a micro-grid energy optimization method and device based on distributed deep reinforcement learning. BACKGROUND

[0002] In the micro-grid environment with wind turbines as the core distributed power supply, due to the strong volatility and intermittency of wind energy, the power output is significantly uncertain due to environmental factors. How to accurately coordinate the energy scheduling of wind turbine clusters, energy storage units and the main grid, and balance the power supply reliability has become the core problem of energy management in such micro-grids.

[0003] In the prior art, the publication number CN114285075A discloses a micro-grid energy online optimization method based on distributed deep reinforcement learning. The deep reinforcement learning algorithm adopted by the method fits the strategy and value function through a neural network, and after training, it can quickly generate an optimal solution for time-varying loads and environmental changes, effectively alleviating the challenges brought by load time variability. On this basis, the distributed deep reinforcement learning method further decomposes the large-scale micro-grid system into multiple sub-modules through a multi-agent framework, and each agent is responsible for the management of local power generation units, energy storage or loads, reducing the learning complexity of a single neural network and improving the applicability of the model in large-scale systems. For example, the Actor network interacts with the local environment to generate action strategies, and the Critic network integrates global information to optimize value estimation, achieving collaborative management of distributed energy production and utilization.

[0004] However, the existing optimization method based on distributed deep reinforcement learning still has certain limitations: its core focuses on the generation and execution of optimization strategies, and does not adequately consider the prediction uncertainty of the wind turbine power supply side. In actual operation, random fluctuations in wind speed and air pressure result in large prediction errors in wind turbine power generation, coupled with power output anomalies caused by wind turbine equipment failures and blade aging, which directly affect the reliability of the optimization strategy. The existing method does not adequately quantify the prediction error characteristics under different wind conditions, and lacks real-time detection and dynamic correction mechanisms for abnormal states of wind turbines, making it difficult to ensure the robustness of energy scheduling in scenarios with strong wind energy volatility and high prediction uncertainty.

[0005] The above information disclosed in the background section is only used to enhance the understanding of the background of the present disclosure, and therefore it can include information that does not constitute prior art known to those of ordinary skill in the art. SUMMARY

[0006] The present application aims to provide a micro-grid energy optimization method and device based on distributed deep reinforcement learning to solve the problems raised in the background.

[0007] To achieve the above object, the present application provides the following technical solutions:

[0008] The micro-grid energy optimization method based on distributed deep reinforcement learning comprises the following specific steps:

[0009] The historical power consumption, historical meteorological data and corresponding power supply of the micro-grid in the past Z consecutive time periods are acquired and normalized, and the historical power consumption feature sequence and the historical meteorological-power supply feature sequence are constructed in time sequence;

[0010] The historical power consumption feature sequence is used to train the LSTM model to obtain a power consumption prediction model, and the historical meteorological-power supply feature sequence is used to train the LSTM model to obtain a power supply prediction model, and the prediction error is obtained by means of a sliding window;

[0011] The meteorological data of the next time period in the future are acquired, and the historical meteorological data and power supply of the previous M1 time periods are input into the power supply prediction model to obtain the power supply prediction value of the next time period, the historical meteorological data are clustered to divide the working conditions, the total power supply intervals are generated, the prediction error is weighted by combining the meteorological data of the period and the similarity of each working condition, the future error distribution is obtained, and the probability that the power supply prediction value belongs to each total power supply interval is calculated as the first type of evidence;

[0012] The similarity of the meteorological data of the next time period in the future and each working condition is taken as the weight, the second type of evidence is allocated to the total power supply interval corresponding to each working condition, the dynamic time warping difference method is adopted to analyze the power supply of each wind turbine in the previous TA time periods of the current time, the abnormal wind turbine is identified, the power supply of the abnormal wind turbine is set to zero, the power supply prediction value of the next time period in the future is corrected, and the power supply prediction value is mapped to the total power supply interval to generate the third type of evidence;

[0013] The three types of evidence are fused by D-S, the third type of evidence is dominant in the evidence conflict, and the power supply prediction value of the next time period in the future is output, which is compared with the power consumption prediction value obtained by inputting the historical power consumption of the previous M2 consecutive time periods of the current time into the power consumption prediction model to make a power supply mode decision.

[0014] The historical power consumption, historical meteorological data and corresponding power supply of the micro-grid in the past Z consecutive time periods are acquired and normalized, and the historical power consumption feature sequence and the historical meteorological-power supply feature sequence are constructed in time sequence;

[0015] The historical power consumption feature sequence is used to train the LSTM model to obtain a power consumption prediction model, and the historical meteorological-power supply feature sequence is used to train the LSTM model to obtain a power supply prediction model, and the prediction error is obtained by means of a sliding window;

[0016] The meteorological data of the next period in the future is acquired, and historical meteorological data of M1 previous periods and power supply amount are input into a power supply amount prediction model to obtain a power supply amount prediction value of the next period, a working condition is divided based on historical meteorological data clustering, a total power supply amount interval is generated, a prediction error is weighted based on meteorological data of the period and a similarity of each working condition, a future error distribution is obtained, and a probability that the power supply amount prediction value belongs to each total power supply amount interval is calculated as a first type of evidence;

[0017] The similarity of the meteorological data of the next period in the future and each working condition is taken as a weight, a second type of evidence is distributed to the total power supply amount interval corresponding to each working condition, a dynamic time warping difference method is used to analyze the power supply amount of each fan in the previous TA periods at the current time, an abnormal fan is identified, and the power supply amount of the abnormal fan is set to zero to correct the power supply amount prediction value of the next period in the future, and the power supply amount prediction value is mapped to the total power supply amount interval to generate a third type of evidence;

[0018] The three types of evidence are fused by D-S fusion, the third type of evidence is dominant in evidence conflict, and a power supply amount prediction value of the next period in the future is output, which is compared with a power consumption prediction value obtained by inputting historical power consumption amount of M2 continuous periods at the current time into a power consumption prediction model to make a power supply mode decision.

[0019] Further, a method for training an LSTM model using a historical meteorological-power supply amount feature sequence to obtain a power supply amount prediction model is as follows:

[0020] First, Z consecutive periods in the past are marked as a historical total period number Z, power supply amount data of M2 continuous periods is extracted from the historical meteorological-power supply amount feature sequence, and is marked as a historical data subset, a training window length is set to M1 periods, and 0

[0021] Further, a method for obtaining a prediction error by a sliding window is as follows:

[0022] The data from independent time periods that were not used in the power supply prediction model training within the past Z consecutive time periods are selected as the error calculation benchmark. A sliding window of length M1, consistent with that used during training, is set. The feature sequence containing M1 consecutive time periods is extracted and input into the power supply prediction model to obtain the predicted value for the next time period. The absolute error is calculated by comparing the predicted value with the actual value of the power supply for that time period. Then, the window is slid forward by one time period along the time axis, and the above prediction and error calculation process is repeated. All the calculated errors are integrated into an error set in chronological order, which is denoted as the prediction error.

[0023] Furthermore, the method for generating mutually exclusive total power supply intervals by clustering and classifying operating conditions based on historical meteorological data is as follows:

[0024] Wind speed and air pressure values ​​for each time period are extracted from a subset of historical data to form a two-dimensional feature vector. Then, the K-means clustering algorithm is used to perform cluster analysis on the standardized feature vector to classify operating conditions. The elbow method is used to determine the optimal number of clusters. Specifically, the number of clusters is predetermined. , , For integers greater than 2, calculate the sum of squared clustering errors and plot the results. The relationship curve with SSE is used, and the value at the elbow inflection point of the curve is selected as the final number of working condition categories. During clustering, Euclidean distance is used as the sample similarity measure, and the cluster centers are iteratively optimized until convergence, ultimately yielding... There are three operating condition categories, and each operating condition corresponds to a cluster center, which is the typical wind speed and air pressure value of that operating condition.

[0025] Power consumption data for each time period was extracted from a subset of historical data. The K-means clustering algorithm was used, and the elbow method was employed to determine the optimal number of clusters. Specifically, the number of clusters was pre-set. , , For integers greater than 2, calculate the sum of squared clustering errors and plot the results. The relationship curve with SSE is used, and the value at the elbow inflection point of the curve is selected as the final number of clusters. During clustering, Euclidean distance is used as the sample similarity measure, and the cluster centers are iteratively optimized until convergence. The cluster centers are sorted in ascending order of numerical value, the minimum value in the power supply data is taken as the starting lower bound minus a buffer value, the buffer value is 5% of the minimum value, the maximum value in the power supply data is taken as the termination upper bound plus the same buffer value, the buffer value is 5% of the maximum value, the dividing points of each total power supply interval are calculated, for the sorted cluster centers, the numerical average value of each two adjacent cluster centers is taken as the interval dividing point between them, that is, the first dividing point is the average value of the first cluster center and the second cluster center after sorting, the second dividing point is the average value of the second cluster center and the third cluster center, and so on, until the average value of the last two adjacent cluster centers is calculated as the second last dividing point, and finally the total power supply interval is formed.

[0026] Further, the method for calculating the probability that the predicted value belongs to each total power supply interval as the first type of evidence is:

[0027] Independent period data not participating in the power supply prediction model training is selected from the past Z consecutive periods, the Euclidean distance of the wind speed and air pressure of each period from each working condition center is calculated, the period is attributed to the nearest working condition category and is attached with a unique working condition label, the corresponding prediction error is matched, all error values corresponding to the same working condition label are extracted as a group by traversing all records with working condition labels, and the error probability distribution corresponding to the working condition is generated, a total of groups;

[0028] The similarity between the future next period and the working condition is calculated, the future next period is recorded as the next period of the current time, and the specific process is as follows: the wind speed and air pressure data of the future next period are obtained from the meteorological bureau, normalized processing is performed, the Euclidean distance between the wind speed and air pressure data of the period and each working condition center is calculated, the distance is converted into similarity by inverse distance method, and the similarity of all working conditions is normalized by dividing by the sum of all similarities;

[0029] The error value of all prediction error data under a certain working condition is extracted, and the kernel density estimation method is used to estimate the probability density corresponding to any error value:

[0030] ;

[0031] In the formula, represents any error value to be estimated, represents the th error value, represents the probability density function, i.e. the probability density of the error value, represents the number of error values, represents the bandwidth, represents the kernel function; ;

[0032] Wherein, the kernel function adopts Gaussian kernel:

[0033] ;

[0034] For the selection of bandwidth, the specific process is that the preset candidate number is generated, each candidate bandwidth is generated, for each candidate bandwidth, based on the error data under the working condition, the above-mentioned probability density function formula is used, each error value is substituted to obtain each estimated density , the deviation square of the discrete data is calculated combined with the frequency

[0035] ;

[0036] In the formula, The deviation square of the first error value, the average value of all sample deviation squares is taken as the mean square error under the bandwidth, and finally the candidate bandwidth which makes the mean square error reach the minimum value is selected as the selection, the frequency of the discrete data is ;

[0037] For the next period in the future, the error probability density function obtained by the kernel density estimation of each working condition is weighted and superimposed with the normalized working condition similarity as the weight, to obtain the comprehensive error probability density function of the period:

[0038] ;

[0039] In the formula, Indicates the comprehensive error probability density function of the period, Indicates the normalized working condition similarity of the period and the first , indicates the error probability density function of the period and the first working condition, ;

[0040] The power supply capacity prediction result of a period is recorded as , and the actual possible value formula of the power supply capacity of the period is constructed:

[0041] ;

[0042] In the formula, Indicates the actual possible value of the power supply capacity of the period, recorded as actual power supply capacity, therefore, the probability density function of the actual power supply capacity of the period can be derived from the comprehensive error distribution:

[0043] ;

[0044] In the formula, Indicates the probability density function when the actual power supply capacity is ; ​

[0045] Calculate the probability that the actual power supply falls within each total power supply range:

[0046] ;

[0047] In the formula, This indicates that the actual power supply falls on the [number]th [number]. The probability within a given total power supply range Indicates the first The upper limit of the total power supply range Indicates the first The lower bound of each total power supply range, where... Indicates the index of the total power supply range. , This represents the number of intervals in the total power supply, calculated above. These are allocated to the corresponding total power supply intervals, thus forming the first type of evidence for that period.

[0048] Furthermore, the method for assigning the second type of evidence to the total power supply interval corresponding to each operating condition by using the similarity between the meteorological data of the next time period and each operating condition as a weight is as follows:

[0049] Data from independent time periods that were not used in the power supply prediction model training were selected from Z consecutive time periods. The power supply and corresponding operating conditions of each time period were grouped to generate samples. For a certain operating condition, the frequency of each power supply data falling within each total power supply interval was calculated.

[0050] ;

[0051] In the formula, Indicates the first The first working condition for the first The basic BPA for each total power supply range, Indicates the first Under what operating conditions does the power supply fall into the first... The number of samples for each total power supply range. Indicates the first Total number of samples under operating conditions;

[0052] With the first Using normalized operating condition similarity as weights, the base BPA of each operating condition is weighted and summed to obtain the second type of evidence for the total power supply range during that period:

[0053] ;

[0054] In the formula, Indicating the second type of evidence, the first BPA for each total power supply range.

[0055] Furthermore, the dynamic time warping difference method is used to analyze the power supply of each fan in the previous TA time periods, identify abnormal fans, and set their power supply to zero to correct the predicted power supply value for the next time period.

[0056] Set the time step for each set of fans under each operating condition. ,and Based on the current time, including the current time and the time going back. A continuous time step ,according to Real-time power supply data of all wind turbines are collected at consecutive time steps to form a continuous power supply sequence. The power supply sequence is converted into a power supply sequence with a mean of 0 and a standard deviation of 1 using the z-score method. The adjacent differences of the power supply sequence are calculated to form a first-order difference power supply change trend sequence. Based on the first-order difference sequence of all wind turbines, the median of each time step is taken as the reference value for that step to form a first-order difference group reference trend sequence.

[0057] Next, the first-order differential power supply trend sequence and the first-order differential group reference trend sequence are further differentially analyzed to obtain the second-order differential power supply trend sequence and the second-order differential group reference trend sequence. The absolute difference between the corresponding positions of the two sequences is calculated to form a distance matrix. Then, the minimum value between the current distance and the previous cumulative distance is used to construct the cumulative distance matrix. The value in the lower right corner of the cumulative matrix is ​​the DTW distance between the wind turbine and the group reference trend. After collecting the DTW distance data of all wind turbines under the current operating conditions, the mean of all distances is first calculated, and then the average of the sum of squares of the deviations of all distances from the mean is calculated. The square root is taken to obtain the standard deviation. The threshold is set to the mean of the DTW distance of all wind turbines plus 3 times the standard deviation. When the DTW distance of a single wind turbine exceeds this threshold, the wind turbine is marked as an abnormal wind turbine. The power supply of the abnormal wind turbine is set to zero, and the power supply prediction value for the next period is corrected.

[0058] Furthermore, mapping to the total power supply range, the method for generating the third type of evidence is as follows:

[0059] The predicted power supply for the next time period, corrected for abnormal wind turbines, is matched with the total power supply range to determine the confidence level for the third type of evidence.

[0060] ;

[0061] In the formula, Indicating the third type of evidence, the first... BPA for each total power supply range, This represents a reliability parameter with a value of 0.9. This represents the predicted power supply for the next time period after correction for abnormal fan operation.

[0062] Further, the method for comparing the predicted power supply value of the next time period with the predicted power consumption value of the historical power consumption of the M2 continuous time periods before the current time obtained by inputting the historical power consumption into the power consumption prediction model and making power supply mode decision is:

[0063] First, the orthogonal sum rule of Dempster-Shafer evidence theory is used to fuse the first type, the second type and the third type of evidence:

[0064] ;

[0065] In the formula, Total power supply intervals corresponding to the three types of evidence are respectively pre-set conflict threshold , and When , it is determined that the evidence has significant conflict, and the third type of evidence is used as the dominant evidence to adopt a weighted fusion strategy:

[0066] ;

[0067] In the formula, The comprehensive basic probability distribution is represented, The weight coefficient is represented, wherein ;

[0068] When , it is determined that the evidence has no significant conflict, and the comprehensive basic probability distribution is calculated:

[0069] ;

[0070] Based on the comprehensive basic probability distribution, the lower limit of the total power supply interval with the highest confidence is selected as the corrected predicted power supply value of the next time period, and the predicted power consumption value of the next time period is compared, when the predicted power supply value of the next time period is higher than the predicted power consumption value, the energy storage mode is prepared to supplement power supply in priority, and vice versa, on the basis of the energy storage mode to supplement power supply, the standby energy and the external power grid are prepared to supplement power supply.

[0071] The application further provides a micro-grid energy optimization device based on distributed deep reinforcement learning, which is used to execute the micro-grid energy optimization method based on distributed deep reinforcement learning.

[0072] The data sequence module is used to acquire the historical power consumption, historical meteorological data and corresponding power supply of the micro-grid in the past Z continuous time periods and normalize the historical power consumption, historical meteorological data and corresponding power supply, and construct the historical power consumption feature sequence and the historical meteorological-power supply feature sequence in time sequence;

[0073] A prediction error statistics module is configured to train an LSTM model using a historical power consumption feature sequence to obtain a power consumption prediction model, train the LSTM model using a historical meteorological-power consumption feature sequence to obtain a power supply prediction model, and obtain prediction errors in a sliding window manner;

[0074] A first type of evidence module is configured to obtain historical power consumption, historical meteorological data and corresponding power supply of the micro-grid in the past Z consecutive time periods, construct a historical power consumption feature sequence and a historical meteorological-power consumption feature sequence in chronological order, obtain a power supply prediction value of a next time period, divide working conditions based on historical meteorological data clustering, generate mutually exclusive total power supply intervals, and combine the meteorological data of the time period and the similarity of each working condition to weight the prediction errors, obtain a future error distribution, and calculate the probability of the power supply prediction value belonging to each total power supply interval as the first type of evidence;

[0075] A second and third type of evidence module is configured to use the similarity of the future next time period meteorological data and each working condition as a weight to distribute the second type of evidence to the total power supply interval corresponding to each working condition, analyze the power supply of each wind turbine in the past TA time periods before the current time by using a dynamic time warping difference method, identify abnormal wind turbines and correct the power supply prediction value of the future next time period by setting the power supply of the abnormal wind turbines to zero, and map the power supply prediction value to the total power supply interval to generate the third type of evidence;

[0076] An evidence fusion module is configured to perform D-S fusion on the three types of evidence, the third type of evidence is dominant in the case of evidence conflict, and outputs a power supply prediction value of a future next time period, compares the power consumption prediction value obtained by inputting the historical power consumption of the past M2 consecutive time periods before the current time into a power consumption prediction model with the power supply prediction value, and makes a power supply mode decision.

[0077] Compared with the prior art, the present application has the following beneficial effects:

[0078] By using a dynamic time warping difference method, the power supply of a single wind turbine in a fixed time window before the current time is analyzed, abnormal wind turbines are identified and their power supply is set to zero, the power supply prediction depends not only on meteorological conditions and historical rules, but also considers the factor of wind turbine failure, the power supply prediction value of the future next time period is corrected, the actual running scene is adapted, and the present application further divides working conditions according to historical meteorological data clustering and generates mutually exclusive total power supply intervals, weights the prediction errors by combining the meteorological data of the time period and the similarity of each working condition to obtain a future error distribution, and then constructs the first type, the second type and the third type of evidence, integrates multiple sources of information by D-S fusion, integrates the prediction error characteristics under different working conditions, reduces the deviation caused by a single prediction model due to failure or meteorological fluctuations, and thus improves the adaptability to uncertainty when making a power supply mode decision. BRIEF DESCRIPTION OF DRAWINGS

[0079] Figure 1The whole method flowchart of the application is shown in the figure;

[0080] Figure 2 The power supply model error statistical chart of the application is shown in the figure;

[0081] Figure 3 The first type of evidence distribution chart of the application is shown in the figure;

[0082] Figure 4 The fault and non-fault period prediction accuracy schematic diagram of the application is shown in the figure;

[0083] Figure 5 The whole device structure schematic diagram of the application is shown in the figure. DETAILED DESCRIPTION

[0084] In order to make the purpose, technical scheme and advantages of the application more clear and explicit, the application is further described in detail below in combination with specific embodiments.

[0085] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the application should be understood as the usual meaning understood by those skilled in the art to which the application belongs. The "first", "second" and similar words used in the application do not represent any order, quantity or importance, but are only used to distinguish different components. "Include" or "contain" and similar words mean that the elements or objects before the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connected" or "connected" and similar words are not limited to physical or mechanical connection, but can include electrical connection, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to represent relative positional relationship, when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0086] EMBODIMENT

[0087] Please refer to Figures 1 to 4 The application provides a technical scheme:

[0088] The micro-grid energy optimization method based on distributed deep reinforcement learning, the specific steps include:

[0089] Step 1: Obtain the historical power consumption, historical meteorological data and corresponding power supply of the micro-grid in the past Z consecutive periods, and normalize the data, and construct the historical power consumption feature sequence and the historical meteorological-power supply feature sequence according to the time sequence;

[0090] The historical power consumption, historical meteorological data and corresponding power supply of the micro-grid in the past Z consecutive time periods are acquired, Z is an integer greater than 0, for the setting of the time period, the output fluctuation rule of the power supply can be captured, for example, the basic time period granularity is set to 15 minutes, even a short-term output drop caused by cloud blocking can be captured, for example, the total historical power consumption, historical meteorological data and corresponding power supply of 168 hours of data collection are collected, the corresponding time period is 672, covering the daily variation period of wind speed, wherein the meteorological data is wind speed and air pressure, the collection frequency is set to 1 minute / time, that is, 900 consecutive wind speed and air pressure data can be collected in each 15-minute time period, the historical power consumption feature sequence and the historical meteorological-power supply feature sequence are constructed based on the power consumption and power supply of each time period in chronological order, and the historical power consumption feature sequence and the historical meteorological-power supply feature sequence are normalized by using the min-max normalization method, for the historical meteorological-power supply feature sequence, the wind speed and air pressure curves of each time period are recorded synchronously.

[0091] Step 2: using the historical power consumption feature sequence to train the LSTM model to obtain a power consumption prediction model, using the historical meteorological-power supply feature sequence to train the LSTM model to obtain a power supply prediction model, and obtaining the prediction error by using the sliding window method;

[0092] The LSTM is a time series model, and the core capability is to learn the correlation between historical time series features and future prediction targets, which is suitable for predicting power supply and power consumption. The power consumption prediction model is constructed, and the specific process is as follows: all time period power consumption data of the historical power consumption feature sequence are extracted, only 80% of the training set and the independent validation set are divided, the power consumption of the first consecutive time periods in the training set is taken as a single input sequence, the actual power consumption of the time period is set as the corresponding prediction label, the input structure is sample number ×1, the sample number is the total number of all time periods of the historical power consumption feature sequence minus , the LSTM model is constructed, the input layer receives the sample number The timing input of X1 is directly transmitted to the hidden layer. Two layers of LSTM hidden layers are set. The number of neurons in each layer is 64. Each LSTM unit selectively remembers and updates information through the input gate, the forgetting gate and the output gate, so as to simultaneously capture short-term fluctuation patterns and long-term trend characteristics in the input time window. The TAnh activation function is used to stabilize the output value in the range of [-1, 1]. A random inactivation layer is separately added after each LSTM network. The dropout rate is set to 0.2, that is, 20% of the neuron outputs are randomly shielded to prevent model overfitting. The output layer has an output dimension of 1. The mean square error is used as the loss function to measure the deviation between the predicted value and the true value. The Adam optimizer is used. The initial learning rate is set to 0.001. The learning rate is attenuated to half of the original value every 50 epochs to balance the learning speed and stability. The iteration training is stopped when the validation set loss does not decrease for 20 consecutive epochs to avoid overfitting.

[0093] The construction of the power supply amount prediction model is as follows. First, M2 consecutive time period power supply amount data is extracted from the historical meteorological-power supply amount feature sequence, denoted as a historical data subset, and the total historical time period number Z is marked. The training window length is set to , and 0 < M1 < M2 < Z. The remaining data not involved in training is left, so that the true deviation of the constructed model to the unfamiliar data can be obtained subsequently. The sample number is calculated by M2 minus M1 plus 1. The power supply amount of each time period in the training time window and the meteorological data of the next time period after the end of the training time window are used as inputs to increase the accuracy of the model prediction and avoid overemphasis on the power supply amount feature, thereby ignoring the key influence of meteorological data on the power supply amount. The processed power supply amount and meteorological data are arranged in chronological order to form a three-dimensional input structure with a sample number, a training window length and a feature dimension of each time step. The meteorological data includes wind speed and air pressure. The feature dimension of each time period is the sum of the data points of wind speed and air pressure in a single time period and the data points of wind speed and air pressure in the next time period after the end of the window. The prediction label corresponding to each sample is the power supply amount of the next time period after the end of the training window to ensure that the model learns the rules associated with the power supply amount under different meteorological data.

[0094] After that, the LSTM model is constructed. The input layer receives a three-dimensional input structure and maintains the same dimension, thereby receiving three-dimensional time series data, preserving the time sequence information and the relationship between meteorological variables and power supply characteristics. Two layers of LSTM hidden layers are set, with 64 neurons in each layer. The first layer extracts short-term and long-term dependencies, and the second layer captures the dynamic coupling relationship between meteorological and power supply within the previous M1 time periods, which is more suitable for learning time series patterns. The TAnh activation function is used, and a random inactivation layer is added separately after each LSTM network, with a dropout rate of 0.2, i.e., randomly masking 20% of the neuron outputs to prevent model overfitting and improve its generalization ability on unseen data. The output layer has an output dimension of 1, which realizes the random masking of part of the neuron outputs with a certain probability during the training process. The linear activation function is used to ensure that the prediction result is a continuous real value. During the model training process, the historical data subset is used as the basis for model training. The time series cross-validation method is used to divide the dataset, with the training set accounting for 70%, the validation set accounting for 20%, and the test set accounting for 10%. The mean squared error is used as the loss function to measure the difference between the predicted value and the true value. The Adam optimizer is used, with an initial learning rate of 0.001 and a decay rate of 0.5 every 50 epochs. The training is stopped when the validation set loss does not decrease for 20 consecutive epochs to avoid invalid iterations and overfitting.

[0095] From the past Z consecutive time periods, independent time period data that did not participate in the power supply prediction model training is selected, for example, M2 consecutive time period power supply data is extracted from the historical meteorological-power supply characteristic sequence for training, which is used as the error calculation benchmark to truly reflect the model's generalization ability in actual prediction scenarios. A sliding window with a length of M1 consistent with the training is set, and the feature sequence input power supply prediction model containing M1 consecutive time periods is intercepted to obtain the predicted value of the next time period. The absolute error is calculated with the actual value of the power supply in that time period. Then the window is shifted by 1 time period along the time axis, and the prediction and error calculation process is repeated. All calculated errors are integrated in chronological order into an error set, denoted as prediction error, which reflects the degree of deviation, thereby correcting the power supply prediction based on the prediction error to reduce the decision risk. Table 1 shows the absolute error between the predicted value and the actual value of the power supply prediction model for the power supply prediction under independent time period data that did not participate in the power supply prediction model training. 40 groups of continuous data containing fan sudden failure conditions from 09:30 to 10:44 are intercepted, and the data table is generated after aggregation;

[0096] Table 1 Prediction error statistics table

[0097]

[0098] AsFigure 2 As shown in the figure, most of the time, the power supply prediction value is highly consistent with the actual value, and the prediction value is highly consistent with the actual value. Under the normal scenario of fan failure, the LSTM prediction model can effectively learn the time sequence rule of power supply. In the period of 09:30-10:29 and 09:45-10:44, the fan suddenly fails and the output drops sharply, resulting in a sharp decrease in actual power supply. The power supply prediction model is trained based on the majority of historical normal data, which makes the error between the prediction value and the actual value increase sharply, and the actual power supply affected by the fan failure needs to be considered.

[0099] Step 3: Obtain the historical power consumption, historical meteorological data and corresponding power supply of the micro-grid in the past Z consecutive periods, and construct the historical power consumption feature sequence and the historical meteorological-power supply feature sequence in time sequence, obtain the power supply prediction value of the next period, divide the working conditions based on the historical meteorological data clustering, generate mutually exclusive total power supply intervals, and combine the meteorological data of the period with the similarity of each working condition to weight the prediction error, obtain the future error distribution, and calculate the probability of the power supply prediction value belonging to each total power supply interval as the first type of evidence;

[0100] Through the short-term forecast function of the micro-grid associated meteorological bureau API or on-site meteorological station, obtain the meteorological data of the next period, normalize the data according to the Min-Max standardization parameters of the training set, and intercept the power supply of the next period and the meteorological data of each period, combine the meteorological data of the next period, and normalize the data according to the Min-Max standardization parameters, integrate them into a three-dimensional input structure, input the power supply prediction model, and obtain the power supply prediction value of the next period;

[0101] The specific process of dividing the working conditions based on historical meteorological data clustering is as follows: in order to cover a large number of meteorological scenes, the average value of the wind speed and air pressure of each period in the past Z consecutive periods is taken as the wind speed value and air pressure value of each period, and the wind speed value and air pressure value of each period are extracted based on the historical data subset to form a two-dimensional feature vector. Wind speed and air pressure are the core meteorological factors affecting micro-grid power supply, but both are continuous variables. If continuous meteorological data is used directly for modeling, overfitting may occur. Therefore, K-means clustering algorithm is used to cluster the standardized feature vectors to divide the working condition categories. Under different working conditions, the error of the predicted power supply is different. The error distribution is more consistent with the actual scene in the future, avoiding the deviation caused by global error. The elbow method is used to determine the optimal number of clusters to ensure that each working condition has enough samples to support statistical rules and can distinguish significantly different meteorological scenes, i.e. working conditions. The specific process is as follows: for the pre-set number of clusters , , For integers greater than 2, calculate the sum of squared clustering errors (SSE) and plot them. The relationship curve with SSE is used, and the value at the elbow inflection point of the curve is selected as the final number of working condition categories. During clustering, Euclidean distance is used as the sample similarity measure, and the cluster centers are iteratively optimized until convergence, ultimately yielding... Each working condition is categorized into several categories, and each working condition corresponds to a cluster center, which is the typical wind speed and air pressure value of that working condition. This provides a basis for subsequent classification of working conditions based on meteorological similarity.

[0102] Power supply data for each time period was extracted from a subset of historical data. The K-means clustering algorithm was used, and the elbow method was used to determine the optimal number of clusters. Power supply is not a random discrete value, but is affected by factors such as weather, load, and equipment status, exhibiting natural clustering characteristics. Artificially setting intervals ignores the distribution pattern of the data itself, resulting in a disconnect between the intervals and the actual power supply level. K-means is an unsupervised learning algorithm that can automatically cluster similar values ​​in power supply into a cluster, which is consistent with the continuous numerical characteristics of power supply. At the same time, any prediction model has prediction errors and cannot guarantee that the predicted value is completely consistent with the actual value. The average value of the centers of adjacent clusters is used as the dividing point to ensure that all actual power supply can be included in a certain interval, providing a unified domain of discourse for the generation of three types of BPA evidence.

[0103] The specific process involves setting a pre-defined number of clusters. , , For integers greater than 2, calculate the sum of squared clustering errors (SSE) and plot them. The relationship curve with SSE is used, and the value at the elbow inflection point of the curve is selected as the final number of clusters. During clustering, Euclidean distance is used as the sample similarity measure, and the cluster centers are iteratively optimized until convergence. The cluster centers are sorted in ascending order of numerical value. The minimum value of the power supply data minus a buffer value is taken as the starting lower bound, where the buffer value is 5% of the minimum value. The maximum value of the power supply data plus an equal buffer value is taken as the ending upper bound, where the buffer value is 5% of the maximum value. The boundary points of each total power supply interval are calculated. For the sorted cluster centers, the average value of every two adjacent cluster centers is taken as the interval boundary point between them. That is, the first boundary point is the average value of the first and second cluster centers after sorting, the second boundary point is the average value of the second and third cluster centers, and so on, until the average value of the last two adjacent cluster centers is calculated, which is taken as the penultimate boundary point, thus forming the total power supply interval.

[0104] In the historical meteorological-power supply characteristic sequence, the data of the period not participating in the power supply prediction model training is selected, so that the generated error distribution is more general. The Euclidean distance of the wind speed and air pressure of each period to each working condition center is calculated. The closer the Euclidean distance of the meteorological data of each period to a working condition center, the more similar the meteorological scene of the period to the working condition, so the period is attributed to the nearest working condition category and is attached with a unique working condition label. The prediction error characteristics of different working conditions are significantly different. In order to accurately capture the error law, the error distribution of each working condition is bound to the specific meteorological scene. After matching the prediction error of the period, all error values corresponding to the same working condition label are extracted and grouped to generate the error probability distribution of the working condition. There are a total of groups;

[0105] The similarity between the future next period and the working condition is calculated, and the future next period is recorded as the next period of the current time. The specific process is as follows: the wind speed and air pressure data of the future next period are obtained from the meteorological bureau, normalized, the distances between the wind speed and air pressure data of the period and each working condition center are calculated by using the Euclidean distance, and the distances are converted into similarities by using the inverse distance method. The similarity of all working conditions is normalized by dividing the sum of all similarities, so as to quantify the correlation strength between the future meteorological scene and the historical typical working condition. If there is no similarity weighting, the error distribution of a single working condition is directly used, which will lead to a large deviation between the predicted error and the actual error.

[0106] The power supply prediction error is affected by the working conditions such as wind speed and air pressure, and its distribution often does not conform to the standard parameter distribution such as normal distribution and uniform distribution. Kernel density estimation is a non-parametric method, which does not need to preset the distribution form, but only fits the curve through the error sample itself, and can adaptively capture the real distribution form of the error under different working conditions. Therefore, the error values of all prediction error data under a certain working condition are extracted, and the probability density of any error value is estimated by using the kernel density estimation method:

[0107] ;

[0108] In the formula, represents any error value to be estimated, represents the th error value, represents the probability density function, i.e. the probability density of the error value, represents the number of error values, represents the bandwidth, represents the kernel function; ;

[0109] Among them, the kernel function adopts Gaussian kernel:

[0110] ;

[0111] For the selection of bandwidth, the specific process is, presetting the candidate number, generating each candidate bandwidth, for each candidate bandwidth, based on the error data under the working condition, using the above probability density function formula, substituting each error value to obtain each estimated density , combining the frequency of discrete data to calculate the deviation square:

[0112] ;

[0113] In the formula, the deviation square of the first error value, taking the average value of all sample deviation squares as the mean square error under the bandwidth, finally selecting the candidate bandwidth which makes the mean square error minimum as the selection, the frequency of the discrete data is ;

[0114] For the next period in the future, the error probability density function obtained by the kernel density estimation of each working condition is weighted and superimposed with the normalized working condition similarity as the weight, to obtain the comprehensive error probability density function of the period:

[0115] ;

[0116] In the formula, the comprehensive error probability density function of the period, the normalized working condition similarity of the period and the first working condition, the error probability density function of the period and the first working condition, ;

[0117] The power supply capacity prediction result of a period is recorded as , the actual possible value formula of the power supply capacity of the period is constructed:

[0118] ;

[0119] In the formula, the actual possible value of the power supply capacity of the period, recorded as the actual power supply capacity, therefore, the probability density function of the actual power supply capacity of the period can be derived from the comprehensive error distribution:

[0120] ;

[0121] In the formula, the probability density function when the actual power supply capacity is ;

[0122] The probability that the actual power supply capacity falls in each total power supply capacity interval is calculated:

[0123] ;

[0124] In the formula, represents the probability that the actual power supply falls in the th total power supply interval, represents the upper limit of the th total power supply interval, represents the lower limit of the th total power supply interval, wherein, represents the index of the total power supply interval, The formula reflects in which range the actual power supply may fall, further improving the reliability of the decision, and the calculated above is respectively assigned to the corresponding total power supply interval, that is, the first type of evidence of the time period is formed, and the risk evidence output can be directly used for scheduling, supporting reliable decision-making.

[0125] As shown in Figure 3 , three working conditions are divided by weather clustering through an unsupervised clustering algorithm, and BPA1, BPA2 and BPA3 respectively correspond to the confidence probability that the current predicted power supply falls in three total power supply intervals. For the target period, the probability distribution of the power supply corresponding to different working conditions is different, and the confidence probability of each power supply interval is generated, which converts weather and prediction error information into probabilistic evidence required for decision-making.

[0126] Step 4: The similarity of the future next period weather data to each working condition is taken as a weight, and the second type of evidence is assigned to the total power supply interval corresponding to each working condition. The dynamic time warping difference method is used to analyze the power supply of each fan in the past TA periods, identify abnormal fans and correct the future next period power supply prediction value by setting the power supply of abnormal fans to zero, and map it to the total power supply interval to generate the third type of evidence;

[0127] From the past Z consecutive periods, independent period data not participating in the training of the power supply prediction model is selected, each period power supply and the corresponding working condition are grouped to generate samples, and the frequency of each power supply data falling in each total power supply interval under a certain working condition is counted:

[0128] ;

[0129] In the formula, represents the base BPA of the th total power supply interval under the th working condition, represents the number of samples of the power supply falling in the th total power supply interval under the th working condition, represents the number of samples of the power supply falling in the The total number of samples under operating conditions is used to statistically analyze the long-term pattern of power supply falling within a certain range of total power supply under specific meteorological conditions.

[0130] With the first Using normalized operating condition similarity as weights, the base BPA of each operating condition is weighted and summed to obtain the second type of evidence for the total power supply range during that period:

[0131] ;

[0132] In the formula, Indicating the second type of evidence, the first By establishing a BPA for each total power supply range, an objective reference system independent of the prediction model is constructed.

[0133] Set the time step for each set of fans under each operating condition. ,and Based on the current time, including the current time and the time going back. A continuous time step ,according to Real-time power supply data of all wind turbines are collected at continuous time steps to form a continuous power supply sequence. This avoids accidental misjudgment due to a small amount of data. The power supply sequence is converted into a power supply sequence with a mean of 0 and a standard deviation of 1 using the z-score method to eliminate individual differences of wind turbines and achieve trend comparability. The adjacent differences of the power supply sequence are calculated to form a first-order differential power supply change trend sequence. Based on the first-order differential sequence of all wind turbines, the median of each time step is taken as the reference value for that step to form a first-order differential group reference trend sequence, which reflects the direction and magnitude of power generation change.

[0134] After that, the first-order difference of the first-order difference power supply trend sequence and the first-order difference of the group reference trend sequence is taken to obtain the second-order difference power supply trend sequence and the second-order difference group reference trend sequence, which essentially reflects the change rate of the trend, thereby realizing early identification of implicit faults. The absolute difference value of the corresponding positions of the two is calculated to form a distance matrix, and the sum of the current distance and the minimum value in the previous step cumulative distance is calculated to build a cumulative distance matrix. The value in the lower right corner of the cumulative matrix is the DTW distance of the fan and the group reference trend. After collecting the DTW distance data of all fans under the current working condition, the mean of all distances is first calculated, and then the average of the sum of squares of the deviations of all distances from the mean is calculated. After square root, the standard deviation is obtained. The threshold is set to be the mean of the DTW distances of all fans plus 3 times the standard deviation, which excludes the occasional fluctuations of normal fans. When the DTW distance of a single fan exceeds the threshold, the trend of the fan deviates from the group, and there is a high probability of its own fault. The fan is marked as an abnormal fan. After setting the power supply of the abnormal fan to zero, the future next period power supply prediction value is corrected. If it is not corrected, the prediction model will still calculate the power supply under the condition that the abnormal fan is running normally. In the early stage of implicit failure, the fan will not shut down immediately, but the power generation will gradually decrease. When the motor burns out or the control system fails, the fan cannot run directly, and the power supply is 0. If this part is not excluded during prediction, the total predicted power supply value will be too high, which may lead to scheduling decision errors.

[0135] Excluding fault interference, i.e. setting the power supply of the abnormal fan to zero, the evidence is converted into interval reliability allocation, which provides a basis for subsequent multi-evidence fusion. The future next period power supply prediction value after the abnormal fan correction is matched with the total power supply interval to allocate the reliability of the third type of evidence:

[0136] ;

[0137] In the formula, BPA represents the BPA of the first total power supply interval in the third type of evidence, BPA represents the BPA of the first total power supply interval in the third type of evidence, BPA represents the BPA of the first total power supply interval in the third type of evidence, BPA represents the BPA of the first total power supply interval in the third type of evidence, wherein BPA represents the BPA of the first total power supply interval in the third type of evidence, wherein

[0138] Step 5: D-S fusion of the three types of evidence, with the third type of evidence as the dominant evidence in case of evidence conflict, and the future next period power supply prediction value is output. The power consumption prediction value obtained by inputting the historical power consumption of the current time and the previous M2 continuous time periods into the power consumption prediction model is compared and the power supply mode decision is made;

[0139] The first type of evidence is affected by dynamic factors such as aging of meteorological equipment, and cannot completely match the current working condition. The second type of evidence is essentially power supply data reflecting real-time non-exceptional fan operation. The third type of evidence is aimed at solving fan abnormalities and avoiding high fan power supply problems. In order to make a conclusion without relying on a single evidence, disperse confidence is concentrated into a consistent power supply interval, and uncertainty is reduced. First, the orthogonal sum rule of Dempster-Shafer evidence theory is used to fuse the first, second and third types of evidence:

[0140] ;

[0141] In the formula, are the total power supply intervals corresponding to the three types of evidence, and the conflict threshold is set in advance , and For example, the conventional setting is 0.6, which can be adjusted according to actual conditions. When , it is determined that there is a significant conflict in the evidence. The reason is that when there is a significant conflict in the evidence, the difference in the judgment of the three types of evidence on the power supply is too large. If the original D-S theory is used for fusion, multiple total power supply intervals will be fused into a meaningless interval, and the result will be unreliable. Therefore, in order to integrate the possibility of equipment failure, close to the actual operation scene, and avoid overestimating the power supply prediction value, it is assumed that the third type of evidence is the most reliable, and the weighted fusion strategy is adopted with the third type of evidence as the dominant:

[0142] ;

[0143] In the formula, represents the comprehensive basic probability distribution, represents the weight coefficient, wherein , which ensures that the weight of the third type of evidence is higher than the average of the first two types of evidence, This item avoids completely ignoring the average credibility of the first two types of evidence;

[0144] When , it is determined that there is no significant conflict in the evidence, which means that there is more overlap in the judgment of the three types of evidence. At this time, the original D-S fusion is effective, and the influence of the third type of evidence on the result is small, avoiding affecting the determination result. The comprehensive basic probability distribution is calculated:

[0145] ;

[0146] In order to avoid the total sum of the comprehensive credibility being less than 1, As a normalization factor, the basic requirement of the probability distribution is met, based on the comprehensive basic probability distribution, the lower limit of the total power supply interval with the highest confidence is selected as the corrected future next time period power supply prediction value, and the future next time period power supply prediction value is obtained by comparing the power consumption prediction model, when the future next time period power supply prediction value is higher than the power consumption prediction value, the energy storage mode is prepared to supplement power supply in priority, otherwise, on the basis of preparing energy storage mode to supplement power supply, standby energy and external power grid are prepared to supplement power supply, table 2 shows the confidence distribution and conflict determination of three kinds of evidence recorded in 40 continuous time periods under the condition that there is part of wind turbine failure in time period 39-40, after collecting the BPA value of three kinds of evidence, whether there is conflict is determined by comparing the preset threshold value, and the evidence conflict determination table is generated after summarizing;

[0147] Table 2 evidence conflict determination table

[0148]

[0149] As Figure 4 shown, when there is no wind turbine failure in the first 38 time periods, the difference between the actual power supply and the predicted power supply fluctuates, but the overall trend matching degree is high, which reflects that the prediction model can effectively capture the power supply relationship under normal scenario, when there is wind turbine failure in 39-40 time periods, the actual value of power supply sharply drops, although the predicted power supply is lower than the actual power supply, but the accuracy of the prediction result is still high, which avoids the problem of lack of early response.

[0150] Please refer to Figure 5 , the application further provides a micro-grid energy optimization device based on distributed deep reinforcement learning, which is used for executing the micro-grid energy optimization method based on distributed deep reinforcement learning, and comprises:

[0151] A data sequence module is configured to acquire historical power consumption, historical meteorological data and corresponding power supply of the micro-grid in the past Z continuous time periods, and normalize the data, and construct historical power consumption feature sequence and historical meteorological-power supply feature sequence in time sequence.

[0152] A prediction error statistical module is configured to train the LSTM model using the historical power consumption feature sequence to obtain a power consumption prediction model, train the LSTM model using the historical meteorological-power supply feature sequence to obtain a power supply prediction model, and acquire prediction errors in a sliding window manner.

[0153] The first type of evidence module is used to obtain historical power consumption of the micro-grid, historical meteorological data and corresponding power supply in the past Z consecutive time periods, and construct a historical power consumption feature sequence and a historical meteorological-power supply feature sequence in chronological order, to obtain a power supply prediction value of the next time period, divide the working conditions based on historical meteorological data clustering, generate mutually exclusive total power supply intervals, and combine the meteorological data of the time period and the similarity of each working condition to weight the prediction error, obtain a future error distribution, and calculate the probability of the power supply prediction value belonging to each total power supply interval as the first type of evidence;

[0154] The second and third types of evidence modules are used to take the similarity of the future next time period meteorological data and each working condition as a weight, to allocate the second type of evidence to the total power supply interval corresponding to each working condition, and use the dynamic time warping difference method to analyze the power supply of each fan in the past TA time periods, identify abnormal fans and correct the power supply prediction value of the future next time period by setting the power supply of the abnormal fans to zero, and map to the total power supply interval to generate the third type of evidence;

[0155] The evidence fusion module is used to perform D-S fusion on the three types of evidence, the third type of evidence is dominant in evidence conflict, and outputs the power supply prediction value of the future next time period, compares the power consumption prediction value obtained by inputting the historical power consumption of the past M2 consecutive time periods into the power consumption prediction model with the power supply prediction value of the future next time period, and makes a power supply mode decision.

[0156] The above formulas are all dimensionless values, and the formulas are obtained by collecting a large amount of data to simulate a formula of the latest real situation, and the preset parameters in the formula are set by a person skilled in the art according to the actual situation.

[0157] The above embodiments can be realized by software, hardware, firmware or any combination thereof, in whole or in part. When realized by software, the above embodiments can be realized in the form of a computer program product in whole or in part. Those skilled in the art can realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are realized by hardware or software methods depends on the specific application and design constraints of the technical solutions.

[0158] The units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, which can be located in one place or distributed on multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the present embodiment.

[0159] The above merely provides the specific implementation of the present application, but the protection scope of the present application is not limited to this. Any person skilled in the art can easily think of the changes or replacements within the technical range disclosed by the present application, which should be covered in the protection scope of the present application.

Claims

1. A microgrid energy optimization method based on distributed deep reinforcement learning, characterized in that, The specific steps include: Obtain the historical power consumption, historical meteorological data, and corresponding power supply of the microgrid within the past Z consecutive time periods, perform normalization processing, and construct a historical power consumption feature sequence and a historical meteorological-power supply feature sequence in chronological order; Use the historical power consumption feature sequence to train the LSTM model to obtain a power consumption prediction model, use the historical meteorological-power supply feature sequence to train the LSTM model to obtain a power supply prediction model, and obtain the prediction error through a sliding window method; Obtain the meteorological data for the next future time period, combine the historical meteorological data and power supply of the previous M1 time periods, input them into the power supply prediction model to obtain the predicted power supply value for the next time period. Based on the historical meteorological data clustering to divide the operating conditions, generate mutually exclusive total power supply intervals, and combine the meteorological data of this time period with the similarity of each operating condition to weight the prediction error to obtain the future error distribution. Calculate the probability that the predicted power supply value belongs to each total power supply interval as the first type of evidence; Use the similarity between the meteorological data of the next future time period and each operating condition as the weight to allocate the second type of evidence to the total power supply intervals corresponding to each operating condition. Adopt the dynamic time warping difference method to analyze the power supply of each fan in the TA time periods before the current moment, identify abnormal fans, set their power supply to zero to correct the predicted power supply value for the next future time period, and map it to the total power supply interval to generate the third type of evidence; Perform D-S fusion on the three types of evidence. If there is evidence conflict, the third type of evidence shall be the dominant one, output the predicted power supply value for the next future time period, compare it with the predicted power consumption value obtained by inputting the historical power consumption of the previous M2 consecutive time periods before the current moment into the power consumption prediction model, and make a power supply mode decision.

2. The microgrid energy optimization method based on distributed deep reinforcement learning according to claim 1, characterized in that: The method of using the historical meteorological-power supply feature sequence to train the LSTM model to obtain a power supply prediction model is as follows: First, mark the past Z consecutive time periods as the total number of historical time periods Z. Extract the power supply data of M2 consecutive time periods from the historical meteorological-power supply feature sequence, denoted as the historical data subset. Set the training window length to M1 time periods, and 0 < M1 < M2 < Z. Calculate the number of samples by M2 minus M1 plus 1. Arrange the processed power supply and meteorological data in chronological order to form a three-dimensional input structure with dimensions of the number of samples, the training window length, and the feature dimension of each time period. Among them, the meteorological data includes the wind speed and air pressure data of each time period within the training window and the next time period after the window ends. The feature dimension of each time period is the sum of the number of data points of wind speed and air pressure within a single time period and the number of data points of wind speed and air pressure in the next time period after the window ends. The prediction label corresponding to each sample is the power supply in the next time period after the training window ends.

3. The microgrid energy optimization method based on distributed deep reinforcement learning according to claim 2, characterized in that: The method of obtaining the prediction error through a sliding window method is as follows: The data from independent time periods that were not used in the power supply prediction model training within the past Z consecutive time periods are selected as the error calculation benchmark. A sliding window of length M1, consistent with that used during training, is set. The feature sequence containing M1 consecutive time periods is extracted and input into the power supply prediction model to obtain the predicted value for the next time period. The absolute error is calculated by comparing the predicted value with the actual value of the power supply for that time period. Then, the window is slid forward by one time period along the time axis, and the above prediction and error calculation process is repeated. All the calculated errors are integrated into an error set in chronological order, which is denoted as the prediction error.

4. The microgrid energy optimization method based on distributed deep reinforcement learning according to claim 2, characterized in that: The method for generating mutually exclusive total power supply intervals based on clustering historical meteorological data to classify operating conditions is as follows: Wind speed and air pressure values ​​for each time period are extracted from a subset of historical data to form a two-dimensional feature vector. Then, the K-means clustering algorithm is used to perform cluster analysis on the standardized feature vector to classify operating conditions. The elbow method is used to determine the optimal number of clusters. Specifically, the number of clusters is predetermined. , , For integers greater than 2, calculate the sum of squared clustering errors and plot the results. The relationship curve with SSE is used, and the value at the elbow inflection point of the curve is selected as the final number of working condition categories. During clustering, Euclidean distance is used as the sample similarity measure, and the cluster centers are iteratively optimized until convergence, ultimately yielding... There are three operating condition categories, and each operating condition corresponds to a cluster center, which is the typical wind speed and air pressure value of that operating condition. Power consumption data for each time period was extracted from a subset of historical data. The K-means clustering algorithm was used, and the elbow method was employed to determine the optimal number of clusters. Specifically, the number of clusters was pre-set. , , For integers greater than 2, calculate the sum of squared clustering errors and plot the results. The relationship curve with SSE is used, and the value at the elbow inflection point of the curve is selected as the final number of clusters. During clustering, Euclidean distance is used as the sample similarity measure, and the cluster centers are iteratively optimized until convergence. The cluster centers are sorted in ascending order of numerical value. The minimum value of the power supply data minus a buffer value is taken as the starting lower bound, where the buffer value is 5% of the minimum value. The maximum value of the power supply data plus an equal buffer value is taken as the ending upper bound, where the buffer value is 5% of the maximum value. The boundary points of each total power supply interval are calculated. For the sorted cluster centers, the average value of every two adjacent cluster centers is taken as the interval boundary point between them. That is, the first boundary point is the average value of the first and second cluster centers after sorting, the second boundary point is the average value of the second and third cluster centers, and so on, until the average value of the last two adjacent cluster centers is calculated, which is taken as the penultimate boundary point, thus forming the total power supply interval.

5. The microgrid energy optimization method based on distributed deep reinforcement learning according to claim 3, characterized in that: The method for calculating the probability that the predicted value belongs to each total power supply interval as the first type of evidence is as follows: From the past Z consecutive time periods, independent time periods not involved in the power supply prediction model training are selected. For each time period, the Euclidean distance between the wind speed and air pressure and the center of each operating condition is calculated. The time period is assigned to the nearest operating condition category and assigned a unique operating condition label. The corresponding prediction error is matched, and all records with operating condition labels are traversed. All error values ​​corresponding to the same operating condition label are extracted and grouped together to generate the error probability distribution corresponding to that operating condition. Group; The similarity between the next time period and the working conditions is calculated. The next time period is referred to as the next time period after the current time. Specifically, the wind speed and air pressure data of the next time period are obtained from the meteorological bureau and normalized. The distance between the wind speed and air pressure data of this time period and the center of each working condition is calculated using Euclidean distance. The distance is converted into similarity using the inverse distance method. The similarity of all working conditions is normalized by dividing by the sum of all similarities. Extract the error values ​​of all prediction error data under a certain working condition, and use the kernel density estimation method to estimate the probability density corresponding to any error value: ； In the formula, Represents any error value to be estimated. Indicates the first One error value, This represents the probability density function, i.e., the probability density of the error values. Indicates the number of error values. Indicates bandwidth. Represents the kernel function; ; The kernel function used is a Gaussian kernel: ； The specific process for selecting bandwidth is as follows: A predetermined number of candidate bandwidths are generated; for each candidate bandwidth, based on the error data under that operating condition, the aforementioned probability density function formula is used to substitute each error value to obtain the estimated density. The squared deviation is calculated by combining the frequency of discrete data: ； In the formula, Indicates the first The squared deviation of each error value is taken as the mean square error (MSE) under that bandwidth, and the candidate bandwidth that minimizes the MSE is selected. The frequency of the discrete data is... ; For the next time period, using normalized operating condition similarity as weights, the error probability density functions obtained from kernel density estimation for each operating condition are weighted and superimposed to obtain the comprehensive error probability density function for that time period: ； In the formula, This represents the overall error probability density function for that period. This indicates that the time period is related to the first... A normalized operating condition similarity, This indicates that the time period is related to the first... The error probability density function for each working condition. ; The power supply forecast result for a certain period is denoted as: Construct the formula for the actual possible values ​​of power supply during this period: ； In the formula, The actual possible value of the power supply during this period is denoted as the actual power supply. Therefore, the probability density function of the actual power supply during this period can be derived from the comprehensive error distribution: ； In the formula, This indicates the actual power supply. The probability density function at time; Calculate the probability that the actual power supply falls within each total power supply range: ； In the formula, This indicates that the actual power supply falls on the [number]th [number]. The probability within a given total power supply range Indicates the first The upper limit of the total power supply range Indicates the first The lower bound of each total power supply range, where... Indicates the index of the total power supply range. , This represents the number of intervals in the total power supply, calculated above. These are allocated to the corresponding total power supply intervals, thus forming the first type of evidence for that period.

6. The microgrid energy optimization method based on distributed deep reinforcement learning according to claim 5, characterized in that: The method for assigning the second type of evidence to the total power supply interval corresponding to each operating condition by using the similarity between the meteorological data of the next time period and each operating condition as a weight is as follows: Data from independent time periods that were not used in the power supply prediction model training were selected from Z consecutive time periods. The power supply and corresponding operating conditions of each time period were grouped to generate samples. For a certain operating condition, the frequency of each power supply data falling within each total power supply interval was calculated. ； In the formula, Indicates the first The first working condition for the first The basic BPA for each total power supply range, Indicates the first Under which the power supply falls into the first operating condition The number of samples for each total power supply range. Indicates the first Total number of samples under operating conditions; With the first Using normalized operating condition similarity as weights, the base BPA of each operating condition is weighted and summed to obtain the second type of evidence for the total power supply range during that period: ； In the formula, Indicating the second type of evidence, the first BPA for each total power supply range.

7. The microgrid energy optimization method based on distributed deep reinforcement learning according to claim 1, characterized in that: The method of using dynamic time warping difference to analyze the power supply of each wind turbine in the previous TA time periods, identifying abnormal wind turbines and setting their power supply to zero to correct the predicted power supply value for the next time period is as follows: Set the time step for each set of fans under each operating condition. ,and Based on the current time, including the current time and the time going back. A continuous time step ,according to Real-time power supply data of all wind turbines are collected at consecutive time steps to form a continuous power supply sequence. The power supply sequence is converted into a power supply sequence with a mean of 0 and a standard deviation of 1 using the z-score method. The adjacent differences of the power supply sequence are calculated to form a first-order difference power supply change trend sequence. Based on the first-order difference sequence of all wind turbines, the median of each time step is taken as the reference value for that step to form a first-order difference group reference trend sequence. Next, the first-order differential power supply trend sequence and the first-order differential group reference trend sequence are further differentially analyzed to obtain the second-order differential power supply trend sequence and the second-order differential group reference trend sequence. The absolute difference between the corresponding positions of the two sequences is calculated to form a distance matrix. Then, the minimum value between the current distance and the previous cumulative distance is used to construct the cumulative distance matrix. The value in the lower right corner of the cumulative matrix is ​​the DTW distance between the wind turbine and the group reference trend. After collecting the DTW distance data of all wind turbines under the current operating conditions, the mean of all distances is first calculated, and then the average of the sum of squares of the deviations of all distances from the mean is calculated. The square root is taken to obtain the standard deviation. The threshold is set to the mean of the DTW distance of all wind turbines plus 3 times the standard deviation. When the DTW distance of a single wind turbine exceeds this threshold, the wind turbine is marked as an abnormal wind turbine. The power supply of the abnormal wind turbine is set to zero, and the power supply prediction value for the next period is corrected.

8. The microgrid energy optimization method based on distributed deep reinforcement learning according to claim 2, characterized in that: The method for generating third-type evidence by mapping to the total power supply range is as follows: The predicted power supply for the next time period, corrected for abnormal wind turbines, is matched with the total power supply range to determine the confidence level for the third type of evidence. ； In the formula, Indicating the third type of evidence, the first... BPA for each total power supply range This represents a reliability parameter with a value of 0.

9. This represents the predicted power supply for the next time period after correction for abnormal fan operation.

9. The microgrid energy optimization method based on distributed deep reinforcement learning according to claim 1, characterized in that: The method for outputting the predicted power supply value for the next time period and comparing it with the predicted power supply value obtained from the power supply prediction model based on the historical power consumption of the previous M2 consecutive time periods to make a power supply mode decision is as follows: First, the orthogonal sum rules of the Dempster-Shafer evidence theory are used to fuse the first, second, and third types of evidence: ； In the formula, For each of the three types of evidence, a conflict threshold is pre-set for the total power supply range. ,and ,when If significant conflict exists in the evidence, a weighted fusion strategy is adopted, with the third type of evidence playing a dominant role. ； In the formula, This represents the overall basic probability distribution. Represents the weighting coefficient, where, ; when If there is no significant conflict in the evidence, calculate the overall basic probability distribution: ； Based on the comprehensive basic probability allocation, the lower bound of the total power supply interval with the highest confidence level is selected as the revised power supply forecast value for the next period. It is compared with the power consumption forecast value for the next period. If the power supply forecast value for the next period is higher than the power consumption forecast value, the backup energy storage mode is given priority to supplement the power supply. Otherwise, in addition to the backup energy storage mode, backup energy and external power grid are used to supplement the power supply.

10. A microgrid energy optimization device based on distributed deep reinforcement learning, characterized in that: The device is used to execute the microgrid energy optimization method based on distributed deep reinforcement learning as described in any one of claims 1-9: The data sequence module is used to acquire and normalize the historical electricity consumption, historical meteorological data and corresponding power supply of the microgrid over the past Z consecutive time periods, and construct the historical electricity consumption feature sequence and the historical meteorological-power supply feature sequence in chronological order. The prediction error statistics module is used to train the LSTM model using historical electricity consumption feature sequences to obtain an electricity consumption prediction model, and to train the LSTM model using historical weather-electricity consumption feature sequences to obtain an electricity consumption prediction model. The prediction error is obtained through a sliding window method. The first type of evidence module is used to obtain the historical electricity consumption, historical meteorological data and corresponding power supply of the microgrid in the past Z consecutive time periods, and construct the historical electricity consumption feature sequence and the historical meteorological-power supply feature sequence in chronological order to obtain the power supply prediction value for the next time period. Based on the historical meteorological data, the operating conditions are divided into clusters to generate mutually exclusive total power supply intervals. The prediction error is weighted by combining the meteorological data of the time period with the similarity of each operating condition to obtain the future error distribution. The probability of the power supply prediction value belonging to each total power supply interval is calculated as the first type of evidence. The second and third type of evidence modules are used to assign the second type of evidence to the total power supply interval corresponding to each working condition by using the similarity between the meteorological data of the next time period and each working condition as a weight. The dynamic time warping difference method is used to analyze the power supply of each wind turbine in the previous TA time periods, identify abnormal wind turbines and set their power supply to zero to correct the predicted value of power supply in the next time period, and map it to the total power supply interval to generate the third type of evidence. The evidence fusion module is used to perform DS fusion on three types of evidence. In case of evidence conflict, the third type of evidence takes precedence. It outputs the power supply prediction value for the next time period and compares it with the power supply prediction value obtained by inputting the historical power consumption of the previous M2 consecutive time periods into the power consumption prediction model to make a power supply mode decision.

Images