A multi-model integrated power distribution network short-term load forecasting method and device

Abstract

This invention belongs to the field of power system technology, specifically relating to a multi-model integrated method and apparatus for short-term load forecasting of distribution networks. The method includes: collecting first historical load data of the distribution network; predicting multiple initial distribution network loads at the forecast time based on the first historical load data using a pre-established load forecasting model; and determining the final distribution network load at the forecast time using the multiple initial distribution network loads. The load forecasting model is constructed using multiple deep learning models based on a self-attention mechanism. The technical solution provided by this invention achieves high-precision and highly robust short-term load forecasting of distribution networks, meeting the needs of real-time dispatching of distribution networks and providing strong technical support for the optimized operation of smart distribution networks.

Description

Technical Field

[0001] This invention belongs to the field of power system technology, specifically relating to a method and apparatus for short-term load forecasting of distribution networks using multi-model integration. Background Technology

[0002] Short-term load forecasting for distribution networks is a key technology for power system operation and dispatch. Its core objective is to accurately predict electricity demand for the next few hours to days by analyzing historical load data and related influencing factors. With the large-scale grid connection of renewable energy, load volatility and uncertainty have increased significantly, highlighting the limitations of traditional forecasting methods. Early methods mainly employed statistical models such as time series analysis and linear regression. These methods perform well when load changes are stable, but are less adaptable to nonlinear and non-stationary characteristics. In recent years, machine learning (such as support vector machines and random forests) and deep learning (such as LSTM and GRU) methods have gradually become research hotspots. Although they can better capture complex features, they still suffer from high data dependence and insufficient generalization ability. Furthermore, existing methods are mostly limited to historical load data and fail to effectively integrate multi-source heterogeneous information such as meteorological and holiday data, leading to a significant decrease in forecast accuracy under extreme weather or special events. At the same time, traditional deep learning models have high computational complexity, making it difficult to meet the needs of real-time dispatch. Therefore, how to improve the adaptability, real-time performance, and robustness of models while ensuring accuracy has become a critical issue that urgently needs to be addressed.

[0003] Multi-source data fusion is a key breakthrough for improving the accuracy of power distribution network load forecasting. With the advancement of new power system construction, the load changes in modern power distribution networks exhibit highly nonlinear characteristics. These fluctuations are not only influenced by historical electricity consumption patterns but also closely related to various complex factors such as meteorological conditions, holiday arrangements, and socio-economic activities. Although existing research attempts to introduce external data sources to improve forecasting performance, most methods remain at the level of simple data splicing or weighted averaging, failing to fully consider the spatiotemporal correlation characteristics and interaction mechanisms between different data sources. For example, the impact of temperature on load often exhibits significant regional differences and time lag effects—air conditioning loads in commercial areas respond to temperature changes significantly faster than those in residential areas, while industrial loads may show completely different temperature sensitivities. Furthermore, special events (such as major sporting events) and holiday work schedule adjustments can disrupt conventional electricity consumption patterns, necessitating the establishment of deep correlation models across dimensions and time periods. However, existing methods rarely systematically model these complex relationships, making it difficult to achieve breakthroughs in forecasting accuracy. In particular, the lack of spatiotemporal feature extraction and dynamic coupling analysis of multi-source heterogeneous data (including meteorological data, calendar information, economic indicators, etc.) makes the prediction model less adaptable to extreme weather or special scenarios. Summary of the Invention

[0004] To overcome the problems existing in the above-mentioned related technologies, the present invention provides a method and device for short-term load forecasting of distribution networks using multi-model integration.

[0005] According to a first aspect of the present invention, a method for short-term load forecasting of a distribution network integrating multiple models is provided, comprising: Collect the first historical load data of the distribution network; Based on the first historical load data, multiple initial distribution network loads at the prediction time are predicted using a pre-established load prediction model. The final distribution network load for the predicted time is determined using the multiple initial distribution network loads. The load prediction model is constructed using multiple deep learning models based on self-attention mechanisms.

[0006] Preferably, the step of predicting multiple initial distribution network loads at the prediction time based on the first historical load data using a pre-established load forecasting model includes: Using the first historical load data as input to each deep learning model based on the self-attention mechanism in the load prediction model, multiple initial distribution network loads at the prediction time are predicted.

[0007] Preferably, determining the final distribution network load for the predicted time using the plurality of initial distribution network loads includes: Using the multiple initial distribution network loads, the average load forecast is calculated, and the average load forecast is the final distribution network load.

[0008] Preferably, the process of establishing the load forecasting model includes: The multiple deep learning models based on the self-attention mechanism are integrated to obtain the initial model; Collect the second historical load data of the power distribution network; The second historical load data is divided into a training set and a test set; The initial model is trained using the training set to obtain the trained initial model; The initial model after training is validated using the test set to obtain the load prediction model.

[0009] Preferably, training the initial model using the training set includes: The dates corresponding to the second historical load data in the training set are divided into first historical dates and first predicted dates; The initial model is trained by using the second historical load data in the training set corresponding to the first historical date as the input layer training sample and the second historical load data in the training set corresponding to the first prediction date as the output layer training sample, thereby obtaining the trained initial model.

[0010] Preferably, validating the trained initial model using the test set includes: The dates corresponding to the second historical load data in the test set are divided into second historical dates and second predicted dates; The second historical load data in the test set corresponding to the second historical date is used as the input of the trained initial model, so that each deep learning model based on the self-attention mechanism in the trained initial model can predict the predicted load of the second prediction date. The mean absolute error is calculated based on all the predicted loads and the second historical load data within the second prediction date; If the mean absolute error meets all preset conditions, the verification is successful, and the trained initial model is the load prediction model; otherwise, the verification fails, the learning rate of the initial model is adjusted using the AdamW optimizer, and the initial model with the adjusted learning rate is retrained until the verification is successful.

[0011] Preferably, the deep learning model based on the self-attention mechanism includes: an embedding layer, a multi-head self-attention layer, and a feedforward neural network layer; The embedding layer employs a positional encoding technique combining sine and cosine functions to extract features from the input data of the deep learning model based on a self-attention mechanism, thereby obtaining a feature vector. The multi-head self-attention layer uses a multi-head self-attention mechanism to calculate the attention weights of the feature vectors and uses the attention weights to quantify the degree of influence of the input data on the current prediction. Based on the degree of influence of the input data on the current prediction, the feedforward neural network layer uses Dropout regularization to obtain the prediction result.

[0012] Preferably, the embedding layer employs a positional encoding technique combining sine and cosine functions to extract features from the input data of the self-attention-based deep learning model, obtaining a feature vector, including: A positional encoding technique combining sine and cosine functions is used to map the temporal information of the input data to the polar coordinate representation of a unit circle, thereby obtaining the encoded input data. The continuous data value features in the encoded input data are normalized, and the discrete time features in the encoded historical load-related data are mapped to... d- A dense vector is used to obtain the feature vector.

[0013] Preferably, the multi-head self-attention layer uses a multi-head self-attention mechanism to calculate the attention weights of the feature vectors and uses the attention weights to quantify the influence of the input data on the current prediction, including: Perform a linear transformation on the feature vector to generate a query vector, a key vector, and a value vector; The attention weights of the feature vectors are calculated using the query vector, key vector, and value vector. The influence of the input data on the current prediction is calculated using the attention weights.

[0014] Preferably, the step of obtaining the prediction result by using Dropout regularization in the feedforward neural network layer based on the degree of influence of the input data on the current prediction includes: A differentiated Dropout strategy is adopted to assign different Monte Carlo dropout rates to the feedforward neural network layers of each deep learning model based on the self-attention mechanism; The influence of the input data on the current prediction is used as the input to the feedforward neural network layer, and the feedforward neural network layer uses Dropout regularization to obtain the prediction result.

[0015] Preferably, the formula for calculating the query vector includes:

[0016] The formula for calculating the key vector includes:

[0017] The formula for calculating the value vector includes:

[0018] In the above formula, , T To predict the total time; Q For query vector, K For key vectors, V For value vectors, h t For the predicted time t The feature vectors are fed into the Transformer. W Q for Q The corresponding weight vector, W K for K The corresponding weight vector, W V for V The corresponding weight vector.

[0019] Preferably, the formula for calculating the attention weight includes:

[0020] In the above formula, , T To predict the total time; , This is a pivotal moment in history. To predict the time t Historical moment i The importance weight of it, i.e., the attention weight; For the predicted time t Features For a historic moment i The characteristic is that T is the transpose. For each head in the attention mechanism, For normalized exponential functions, =Attention(Q,K,V), where Attention(Q,K,V) is the result of attention calculation.

[0021] Preferably, the formula for calculating the impact of historical loads on current forecasts includes:

[0022] In the above formula, To predict the time t Historical moment i Its importance weight, O The extent to which the historical load affects the current forecast. V It is a value vector.

[0023] According to a second aspect of the present invention, a multi-model integrated distribution network short-term load forecasting device is provided, comprising: The data acquisition unit is used to collect the first historical load data of the distribution network; The prediction unit is used to predict multiple initial distribution network loads at the prediction time based on the first historical load data and using a pre-established load prediction model. A determining unit is used to determine the final distribution network load for the predicted time using the plurality of initial distribution network loads; The load prediction model is constructed using multiple deep learning models based on self-attention mechanisms.

[0024] Preferably, the prediction unit is specifically used for: Using the first historical load data as input to each deep learning model based on the self-attention mechanism in the load prediction model, multiple initial distribution network loads at the prediction time are predicted.

[0025] Preferably, the determining unit is specifically used for: Using the multiple initial distribution network loads, the average load forecast is calculated, and the average load forecast is the final distribution network load.

[0026] Preferably, the apparatus further includes: a modeling unit for modeling the load forecasting model; the modeling unit includes: An integration module is used to integrate the multiple deep learning models based on the self-attention mechanism to obtain an initial model; The data acquisition module is used to collect the second historical load data of the power distribution network. A partitioning module is used to divide the second historical load data into a training set and a test set; The training module is used to train the initial model using the training set to obtain the trained initial model; The verification module is used to verify the trained initial model using the test set to obtain the load prediction model.

[0027] Preferably, the training module includes: The first partitioning submodule is used to divide the dates corresponding to the second historical load data in the training set into a first historical date and a first predicted date. The training submodule is used to train the initial model by using the second historical load data in the training set corresponding to the first historical date as the input layer training sample of the initial model and the second historical load data in the training set corresponding to the first prediction date as the output layer training sample of the initial model, so as to obtain the trained initial model.

[0028] Preferably, the verification module includes: The second partitioning submodule is used to divide the dates corresponding to the second historical load data in the test set into second historical dates and second predicted dates. The acquisition submodule is used to take the second historical load data in the test set corresponding to the second historical date as the input of the trained initial model, so that each deep learning model based on the self-attention mechanism in the trained initial model can predict the predicted load of the second prediction date. The calculation submodule is used to calculate the mean absolute error based on all the predicted loads and the second historical load data within the second prediction date; The verification submodule is used to verify the initial model as the load prediction model if the mean absolute error meets the preset conditions; otherwise, the verification fails, the AdamW optimizer is used to adjust the learning rate of the initial model, and the initial model with the adjusted learning rate is retrained until the verification is successful.

[0029] Preferably, the deep learning model based on the self-attention mechanism includes: an embedding layer, a multi-head self-attention layer, and a feedforward neural network layer; The embedding layer employs a positional encoding technique combining sine and cosine functions to extract features from the input data of the deep learning model based on a self-attention mechanism, thereby obtaining a feature vector. The multi-head self-attention layer uses a multi-head self-attention mechanism to calculate the attention weights of the feature vectors and uses the attention weights to quantify the degree of influence of the input data on the current prediction. Based on the degree of influence of the input data on the current prediction, the feedforward neural network layer uses Dropout regularization to obtain the prediction result.

[0030] Preferably, the embedding layer includes: The first acquisition module is used to map the timing information of the input data to the polar coordinate representation of the unit circle using a position encoding technique combining sine and cosine functions, thereby obtaining the encoded input data. The second acquisition module is used to normalize the continuous data value features in the encoded input data and map the discrete time features in the encoded historical load-related data to... d - A dense vector is used to obtain the feature vector.

[0031] Preferably, the multi-head self-attention layer includes: The generation module is used to perform a linear transformation on the feature vector to generate a query vector, a key vector, and a value vector; The first calculation module is used to calculate the attention weight of the feature vector using the query vector, key vector, and value vector; The second calculation module is used to calculate the degree of influence of the input data on the current prediction using the attention weights.

[0032] Preferably, the feedforward neural network layer includes: The allocation module is used to assign different Monte Carlo dropout rates to the feedforward neural network layers of each self-attention-based deep learning model using a differentiated Dropout strategy. The third acquisition module is used to take the influence of the input data on the current prediction as the input of the feedforward neural network layer, and the feedforward neural network layer uses Dropout regularization to obtain the prediction result.

[0033] Preferably, the formula for calculating the query vector includes:

[0034] The formula for calculating the key vector includes:

[0035] The formula for calculating the value vector includes:

[0036] In the above formula, , T To predict the total time; Q For query vector, K For key vectors, V For value vectors, h t For the predicted time t The feature vectors are fed into the Transformer. W Q for Q The corresponding weight vector, W K for K The corresponding weight vector, W V for V The corresponding weight vector.

[0037] Preferably, the formula for calculating the attention weight includes:

[0038] In the above formula, , T To predict the total time; , This is a pivotal moment in history. To predict the time t Historical moment i The importance weight of it, i.e., the attention weight; For the predicted time t Features For a historic moment i The characteristic is that T is the transpose. For each head in the attention mechanism, For normalized exponential functions, =Attention(Q,K,V), where Attention(Q,K,V) is the result of attention calculation.

[0039] Preferably, the formula for calculating the impact of historical loads on current forecasts includes:

[0040] In the above formula, To predict the time t Historical moment i Its importance weight, O The extent to which the historical load affects the current forecast. V It is a value vector.

[0041] According to a third aspect of the present invention, an electronic device is provided, comprising: at least one processor and a memory; the memory and the processor are connected via a bus; The memory is used to store one or more programs; When the one or more programs are executed by the at least one processor, the multi-model integrated short-term load forecasting method for distribution networks is implemented.

[0042] According to a fourth aspect of the present invention, a readable storage medium is provided having an executable program stored thereon, which, when executed, implements the multi-model integrated distribution network short-term load forecasting method.

[0043] The technical solution provided by this invention has the following beneficial effects: This invention provides a multi-model integrated method and apparatus for short-term load forecasting of distribution networks, comprising: collecting first historical load data of the distribution network; predicting multiple initial distribution network loads at the forecast time based on the first historical load data using a pre-established load forecasting model; and determining the final distribution network load at the forecast time using the multiple initial distribution network loads. The load forecasting model is constructed using multiple deep learning models based on a self-attention mechanism. The technical solution provided by this invention achieves high-precision and highly robust short-term load forecasting of distribution networks, meets the needs of real-time dispatching of distribution networks, and provides strong technical support for the optimized operation of smart distribution networks. Attached Figure Description

[0044] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0045] Figure 1 This is a flowchart of a multi-model integrated short-term load forecasting method for distribution networks provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of the model construction and training process provided in the embodiments of the present invention; Figure 3 This is a schematic diagram of the model prediction process provided in an embodiment of the present invention; Figure 4 This is a structural block diagram of a multi-model integrated distribution network short-term load forecasting device provided in an embodiment of the present invention; Figure 5 This is a structural block diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0046] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the following embodiments are only some embodiments of this invention, not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0047] Example 1 This invention provides a multi-model integrated method for short-term load forecasting in distribution networks, such as... Figure 1 As shown, it includes the following steps: Step 1: Collect the first historical load data of the distribution network; Step 2: Based on the first historical load data, use the pre-established load forecasting model to predict multiple initial distribution network loads at the forecast time; Step 3: Determine the final distribution network load for the forecast time using multiple initial distribution network loads; The load prediction model is constructed using multiple deep learning models based on self-attention mechanisms.

[0048] Further, step 2 includes: Step 21: Using the first historical load data as input to each deep learning model based on the self-attention mechanism in the load prediction model, multiple initial distribution network loads at the prediction time are predicted.

[0049] Further, step 3 includes: Step 31: Calculate the average load forecast using multiple initial distribution network loads. The average load forecast is the final distribution network load.

[0050] Furthermore, the process of establishing a load forecasting model includes: Step 41: Integrate multiple deep learning models based on self-attention mechanisms to obtain the initial model; Step 42: Collect the second historical load data of the distribution network; Step 43: Divide the second historical load data into a training set and a test set; Step 44: Train the initial model using the training set to obtain the trained initial model; Step 45: Validate the trained initial model using the test set to obtain the load prediction model.

[0051] This invention constructs an intelligent fusion framework that considers multi-scale spatiotemporal correlations, deeply mines the nonlinear coupling relationships between data, and improves the accuracy of load forecasting.

[0052] Further, step 44 includes: Step 441: Divide the dates corresponding to the second historical load data in the training set into first historical dates and first predicted dates; Step 441: Use the second historical load data in the training set corresponding to the first historical date as the input layer training sample of the initial model, and use the second historical load data in the training set corresponding to the first prediction date as the output layer training sample of the initial model to train the initial model and obtain the trained initial model.

[0053] Further, step 45 includes: Step 451: Divide the dates corresponding to the second historical load data in the test set into second historical dates and second predicted dates; Step 452: Use the second historical load data in the test set corresponding to the second historical date as the input of the trained initial model, so that each deep learning model based on the self-attention mechanism in the trained initial model can predict the predicted load of the second prediction date. Step 453: Calculate the mean absolute error based on all predicted loads and the second historical load data within the second predicted date; Step 454: If the mean absolute error meets the preset conditions, the verification is successful and the trained initial model is the load prediction model; otherwise, the verification fails. The AdamW optimizer is used to adjust the learning rate of the initial model, and the initial model with the adjusted learning rate is retrained until the verification is successful.

[0054] It should be noted that the calculation method of the "mean absolute error" involved in this invention is well known to those skilled in the art, therefore, its specific implementation will not be described in detail.

[0055] Furthermore, deep learning models based on self-attention mechanisms include: embedding layers, multi-head self-attention layers, and feedforward neural network layers; The embedding layer employs a positional encoding technique combining sine and cosine functions to extract features from the input data of the self-attention-based deep learning model, thereby obtaining feature vectors. The multi-head self-attention layer uses a multi-head self-attention mechanism to calculate the attention weights of the feature vectors and uses the attention weights to quantify the degree of influence of the input data on the current prediction. Based on the degree of influence of the input data on the current prediction, the feedforward neural network layer uses Dropout regularization to obtain the prediction result.

[0056] Furthermore, the embedding layer employs a positional encoding technique combining sine and cosine functions to extract features from the input data of the self-attention-based deep learning model, obtaining feature vectors, including: Step 511: Using a positional encoding technique combining sine and cosine functions, the temporal information of the input data is mapped to the polar coordinate representation of the unit circle to obtain the encoded input data; Step 512: Normalize the continuous data value features in the encoded input data, and map the discrete time features in the encoded historical load-related data to... d -Dense vectors are used to obtain eigenvectors.

[0057] Furthermore, the multi-head self-attention layer employs a multi-head self-attention mechanism to calculate the attention weights of the feature vectors and uses these attention weights to quantify the influence of the input data on the current prediction, including: Step 521: Perform a linear transformation on the feature vector to generate the query vector, key vector, and value vector; Specifically, the formula for calculating the query vector includes:

[0058] The formula for calculating the key vector includes:

[0059] The formulas for calculating value vectors include:

[0060] In the above formula, , To predict the total time; Q For query vector, K For key vectors, V For value vectors, h t For the predicted time t The feature vectors are fed into the Transformer. W Q for Q The corresponding weight vector, WK for K The corresponding weight vector, W V for V The corresponding weight vector; Step 522: Calculate the attention weights of the feature vectors using the query vector, key vector, and value vector; Specifically, the formula for calculating attention weights includes:

[0061] In the above formula, , To predict the total time; , This is a pivotal moment in history. To predict the time t Historical moment i The importance weight of it, i.e., the attention weight; For the predicted time t Features For a historic moment i The characteristic is that T is the transpose. For each head in the attention mechanism, For normalized exponential functions, =Attention(Q,K,V), where Attention(Q,K,V) is the result of attention calculation; Step 523: Calculate the degree of influence of the input data on the current prediction using attention weights; Specifically, the formula for calculating the impact of historical loads on current forecasts includes:

[0062] In the above formula, , To predict the total time; , This is a pivotal moment in history. To predict the time t Historical moment i Its importance weight, O To determine the extent to which historical loads influence current forecasts, V It is a value vector.

[0063] Furthermore, based on the degree of influence of the input data on the current prediction, the feedforward neural network layer uses Dropout regularization to obtain the prediction results, including: Step 531: Employ a differentiated Dropout strategy to assign different Monte Carlo dropout rates to the feedforward neural network layers of each deep learning model based on the self-attention mechanism; Step 532: Use the degree of influence of the input data on the current prediction as the input to the feedforward neural network layer, and use Dropout regularization in the feedforward neural network layer to obtain the prediction result.

[0064] This invention provides a multi-model integrated method for short-term load forecasting in distribution networks, which solves a key technical bottleneck in current short-term load forecasting technologies for distribution networks. (1) To address the problem that existing models are not adaptable to complex load features, a feature extraction architecture based on Transformer is constructed, which uses its self-attention mechanism to dynamically capture nonlinear temporal dependence and spatial correlation features in the load sequence, thereby significantly improving the model's ability to model complex scenarios such as load fluctuations and seasonal changes. (2) To address the problem of insufficient fusion of multi-source heterogeneous data, a multimodal data embedding layer and a cross-modal attention fusion module are designed to realize deep feature interaction and collaborative computing of multi-dimensional data such as meteorological factors, electricity price signals, and user behavior, thus overcoming the limitations of information loss caused by simple data splicing in traditional methods.

[0065] Example 2 To further illustrate the aforementioned multi-model integrated short-term load forecasting method for distribution networks, this invention provides a specific example, including: First, constructing a Transformer-based embedding layer, with temporal location encoding used to extract the periodic characteristics of the load; second, employing a multi-head self-attention mechanism to establish multi-dimensional temporal dependencies, and quantifying the impact of historical loads on the current forecast through a dynamic weight matrix; then, introducing a differentiated Dropout strategy into the feedforward network to train multiple sub-models, achieving model integration; finally, simulating the Monte Carlo process through multiple forward propagations in the forecasting phase to accurately capture the random fluctuation characteristics of composite loads. This method significantly improves forecast accuracy through feature fusion and model integration optimization. Specifically, it is divided into two parts: the model building and training phase and the forecasting phase. The key steps of the model building and training phase are as follows: Figure 2 As shown, the key steps in the prediction phase are as follows: Figure 3 As shown in the figure, Embedding is the embedding layer, MHA is the multi-head attention layer, FFN is the feedforward neural network layer, and Dropout is the regularization layer.

[0066] (1) The model building and training phase includes the following key steps: Step a: Building the embedding layer 1) Sine-Cosine Position Encoding: A positional encoding technique combining sine and cosine functions is employed to map time series information to the polar coordinate representation of a unit circle, explicitly preserving the periodicity of the time series and enhancing the model's sensitivity to time series location.

[0067] The model directly processes continuous numerical features (such as electricity consumption) through normalization or linear layers, preserving their numerical sensitivity; for discrete-time features (such as weekdays, months, holidays, etc.) and category features (such as user type, region, etc.), it maps them to... d -Dense vectors are used to learn semantic associations, and periodic encoding is combined to strengthen temporal priors. Embedded layers explicitly model temporal patterns and category semantics, and are co-optimized with continuous features to enhance the model's ability to represent temporal dependencies and external covariates, thereby improving prediction performance.

[0068] Given timing input data with a time length of T The sequence, time t The original feature vector It can be represented as: (1) In the above formula, For time sequence t The corresponding year sequence date represents the cumulative day sequence number of that day within the current year; Representing continuous eigenvectors, For a moment t Continuous eigenvectors, n For feature dimension, For the feature space.

[0069] variable day This variable exhibits periodic characteristics and is classified as a discrete, ordered categorical variable. Directly using numerical encoding may fail to effectively represent its periodicity, leading to incorrect modeling of the ordinal relationships of the features. To address this issue, a sine-cosine positional encoding is introduced, mapping the time-series information to polar coordinates on a unit circle. This explicitly preserves the periodicity of the time series, enabling the model to capture the seasonal influences and cyclical patterns of different dates within a year. (2) In the above formula, The vector is the result of sin-cos encoding the variable day.

[0070] 2) Data embedding layer construction: The design incorporates a fusion embedding layer that embeds numerical and periodic features of the load data in a high-dimensional manner to form a unified feature representation, thereby enhancing the model's ability to capture information from multiple sources.

[0071] Original continuous features By using a linear embedding layer for unified projection, discrete features are unified into the same vector space for subsequent self-attention calculations. , (3) After integration, at any time t The input can be represented as: (4) In the above formula, Indicates feature splicing, h t It is a moment t The feature vectors are fed into the Transformer. For embedding features, This is the weight matrix of the Embedding layer. For original continuous features, For the bias term of the Embedding layer, for Feature space, for[ e day , e cont Feature space, for e day Dimensions is the embedding dimension used for continuous features.

[0072] Step b: Self-attention temporal modeling By stacking multi-head self-attention layers, the model adaptively learns the long-term dependence and short-term fluctuations of the load sequence and constructs a dynamic time-dependent weight matrix, thereby enhancing the model's ability to model complex time-series patterns.

[0073] 1) Linear transformation: Input embedding h t Perform a linear transformation to generate a query vector. Q Key vector K Sum value vector V : , , (5) In the above formula, W Q , W K and W V It is a learnable weight matrix. W Q for Q The corresponding learnable weight matrix, W K for K The corresponding learnable weight matrix, W V for VThe corresponding learnable weight matrix.

[0074] 2) Attention weight calculation: Self-attention mechanisms achieve feature representation by calculating the interaction weights between elements within a sequence: (6) In the above formula, Q ( Query ) represents the characteristics of the current prediction time; K ( Key This represents the characteristics of each moment in history; V ( Value This represents the electricity consumption characteristics at each historical moment. For the attention calculation results, for K Dimensions softmax ( ) is the normalized exponential function.

[0075] Attention weights quantify the correlation strength between different positions in a sequence, and their calculation process is described as follows: (7) In the above formula, , To predict the total time; , This is a pivotal moment in history. α t,i This indicates that the current moment is being predicted. t Historical moment i Its importance weight, For a moment t The query vector, For the first i A vector of values.

[0076] 3) Weighted summation: Based on attention weight α t,i , value vector V Perform a weighted summation to obtain the output. O : (8) The Transformer model captures multidimensional temporal dependencies in load forecasting through its self-attention mechanism, quantifies the dynamic impact of historical observations at different times on the current load, and constructs time-dependent weight matrices with long and short cycles. Through feature space mapping, it identifies seasonal patterns (such as the year-round correlation between summer load and historical summer data for the same period) and periodic patterns (such as the weekly cycle pattern of weekend load characteristics). Furthermore, its multi-head attention architecture effectively separates and characterizes the nonlinear impact of special events (such as extreme weather, holidays, and other abnormal operating conditions) on the load, enabling hierarchical analysis of load temporal characteristics.

[0077] Step c: Feature mapping and regularization 1) Feedforward neural network mapping: By using a feedforward neural network to perform a nonlinear transformation on the time-series representation output by the self-attention mechanism, and mapping it to the feature space of the load forecasting task, high-precision short-term load forecasting can be achieved.

[0078] Feedforward neural networks employ a "dilation-compression" architecture through location-aware feedforward computation. They achieve feature decoupling via a high-dimensional latent space, introduce higher-order interaction terms through the ReLU nonlinear activation function, and finally project back into the original space to complete feature recombination. This structural characteristic enables them to model complex nonlinear dynamics in load forecasting, including: 1. the superposition effects of multi-period seasonality and sudden events; 2. the cross-modal coupling relationship between covariates (such as meteorological parameters, economic indicators, etc.) and historical loads; and 3. the non-stationarity and state-switching behavior of load curves.

[0079] (9) In the above formula, This represents the hidden vector after self-attention (containing global historical weighted information); Indicates the weight of the first fully connected component; b 1. b 2 indicates bias; Indicates ReLU activation; This represents the weight of the second fully connected layer. For the output of the feedforward neural network layer, The hidden vector after self-attention. for x Dimensions For the output dimension of the FFN layer, For the FFN layer weight space, This represents the weight space of the FFN layer.

[0080] 2) Dropout regularization: A regularization strategy is adopted by adding a Dropout layer after the ReLU activation function of the feedforward neural network. This strategy uses probability...p Randomly masking the output of some neurons effectively reduces the co-adaptation among neurons, thereby suppressing the model's overfitting tendency and enhancing its generalization ability.

[0081] (10) In the above formula, p It is the discard rate (usually 0 < 0). p <1); m i It is a Bernoulli random variable. ~ ,Right now m i = The probability of 1 is 1- p (Preserving neurons) m i = The probability of 0 is p (Discard neurons); Denominator 1- p It is a scaling factor used to keep the expected value of the output constant (during training). Output the results for the dropout layer. For dropout i One input value, i For serial numbers.

[0082] Step d: Model optimization and ensemble training 1) Loss function optimization: By using mean absolute error as the training loss function, the impact of outliers on model training is reduced, thereby enhancing prediction robustness.

[0083] Mean absolute error (MAE) is used to evaluate model performance by calculating the arithmetic mean of the absolute deviations between predicted and observed values. Its mathematical definition can be expressed as: (11) In the above formula, n For the sample size, y i For the first i A true value, For the first i There are 10 predicted values, and MAE is the mean absolute error.

[0084] 2) AdamW optimization strategy: The AdamW optimizer is used to dynamically adjust the learning rate, and the model parameters are optimized by combining the weight decay strategy, thereby improving training stability and convergence efficiency.

[0085] (12) In the above formula, For a momentt Model parameters, The gradient of the loss function. m t For a moment t The first moment estimate of the gradient, m t-1 For a moment t First-moment estimate of the gradient of -1 v t For a moment t The second moment estimate of the gradient, v t-1 For a moment t Second-moment estimate of the gradient of -1 β 1 represents the exponential decay coefficient used to calculate the first-moment estimate. β 2 represents the exponential decay coefficient used to calculate the second-order moment estimate. For a moment t The bias-corrected first-moment estimate For a moment t The bias-corrected second-order moment estimate, η For learning rate, λ This is the weight decay coefficient. To customize hyperparameters, To customize hyperparameters, These are the model parameters for the next time step. For custom hyperparameters.

[0086] 3) Model training: By setting a differentiated Dropout probability set { p 1, p 2, ..., p k This method trains multiple independent models in parallel, each using a fixed but different dropout rate, forming a group of sub-models required for the ensemble model in the prediction phase. By adjusting the Dropout probability, this method achieves differentiated control over model sparsity and learning ability, enabling each sub-model to learn diverse feature representations. Compared to traditional models with a single Dropout rate, this strategy maintains model simplicity while better mitigating overfitting and enhancing generalization performance.

[0087] Step e: The prediction phase includes the following key steps: 1) Monte Carlo discard rate integration: In the sub-model groups obtained during the training phase, multiple random forward propagations are performed on each test sample. By integrating models with different dropout rates and their diverse activation modes, model diversity enhancement and Monte Carlo uncertainty estimation are achieved. For each sub-model (with different dropout rates), Dropout is enabled. After a random forward propagation, the prediction set is obtained: (14) In the above formula, p For the discard rate, k The number of sub-models, p 1 represents the dropout rate of the first sub-model. p 2 represents the dropout rate of the second sub-model. p k For the first k The dropout rate of each sub-model This is a predicted value; To predict the number of times, That is, total prediction Second-rate.

[0088] Mean (Final Forecast): (15) Variance (uncertainty estimation): (16) In the above formula, This is the average value. for y The variance.

[0089] 2) Predictive stability analysis: Based on the statistical characteristics of multiple prediction results, the mean absolute error is used to quantitatively evaluate the model's predictive performance, and different... p The impact of the value on prediction stability.

[0090] (17) In the above formula, , This is a pivotal moment in history. n The total number of samples; y i For the first i The actual value of the load at any given time. For the first i Forecast values ​​of load at any given time. The mean absolute percentage error.

[0091] This invention provides a multi-model integrated method for short-term load forecasting in distribution networks. By combining the feature extraction capabilities of the Transformer model, multi-head self-attention mechanism, differentiated Dropout strategy, and Monte Carlo simulation forecasting, it significantly improves the accuracy and robustness of load forecasting. Specific effects are as follows: (1) Efficient fusion of features Traditional load forecasting methods typically handle temporal and spatial features separately, making it difficult to fully explore the complex relationships between distribution network nodes. This invention effectively solves the problem of feature-coupled modeling by constructing a Transformer-based embedding layer and combining it with time-location encoding to extract periodic and trend features from load data, thereby enhancing the model's ability to represent complex load patterns.

[0092] (2) Accurate modeling of multidimensional temporal dependencies Employing a multi-head self-attention mechanism, this invention enables parallel learning of load variation patterns across different time scales and quantifies the impact of historical load on current predictions through a dynamic weight matrix. This avoids the gradient vanishing or exploding problems caused by long-sequence training in traditional recurrent neural networks. Compared to traditional time-series forecasting methods (such as ARIMA and LSTM), this invention can more accurately capture the long-term and short-term dependencies of load data, significantly reducing prediction errors.

[0093] (3) Improved model generalization ability By introducing a differentiated dropout strategy into the feedforward network, multiple sub-models are trained by randomly dropping different combinations of neurons, effectively mitigating the overfitting problem and improving the model's generalization performance. Furthermore, the model ensemble strategy further enhances the stability of the predictions, enabling them to maintain high prediction accuracy even when facing different distribution network structures and load fluctuations.

[0094] (4) Accurate capture of random fluctuation characteristics In the forecasting phase, the Monte Carlo simulation method is employed to calculate the probability distribution of the forecast results through multiple forward propagations. This not only outputs the forecast values ​​but also quantifies the uncertainty of the forecasts. This characteristic is particularly suitable for composite load forecasting after the integration of distributed energy sources such as wind and solar power, and can more comprehensively reflect the random fluctuation characteristics of the load, providing more reliable data support for distribution network dispatching decisions.

[0095] (5) Significant practical application results Experimental results show that, compared with traditional load forecasting methods, the present invention reduces the mean absolute error and root mean square error by 15% to 30% on multiple real distribution network datasets, and significantly improves forecasting stability. Furthermore, due to the parallel computing characteristics of the Transformer, the model training and prediction efficiency are high, meeting the needs of real-time distribution network scheduling.

[0096] In summary, this invention achieves high-precision and robust short-term load forecasting for distribution networks through feature fusion, dynamic weight modeling, differentiated Dropout integration, and Monte Carlo uncertainty analysis, providing strong technical support for the optimized operation of smart distribution networks.

[0097] Example 3 This invention also provides a multi-model integrated short-term load forecasting device for distribution networks, such as... Figure 4 As shown, it includes: The data acquisition unit is used to collect the first historical load data of the distribution network; The prediction unit is used to predict multiple initial distribution network loads at the prediction time based on the first historical load data and using a pre-established load prediction model. A determination unit is used to determine the final distribution network load at the predicted time using multiple initial distribution network loads; The load prediction model is constructed using multiple deep learning models based on self-attention mechanisms.

[0098] Furthermore, the prediction unit is specifically used for: Using the first historical load data as input to each deep learning model based on the self-attention mechanism in the load prediction model, multiple initial distribution network loads at the prediction time are predicted.

[0099] Furthermore, the unit is defined, specifically for: Using multiple initial distribution network loads, the average load forecast is calculated, and the average load forecast is the final distribution network load.

[0100] Furthermore, the device also includes: a modeling unit for establishing a load forecasting model; the modeling unit includes: The integration module is used to integrate multiple deep learning models based on the self-attention mechanism to obtain an initial model; The data acquisition module is used to collect the second historical load data of the power distribution network. The partitioning module is used to divide the second historical load data into training and test sets; The training module is used to train the initial model using the training set to obtain the trained initial model; The validation module is used to validate the trained initial model using a test set to obtain the load prediction model.

[0101] Furthermore, the training module includes: The first partitioning submodule is used to divide the dates corresponding to the second historical load data in the training set into the first historical date and the first predicted date. The training submodule is used to train the initial model by using the second historical load data in the training set corresponding to the first historical date as the input layer training sample of the initial model and the second historical load data in the training set corresponding to the first prediction date as the output layer training sample of the initial model, so as to obtain the trained initial model.

[0102] Furthermore, the verification module includes: The second partitioning submodule is used to divide the dates corresponding to the second historical load data in the test set into the second historical date and the second predicted date. The acquisition submodule is used to take the second historical load data in the test set corresponding to the second historical date as the input of the trained initial model, so that each deep learning model based on the self-attention mechanism in the trained initial model can predict the predicted load of the second prediction date. The calculation submodule is used to calculate the mean absolute error based on all the predicted loads and the second historical load data within the second prediction date. The validation submodule is used to verify the model if the mean absolute error meets the preset conditions. If the verification is successful, the trained initial model is the load prediction model. Otherwise, if the verification fails, the AdamW optimizer is used to adjust the learning rate of the initial model, and the initial model with the adjusted learning rate is retrained until the verification is successful.

[0103] Furthermore, deep learning models based on self-attention mechanisms include: embedding layers, multi-head self-attention layers, and feedforward neural network layers; The embedding layer employs a positional encoding technique combining sine and cosine functions to extract features from the input data of the self-attention-based deep learning model, thereby obtaining feature vectors. The multi-head self-attention layer uses a multi-head self-attention mechanism to calculate the attention weights of the feature vectors and uses the attention weights to quantify the degree of influence of the input data on the current prediction. Based on the degree of influence of the input data on the current prediction, the feedforward neural network layer uses Dropout regularization to obtain the prediction result.

[0104] Furthermore, the embedding layer includes: The first acquisition module is used to map the timing information of the input data to the polar coordinate representation of the unit circle using a position encoding technique combining sine and cosine functions, thereby obtaining the encoded input data. The second acquisition module is used to normalize the continuous data value features in the encoded input data and map the discrete time features in the encoded historical load-related data to... d -Dense vectors are used to obtain eigenvectors.

[0105] Furthermore, the bullish self-attention layer includes: The generation module is used to perform linear transformations on the feature vectors to generate query vectors, key vectors, and value vectors; The first calculation module is used to calculate the attention weights of the feature vector using the query vector, key vector, and value vector. The second calculation module is used to calculate the degree of influence of the input data on the current prediction using attention weights.

[0106] Furthermore, the feedforward neural network layer includes: The allocation module is used to assign different Monte Carlo dropout rates to the feedforward neural network layers of each self-attention-based deep learning model using a differentiated Dropout strategy. The third acquisition module is used to take the degree of influence of the input data on the current prediction as the input of the feedforward neural network layer, and the feedforward neural network layer uses Dropout regularization to obtain the prediction result.

[0107] Furthermore, the formula for calculating the query vector includes:

[0108] The formula for calculating the key vector includes:

[0109] The formulas for calculating value vectors include:

[0110] In the above formula, , To predict the total time; Q For query vector, K For key vectors, V For value vectors, h t For the predicted time t The feature vectors are fed into the Transformer. W Q for Q The corresponding weight vector, W K for K The corresponding weight vector, W V for V The corresponding weight vector.

[0111] Furthermore, the formula for calculating attention weights includes:

[0112] In the above formula, , To predict the total time; , This is a pivotal moment in history. To predict the time t Historical moment i The importance weight of it, i.e., the attention weight; For the predicted time t Features For a historic moment iThe characteristic is that T is the transpose. For each head in the attention mechanism, For normalized exponential functions, =Attention(Q,K,V), where Attention(Q,K,V) is the result of attention calculation.

[0113] Furthermore, the formula for calculating the extent to which historical loads influence current forecasts includes:

[0114] In the above formula, , To predict the total time; , This is a pivotal moment in history. To predict the time t Historical moment i Its importance weight, O To determine the extent to which historical loads influence current forecasts, V It is a value vector.

[0115] It is understood that the device embodiments provided above correspond to the method embodiments described above, and the specific details can be referred to each other, which will not be repeated here.

[0116] It is understood that the same or similar parts in the above embodiments can be referred to each other, and the contents not described in detail in some embodiments can be referred to the same or similar contents in other embodiments.

[0117] Example 4 like Figure 5 As shown, the present invention also provides an electronic device, which may be a computer device, a microcontroller device, a smart mobile device, etc. The electronic device in this embodiment may include a processor, a memory, a transceiver component, etc. The memory, processor, and transceiver component are connected via a bus; the memory can be used to store executable programs, and an exemplary executable program may include instructions; the processor is used to execute the instructions stored in the memory. The memory can also be used to store data, which can be accessed and / or modified when instructions are executed.

[0118] The processor may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It is the computing and control core of the terminal, and is suitable for implementing one or more instructions. Specifically, it is suitable for loading and executing one or more instructions in the storage medium to implement the corresponding method flow or corresponding function, so as to realize the steps of the multi-model integrated distribution network short-term load forecasting method in the above embodiments.

[0119] Example 5 Based on the same inventive concept, this invention also provides a readable storage medium, specifically an electronic device readable storage medium (Memory). This readable storage medium is a memory device within an electronic device used to store programs and data. It is understood that the storage medium here can include both built-in storage media within the electronic device and extended storage media supported by the electronic device. The storage medium provides storage space, which stores the terminal's operating system. Furthermore, this storage space also stores one or more instructions suitable for loading and execution by a processor. These instructions can be one or more executable programs (including program code). It should be noted that the storage medium here can be high-speed RAM or non-volatile memory, such as at least one disk storage device. Loading and executing one or more instructions stored in the storage medium by the processor can implement the steps of the multi-model integrated distribution network short-term load forecasting method described in the above embodiments.

[0120] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0121] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0122] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0123] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0124] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A multi-model integrated method for short-term load forecasting of distribution networks, characterized in that, include: Collect the first historical load data of the power distribution network; Based on the first historical load data, multiple initial distribution network loads at the prediction time are predicted using a pre-established load prediction model. The final distribution network load for the predicted time is determined using the multiple initial distribution network loads. The load prediction model is constructed using multiple deep learning models based on self-attention mechanisms.

2. The method according to claim 1, characterized in that, The step of predicting multiple initial distribution network loads at the prediction time based on the first historical load data using a pre-established load forecasting model includes: Using the first historical load data as input to each deep learning model based on the self-attention mechanism in the load prediction model, multiple initial distribution network loads at the prediction time are predicted.

3. The method according to claim 1, characterized in that, The process of determining the final distribution network load for the predicted time using the plurality of initial distribution network loads includes: Using the multiple initial distribution network loads, the average load forecast is calculated, and the average load forecast is the final distribution network load.

4. The method according to claim 1, characterized in that, The process of establishing the load forecasting model includes: The multiple deep learning models based on the self-attention mechanism are integrated to obtain the initial model; Collect the second historical load data of the power distribution network; The second historical load data is divided into a training set and a test set; The initial model is trained using the training set to obtain the trained initial model; The initial model after training is validated using the test set to obtain the load prediction model.

5. The method according to claim 4, characterized in that, The step of training the initial model using the training set includes: The dates corresponding to the second historical load data in the training set are divided into first historical dates and first predicted dates; The initial model is trained by using the second historical load data in the training set corresponding to the first historical date as the input layer training sample and the second historical load data in the training set corresponding to the first prediction date as the output layer training sample, thereby obtaining the trained initial model.

6. The method according to claim 4, characterized in that, The validation of the trained initial model using the test set includes: The dates corresponding to the second historical load data in the test set are divided into second historical dates and second predicted dates; The second historical load data in the test set corresponding to the second historical date is used as the input of the trained initial model, so that each deep learning model based on the self-attention mechanism in the trained initial model can predict the predicted load of the second prediction date. The mean absolute error is calculated based on all the predicted loads and the second historical load data within the second prediction date; If the mean absolute error meets all preset conditions, the verification is successful, and the trained initial model is the load prediction model; otherwise, the verification fails, the learning rate of the initial model is adjusted using the AdamW optimizer, and the initial model with the adjusted learning rate is retrained until the verification is successful.

7. The method according to claim 1, characterized in that, The deep learning model based on the self-attention mechanism includes: an embedding layer, a multi-head self-attention layer, and a feedforward neural network layer; The embedding layer employs a positional encoding technique combining sine and cosine functions to extract features from the input data of the deep learning model based on a self-attention mechanism, thereby obtaining a feature vector. The multi-head self-attention layer uses a multi-head self-attention mechanism to calculate the attention weights of the feature vectors and uses the attention weights to quantify the degree of influence of the input data on the current prediction. Based on the degree of influence of the input data on the current prediction, the feedforward neural network layer uses Dropout regularization to obtain the prediction result.

8. The method according to claim 7, characterized in that, The embedding layer employs a positional encoding technique combining sine and cosine functions to extract features from the input data of the self-attention-based deep learning model, obtaining a feature vector, including: A positional encoding technique combining sine and cosine functions is used to map the temporal information of the input data to the polar coordinate representation of a unit circle, thereby obtaining the encoded input data. The continuous data value features in the encoded input data are normalized, and the discrete time features in the encoded historical load-related data are mapped to... d - A dense vector is used to obtain the feature vector.

9. The method according to claim 7, characterized in that, The multi-head self-attention layer uses a multi-head self-attention mechanism to calculate the attention weights of the feature vectors and uses these attention weights to quantify the influence of the input data on the current prediction, including: Perform a linear transformation on the feature vector to generate a query vector, a key vector, and a value vector; The attention weights of the feature vectors are calculated using the query vector, key vector, and value vector. The influence of the input data on the current prediction is calculated using the attention weights.

10. The method according to claim 7, characterized in that, The prediction result obtained by the feedforward neural network layer using Dropout regularization based on the influence of the input data on the current prediction includes: A differentiated Dropout strategy is adopted to assign different Monte Carlo dropout rates to the feedforward neural network layers of each deep learning model based on the self-attention mechanism; The influence of the input data on the current prediction is used as the input to the feedforward neural network layer, and the feedforward neural network layer uses Dropout regularization to obtain the prediction result.

11. The method according to claim 9, characterized in that, The formula for calculating the query vector includes: The formula for calculating the key vector includes: The formula for calculating the value vector includes: In the above formula, , To predict the total time; Q For query vector, K For key vectors, V For value vectors, h t For the predicted time t The feature vectors are fed into the Transformer. W Q for Q The corresponding weight vector, W K for K The corresponding weight vector, W V for V The corresponding weight vector.

12. The method according to claim 9, characterized in that, The formula for calculating the attention weight includes: In the above formula, , To predict the total time; , This is a pivotal moment in history. To predict the time t Historical moment i The importance weight of it, i.e., the attention weight; For the predicted time t Features For a historic moment i The characteristic is that T is the transpose. For each head in the attention mechanism, For normalized exponential functions, =Attention(Q,K,V), where Attention(Q,K,V) is the result of attention calculation.

13. The method according to claim 9, characterized in that, The formula for calculating the impact of historical loads on current forecasts includes: In the above formula, , To predict the total time; , This is a pivotal moment in history. To predict the time t Historical moment i Its importance weight, O The extent to which the historical load affects the current forecast. V It is a value vector.

14. A multi-model integrated short-term load forecasting device for distribution networks, characterized in that, include: The data acquisition unit is used to collect the first historical load data of the distribution network; The prediction unit is used to predict multiple initial distribution network loads at the prediction time based on the first historical load data and using a pre-established load prediction model. A determining unit is used to determine the final distribution network load for the predicted time using the plurality of initial distribution network loads; The load prediction model is constructed using multiple deep learning models based on self-attention mechanisms.

15. The apparatus according to claim 14, characterized in that, The prediction unit is specifically used for: Using the first historical load data as input to each deep learning model based on the self-attention mechanism in the load prediction model, multiple initial distribution network loads at the prediction time are predicted.

16. The apparatus according to claim 14, characterized in that, The determining unit is specifically used for: Using the multiple initial distribution network loads, the average load forecast is calculated, and the average load forecast is the final distribution network load.

17. The apparatus according to claim 14, characterized in that, It also includes: a setup unit for setting up the load forecasting model; the setup unit includes: An integration module is used to integrate the multiple deep learning models based on the self-attention mechanism to obtain an initial model; The data acquisition module is used to collect the second historical load data of the power distribution network. A partitioning module is used to divide the second historical load data into a training set and a test set; The training module is used to train the initial model using the training set to obtain the trained initial model; The verification module is used to verify the trained initial model using the test set to obtain the load prediction model.

18. The apparatus according to claim 17, characterized in that, The training module includes: The first partitioning submodule is used to divide the dates corresponding to the second historical load data in the training set into a first historical date and a first predicted date. The training submodule is used to train the initial model by using the second historical load data in the training set corresponding to the first historical date as the input layer training sample of the initial model and the second historical load data in the training set corresponding to the first prediction date as the output layer training sample of the initial model, so as to obtain the trained initial model.

19. The apparatus according to claim 17, characterized in that, The verification module includes: The second partitioning submodule is used to divide the dates corresponding to the second historical load data in the test set into second historical dates and second predicted dates. The acquisition submodule is used to take the second historical load data in the test set corresponding to the second historical date as the input of the trained initial model, so that each deep learning model based on the self-attention mechanism in the trained initial model can predict the predicted load of the second prediction date. The calculation submodule is used to calculate the mean absolute error based on all the predicted loads and the second historical load data within the second prediction date; The verification submodule is used to verify the initial model as the load prediction model if the mean absolute error meets the preset conditions; otherwise, the verification fails, the AdamW optimizer is used to adjust the learning rate of the initial model, and the initial model with the adjusted learning rate is retrained until the verification is successful.

20. The apparatus according to claim 14, characterized in that, The deep learning model based on the self-attention mechanism includes: an embedding layer, a multi-head self-attention layer, and a feedforward neural network layer; The embedding layer employs a positional encoding technique combining sine and cosine functions to extract features from the input data of the deep learning model based on a self-attention mechanism, thereby obtaining a feature vector. The multi-head self-attention layer uses a multi-head self-attention mechanism to calculate the attention weights of the feature vectors and uses the attention weights to quantify the degree of influence of the input data on the current prediction. Based on the degree of influence of the input data on the current prediction, the feedforward neural network layer uses Dropout regularization to obtain the prediction result.

21. The apparatus according to claim 20, characterized in that, The embedding layer includes: The first acquisition module is used to map the timing information of the input data to the polar coordinate representation of the unit circle using a position encoding technique combining sine and cosine functions, thereby obtaining the encoded input data. The second acquisition module is used to normalize the continuous data value features in the encoded input data and map the discrete time features in the encoded historical load-related data to... d - A dense vector is used to obtain the feature vector.

22. The apparatus according to claim 20, characterized in that, The multi-head self-attention layer includes: The generation module is used to perform a linear transformation on the feature vector to generate a query vector, a key vector, and a value vector; The first calculation module is used to calculate the attention weight of the feature vector using the query vector, key vector, and value vector; The second calculation module is used to calculate the degree of influence of the input data on the current prediction using the attention weights.

23. The apparatus according to claim 20, characterized in that, The feedforward neural network layer includes: The allocation module is used to assign different Monte Carlo dropout rates to the feedforward neural network layers of each self-attention-based deep learning model using a differentiated Dropout strategy. The third acquisition module is used to take the influence of the input data on the current prediction as the input of the feedforward neural network layer, and the feedforward neural network layer uses Dropout regularization to obtain the prediction result.

24. The apparatus according to claim 22, characterized in that, The formula for calculating the query vector includes: The formula for calculating the key vector includes: The formula for calculating the value vector includes: In the above formula, , To predict the total time; Q For query vector, K For key vectors, V For value vectors, h t For the predicted time t The feature vectors are fed into the Transformer. W Q for Q The corresponding weight vector, W K for K The corresponding weight vector, W V for V The corresponding weight vector.

25. The apparatus according to claim 22, characterized in that, The formula for calculating the attention weight includes: In the above formula, , To predict the total time; , This is a pivotal moment in history. To predict the time t Historical moment i The importance weight of it, i.e., the attention weight; For the predicted time t Features For a historic moment i The characteristic is that T is the transpose. For each head in the attention mechanism, For normalized exponential functions, =Attention(Q,K,V), where Attention(Q,K,V) is the result of attention calculation.

26. The apparatus according to claim 22, characterized in that, The formula for calculating the impact of historical loads on current forecasts includes: In the above formula, , To predict the total time; , This is a pivotal moment in history. To predict the time t Historical moment i Its importance weight, O The extent to which the historical load affects the current forecast. V It is a value vector.

27. An electronic device, characterized in that, include: At least one processor and memory; The memory and processor are connected via a bus; The memory is used to store one or more programs; When the one or more programs are executed by the at least one processor, the method for short-term load forecasting of distribution networks with multi-model integration as described in any one of claims 1 to 13 is implemented.

28. A readable storage medium, characterized in that, It contains an execution program, which, when executed, implements the multi-model integrated short-term load forecasting method for distribution networks as described in any one of claims 1 to 13.

Images