Attention mechanism and RIME optimized CNN-BiLSTM-based transformer area ultra-short-term load prediction method

Abstract

The invention discloses a transformer area ultra-short-term load prediction method based on an attention mechanism and RIME optimization CNN-BiLSTM, and the method comprises the steps: collecting transformer area load and meteorological sample data, and carrying out the abnormal value detection and normalization processing; extracting load information local features based on a CNN network; after local features of the extracted load information are output, a BiLSTM network is adopted to process a feature sequence to capture a time dependency relationship; dynamic weight distribution is carried out based on an Att mechanism, and the effect of key information in prediction is enhanced; the key hyper-parameters are optimized on the basis of an RIME algorithm, and an RIME-CNN-BiLSTM-Att model is constructed; and on the basis of an RIME-CNN-BiLSTM-Att model, ultra-short-term load prediction of the transformer area is carried out. According to the method, various factors can be comprehensively considered, data features are fully mined, a time sequence dependency relationship is effectively captured, and hyper-parameters are automatically optimized, so that the precision and reliability of ultra-short-term load prediction of the transformer area are improved, and safe and stable operation and economic dispatching of a power system are guaranteed.

Description

Technical Field

[0001] This invention relates to the field of power system load forecasting technology, specifically to a method for ultra-short-term load forecasting of distribution areas based on attention mechanism and RIME-optimized CNN-BiLSTM. Background Technology

[0002] Ultra-short-term load forecasting refers to predicting load changes over the next few minutes to hours. In power grid operation, ultra-short-term load forecasting helps dispatchers provide timely and accurate information on load changes, enabling rapid adjustments to generation plans and ensuring a balance between supply and demand. Improved forecast accuracy significantly enhances the stability of power grid operation and the economic efficiency of resource allocation.

[0003] With the development of artificial intelligence technology, deep learning-based load forecasting methods have become mainstream. Existing solutions often employ combined models to integrate the advantages of different structures. For example, convolutional neural networks (CNNs) are used to extract local spatial features of load data; bidirectional long short-term memory networks (BiLSTM) are used to capture long-term and short-term bidirectional dependencies in time series data; and attention mechanisms are introduced to dynamically weight key feature information, thereby improving the model's expressive power. To further improve model performance, researchers typically introduce intelligent optimization algorithms to tune hyperparameters such as learning rate and number of network units, for example, using particle swarm optimization or gray wolf optimization algorithms.

[0004] The existing technical solutions mentioned above still have significant defects and limitations in practical applications. Although the existing models are structurally complex, they fail to fully analyze the deep correlation between load variation characteristics and the internal mechanisms of the models. Most methods focus on the stacking and combination of models, lacking mechanistic analysis of how the complex characteristics of load sequences, such as non-stationarity, abrupt changes, and periodicity, affect model feature extraction and weight allocation. This results in insufficient generalization ability of the models when facing complex and variable load scenarios in actual power grid operation, and the stability and accuracy of prediction results are difficult to reliably guarantee. Secondly, model performance is highly dependent on hyperparameters, and traditional optimization methods are inefficient and have inherent defects, which seriously restricts their engineering applicability. Currently commonly used parameter optimization algorithms (such as the aforementioned particle swarm optimization and gray wolf optimization algorithms) are prone to getting trapped in local optima during the search process, exhibiting premature convergence problems and making it difficult to achieve global optimization. At the same time, the parameter tuning process of these algorithms often relies on human experience, requiring a large number of repetitive and time-consuming experiments, with low automation and adaptability. This leads to difficulties in parameter tuning, long optimization cycles, and poor portability when deploying existing forecasting models on load data from different regions and seasons, making it difficult to meet the real-time and efficient engineering application requirements of power systems. Therefore, how to construct a load forecasting model that can deeply integrate load time-series characteristics and adaptively configure parameters through efficient and robust optimization methods to solve the bottleneck problems of limited forecasting accuracy, strong parameter dependence, and low optimization efficiency in existing technologies has become a technical challenge that urgently needs to be overcome in this field. Summary of the Invention

[0005] To address the aforementioned technical problems, this invention provides a method for ultra-short-term load forecasting of distribution areas based on attention mechanism and RIME-optimized CNN-BiLSTM. This method can comprehensively consider multiple factors, fully mine data features, effectively capture temporal dependencies, and automatically optimize hyperparameters, thereby improving the accuracy and reliability of ultra-short-term load forecasting of distribution areas and ensuring the safe, stable operation and economical dispatch of the power system.

[0006] The technical solution adopted in this invention is as follows: A method for ultra-short-term load prediction in transformer substations based on attention mechanisms and RIME-optimized CNN-BiLSTM includes the following steps: Step 1: Collect load and meteorological sample data for the transformer area, and perform outlier detection and normalization. Step 2: Extract local features of the load information based on the CNN network; Step 3: After extracting local features of the load information based on Step 2, use a BiLSTM network to process the feature sequences and capture time dependencies; Step 4: Dynamic weight allocation based on the Att mechanism to enhance the role of key information in prediction; Step 5: Optimize key hyperparameters based on the RIME algorithm and construct the RIME-CNN-BiLSTM-Att model; Step 6: Perform ultra-short-term load forecasting for the transformer area based on the RIME-CNN-BiLSTM-Att model.

[0007] In step 1, due to reasons such as data acquisition equipment failure and data storage problems, some abnormal conditions may occur in the power load and its related characteristic data. These conditions will cause some abnormal data, which will directly interfere with the accuracy of model prediction. Therefore, it is necessary to remove outliers and impute missing values ​​in the sample data.

[0008] This invention employs a hybrid strategy combining correction and analytical methods for data cleaning. First, outliers significantly deviating from the normal fluctuation range are identified using the 3σ principle based on statistical distribution in analytical analysis. Then, for the identified outliers and missing values ​​in the original data, denoted as NaN or Null, a time-series horizontal replacement method is uniformly used to fill them in. Specifically: For a time Abnormal or missing data points Its correction value Determined by the following formula: (1); That is, the observation value at the same time on the previous day is used for replacement.

[0009] If the corresponding time of the previous day If the data is also unavailable, abnormal, or missing, then the search continues recursively through earlier historical data from the same period until a valid, usable value is found. This rule can be formally expressed as: (2); in, It is a positive integer. And satisfy These are normal, non-missing values. The search limit can be set according to the actual situation, such as k. max =7, meaning it can trace back a maximum of one week.

[0010] To eliminate the dimensional differences between electricity load (unit: kilowatt) and various meteorological characteristics (such as temperature and humidity, which have different units and magnitudes), and to accelerate the gradient descent convergence process of subsequent deep learning models, all features are subjected to Min-Max normalization, linearly mapping them to the (0,1) interval. For any feature sequence {x} in the dataset... i Its normalization formula is: (3); In equation (3), Indicates the first The result after normalization of each sample; and These represent the maximum and minimum values ​​of the feature in the entire training dataset, respectively. The processed sample data needs to be constructed into an input format suitable for time series prediction models. This invention organizes the data into a three-dimensional tensor. The specific reconstruction process is as follows: Suppose there are N total samples, each of which uses historical data from the past T consecutive time steps (time step lengths) to predict the load for one or more future time steps. Each time step contains F features (such as load value, temperature, humidity, etc.).

[0011] One-dimensional time-series data is sampled using a sliding window of fixed length T to generate multiple consecutive sequence segments. The final input tensor shape is the number of samples, time step, and feature dimension, i.e., N, T, F.

[0012] Where: Number of samples (N): The total number of sequence segments generated by the sliding window; Time step (T): The historical length of each input sequence, representing the historical range observed by the model; Feature dimension (F): The number of feature variables observed at each time point.

[0013] This three-dimensional tensor serves as the standard input to the constructed RIME-CNN-BiLSTM-Att model for training and prediction.

[0014] In step 2, the CNN network possesses superior local feature extraction capabilities, demonstrating outstanding performance in load forecasting for power systems. In this invention, the CNN network processes data through convolutional layers, extracting useful information from the raw input for feature representation, and automatically capturing local features in the load data, as detailed below: 2.1: This invention utilizes convolutional layers to learn features from an input three-dimensional tensor. The input data can be viewed as a tensor of shape (N,T,F). The convolutional layer slides along the time axis using multiple one-dimensional convolutional kernels (filters) of different sizes to capture the local dependencies and nonlinear patterns of the load sequence at different time scales.

[0015] For an input feature sequence , A two-dimensional matrix representing the time step T and feature dimension F; after passing through a convolution kernel of size k with output channels of size k, ... One-dimensional convolution kernel After the operation with bias b, the output feature map Z is obtained. tThe value of the c-th channel and the j-th time position It is calculated using the following convolution formula: (4); In equation (4), This represents the weight parameters of the convolution kernel at the m-th time offset, the n-th input feature channel, and the corresponding c-th output channel; This represents the value of the input sequence at time position j+m and feature channel n; This is the bias term for the c-th output channel; Represents the ReLU nonlinear activation function, used to introduce a nonlinear transformation, enabling the model to learn complex local patterns; summation operation This reflects the weighted aggregation of the convolution operation over the local time window k and all input features F.

[0016] By setting up multiple parallel convolutional layers with different kernel widths k, the model can simultaneously capture local load features at various time scales, such as short-term (hourly fluctuations) and medium-term (hourly trends), forming rich multi-scale feature representations.

[0017] 2.2: Pooling layers are used to downsample the extracted feature maps, preserving key feature information and reducing the feature dimension of the data sequence. Specifically: Pooling layers use non-overlapping or overlapping sliding windows to process feature maps Z. t A scan is performed, and a summary statistic is calculated within each window. Max pooling then takes the maximum value within that window. The operation of one-dimensional max pooling can be formally described as follows: (5); In equation (5), It is the value of the feature map at the s-th position and the c-th channel after pooling; It is the size of the pooling window; It is the sliding step size, when = When it is non-overlapping pooling, < This is overlapping pooling; the max operation is performed within the window coverage area. Select the maximum value within each time point; This indicates that the original input feature map Z... t In the middle, based on the current pooling window position, retrieve the value at the corresponding position; This represents a local offset index used to traverse each position within the pooled window; it is a temporary variable. ; This indicates that the pooling window covers p consecutive positions, from the first position (r=0) to the last position (r=p-1) within the window.

[0018] After processing by the convolutional and pooling layers, the original load time series data is transformed into a set of feature sequences that retain key local patterns, have lower dimensionality, and are more representative, thus preparing for subsequent capture of long-term temporal dependencies.

[0019] In step 3, after the convolution and pooling processes in step 2, the original temporal data is transformed into a feature sequence rich in local patterns. To further capture the complex temporal dynamics and long-range dependencies in the load data, this invention inputs this feature sequence into a BiLSTM network. Specifically: The input to the BiLSTM network is the three-dimensional tensor output from step 2, whose dimensions are the number of samples, the time step, and the number of feature channels. For a specific sample, it can be represented as a feature sequence with a time step of T. , Let represent the feature vectors at the t-th time step.

[0020] Wherein, the feature vector of each time step t It contains local multidimensional features extracted by CNN corresponding to that time moment, where d is the number of feature channels. This sequence of feature vectors is arranged in chronological order and forms the basis for time series processing by the BiLSTM network.

[0021] In step 3, the BiLSTM network consists of forward LSTM units and backward LSTM units. The BiLSTM network structure is as follows: Figure 2 As shown, the BiLSTM network transmits information and extracts features from the time series data in both forward and backward directions.

[0022] By utilizing historical information in the load data through forward LSTM units, the time-series dependencies from the past to the present are captured; specifically as follows: The feedforward LSTM unit receives the current input feature vector x at time t. t and the hidden state of the previous moment. and cell state It utilizes its internal gating mechanism, including input gates. Forgotten Gate Output gate and candidate cell status Calculate the hidden state at the current time step. and cell state The core calculation process is as follows: Input Gate: (6); Forgotten Gate: (7); Output gate: (8); Candidate cell status: (9); Cell status update: (10); Hidden state output: (11); In the above formula, It is the Sigmoid activation function, which compresses the gate signal into the (0,1) interval to control the information flow. It is the hyperbolic tangent activation function; This represents element-wise multiplication (Hadamard product). These correspond to the input x. t and the previous hidden state The weight matrix; It is the corresponding bias vector; , This represents the four different types of gating mechanisms in LSTM; the forward LSTM... It contains the encoding of all historical information from the beginning of the sequence to the current time t.

[0023] The backward LSTM unit utilizes future information in the feature sequence to capture temporal dependencies from the future to the present. Specifically: Backward LSTM uses the reverse time order ( It processes the same sequence. At time t, it receives input x. t And the hidden state at the "next" time step (i.e., time step t+1 in the original order). and cell state Its computation process is completely symmetrical to that of the forward LSTM: Input Gate: (12); Forgotten Gate: (13); Output gate: (14); Candidate cell status: (15); Cell status update: (16); Hidden state output: (17); In the above formula, the meanings of the symbols are similar to those in the forward LSTM, with the superscript 'b' indicating a backward layer. The backward LSTM... It contains the encoding of all "future" information from the end of the sequence to the current time t.

[0024] At each time step t, the BiLSTM network feeds forward the output of the LSTM unit. Output of the backward LSTM unit Concatenation is performed to form the final bidirectional context-aware feature representation at that moment. : (18); In formula (18): This represents a vector concatenation operation; ,in, It is the number of hidden units in a single LSTM layer. It also incorporates the complete historical context up to time t (from... (carried) and the future context backtracking from the end of the sequence to t (by) carry).

[0025] Through this bidirectional deep modeling, the model can more comprehensively and accurately understand the meaning of the load features at each time point in the context of the entire sequence. In particular, for load abrupt change points that are affected by both preceding and following events, BiLSTM can provide a more robust feature representation than a unidirectional model.

[0026] In step 4, after processing in step 3, a feature sequence containing complete bidirectional temporal dependencies is obtained. To further enhance the model's ability to perceive key information and reduce interference from irrelevant or redundant information, this invention inputs the feature sequence output by BiLSTM into the attention mechanism layer. Specifically: The feature sequence output by BiLSTM in step 3 , Let represent the BiLSTM synthesized feature vector at time step t, where each is a eigenvector. This sequence serves as the input for the attention mechanism. It integrates local spatial features extracted from CNNs and long-short-term bidirectional temporal dependencies extracted from BiLSTMs. The core idea of ​​the attention mechanism is that the model should not treat sequences equally. Instead of focusing on features at all time steps, we should learn to dynamically assign different importance weights to features at different time steps based on the needs of the current prediction task.

[0027] The ATT mechanism dynamically assigns weights over time. By analyzing the correlation between the query vector Q, key vector K, and value vector V, it transforms the influence of the input feature sequence on the final prediction result into the basis for weight allocation of the hidden units in the neural network. Specifically: The query vector Q typically represents the target or context of interest in the current decoding or prediction task. In the sequence-to-point prediction framework of this invention, a learnable global context vector q can be used as the query, or, for simplicity, the output of the last time step of the BiLSTM can be used directly. The result is a query after a linear transformation. Its dimension is... .

[0028] Key vector K and value vector V are both obtained from the input feature sequence H through different linear transformations. (19); In formula (19): It is a learnable weight matrix; and They respectively constitute the key sequence and the value sequence. Let represent the key vector at time step t; Let k represent the value vector and key vector at time step t, respectively. t Used to calculate relevance with the query vector q, i.e., the attention score, value vector v t It contains the actual feature information of the corresponding time step and will be weighted and summed according to the score.

[0029] The attention mechanism first calculates the relationship between the query vector q and each key vector k in the sequence. i The correlation score between them. This correlation score reflects the importance of the feature information at the i-th time step to the current prediction task, as follows: Let the attention variable a represent the index of the selected data (i.e., the element in the value vector sequence V), and a=i indicate that the model focuses attention at the i-th time step.

[0030] Given a query q, the i-th data element (i.e., the key-value pair (k)) i ,v i The attention between )) and q is obtained as s(q,k) i It can be formally computed as: (20); In formula (20): Represents the query vector q and the key vector k i The dot product (inner product) is used to measure the similarity between two objects. The higher the similarity, the higher the score. It is the key vector k i The dimension; It is a scaling factor, introduced to prevent scaling when d... k When the value is large, the dot product result may be too large, causing the gradient after passing through the Softmax function to become too small (the gradient vanishing problem).

[0031] The scores at all time steps are normalized using the Softmax function, transforming them into a probability distribution, i.e., the attention weights. : (twenty one); In formula (21): This represents the natural exponential function. , where e≈2.71828; This indicates the last time step.

[0032] The final output of the attention mechanism is a context vector. This vector is the sequence of value vectors V, where each element v is a distinct element. i The weighted sum, where the weights are the attention weights calculated in the previous step. : (twenty two); In formula (22): This represents summing over all time steps i from 1 to T; The attention weights at the i-th time step satisfy the following conditions: , The larger the value, the higher the corresponding time. The more important the feature information is in the current prediction task; It is the value vector corresponding to the i-th time step, which contains the original feature information at that moment; It is the output context vector, which is no longer a fixed-length sequence, but a fixed-dimensional vector (dimension and...). (Similarly), this vector dynamically and selectively aggregates the most relevant information from the entire input sequence H.

[0033] In step 5, the RIME algorithm is a metaheuristic algorithm proposed in recent years. Its inspiration does not come from common animal group behavior, but rather from simulating the entire process of frost formation and growth in a low-temperature environment. The RIME algorithm imagines the hyperparameters to be optimized as supercooled water vapor suspended on a cold surface; they form "nuclei" at random locations, then diffuse randomly in the form of "soft frost," and then rapidly fill along the dominant direction in the form of "hard frost," finally leaving behind a dense "ice layer" with the lowest energy, in order to seek the global optimal solution to the corresponding problem.

[0034] To overcome the problems of complex network models like CNN-BiLSTM-Att being sensitive to hyperparameters (such as learning rate and network structure parameters), and the difficulty and inefficiency of manual tuning, this invention innovatively introduces the RIME algorithm as an automatic hyperparameter optimizer. Specifically, a set of hyperparameters (such as learning rate and number of BiLSTM units) of the RIME-CNN-BiLSTM-Att model to be optimized are encoded as a position vector Xi of a "frost particle." The RIME algorithm is used to simulate two core mechanisms of frost growth—"soft frost search" and "hard frost puncture"—to iteratively update the particle swarm. Details are as follows: Soft Frost Search Phase: This phase simulates the early amorphous diffusion and growth of frost, encouraging particles to conduct large-scale random exploration to escape local optima. In generation t, the position update formula for the i-th particle in the j-th dimension is as follows: (twenty three); In formula (23): These are the candidate positions after the soft cream search update; It is the position of the currently globally optimal particle in the j-th dimension; , It is a random number within the interval [0,1]. It is a random cosine value in the interval [-1,1], used to introduce uncertainty in the search direction; It is an ambient temperature-related disturbance factor that typically decays with increasing iteration count, thus controlling the exploration intensity; It is a random number within the interval [0,1], used for random sampling within the solution space boundary; and These are the upper and lower bounds of the search for the j-th dimension parameter, respectively; This is the frost growth probability (or surface adhesion coefficient), an increasing linear function that controls the probability of particles choosing "soft frost search" or "hard frost puncture." When a random number... At that time, perform a soft cream search.

[0035] Hard Frost Piercing Phase: Once frost crystals form a dominant growth direction, this phase simulates their rapid, directional growth along that direction to perform a fine-grained search within the most promising region, accelerating convergence. The position update formula is as follows: (twenty four); In formula (24): These are candidate sites updated after frost puncture; It is the position of the particle before the update; It is a random factor that controls the step size; This represents a random number that follows a standard normal distribution (mean 0, standard deviation 1), introducing random perturbation into the local search and preventing stagnation; when At that time, perform epidural puncture.

[0036] In step 5, a weighted fitness function is defined to account for the characteristics of ultra-short-term load forecasting. as follows: (25); In formula (25): It is the fitness value to be minimized; It is a vector of hyperparameters to be optimized, including the learning rate, the number of BiLSTM hidden units, the number of attention heads, and the L2 regularization coefficient, etc. It is a weighting factor used to balance the precision and complexity terms. In this invention, it is taken as... =0.9 indicates that prediction accuracy is prioritized; It is the squared L2 norm of the hyperparameter vector, serving as an indirect measure of model complexity (e.g., a larger number of network units usually means more parameters and higher complexity). It is the normalization cardinality of the parameter count, a large constant used to scale the complexity terms to a scaled value. Similar magnitudes ensure that the contributions of the two factors to the fitness value are balanced.

[0037] Mean absolute error measures the accuracy of a model's predictions on the validation set. ,in: This is the load forecast value. Where N is the actual load value, and N is the sample size. Based on the above steps, the construction and training process of the RIME-CNN-BiLSTM-Att model is as follows: Set the parameters of the RIME algorithm, including population size, maximum number of iterations, initial value and variation pattern of the soft / hard frost transition threshold E, and perturbation factor. Meanwhile, the search space [Lb, Ub] for the hyperparameters X to be optimized in the CNN-BiLSTM-Att model is defined.

[0038] For each generation iteration: a. For each frost particle in the population, i.e., a set of hyperparameters The position is updated according to formulas (23) and (24).

[0039] b. Use the updated hyperparameters Construct and initialize a CNN-BiLSTM-Att model, as shown in steps 2 to 4.

[0040] c. Train the model briefly using the training set, and compute its fitness value F(X) on the validation set. i See formula (25).

[0041] d. Update the individual's historical best solution and the global best solution based on the fitness value. .

[0042] When the RIME algorithm reaches its maximum number of iterations, it outputs the globally optimal combination of hyperparameters, X. best Using X best Initialize the final CNN-BiLSTM-Att model and train it fully on the complete training set to obtain the final RIME-CNN-BiLSTM-Att model that can be used for prediction.

[0043] The constructed RIME-CNN-BiLSTM-Att model requires setting multiple parameters, including the maximum number of iterations, initial learning rate, optimizer, data preprocessing, activation function of convolutional layers, number of hidden units in BiLSTM, number of attention mechanism keys, regularization parameters, pooling layer type, and dropout rate. The specific configuration of each parameter is shown in Table 1.

[0044]

[0045] In step 6, the constructed RIME-CNN-BiLSTM-Att model is a hybrid model, and the structure of the constructed hybrid model is shown below. Figure 3 Meteorological and load data from a city in southern China from January 1st of one year to December 31st of the following year were used as the training and validation sets. Load data was recorded every 15 minutes, generating 96 data points per day. This data included temperature (°C), relative humidity (%), rainfall (mm), and load (kW). To ensure model effectiveness, a one-time retention validation method was used for performance evaluation. The entire dataset was statically divided into a training set and a validation set in a 5:1 ratio. The training set was used for model learning and fitting, while the validation set was used to simulate unknown data to evaluate the model's generalization ability. All data from the following year was separately allocated to a test set to evaluate the model's prediction accuracy and generalization ability. 80% of the data from that year was used for model training, and the remaining 20% ​​was used as the test set.

[0046] To evaluate the performance of the RIME-CNN-BiLSTM-Att model, common metrics for evaluating the accuracy of power load forecasting were selected, including: root mean square error (RMSE), mean absolute error (MAE), mean absolute percentage error (MAPE), and coefficient of determination (R²). 2 ( ) is used as a performance evaluation metric. The model's prediction accuracy is compared with RMSE, MAE, MAPE, and R... 2 Closely related, the lower the values ​​of RMSE, MAE, and MAPE, the better the R... 2 The closer to 1, the higher the model's prediction accuracy; (26); (27); (28); (29); In the formula, Indicates the number of samples; Indicates the actual value; Indicates the predicted value; This represents the actual average value; This represents the predicted average.

[0047] This invention discloses a method for ultra-short-term load prediction of transformer substations based on attention mechanism and RIME-optimized CNN-BiLSTM, with the following technical effects: 1) Advantages of step 1 of the present invention: Based on the outlier detection and normalization processing of the load and meteorological data of the substation area, it can effectively handle outliers in the data, and the normalization processing can significantly improve the comparability of the data and the efficiency of model training.

[0048] 2) Advantages of step 2 of the present invention: Based on the CNN network, the local features of the load information are automatically extracted, which can effectively capture the local features in the data and provide a feature basis for subsequent time series modeling.

[0049] 3) Advantages of step 3 of the present invention: Based on BiLSTM network processing of feature sequences to capture time dependencies, it can make full use of its advantages to capture long-term dependencies in time series data from both forward and backward directions.

[0050] 4) Advantages of step 4 of the present invention: Based on the Att mechanism, dynamic weight allocation enhances key information, which can automatically identify and enhance key information that has a significant impact on the prediction results, improve the model's sensitivity to key information, and enhance the prediction accuracy and reliability of the prediction model.

[0051] 5) Advantages of step 5 of the present invention: Based on the RIME algorithm, key hyperparameters are automatically optimized, which can automatically adjust the key hyperparameters of the model, reduce the workload of manually debugging complex parameters, and improve the generalization ability and adaptability of the prediction model.

[0052] 6) Advantages of step 6 of the present invention: The analysis of the ultra-short-term load forecast results of the transformer area based on the hybrid model can comprehensively consider the influence of multiple factors, compare multiple models, and better demonstrate the superior performance of the proposed model. Attached Figure Description

[0053] The present invention will be further described below with reference to the accompanying drawings and examples; Figure 1 This is a diagram of the CNN architecture.

[0054] Figure 2 This is a diagram of the BiLSTM structure.

[0055] Figure 3 This is a flowchart of a method for predicting ultra-short-term load in transformer areas based on the RIME-CNN-BiLSTM-Att model.

[0056] Figure 4 This is a comparison chart of the prediction results of the RIME-CNN-BiLSTM-Att model with other models.

[0057] Figure 5 Prediction error plots for different prediction models.

[0058] Figure 6 The image shows a comparison of prediction results between the CNN-BiLSTM-Att model optimized using the GWO and PSO algorithms and the CNN-BiLSTM-Att model optimized using the RIME algorithm.

[0059] Figure 7(a) shows the error diagram of the two-step prediction model; Figure 7(b) shows the error graph of the two-step prediction model. Detailed Implementation

[0060] A method for ultra-short-term load forecasting of power distribution areas based on attention mechanisms and RIME-optimized CNN-BiLSTM is proposed. This method involves steps such as outlier detection and normalization of load and meteorological data, automatic hyperparameter optimization, local feature extraction, temporal dependency capture, key information enhancement, and forecast result analysis. The method includes: outlier detection and normalization of load and meteorological data; automatic extraction of local features based on CNN networks to fully exploit spatial information in the data; processing of feature sequences using BiLSTM networks to capture their long-term and bidirectional temporal dependencies; dynamic weight allocation through the Att mechanism to enhance the role of key information in forecasting; automatic optimization of hyperparameters using the RIME algorithm; and analysis of the ultra-short-term load forecasting results based on a hybrid model, comprehensively considering the influence of multiple factors to achieve high-precision load forecasting. The method includes the following steps: Step 1: Collect load and meteorological sample data for the transformer area, and perform outlier detection and normalization. Step 2: Extract local features of the load information based on the CNN network; Step 3: After extracting local features of the load information based on Step 2, use a BiLSTM network to process the feature sequences and capture time dependencies; Step 4: Dynamic weight allocation based on the Att mechanism to enhance the role of key information in prediction; Step 5: Optimize key hyperparameters based on the RIME algorithm and construct the RIME-CNN-BiLSTM-Att model; Step 6: Perform ultra-short-term load forecasting for the transformer area based on the RIME-CNN-BiLSTM-Att model.

[0061] In step 1, technical issues such as data acquisition equipment malfunctions and data storage problems may lead to anomalies in power load and related characteristic data. These anomalies can directly interfere with the accuracy of model predictions. Therefore, it is necessary to remove outliers and impute missing values ​​in the sample data.

[0062] This invention employs a hybrid strategy combining correction and analytical methods for data cleaning. First, outliers significantly deviating from the normal fluctuation range are identified using the 3σ principle based on statistical distribution in analytical analysis. Then, for the identified outliers and missing values ​​(NaN or Null) in the original data, a time-series horizontal replacement method is uniformly used for imputation. The specific rules are as follows: For a time Abnormal or missing data points Its correction value Determined by the following formula: (1); That is, the observation value at the same time on the previous day is used for replacement.

[0063] If the corresponding time of the previous day If the data is also unavailable (either abnormal or missing), then recursively search for earlier historical data from the same period until a valid, usable value is found. This rule can be formally expressed as: (2); in, It is a positive integer. And satisfy These are normal, non-missing values. The search limit can be set according to the actual situation, such as k. max =7 (meaning it can go back a maximum of one week).

[0064] To eliminate the dimensional differences between electricity load (unit: kilowatt) and various meteorological characteristics (such as temperature and humidity, which have different units and magnitudes), and to accelerate the gradient descent convergence process of subsequent deep learning models, all features are subjected to Min-Max normalization, linearly mapping them to the (0,1) interval. For any feature sequence {x} in the dataset... i Its normalization formula is: (3); In equation (3), Indicates the first The result after normalization of each sample; and These represent the maximum and minimum values ​​of the feature in the entire training dataset, respectively. The processed sample data needs to be constructed into an input format suitable for time series prediction models. This invention organizes the data into a three-dimensional tensor. The specific reconstruction process is as follows: Suppose there are N total samples, each of which uses historical data from the past T consecutive time steps (time step lengths) to predict the load for one or more future time steps. Each time step contains F features (such as load value, temperature, humidity, etc.).

[0065] One-dimensional time-series data is sampled using a sliding window of fixed length T to generate multiple consecutive sequence segments. The final input tensor shape is N, T, and F (number of samples, time step, feature dimension).

[0066] Where: Number of samples (N): Total number of sequence segments generated by the sliding window; Time step (T): Historical length of each input sequence, representing the historical range observed by the model; Feature dimension (F): Number of feature variables observed at each time point. This three-dimensional tensor serves as the standard input to the constructed RIME-CNN-BiLSTM-Att model for training and prediction.

[0067] In step 2, the known load data and meteorological data undergo outlier detection and normalization processing, which are then used as input to the CNN network. This data is then flattened into a three-dimensional tensor format, which includes the number of samples, time step, and feature dimension.

[0068] This invention utilizes convolutional layers to learn features from an input three-dimensional tensor. The input data can be viewed as a tensor of shape (N,T,F). The convolutional layers slide along the time axis using multiple one-dimensional convolutional kernels (filters) of different sizes to capture the local dependencies and nonlinear patterns of the load sequence at different time scales.

[0069] For an input feature sequence , This represents a two-dimensional matrix with time step T and feature dimension F, which is processed by a convolutional kernel of size k (kernel width) and has an output channel number of k. One-dimensional convolution kernel After the operation with bias b, the output feature map Z is obtained. t The value Z of the c-th channel and the j-th time position t (j,c) is calculated using the following convolution formula: (4); In the formula, This represents the weight parameters of the convolution kernel at the m-th time offset, the n-th input feature channel, and the corresponding c-th output channel; This represents the value of the input sequence at time position j+m and feature channel n; This is the bias term for the c-th output channel; Represents the ReLU nonlinear activation function, used to introduce a nonlinear transformation, enabling the model to learn complex local patterns; summation operation This reflects the weighted aggregation of the convolution operation over the local time window k and all input features F.

[0070] By setting up multiple parallel convolutional layers with different kernel widths k, the model can simultaneously capture local load features at various time scales, such as short-term (hourly fluctuations) and medium-term (hourly trends), forming rich multi-scale feature representations.

[0071] Pooling layers are used to downsample the extracted feature maps, preserving key feature information and reducing the feature dimension of the data sequence.

[0072] Pooling layers use non-overlapping or overlapping sliding windows to process feature maps Z. t Perform a scan and calculate a summary statistic within each window (max pooling involves taking the maximum value within the window). The operation of one-dimensional max pooling can be formally described as: (5); In the formula; It is the value of the feature map at the s-th position and the c-th channel after pooling; It is the size of the pooling window; It is the sliding step size, when = When it is non-overlapping pooling, < This is overlapping pooling; the max operation is performed within the window coverage area. Select the maximum value within each time point; This indicates that the original input feature map Z... t In the middle, based on the current pooling window position, retrieve the value at the corresponding position; This represents a local offset index used to traverse each position within the pooled window; it is a temporary variable. , This indicates that the pooling window covers p consecutive positions, from the first position (r=0) to the last position (r=p-1) within the window.

[0073] After processing by the convolutional and pooling layers, the original load time series data is transformed into a set of feature sequences that retain key local patterns, have lower dimensionality, and are more representative, thus preparing for subsequent capture of long-term temporal dependencies.

[0074] Step 3 includes: 3.1: After convolution and pooling in step 2, the original temporal data is transformed into a feature sequence rich in local patterns. To further capture the complex temporal dynamics and long-range dependencies in the load data, this invention inputs this feature sequence into a BiLSTM network.

[0075] The input to the BiLSTM network is the three-dimensional feature tensor output from step 2, whose dimensions are the number of samples, the time step, and the number of feature channels. For a specific sample, it can be represented as a feature sequence with a time step of T. , Let represent the feature vector at time step t, where the feature vector at each time step t is denoted as . It contains local multidimensional features (d is the number of feature channels) extracted by CNN corresponding to that time moment. This sequence of feature vectors is arranged in chronological order and forms the basis of the time series processing by BiLSTM.

[0076] 3.2: The BiLSTM network consists of forward LSTM units and backward LSTM units. The BiLSTM network transmits information and extracts features from the time series data in both forward and backward directions.

[0077] By utilizing historical information in the load data through forward LSTM units, the time-series dependencies from the past to the present can be captured.

[0078] The forward LSTM receives the current input feature vector x at time t. t and the hidden state of the previous moment. and cell state It uses its internal gating mechanism (input gate) Forgotten Gate Output gate and candidate cell status Calculate the hidden state at the current time step. and cell state The core calculation process is as follows: Input Gate: (6); Forgotten Gate: (7); Output gate: (8); Candidate cell status: (9); Cell status update: (10); Hidden state output: (11); In the formula, It is the Sigmoid activation function, which compresses the gate signal into the (0,1) interval to control the information flow. It is the hyperbolic tangent activation function; This represents element-wise multiplication (Hadamard product). These correspond to the input x. t and the previous hidden state The weight matrix; It is the corresponding bias vector; , This represents the four different types of gating mechanisms in LSTM; the forward LSTM... It contains the encoding of all historical information from the beginning of the sequence to the current time t.

[0079] Backward LSTM units utilize future information in feature sequences to capture temporal dependencies from the future to the present.

[0080] Backward LSTM uses the reverse time order ( It processes the same sequence. At time t, it receives input x. t And the hidden state at the "next" time step (i.e., time step t+1 in the original order). and cell state Its computation process is completely symmetrical to that of the forward LSTM: Input Gate: (12); Forgotten Gate: (13); Output gate: (14); Candidate cell status: (15); Cell status update: (16); Hidden state output: (17); In the formula, the meanings of the symbols are similar to those in the forward LSTM, with the superscript 'b' indicating a backward layer. The backward LSTM... It contains the encoding of all "future" information from the end of the sequence to the current time t.

[0081] At each time step t, the BiLSTM network feeds forward the output of the LSTM. Output of the backward LSTM Concatenation is performed to form the final bidirectional context-aware feature representation at that moment. : (18); In the formula, This represents a vector concatenation operation; ,in It is the number of hidden units in a single LSTM layer. It also incorporates the complete historical context up to time t (from... (carried) and the future context backtracking from the end of the sequence to t (by) carry).

[0082] Step 4 includes: 4.1: After processing in step 3, a feature sequence containing complete bidirectional temporal dependencies is obtained. To further enhance the model's ability to perceive key information and reduce interference from irrelevant or redundant information, this invention inputs the feature sequence output by BiLSTM into the attention mechanism layer.

[0083] The feature sequence output by BiLSTM in step 3 , Let represent the BiLSTM synthesized feature vector at time step t, where each is a eigenvector. This sequence serves as the input for the attention mechanism. It integrates local spatial features extracted from CNNs and long-short-term bidirectional temporal dependencies extracted from BiLSTMs. The core idea of ​​the attention mechanism is that the model should not treat sequences equally. Instead of focusing on features at all time steps, we should learn to dynamically assign different importance weights to features at different time steps based on the needs of the current prediction task.

[0084] The ATT mechanism dynamically assigns weights over time. By analyzing the correlation between the query vector Q, key vector K, and value vector V, it transforms the influence of the input feature sequence on the final prediction result into the basis for weight allocation of the hidden units in the neural network. Specifically: Query vector (Q): Typically represents the target or context of interest in the current decoding or prediction task. In the sequence-to-point prediction framework of this invention, a learnable global context vector q can be used as the query, or, for simplicity, the output of the last time step of the BiLSTM can be used directly. The result is a query after a linear transformation. Its dimension is... .

[0085] Key vector (K) and value vector (V): Both are obtained from the input feature sequence H through different linear transformations. (19); In the formula, It is a learnable weight matrix; and They respectively constitute the key sequence and the value sequence. Let represent the key vector at time step t; Let k represent the value vector and key vector at time step t, respectively. t Used to calculate relevance (attention score) with query vector q, value vector v t It contains the actual feature information of the corresponding time step and will be weighted and summed according to the score.

[0086] 4.2: Attention Mechanism First, it calculates the relationship between the query vector q and each key vector k in the sequence. i The correlation score between the features is used to determine the importance of the feature information at the i-th time step to the current prediction task.

[0087] Let the attention variable a represent the index of the selected data (i.e., the element in the value vector sequence V), and a=i indicate that the model focuses attention at the i-th time step.

[0088] Given a query q, the i-th data element (i.e., the key-value pair (k)) i ,v i The attention between )) and q is obtained as s(q,k) i It can be formally computed as: (20); In the formula, Represents the query vector With key vector The dot product (inner product) is used to measure the similarity between two objects. The higher the similarity, the higher the score. It is a key vector The dimension; It is a scaling factor, introduced to prevent scaling when d... k When the value is large, the dot product result may be too large, causing the gradient after passing through the Softmax function to become too small (the gradient vanishing problem).

[0089] The scores at all time steps are normalized using the Softmax function, transforming them into a probability distribution, i.e., the attention weights. : (twenty one); In the formula, This represents the natural exponential function. , where e≈2.71828; This indicates the last time step.

[0090] 4.3: The final output of the attention mechanism is a context vector. This vector is the sequence of value vectors V, where each element v is a distinct element. i The weighted sum, where the weights are the attention weights calculated in the previous step. : (twenty two); In the formula, This represents summing over all time steps i from 1 to T; The attention weights at the i-th time step satisfy the following conditions: , The larger the value, the higher the corresponding time. The more important the feature information is in the current prediction task; It is the value vector corresponding to the i-th time step, which contains the original feature information at that moment; It is the output context vector, which is no longer a fixed-length sequence, but a fixed-dimensional vector (dimension and...). (Similarly), this vector dynamically and selectively aggregates the most relevant information from the entire input sequence H.

[0091] Step 5 includes: 5.1: The RIME algorithm is a metaheuristic algorithm proposed in recent years. Its inspiration does not come from common animal group behavior, but rather from simulating the entire process of frost formation and growth in low-temperature environments. The RIME algorithm imagines the hyperparameters to be optimized as supercooled water vapor suspended on a cold surface; they form "nuclei" at random locations, then diffuse randomly as "soft frost," and then rapidly fill along the dominant direction as "hard frost," ultimately leaving behind a dense, lowest-energy "ice layer" to seek the global optimal solution to the corresponding problem.

[0092] To overcome the problems of complex network models like CNN-BiLSTM-Att being sensitive to hyperparameters (such as learning rate and network structure parameters), and the difficulty and inefficiency of manual tuning, this invention innovatively introduces the RIME algorithm as an automatic hyperparameter optimizer. Specifically, a set of hyperparameters (such as learning rate and number of BiLSTM units) of the RIME-CNN-BiLSTM-Att model to be optimized are encoded as a position vector Xi of a "frost particle." The RIME algorithm is used to simulate two core mechanisms of frost growth—"soft frost search" and "hard frost puncture"—to iteratively update the particle swarm. Details are as follows: Soft Frost Search Phase: This phase simulates the early amorphous diffusion and growth of frost, encouraging particles to conduct large-scale random exploration to escape local optima. In generation t, the position update formula for the i-th particle in the j-th dimension is as follows: (twenty three); In the formula, These are the candidate positions after the soft cream search update; It is the position of the currently globally optimal particle in the j-th dimension; , It is a random number within the interval [0,1]. It is a random cosine value in the interval [-1,1], used to introduce uncertainty in the search direction; It is an ambient temperature-related disturbance factor that typically decays with increasing iteration count, thus controlling the exploration intensity; It is a random number within the interval [0,1], used for random sampling within the solution space boundary; and These are the upper and lower bounds of the search for the j-th dimension parameter, respectively; This is the frost growth probability (or surface adhesion coefficient), an increasing linear function that controls the probability of particles choosing "soft frost search" or "hard frost puncture." When a random number... At that time, perform a soft cream search.

[0093] Hard Frost Piercing Phase: Once frost crystals form a dominant growth direction, this phase simulates their rapid, directional growth along that direction to perform a fine-grained search within the most promising region, accelerating convergence. The position update formula is as follows: (twenty four); In the formula, These are candidate sites updated after frost puncture; It is the position of the particle before the update; It is a random factor that controls the step size; This represents a random number that follows a standard normal distribution (mean 0, standard deviation 1), introducing random perturbation into the local search and preventing stagnation; when At that time, perform epidural puncture.

[0094] 5.2: A weighted fitness function is defined to address the characteristics of ultra-short-term load forecasting. as follows: (25); In the formula, It is the fitness value to be minimized; It is a vector of hyperparameters to be optimized, including the learning rate, the number of BiLSTM hidden units, the number of attention heads, and the L2 regularization coefficient, etc. Mean absolute error measures the accuracy of a model's predictions on the validation set. ,in, This is the load forecast value. Where N is the actual load value, and N is the sample size. It is a weighting factor used to balance the precision and complexity terms. In this invention, it is taken as... =0.9 indicates that prediction accuracy is prioritized; It is the squared L2 norm of the hyperparameter vector, serving as an indirect measure of model complexity (e.g., a larger number of network units usually means more parameters and higher complexity). It is the normalization cardinality of the parameter count, a large constant used to scale the complexity terms to a scaled value. Similar magnitudes ensure that the contributions of the two factors to the fitness value are balanced.

[0095] 5.3: Based on the above steps, the construction and training process of the RIME-CNN-BiLSTM-ATt model is as follows: Set the parameters of the RIME algorithm, including population size, maximum number of iterations, initial value and variation pattern of the soft / hard frost transition threshold E, and perturbation factor. Meanwhile, the search space [Lb, Ub] for the hyperparameters X to be optimized in the CNN-BiLSTM-Att model is defined.

[0096] For each generation iteration: a. For each frost particle in the population (i.e., a set of hyperparameters) ), and update its position according to formula (23) and formula (24).

[0097] b. Use the updated hyperparameters Build and initialize a CNN-BiLSTM-Att model (structure as described in steps 2 to 4).

[0098] c. Train the model briefly using the training set, and compute its fitness value F(X) on the validation set. i ) (Formula (25)).

[0099] d. Update the individual's historical best solution and the global best solution based on the fitness value. .

[0100] When the RIME algorithm reaches its maximum number of iterations, it outputs the globally optimal combination of hyperparameters, X. best Using X best Initialize the final CNN-BiLSTM-Att model and train it fully on the complete training set to obtain the final RIME-CNN-BiLSTM-Att model that can be used for prediction.

[0101] 5.4: The constructed RIME-CNN-BiLSTM-Att model requires setting multiple parameters, including the maximum number of iterations, initial learning rate, optimizer, data preprocessing, activation function of convolutional layers, number of hidden units in BiLSTM, number of attention mechanism keys, regularization parameters, pooling layer type, and dropout rate. The specific configuration of each parameter is shown in Table 1.

[0102] Step 6, the analysis of the ultra-short-term load forecast results for the transformer area based on the hybrid model, includes: 6.1: By combining the above models into a hybrid model, the structure of the constructed hybrid model is shown in [reference needed]. Figure 3Meteorological and load data from a city in southern China from January 1, 2016 to December 31, 2017 were used as the training and validation sets. Load data was recorded every 15 minutes, generating 96 data points per day. This data included temperature (°C), relative humidity (%), rainfall (mm), and load (kW). To ensure model effectiveness, a one-time retention-validation method was used for performance evaluation. The entire dataset was statically divided into a training set and a validation set in a 5:1 ratio. The training set was used for model learning and fitting, while the validation set was used to simulate unknown data to evaluate the model's generalization ability. All data from 2018 was used as a separate test set to evaluate the model's prediction accuracy and generalization ability. 80% of the data from that year was used for model training, and the remaining 20% ​​was used as the test set.

[0103] 6.2: To evaluate the performance of the RIME-CNN-BiLSTM-ATt model, common metrics for evaluating the accuracy of power load forecasting were selected, including root mean square error (RMSE), mean absolute error (MAE), mean absolute percentage error (MAPE), and coefficient of determination (R²). 2 ( ) is used as a performance evaluation metric. The model's prediction accuracy is compared with RMSE, MAE, MAPE, and R... 2 Closely related, the lower the values ​​of RMSE, MAE, and MAPE, the better the R... 2 The closer to 1, the higher the model's prediction accuracy.

[0104] (26); (27); (28); (29); In the formula, Indicates the number of samples; Indicates the actual value; Indicates the predicted value; This represents the actual average value; This represents the predicted average.

[0105] 6.3: To verify the accuracy of the RIME-CNN-BiLSTM-ATt model in predicting ultra-short-term load in transformer substations, its performance was compared with that of CNN, BiLSTM, LSTM, and CNN-BiLSTM models. The performance differences of each method were compared. Figure 4 As shown. By Figure 4It is known that under low load conditions, the prediction results of most prediction models are quite close to the actual values, while under high load conditions, the deviation between the predicted and actual values ​​increases significantly. However, the prediction model proposed in this invention exhibits excellent prediction performance under both low and high load conditions, and its prediction accuracy is significantly better than other models.

[0106] 6.4: In the study of ultra-short-term load forecasting for distribution areas, in order to comprehensively evaluate the performance of various models and verify their accuracy and generalization ability, detailed forecasting error analyses were conducted on several different forecasting models. The RMSE, MAE, MAPE, and R of different forecasting models were analyzed. 2 The comparison results are shown in Table 2 and Figure 5 As shown in the figure, compared with the traditional LSTM model, the BiLSTM model acquires data information through bidirectional LSTM units, resulting in a significant improvement in prediction performance. RMSE, MAE, and MAPE are all reduced, and R... 2 The performance of the CNN-BiLSTM model improved significantly compared to the BiLSTM model, with a 0.42% decrease in MAPE compared to LSTM. RMSE, MAE, and MAPE were all significantly reduced. 2 The increases were significant, with RMSE, MAE, and MAPE decreasing by 25.68%, 44.04%, and 1.52%, respectively. 2 The 4% increase indicates that the CNN-BiLSTM combined model can help improve the accuracy of ultra-short-term load forecasting. Using the CNN-BiLSTM combined model as the base model, the RIME-CNN-BiLSTM-Att model, obtained by adding the RIME frost optimization algorithm and attention mechanism, shows a significant decrease in all evaluation metrics compared to the base model. The model's prediction accuracy is significantly improved; RMSE, MAE, and MAPE are reduced by 50.35%, 42.58%, and 42.04%, respectively, compared to the base model. 2 It increased by 5%. From the above, it can be concluded that the model proposed in this invention has good predictive performance.

[0107]

[0108] 6.5: To evaluate the superiority of the RIME algorithm in model optimization, the GWO and PSO algorithms were used to optimize the CNN-BiLSTM-Att model, and the RIME algorithm was used to optimize the CNN-BiLSTM-Att model for comparative analysis. The comparative analysis is as follows: Figure 6 As shown. By Figure 6It is evident that the prediction models optimized by the RIME, GWO, and PSO algorithms can all effectively fit the general trend of load changes. However, compared with the GWO and PSO algorithms, the prediction curve of the model optimized by the RIME algorithm fits the true value better, demonstrating that the RIME algorithm has a better optimization effect.

[0109] 6.6: To further evaluate the stability and generalization ability of the proposed model, multi-step prediction experiments were conducted based on the single-step prediction experiments. These experiments more comprehensively demonstrate the model's ability to mine deep features when processing time series data. Two-step and three-step prediction experiments were the main focus, and the final results are shown in Table 2 and Figures 7(a) and 7(b). The results show that the error gradually increases with the number of prediction steps. The main reason for this phenomenon is that the correlation between the load change component and historical data gradually weakens over time, making it difficult to effectively capture valuable time-series features, which significantly increases the difficulty of multi-step prediction. In the multi-step prediction experiments, the RIME-CNN-BiLSTM-Att model achieved an RMSE of 182.04kW, a MAE of 149.43kW, a MAPE as low as 1.70%, and an R² of 0.97 in predictions with a step size of 2. In predictions with a step size of 3, the RMSE was 218.20kW, the MAE was 164.52kW, the MAPE as low as 1.90%, and the R² was 0.96, making it the best in all evaluation metrics. In conclusion, the RIME-CNN-BiLSTM-Att model exhibits excellent stability and generalization ability.

Claims

1. A method for ultra-short-term load prediction of transformer substations based on attention mechanism and RIME-optimized CNN-BiLSTM, characterized in that... Includes the following steps: Step 1: Collect load and meteorological sample data for the transformer area, and perform outlier detection and normalization. Step 2: Extract local features of the load information based on the CNN network; Step 3: After extracting local features of the load information based on Step 2, use a BiLSTM network to process the feature sequences and capture time dependencies; Step 4: Dynamic weight allocation based on the Att mechanism to enhance the role of key information in prediction; Step 5: Optimize key hyperparameters based on the RIME algorithm and construct the RIME-CNN-BiLSTM-Att model; Step 6: Perform ultra-short-term load forecasting for the transformer area based on the RIME-CNN-BiLSTM-Att model.

2. The method for ultra-short-term load prediction of transformer substations based on attention mechanism and RIME-optimized CNN-BiLSTM according to claim 1, characterized in that: In step 1, a hybrid strategy combining correction and analytical analysis is used for data cleaning. First, outliers that significantly deviate from the normal fluctuation range are identified using the 3σ principle based on statistical distribution in analytical analysis. Then, for the identified outliers and missing values ​​in the original data, denoted as NaN or Null, the time series horizontal replacement method is uniformly used to fill them in. For a time Abnormal or missing data points Its correction value Determined by the following formula: （1）； That is, replace it with the observation value at the same time the previous day; If the corresponding time of the previous day If the data is also unavailable, abnormal, or missing, then the search continues recursively through earlier historical data from the same period until a valid, usable value is found. This rule is formally expressed as: （2）； in, Let be a positive integer, and satisfy... Normal, non-missing values; To eliminate the dimensional differences between electricity load and various meteorological characteristics, all features are subjected to Min-Max normalization, linearly mapping them to the (0,1) interval; for any feature sequence {x} in the dataset... i Its normalization formula is: （3）； In equation (3), Indicates the first The result after normalization of each sample; and These represent the maximum and minimum values ​​of the feature in the entire training dataset, respectively.

3. The method for ultra-short-term load prediction of transformer substations based on attention mechanism and RIME-optimized CNN-BiLSTM according to claim 2, characterized in that: The processed sample data needs to be constructed into an input format suitable for time series prediction models; the data is organized into three-dimensional tensors; the specific reconstruction process is as follows: Assume there are N total samples, each sample uses historical data from the past T consecutive time steps to predict the load for one or more future time steps; each time step contains F features, including load value, temperature, and humidity; One-dimensional time series data is sampled by a sliding window of fixed length T to generate multiple consecutive sequence segments; the final input tensor shape is the number of samples, time step, and feature dimension, i.e., N, T, F. Where: Sample number N: represents the total number of sequence segments generated by the sliding window; Time step T: represents the historical length of each input sequence, indicating the historical range observed by the model; Feature dimension F: represents the number of feature variables observed at each time point; This three-dimensional tensor serves as the standard input to the constructed RIME-CNN-BiLSTM-Att model for training and prediction.

4. The method for ultra-short-term load prediction of transformer substations based on attention mechanism and RIME-optimized CNN-BiLSTM according to claim 3, characterized in that: In step 2, the CNN network processes data through convolutional layers to automatically capture local features in the payload data, as follows: 2.1: Feature learning is performed on the input three-dimensional tensor using convolutional layers; the input data can be regarded as a tensor of shape (N,T,F); the convolutional layer slides and scans along the time axis using multiple one-dimensional convolutional kernels of different sizes to capture the local dependencies and nonlinear patterns of the load sequence at different time scales; For an input feature sequence , A two-dimensional matrix representing the time step T and feature dimension F; after passing through a convolution kernel of size k with output channels of size k, ... One-dimensional convolution kernel After the operation with bias b, the output feature map Z is obtained. t The value of the c-th channel and the j-th time position It is calculated using the following convolution formula: （4）； In equation (4), This represents the weight parameters of the convolution kernel at the m-th time offset, the n-th input feature channel, and the corresponding c-th output channel; This represents the value of the input sequence at time position j+m and feature channel n; This is the bias term for the c-th output channel; Represents the ReLU nonlinear activation function, used to introduce a nonlinear transformation, enabling the model to learn complex local patterns; summation operation This reflects the weighted aggregation of the convolution operation over the local time window k and all input features F; 2.2: Pooling layers are used to downsample the extracted feature maps, preserving key feature information and reducing the feature dimension of the data sequence; specifically as follows: Pooling layers use non-overlapping or overlapping sliding windows to process feature maps Z. t Perform a scan, calculate a summary statistic within each window, and max pooling takes the maximum value within the window; the operation of one-dimensional max pooling can be formally described as: （5）； In equation (5), It is the value of the feature map at the s-th position and the c-th channel after pooling; It is the size of the pooling window; It is the sliding step size, when = When it is non-overlapping pooling, < This is overlapping pooling; the max operation is performed within the window coverage area. Select the maximum value within each time point; This indicates that the original input feature map Z... t In the middle, based on the current pooling window position, retrieve the value at the corresponding position; This represents a local offset index used to traverse each position within the pooled window; it is a temporary variable. ; This indicates that the pooling window covers p consecutive positions, from the first position to the last position within the window.

5. The method for ultra-short-term load prediction of transformer substations based on attention mechanism and RIME-optimized CNN-BiLSTM according to claim 4, characterized in that: In step 3, after the convolution and pooling processes in step 2, the original temporal data is converted into a feature sequence rich in local patterns. To further capture the complex temporal dynamics and long-range dependencies in the load data, this invention inputs this feature sequence into a BiLSTM network; specifically as follows: The input to the BiLSTM network is a three-dimensional tensor output from step 2, with dimensions representing the number of samples, the time step, and the number of feature channels. For a specific sample, it is represented as a feature sequence with a time step of T. , These represent the feature vectors at the t-th time step; Wherein, the feature vector of each time step t It contains local multidimensional features extracted by CNN corresponding to that time moment, where d is the number of feature channels; this feature vector sequence is arranged in chronological order and forms the basis for time series processing by the BiLSTM network. The BiLSTM network consists of forward LSTM units and backward LSTM units. The BiLSTM network transmits information and extracts features from the time series data in both forward and backward directions. The forward LSTM units utilize historical information in the load data to capture the dependencies between historical and current time series data. Specifically: The feedforward LSTM unit receives the current input feature vector x at time t. t and the hidden state of the previous moment. and cell state It utilizes its internal gating mechanism, including input gates. Forgotten Gate Output gate and candidate cell status Calculate the hidden state at the current time step. and cell state ; The backward LSTM unit utilizes future information in the feature sequence to capture temporal dependencies from the future to the present; specifically as follows: Backward LSTM uses the reverse time order ( It processes the same sequence; at time t, it receives input x. t And the hidden state in the next moment. and cell state Its computation process is completely symmetrical with that of the forward LSTM. At each time step t, the BiLSTM network feeds forward the output of the LSTM unit. Output of the backward LSTM unit The features are concatenated to form the final bidirectional context-aware feature representation at that moment. : （18）； In formula (18): This represents a vector concatenation operation; ,in, It is the number of hidden units in a single LSTM layer; It also integrates the complete historical context up to time t and the future context back to t from the end of the sequence.

6. The method for ultra-short-term load prediction of transformer substations based on attention mechanism and RIME-optimized CNN-BiLSTM according to claim 5, characterized in that: In step 4, the feature sequence output by BiLSTM is input into the attention mechanism layer; specifically as follows: The feature sequence output by BiLSTM in step 3 , Let represent the BiLSTM synthesized feature vector at time step t, where each is a eigenvector. This sequence serves as the input to the attention mechanism; It has integrated local spatial features extracted from CNN and long-short-term bidirectional temporal dependencies extracted from BiLSTM; The ATT mechanism dynamically allocates weights over time; by analyzing the correlation between the query vector Q, key vector K, and value vector V, it transforms the influence of the input feature sequence on the final prediction result into the basis for weight allocation of the hidden units in the neural network; specifically as follows: Query vector Q: represents the target or context of interest in the current decoding or prediction task; Key vector K and value vector V are both obtained from the input feature sequence H through different linear transformations. （19）； In formula (19): It is a learnable weight matrix; and They respectively constitute the key sequence and the value sequence. Let represent the key vector at time step t; Let k represent the value vector and key vector at time step t, respectively. t Used to calculate relevance with the query vector q, i.e., the attention score, value vector v t It contains the actual feature information of the corresponding time step, and will be weighted and summed according to the score; The attention mechanism first calculates the relationship between the query vector q and each key vector k in the sequence. i The correlation score between them; this correlation score reflects the importance of the feature information at the i-th time step to the current prediction task, as follows: Let the attention variable 'a' represent the index of the selected data, and 'a=i' indicate that the model focuses its attention at the i-th time step. Given a query q, the attention between the i-th data element and q is s(q,k). i It can be formally computed as: （20）； In formula (20): Represents the query vector q and the key vector k i The dot product (inner product) is used to measure the similarity between two objects; the higher the similarity, the higher the score. It is the key vector k i The dimension; It is the scaling factor; The scores at all time steps are normalized using the Softmax function, transforming them into a probability distribution, i.e., the attention weights. : （21）； In formula (21): This represents the natural exponential function. , Indicates the last time step; The final output of the attention mechanism is a context vector. This vector is the sequence of value vectors V, where each element v is a value. i The weighted sum, where the weights are the attention weights calculated in the previous step. : （22）； In formula (22): This represents summing over all time steps i from 1 to T; The attention weights at the i-th time step satisfy the following conditions: , The larger the value, the higher the corresponding time. The more important the feature information is in the current prediction task; It is the value vector corresponding to the i-th time step, which contains the original feature information at that moment; It is the output context vector.

7. The method for ultra-short-term load prediction of transformer substations based on attention mechanism and RIME-optimized CNN-BiLSTM according to claim 6, characterized in that: In step 5, the RIME algorithm is introduced as an automatic hyperparameter optimizer. Specifically, a set of hyperparameters of the RIME-CNN-BiLSTM-ATt model to be optimized is encoded into a position vector Xi of a "frost particle". The particle swarm is iteratively updated using the two core mechanisms of frost growth—"soft frost search" and "hard frost puncture"—simulated by the RIME algorithm. The details are as follows: Soft Frost Search Phase: This phase simulates the early amorphous diffusion and growth of frost, encouraging particles to conduct large-scale random exploration to escape local optima; in generation t, the position update formula for the i-th particle in the j-th dimension is as follows: （23）； In formula (23): These are the candidate positions after the soft cream search update; It is the position of the currently globally optimal particle in the j-th dimension; , It is a random number within the interval [0,1]. It is a random cosine value in the interval [-1,1], used to introduce uncertainty in the search direction; It is an ambient temperature-related disturbance factor that typically decays with increasing iteration count, thus controlling the exploration intensity; It is a random number within the interval [0,1], used for random sampling within the solution space boundary; and These are the upper and lower bounds of the search for the j-th dimension parameter, respectively; It represents the frost growth probability, an increasing linear function that controls the probability of particles choosing "soft frost search" or "hard frost piercing"; when a random number... At that time, perform a soft cream search; Hard Frost Piercing Phase: Once frost crystals form a dominant growth direction, this phase simulates their rapid, directional growth along that direction to perform a fine-grained search within the most promising region, accelerating convergence; the position update formula is as follows: （24）； In formula (24): These are candidate sites updated after frost puncture; It is the position of the particle before the update; It is a random factor that controls the step size; This represents a random number that follows a standard normal distribution, introducing random perturbation into the local search and preventing it from getting stuck; when At that time, perform epidural puncture.

8. The method for ultra-short-term load prediction of transformer areas based on attention mechanism and RIME-optimized CNN-BiLSTM according to claim 7, characterized in that: In step 5, a weighted fitness function is defined to account for the characteristics of ultra-short-term load forecasting. as follows: （25）； In formula (25): It is the fitness value to be minimized; It is a vector of hyperparameters to be optimized, including the learning rate, the number of BiLSTM hidden units, the number of attention heads, and the L2 regularization coefficient, etc. It is a weighting factor used to balance the accuracy and complexity terms, indicating that prediction accuracy is given priority. It is the squared L2 norm of the hyperparameter vector, used as an indirect measure of model complexity; It is the parameter normalization cardinality, used to scale complexity terms to a scaled value. Similar magnitude; Mean absolute error measures the accuracy of a model's predictions on the validation set. ,in: This is the load forecast value. Where N is the actual load value and N is the number of samples.

9. The method for ultra-short-term load prediction of transformer areas based on attention mechanism and RIME-optimized CNN-BiLSTM according to claim 8, characterized in that: The construction and training process of the RIME-CNN-BiLSTM-ATt model is as follows: Set the parameters of the RIME algorithm, including population size, maximum number of iterations, initial value and variation pattern of the soft / hard frost transition threshold E, and perturbation factor. Meanwhile, the search space [Lb,Ub] for the hyperparameters X to be optimized in the CNN-BiLSTM-Att model is defined. For each generation iteration: a. For each frost particle in the population, i.e., a set of hyperparameters Update its position according to formulas (23) and (24); b. Use the updated hyperparameters Build and initialize a CNN-BiLSTM-Att model; c. Train the model briefly using the training set, and compute its fitness value F(X) on the validation set. i See formula (25); d. Update the individual's historical best solution and the global best solution based on the fitness value. ; When the RIME algorithm reaches its maximum number of iterations, it outputs the globally optimal combination of hyperparameters, X. best Use X best Initialize the final CNN-BiLSTM-Att model and train it fully on the complete training set to obtain the final RIME-CNN-BiLSTM-Att model that can be used for prediction; The constructed RIME-CNN-BiLSTM-Att model requires setting multiple parameters, including the maximum number of iterations, initial learning rate, optimizer, data preprocessing, activation function of convolutional layers, number of hidden units in BiLSTM, number of attention mechanism keys, regularization parameters, pooling layer type, and Dropout rate.

10. The method for ultra-short-term load prediction of transformer substations based on attention mechanism and RIME-optimized CNN-BiLSTM according to claim 9, characterized in that: In step 6, the constructed RIME-CNN-BiLSTM-Att model is a hybrid model. Meteorological data and load data from a city in southern China from January 1st of one year to December 31st of the following year are used as samples to set it as the training set and validation set. Load data is recorded every 15 minutes, thus generating 96 data points per day. These data include temperature, relative humidity, rainfall, and load information. To ensure the effectiveness of the model, the entire dataset was statically divided into a training set and a validation set in a 5:1 ratio. The training set was used for model learning and fitting, while the validation set was used to simulate unknown data to evaluate the model's generalization ability. All data from the following year was divided into a separate independent test set to evaluate the model's prediction accuracy and generalization ability. 80% of the data from that year was used for model training, and the remaining 20% ​​was used as the test set. To evaluate the performance of the RIME-CNN-BiLSTM-Att model, common metrics for evaluating the accuracy of power load forecasting were selected, including: root mean square error (RMSE), mean absolute error (MAE), mean absolute percentage error (MAPE), and coefficient of determination (R²). 2 () is used as an indicator to evaluate performance; the model's prediction accuracy is related to RMSE, MAE, MAPE, and R... 2 Closely related, the lower the values ​​of RMSE, MAE, and MAPE, the better the R... 2 The closer to 1, the higher the model's prediction accuracy; （26）； （27）； （28）； （29）； In the formula, Indicates the number of samples; Indicates the actual value; Indicates the predicted value; This represents the actual average value; This represents the predicted average.

Images