A power load prediction method and system based on a parallel fusion architecture and transfer learning

Abstract

The application relates to a power load prediction method and system based on a parallel fusion architecture and transfer learning. The method comprises the following steps: acquiring source domain power load time series data, and performing overall training on an original hybrid prediction model to obtain source domain pre-training parameters; constructing a hybrid prediction model with the same structure as the original hybrid prediction model, and loading the source domain pre-training parameters; freezing the network layer parameters of the Transformer branch of the hybrid prediction model; acquiring target domain power load time series data and performing preprocessing, and then inputting the TSMixer branch and the Transformer branch to perform time and global dimension feature extraction, respectively; further performing feature fusion and model prediction, and fine-tuning the modules in a trainable state until training convergence, so that a trained hybrid prediction model is obtained. Compared with the prior art, the application can realize high-precision prediction in a small sample scene through transfer learning.

Description

Technical Field

[0001] This invention relates to the field of power system technology, and in particular to a power load forecasting method and system based on a parallel fusion architecture and transfer learning. Background Technology

[0002] At this critical juncture of energy transition and the construction of new power systems, power load forecasting is a key link in the operation and dispatch management of smart grids. Accurate load forecasting is of great significance for ensuring the safe and stable operation of the power system, optimizing power generation plans, reducing operating costs, and promoting the consumption of new energy sources.

[0003] Traditional power load forecasting methods mainly include time series analysis methods (such as ARIMA and SARIMA) and traditional machine learning methods (such as support vector regression and random forest). These methods typically rely on manually designed statistical features and linear assumptions, making it difficult to effectively capture the complex nonlinear relationships and long-term time series dependencies in power load data, resulting in limited forecast accuracy.

[0004] In recent years, with the development of deep learning technology, prediction methods based on time series models such as Recurrent Neural Networks (RNNs), Long Short-Term Memory Networks (LSTMs), and Gated Recurrent Units (GRUs) have been widely used. These models can effectively extract temporal dependencies in time series, but they still suffer from gradient vanishing or exploding problems when dealing with extremely long sequences, and are difficult to parallelize, resulting in low training efficiency. Furthermore, a single model structure is often too simple, risking overfitting and failing to capture the deep features of the time series. For example, the Transformer architecture utilizes a multi-head self-attention mechanism to model global dependencies, demonstrating excellent performance in long-term time series prediction tasks, but its ability to extract local detailed features is relatively weak, often failing to accurately capture short-term high-frequency fluctuations in power load data caused by factors such as user behavior and weather changes. Lightweight mixer architectures, represented by TSMixer, can efficiently extract local cross features and short-term dependencies between multivariate variables through fully connected operations along the time and feature dimensions, but due to their model structure, their ability to model global patterns is somewhat lacking when dealing with extremely long sequences.

[0005] Furthermore, electricity load forecasting faces the dual challenges of data distribution bias and small sample sizes in practice. On the one hand, factors such as seasonal changes, unexpected events, and adjustments in electricity consumption structure can lead to distributional differences between training and forecast data, resulting in decreased model performance. On the other hand, for newly established target scenarios, historical load data is often extremely limited, making it difficult to train a high-precision forecasting model. Existing methods typically employ transfer learning strategies, but traditional transfer learning methods are prone to overfitting when data is scarce. How to effectively adapt to the target domain features while preserving general knowledge from the source domain has become a key challenge in current research on transfer learning for time series forecasting.

[0006] The invention disclosed in CN120541620A is a method and apparatus for short-term power load forecasting, designed to address the technical problem that existing short-term power load forecasting methods lack the ability to recognize long-term changes and global trends. The method includes preprocessing power load time series data to form target power load time series data; subsequently, a time-domain feature sequence is generated using a multi-scale convolutional layer and a time-series feature extraction module in a pre-set short-term power load forecasting model; simultaneously, a frequency-domain feature enhancement module in the pre-set short-term power load forecasting model is used to perform frequency-domain transformation on the original data and time-domain features respectively, obtaining two types of frequency-domain features; finally, the time-domain and frequency-domain information are integrated through a feature fusion module in the pre-set short-term power load forecasting model, and the prediction result is output through a detection head in the pre-set short-term power load forecasting model. However, this scheme cannot effectively extract local cross-features and short-term dependencies between multivariate variables. Summary of the Invention

[0007] The purpose of this invention is to overcome the shortcomings of the existing technology and provide a power load forecasting method and system based on parallel fusion architecture and transfer learning.

[0008] The objective of this invention can be achieved through the following technical solutions: A power load forecasting method based on a parallel fusion architecture and transfer learning includes: Acquire power load time series data and perform real-time preprocessing; input the real-time preprocessed power load time series data into a pre-built and trained hybrid prediction model to obtain power load prediction results; The hybrid prediction model includes a TSMixer branch and a Transformer branch; the training process of the hybrid prediction model includes: acquiring target domain power load time series data and preprocessing it; inputting the preprocessed target domain power load time series data into the TSMixer branch and extracting features in the time dimension to obtain a short-term time-series dependency feature vector; based on the short-term time-series dependency feature vector, extracting the cross features between multivariate variables to obtain a time-series variable cross feature vector. Simultaneously, the preprocessed target domain power load time series data is input into the Transformer branch and global time series features are extracted to obtain a global time series feature vector; power load prediction is performed based on the time series variable cross feature vector and the global time series feature vector; the model parameters of the hybrid prediction model are further updated until the training converges to obtain the trained hybrid prediction model.

[0009] Furthermore, the real-time preprocessing process includes reversible instance normalization of the power load time series data. The corresponding processing steps include: calculating the local mean and local variance based on the power load time series data; and normalizing the power load time series data based on the local mean and local variance to obtain normalized power load time series data.

[0010] Furthermore, the preprocessed target domain power load time series data is input into the TSMixer branch and time-dimensional feature extraction is performed to obtain short-term time-series dependent feature vectors. The corresponding feature extraction expression is: in, This represents the feature matrix after mixing along the time dimension; Presentation layer normalization operation; This represents a multilayer perceptron operating in the time dimension. This is the input feature matrix.

[0011] Furthermore, based on the short-term time-dependent feature vector, high-frequency cross features between multivariate variables are extracted to obtain the time-series variable cross feature vector, and the corresponding extraction expression is: in, This represents the time-series variable cross-feature matrix of the final output of the current mixer layer; This represents a multilayer perceptron acting on the feature dimension. This represents the feature matrix after mixing along the time dimension. Presentation layer normalization operation.

[0012] Furthermore, the preprocessed target domain power load time series data is input into the Transformer branch and global time series feature extraction is performed to obtain a global time series feature vector. The corresponding global time series feature extraction expression is: in, , , These represent the query matrix, key matrix, and value matrix obtained from the input feature mapping, respectively. This represents the transpose of the key matrix. This represents the dimension of the key vector. This represents the output matrix after the input features have passed through a multi-head self-attention layer and have residual connections added. This represents the feedforward neural network module. This represents the final global temporal feature output matrix of the Transformer branch. This indicates a multi-head self-attention processing module. Represents the normalized exponential function, Presentation layer normalization operation, This is the input feature matrix.

[0013] Furthermore, based on the aforementioned time-series variable cross feature vector and global time-series feature vector, the gating weight is calculated, and the corresponding calculation formula is as follows: in, This represents the calculated gate weight matrix. This indicates that the output features of the two branches are concatenated along the feature dimension. This represents the learnable weight parameter matrix in the gated network; This represents the learnable bias parameters in the gated network; This represents the Sigmoid activation function.

[0014] Furthermore, based on the gating weights, the cross-feature vectors of the temporal variables and the global temporal feature vectors are weighted and fused to obtain a comprehensive temporal feature vector. The corresponding fusion expression is as follows: in, This represents the integrated time-series feature matrix obtained through fusion. Represents the Hadamard product. This represents the final output feature of the time series mixer model.

[0015] Furthermore, using the local mean and local variance, an inverse instance normalization transformation is performed on the power load forecast results to obtain the actual power load forecast values ​​mapped back to the original physical dimension. The corresponding transformation expression is: in, To standardize the prediction results, It is a local mean. For local variance, Scaling factor The translation factor is... This is the standardized prediction result after transformation.

[0016] Furthermore, based on the local mean and local variance, the power load time series data is normalized. The corresponding normalization expression is as follows: in, For normalized power load time series data, Scaling factor The translation factor is... To prevent division by zero constants, It is a local mean. For local variance, This represents time series data.

[0017] The present invention also provides a power load forecasting system based on parallel fusion architecture and transfer learning, including a memory and a processor. The memory stores a computer program, and the processor calls the computer program to execute the steps of the method described above.

[0018] Compared with the prior art, the present invention has the following advantages: (1) This invention creatively combines the Transformer architecture, which has a powerful global receptive field and long-range periodicity capture capability, with the TSMixer architecture, which excels at handling multivariate cross-dimensional interactions and extracting local high-frequency jump features. Furthermore, through a dynamic adaptive gating mechanism, it assigns fusion weights to the cross-feature vectors of time-series variables and the global time-series feature vectors in real time, achieving complementary advantages in deep and shallow layers, global and local features, and long-term and short-term features. This significantly improves the model's representation capability in complex power load fluctuation scenarios. It effectively avoids the dilemma of a single model being unable to address both high-frequency noise and low-frequency trends when facing complex power load signals, and significantly improves the model's comprehensive perception capability of different frequency domain information in power load data, laying a rich and complementary data foundation for subsequent multi-source feature fusion and accurate prediction.

[0019] (2) Based on the acquisition of short-term time-dependent feature vectors, this invention further extracts high-frequency cross features between multivariate variables to obtain time-series variable cross feature vectors, which can effectively capture the highly nonlinear and strongly time-varying interaction relationships in power load data. Traditional prediction models often simply concatenate features of external variables or approximate the relationship between variables through shallow linear transformations, which makes it difficult to capture the intense dynamic interactions between variables at the instantaneous scale. This invention designs a dedicated high-frequency cross feature extraction module to perform pairwise or higher-order interaction modeling of multivariate variables in the semantic space of short-term dependent features, which greatly improves the model's response sensitivity and prediction accuracy in non-stationary and abrupt scenarios.

[0020] (3) The present invention adopts a training mechanism that freezes only the Transformer branch with a large number of parameters and the general generalization law during the transfer stage, while keeping the TSMixer branch with a smaller number of parameters and the general local characteristics trainable. This mechanism not only utilizes the general temporal physical laws contained in the big data of the source domain, but also accurately fits the unique features of the target domain with a small computational cost, thus achieving high-precision and fast transfer in small sample scenarios.

[0021] (4) In this invention, the statistics of independent instances are calculated during data input to smooth and homogenize the data, and the saved statistical parameters are used to reverse the process at the output end. This mechanism not only fundamentally unifies the underlying distribution of features in the source and target domains, greatly reducing the difficulty of optimizing gradients in deep networks, but also ensures that the prediction results are accurately mapped back to the original physical range, and outputs accurate prediction values ​​with extremely high practical engineering application value. Attached Figure Description

[0022] Figure 1 This is a flowchart of a power load forecasting method based on parallel fusion architecture and transfer learning provided in an embodiment of the present invention. Figure 2 This is a flowchart illustrating the overall technical process of a power load forecasting method based on a parallel fusion architecture and transfer learning, as provided in this embodiment of the invention. Figure 3 This is a TSMixer architecture diagram of a power load forecasting method based on parallel fusion architecture and transfer learning provided in an embodiment of the present invention. Figure 4 This is a Transformer architecture diagram of a power load forecasting method based on parallel fusion architecture and transfer learning provided in an embodiment of the present invention. Figure 5 The RevIN-TSMixer-Transformer model structure diagram is provided in an embodiment of the present invention for a power load forecasting method based on parallel fusion architecture and transfer learning. Figure 6 This is a schematic diagram of transfer learning freeze fine-tuning for a power load forecasting method based on parallel fusion architecture and transfer learning provided in an embodiment of the present invention. Figure 7 This is a comparison chart of the prediction results of the power load prediction method based on parallel fusion architecture and transfer learning provided in this embodiment of the invention on the source domain dataset, comparing the prediction results of the model of the present invention with those of other comparative models. Figure 8 This is a comparison chart of prediction results for a power load prediction method based on parallel fusion architecture and transfer learning provided in an embodiment of the present invention, under the target domain dataset. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0024] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0025] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0026] Example 1 like Figure 1 As shown, this embodiment provides a power load forecasting method based on a parallel fusion architecture and transfer learning. The method includes the following steps: S1: Acquire power load time series data and perform real-time preprocessing; input the real-time preprocessed power load time series data into the pre-built and trained hybrid prediction model to obtain power load prediction results; Preferred, The real-time preprocessing process includes reversible instance normalization of the power load time series data. The corresponding processing steps include: calculating the local mean and local variance based on the power load time series data; and normalizing the power load time series data based on the local mean and local variance to obtain the normalized original power load time series data. After obtaining the prediction results, the power load prediction results are subjected to inverse instance normalization transformation using local mean and local variance to obtain the real power load prediction values ​​mapped back to the original physical dimension.

[0027] S2: The hybrid prediction model includes the TSMixer branch and the Transformer branch; the training process of the hybrid prediction model includes: acquiring the target domain power load time series data and preprocessing it; inputting the preprocessed target domain power load time series data into the TSMixer branch and extracting features in the time dimension to obtain short-term time-series dependency feature vectors; based on the short-term time-series dependency feature vectors, extracting high-frequency cross features between multivariate variables to obtain time-series variable cross feature vectors; S3: Simultaneously, the preprocessed target domain power load time series data is input into the Transformer branch and global time series features are extracted to obtain a global time series feature vector; prediction is performed based on the time series variable cross feature vector and the global time series feature vector; and the model parameters are updated until the training converges to obtain a trained hybrid prediction model.

[0028] Preferred, The dual-branch output features are weighted and fused using a learnable gating network, with the gating weights dynamically calculated based on the current state of the input features.

[0029] Preferred, The frozen network layers in the Transformer branch include the multi-head self-attention layer, the feedforward neural network layer, and the corresponding normalization layer; all parameters of the reversible instance normalization module, the TSMixer branch, the dynamic adaptive gating fusion module, and the terminal output layer are in a trainable state.

[0030] This embodiment also provides a power load forecasting system based on parallel fusion architecture and transfer learning, including a memory and a processor. The memory stores a computer program, and the processor calls the computer program to execute the steps of the method described above.

[0031] Example 2 like Figure 2 As shown in the figure, this embodiment provides a specific implementation process of a power load forecasting method based on parallel fusion architecture and transfer learning. The overall technical process is mainly divided into five parts: data processing, reversible instance normalization, model training, transfer learning, and forecast result output.

[0032] Step S1: Data Acquisition and Preprocessing Step S101: Obtain raw power load data, covering historical power load values ​​of the source domain (large data volume, such as more than 1 year) and the target domain (small data volume, such as 1 month), as well as possible covariates (such as meteorological data such as temperature, humidity, wind speed, etc., and time coding features such as days of the week and holidays). Step S102: Preprocess the source and target domain data separately. Specifically, use... The principle is to identify and remove abnormal extreme points caused by sensor failures; for the missing values ​​after removal or missing system values ​​during the acquisition process, linear interpolation is used to fill them in reasonably to ensure the continuity of the time series. Step S103: Ensure that the sampling frequency (e.g., uniform resampling at 15 minutes / point) and feature dimensions of the target domain and source domain data are absolutely consistent.

[0033] Step S2: Reversible Instance Normalization (RevIN) Processing To eliminate the non-stationarity of time series data and the distribution shift between the source and target domains, independent local mean and variance smoothing is performed on each time series instance of the input model during the forward propagation process: Step S201: For any time series instance of the current input Calculate their local means respectively and local variance : in, The total length of time steps for a time series instance (i.e., the size of the time window for the series). Indicates the current time step index. ; Indicates the first Input sequence observations at each time step; Step S202: Utilize learnable affine transformation parameters (scaling factors) Translation factor The sequence data is normalized to obtain a normalized sequence. : in, To prevent division by zero of small constants.

[0034] Step S203: Accurately save the local mean calculated in step S201. and local variance We construct instance-independent statistical parameter dictionaries for use in the reverse reconstruction of model output results in subsequent steps.

[0035] Step S3, as follows Figure 5 As shown, a hybrid prediction model based on a parallel fusion architecture is constructed. Step S301: As Figure 3 As shown, a parallel TSMixer mixer branch is constructed, which contains multiple stacked mixer layers. Let the input feature matrix be... ; Step S3011: Perform temporal feature mixing. Each mixer layer sequentially performs fully connected operations along the time dimension to extract short-term temporal mixture dependencies among multiple variables. in, This represents the feature matrix after mixing along the time dimension; Presentation layer normalization operation; This represents the matrix transpose operation; The branch representing a multilayer perceptron (fully connected network) operating in the time dimension, after being stacked with multiple layers, has its final output denoted as... ; Step S3012: Perform variable feature mixing. Based on the temporal mixing output, perform a fully connected operation along the feature dimension to extract high-frequency cross features between multivariate variables. in, This represents the time-series and variable cross-feature matrix of the final output of the current mixer layer; This represents a multilayer perceptron acting on the feature dimension; Step S302: As Figure 4 As shown, a parallel Transformer branch is constructed: this branch includes a multi-head attention mechanism and a feedforward neural network module (FFN) for mining long-range temporal dependencies and intertemporal periodic patterns within the global receptive field. in, , , These represent the query matrix, key matrix, and value matrix obtained from the input feature mapping, respectively. Represents the transpose of the key matrix; This indicates the dimension of the key vector, used to scale the dot product result to prevent gradient vanishing; This represents the normalized exponential function; This represents the output matrix after the input features pass through a multi-head self-attention layer and are connected by residual connections. This represents a feedforward neural network module; This represents the final global temporal feature output matrix of the Transformer branch; Step S303: Output features of the Transformer branch through a learnable gating network. and TSMixer branch output characteristics Perform dynamic weighted fusion; Step S3031: Perform dynamic weight calculation and gating weights. Dynamically calculated based on the current input feature state: in, This represents the calculated gate weight matrix, whose value range is between 0 and 1; This indicates that the output features of the two branches are concatenated along the feature dimension. This represents the learnable weight parameter matrix in the gated network; This represents the learnable bias parameters in the gated network; This represents the Sigmoid activation function; Step S3032: Dual-branch feature fusion: Based on the calculated weight matrix, perform the feature fusion operation. in, This represents the deep integrated temporal feature matrix obtained after the gateway adaptively allocates weights. This represents the Hadamard product (an element-wise matrix multiplication operation).

[0036] Step S4: Transfer Learning and Model Fine-tuning Based on the model architecture constructed in step S3, a two-stage training transfer strategy is executed, from source domain generalization to target domain fine-tuning: Step S401: Use the large source domain data obtained in step S1 to enter the model. After the instance normalization process in step S2, pre-train the overall network model. Update all network layer parameters through the backpropagation algorithm to obtain source domain pre-training parameters with high generalization ability. Step S402: In the specific task of predicting the target domain, construct a hybrid prediction model that is isomorphic to the network in the pre-training stage, and fully load the source domain pre-training parameters learned in step S401 into the hybrid prediction model. Step S403: Activate the unilateral parameter freeze setting; Step S4031: As Figure 6As shown, only the parameters of the Transformer branches with global receptive fields (including multi-head self-attention layers, feedforward neural network layers, and corresponding normalization layers) are forced to be frozen and non-differentiable (i.e., parameter updates are stopped). Step S4032: Define the fine-tuning layer, keeping the reversible instance normalization module, the TSMixer branch which is good at local Gaussian extraction, the dynamic adaptive gating fusion module, and the terminal output layer in a trainable active state; Step S404: Using the target domain time series data, after the instance normalization process in step S2, calculate the prediction error loss function, and only perform incremental parameter updates on the trainable network modules specified in step S4032 to quickly adapt to the unique local high-frequency characteristics of the target domain. in, This represents the mean squared error loss value; This represents the total number of sample batches that participated in the fine-tuning training. Indicates the sample index; This represents the output of the model during the forward propagation process. One normalized predicted value; Indicates the first A true normalized power load label value.

[0037] Step S5: Prediction and Inverse Normalization Output Step S501: Model prediction: Input the sequence to be tested in the target domain into the fine-tuned hybrid prediction model to obtain the deep temporal features after parallel dual-branch extraction and dynamic gating fusion. Step S502: Linear Representation Mapping: The fused temporal features are dimensionally mapped through the terminal projection layer to obtain the standardized prediction result. ; Step S503: Reverse normalization restoration: Call the local mean saved in the corresponding input instance in step S203. and local variance ,right Perform reverse instance normalization transformation: Step S504: The final output maps back to the original physical dimension of the predicted power load. .

[0038] Example 3 To verify the effectiveness of the method of the present invention, this embodiment uses two power load datasets with different amounts of data for experimentation.

[0039] The proposed RevIN-TSMixer-Transformer model is compared with several benchmark models, including single models (TSMixer, Transformer) and different combinations of models (RevIN-TSMixer, TSMixer-Transformer, etc.). The coefficient of determination (R²) is used. 2 The root mean square error (RMSE), mean absolute error (MAE), and mean absolute percentage error (MAPE) are used as evaluation indicators.

[0040] Table 1 Comparison of prediction model accuracy for source domain datasets To further verify that the proposed model improves prediction accuracy, a comparative analysis of the accuracy of various prediction models was conducted on the source domain dataset. When a parallel fusion architecture of TSMixer and Transformer is adopted, the prediction accuracy is further improved. Compared to the single TSMixer model, R... 2 The R² improved from 0.865 to 0.872, the RMSE decreased from 671.75 kW to 654.87 kW, and the MAE decreased from 530.27 kW to 500.12 kW, verifying that the dual-branch parallel fusion can effectively integrate local features and global dependencies, achieving complementary advantages. Further introduction of the Reversible Instance Normalization (RevIN) module significantly improved the prediction accuracy of all models. Taking the RevIN-TSMixer model as an example, compared to the TSMixer model without RevIN, R² improved significantly. 2 The R² value increased significantly from 0.865 to 0.936, RMSE decreased from 671.75 kW to 461.15 kW (a reduction of 210.60 kW), MAE decreased from 530.27 kW to 347.57 kW (a reduction of 182.70 kW), and MAPE decreased from 1.156% to 0.755%. Furthermore, the complete model RevIN-TSMixer-Transformer proposed in this invention achieved optimal performance across all metrics. Compared to the TSMixer-Transformer model without RevIN, R² significantly improved. 2 The RMSE increased further from 0.872 to 0.956, while the MAE decreased significantly from 654.87 kW to 406.81 kW and from 500.12 kW to 312.86 kW. For example... Figure 7The results shown above demonstrate that the RevIN module can effectively eliminate the non-stationarity and distribution shift of the data. The parallel dual-branch fusion architecture of TSMixer and Transformer, combined with the dynamic gating mechanism, can comprehensively capture the local high-frequency features and global long-range dependencies in the power load time series data. The synergistic effect of each module achieves the optimal prediction accuracy, laying a good pre-training foundation for transfer learning in the target domain with small sample scenarios.

[0041] Table 2 Comparison of prediction accuracy for target domain datasets Based on the experimental results in Table 2, the effectiveness of the proposed transfer learning method was verified in a small-sample scenario in the target domain. When the pre-trained RevIN-TSMixer-Transformer model is directly applied to the target domain for prediction, the model's prediction accuracy is poor due to the data distribution differences between the source and target domains. However, by using the proposed transfer learning method based on a one-sided freezing mechanism—that is, freezing the Transformer branch parameters to retain the general long-term patterns learned in the source domain, while keeping the RevIN module, TSMixer branch, and dynamic gating fusion module in a fine-tunable state—the model's prediction accuracy is significantly improved after fine-tuning with a small amount of target domain data: R 2 The power efficiency index (MEX) increased significantly from 0.708 to 0.891, a relative increase of 25.8%; the power reserve ratio (RMSE) decreased from 689.53 kW to 504.13 kW, a relative decrease of 26.9%; and the power efficiency index (MAE) decreased from 523.21 kW to 309.87 kW, a relative decrease of 40.8%. Figure 8 The results shown demonstrate that the experimental data fully validates the effectiveness of the proposed one-sided frozen transfer learning mechanism in small sample target domain scenarios. By preserving the general global temporal dependency rules obtained by the Transformer branch and making targeted fine-tuning for local high-frequency features in the target domain, the knowledge learned from large-scale data in the source domain can be efficiently transferred to the target domain, significantly improving the prediction accuracy in small sample scenarios. This effectively solves the technical problems of decreased prediction accuracy under data distribution shifts and poor prediction performance due to insufficient model transfer capability in small sample target domain scenarios.

[0042] Example 4 The electronic device of this invention includes a central processing unit (CPU), which can perform various appropriate actions and processes according to computer program instructions stored in read-only memory (ROM) or loaded from a storage unit into random access memory (RAM). The RAM may also store various programs and data required for device operation. The CPU, ROM, and RAM are interconnected via a bus. Input / output (I / O) interfaces are also connected to the bus.

[0043] Multiple components in the device are connected to the I / O interface, including: input units such as keyboards and mice; output units such as various types of displays and speakers; storage units such as disks and optical discs; and communication units such as network interface cards (NICs), modems, and wireless transceivers. The communication unit allows the device to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0044] The functions described in this invention can be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that can be used include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0045] The program code used to implement the methods of the present invention can be written in any combination of one or more programming languages. This program code can be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing device, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code can be executed entirely on the machine, partially on the machine, as a standalone software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0046] In the context of this invention, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0047] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this disclosure can be achieved, and this is not limited herein.

[0048] The preferred embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make numerous modifications and variations based on the concept of the present invention without creative effort. Therefore, all technical solutions that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning, or limited experimentation on the basis of existing technology should be within the scope of protection defined by the claims.

Claims

1. A power load forecasting method based on parallel fusion architecture and transfer learning, characterized in that, include: Acquire time-series power load data and perform real-time preprocessing; The pre-processed power load time series data is input into a pre-built and trained hybrid prediction model to obtain power load prediction results. The hybrid prediction model includes TSMixer branches and Transformer branches; The training process of the hybrid prediction model includes: acquiring time series data of target domain power load and preprocessing it; inputting the preprocessed time series data of target domain power load into the TSMixer branch and extracting features in the time dimension to obtain a short-term time-dependent feature vector; and extracting the cross features between multivariate variables based on the short-term time-dependent feature vector to obtain a time-series variable cross feature vector. Simultaneously, the preprocessed target domain power load time series data is input into the Transformer branch and global time series features are extracted to obtain a global time series feature vector; power load prediction is performed based on the time series variable cross feature vector and the global time series feature vector; the model parameters of the hybrid prediction model are further updated until the training converges to obtain the trained hybrid prediction model.

2. The power load forecasting method based on parallel fusion architecture and transfer learning according to claim 1, characterized in that, The real-time preprocessing process includes reversible instance normalization of the power load time series data. The corresponding processing steps include: calculating the local mean and local variance based on the power load time series data; and normalizing the power load time series data based on the local mean and local variance to obtain normalized power load time series data.

3. The power load forecasting method based on parallel fusion architecture and transfer learning according to claim 1, characterized in that, The preprocessed target domain power load time series data is input into the TSMixer branch and time-dimensional feature extraction is performed to obtain a short-term time-series dependency feature vector. The corresponding feature extraction expression is: in, This represents the feature matrix after mixing along the time dimension; Presentation layer normalization operation; This represents a multilayer perceptron operating in the time dimension. This is the input feature matrix.

4. The power load forecasting method based on parallel fusion architecture and transfer learning according to claim 3, characterized in that, Based on the short-term time-series dependency feature vector, high-frequency cross features between multivariate variables are extracted to obtain the time-series variable cross feature vector. The corresponding extraction expression is as follows: in, This represents the time-series variable cross-feature matrix of the final output of the current mixer layer; This represents a multilayer perceptron acting on the feature dimension. This represents the feature matrix after mixing along the time dimension. Presentation layer normalization operation.

5. The power load forecasting method based on parallel fusion architecture and transfer learning according to claim 1, characterized in that, The preprocessed target domain power load time series data is input into the Transformer branch and global time series feature extraction is performed to obtain a global time series feature vector. The corresponding global time series feature extraction expression is: in, , , These represent the query matrix, key matrix, and value matrix obtained from the input feature mapping, respectively. This represents the transpose of the key matrix. This represents the dimension of the key vector. This represents the output matrix after the input features have passed through a multi-head self-attention layer and have residual connections added. This represents the feedforward neural network module. This represents the final global temporal feature output matrix of the Transformer branch. This indicates a multi-head self-attention processing module. Represents the normalized exponential function, Presentation layer normalization operation, This is the input feature matrix.

6. The power load forecasting method based on parallel fusion architecture and transfer learning according to claim 1, characterized in that, Based on the aforementioned time-series variable cross feature vector and global time-series feature vector, the gating weight is calculated using the following formula: in, This represents the calculated gate weight matrix. This indicates that the output features of the two branches are concatenated along the feature dimension. This represents the learnable weight parameter matrix in the gated network; This represents the learnable bias parameters in the gated network; This represents the Sigmoid activation function.

7. The power load forecasting method based on parallel fusion architecture and transfer learning according to claim 6, characterized in that, Based on the gating weights, the cross-feature vectors of the time-series variables and the global time-series feature vectors are weighted and fused to obtain a comprehensive time-series feature vector. The corresponding fusion expression is as follows: in, This represents the integrated time-series feature matrix obtained through fusion. Represents the Hadamard product. This represents the final output feature of the time series mixer model.

8. The power load forecasting method based on parallel fusion architecture and transfer learning according to claim 2, characterized in that, Using the local mean and local variance, an inverse instance normalization transformation is performed on the power load forecast results to obtain the actual power load forecast values ​​mapped back to the original physical dimension. The corresponding transformation expression is: in, To standardize the prediction results, It is a local mean. For local variance, Scaling factor The translation factor is... This is the standardized prediction result after transformation.

9. The power load forecasting method based on parallel fusion architecture and transfer learning according to claim 2, characterized in that, Based on the local mean and local variance, the power load time series data is normalized. The corresponding normalization expression is as follows: in, For normalized power load time series data, Scaling factor The translation factor is... To prevent division by zero constants, It is a local mean. For local variance, This represents time series data.

10. A power load forecasting system based on a parallel fusion architecture and transfer learning, characterized in that, It includes a memory and a processor, the memory storing a computer program, the processor invoking the computer program to perform the steps of the method as described in any one of claims 1 to 9.

Images