Ultra-short-term load prediction method and system based on tensor decomposition personalized federated learning, and storage medium
By adopting a tensor decomposition-based personalized federated learning approach, the problems of data privacy protection, slow model convergence speed, and weak generalization ability in power load forecasting are solved, achieving faster training speed and stronger generalization ability.
Patent Information
- Application Number
- CN202510742130.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-05
- Publication Date
- 2025-11-14
AI Technical Summary
Existing power load forecasting methods have shortcomings in terms of data privacy protection, model convergence speed, and generalization ability. Centralized collaborative modeling is risky for data leakage and costly, while distributed collaborative modeling ignores communication costs and model convergence efficiency.
We adopt a personalized federated learning approach based on tensor decomposition, which constructs a global model and a personalized model through a deep neural network, uses CP tensor decomposition to compress model parameters, and designs regularization terms and ensemble tensor aggregation strategies to optimize the model training process.
The model training speed was improved, communication overhead was reduced, the generalization ability of the model was enhanced, and the convergence of the linear acceleration was proved through theoretical analysis.
Smart Images
Figure CN120952102A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of smart grid technology, specifically relating to an ultra-short-term load forecasting method, system, and storage medium based on tensor decomposition and personalized federated learning. Background Technology
[0002] With the continued growth in electricity demand, accurate ultra-short-term load forecasting (USTLF) is crucial for the management and dispatch of modern power systems. USTLF can not only optimize grid operation and reduce operating costs, but also improve the reliability and flexibility of power supply.
[0003] However, the spatiotemporal fluctuations in electricity demand pose a significant challenge to accurately sensing these changes. To effectively address the instability of the electricity market, power companies and retailers need to leverage advanced data analytics to gain a deep understanding of the electricity consumption patterns and load demands of various user groups, thereby providing personalized services. From a data storage perspective, collaborative modeling methods for electricity load forecasting are mainly divided into centralized collaborative modeling and distributed collaborative modeling.
[0004] Centralized collaborative modeling methods store data from various operators on a server for unified modeling and analysis, enabling model training in a single environment. However, this approach faces risks of data leakage during data transmission and centralized storage, and centralized storage is costly. Distributed collaborative modeling methods, primarily represented by federated learning, utilize larger datasets for collaborative training while protecting data privacy, thereby improving prediction accuracy. However, existing federated learning methods often neglect the communication costs between clients and servers, as well as model convergence efficiency. Furthermore, during global model aggregation, they tend to employ average parameter aggregation strategies, failing to fully consider the impact of data heterogeneity among clients, resulting in slow model convergence and weak generalization ability. Summary of the Invention
[0005] The purpose of this invention is to overcome the shortcomings of the prior art and provide a method, system and storage medium for ultra-short-term load forecasting based on tensor decomposition personalized federated learning, which can solve problems such as data privacy protection, slow model convergence speed and weak generalization ability in the training process of ultra-short-term forecasting models for power systems.
[0006] The technical problem solved by this invention is achieved through the following technical solution:
[0007] A method for ultra-short-term load forecasting based on tensor decomposition and personalized federated learning, the steps of which are as follows:
[0008] S1. Based on the spatiotemporal coupling characteristics of ultra-short-term load data, we delve into the temporal correlation and spatial distribution features of the data, and design a deep neural network to construct a global model and a personalized model for ultra-short-term load prediction.
[0009] S2. Broadcast the constructed ultra-short-term load forecast global model to the clients as their personalized models.
[0010] S3. Use CP tensor decomposition to extract and compress features from the personalized model parameters, and use the decomposed model as the tensor local model for each client.
[0011] S4. Design the objective function of the tensor decomposition personalized federated learning ultra-short-term load prediction model, and introduce a regularization term to control the Euclidean distance between the personalized model and the tensor local model.
[0012] S5. Design a training strategy for a tensor decomposition personalized federated learning ultra-short-term load prediction model. Train the personalized model on the client side, update the personalized model parameters by minimizing the objective function, and decouple the tensor local model to complete the tensor local model parameter update with the trained personalized model as the target.
[0013] S6. Based on the characteristics of the tensor decomposition personalized federated learning ultra-short-term load forecasting model, an integrated tensor aggregation strategy is designed. Each client transmits the updated tensor local model parameters to the server to realize tensor parameter aggregation, complete the global model parameter update for ultra-short-term load forecasting, and realize ultra-short-term load forecasting.
[0014] Moreover, S1 specifically refers to:
[0015] The input feature, i.e., the historical load sequence x, is first mapped to the intermediate feature space through the first fully connected layer. The calculation process is as follows:
[0016] h1 = RELU(W1x + b1);
[0017] in: This is the weight matrix; It is the bias vector; For the real number space; d mid d represents the dimension of the intermediate feature space. in is the dimension of the input features; x is the historical load sequence; ReLU is the activation function used to introduce nonlinearity and enhance the model's ability to represent the complex semantic relationships between data caused by high load volatility;
[0018] The output h1 of the intermediate layer is mapped to the output layer through the second fully connected layer. The calculation formula is as follows:
[0019] y = W2h1 + b2;
[0020] Where: y is the load value predicted by the model; This is the weight matrix for the second layer; This is the corresponding bias vector; For the real number space; d out d represents the dimension of the output feature. mid Let be the dimension of the intermediate feature space.
[0021] Moreover, S3 specifically refers to:
[0022] The weights of the fully connected layers of the prediction model are decomposed using CP tensors. The decomposed prediction model weights are represented as the product of multiple low-rank tensor factors:
[0023]
[0024] Where: W is the weight matrix; A (1) and A (2) For factor matrix; α r (1) and ɑ r (2) A respectively (1) and A (2) The r-th column vector; R is the rank of the weight matrix;
[0025] Each element w in the weight matrix W i,j All can be obtained through factor matrix A (1) and A (2) The sum of the products of the corresponding column vector elements is represented as:
[0026]
[0027] The fully connected layer after tensor decomposition is represented as:
[0028] y′≈A (1) (A (2) ) T x+b;
[0029] Where: y′ is the predicted load value of the fully connected layer; A (1) and A (2) is the factor matrix; x is the historical load sequence; b is the bias vector of the fully connected layer.
[0030] Moreover, S4 specifically refers to:
[0031] The original high-dimensional weight parameters Let denot be an N-order tensor, and convert it into a CP tensor decomposition of rank R, denoted as .
[0032] Where: factor matrix The r-th column vector is ɑ r (n) , n=1,…,N, and A (n) =[ɑ1 (n) ,…,ɑ R (n) ]; For the real number space; I1~I N R represents the length of each dimension; R is the rank of the factor matrix.
[0033] The objective function is defined as:
[0034]
[0035] in: It is the local dataset of the kth client; Indicates the number of local samples; x i It is the input sequence of the sample; y i It is the target value; F k The local objective function of the k-th client is defined as:
[0036]
[0037] Where: θ k It is the personalized model for the kth client, f k (θ k ) is the expected value of the loss prediction of the personalized model on the local data distribution of client k. It is a local model of tensors; F k (·) includes an l2 norm regularization term, which controls the distance between the personalized model and the tensor local model, and λ controls the degree of regularization.
[0038] Moreover, S5 specifically refers to:
[0039] On the client side, the tensor local model is first used as the reference center point, and the objective function F is then used to... k The personalized model is trained by minimizing the input; then, based on the optimal personalized model, the tensor local model is decoupled, and the factor matrix of the tensor local model is uploaded to the server; after the server aggregates the factor matrix, it broadcasts the aggregated tensor and factor matrix to the client respectively, as the personalized and tensor local models, until the iteration is complete.
[0040] 1) Initialization: The server first initializes a global model for ultra-short-term load forecasting. And broadcast it to the client;
[0041] 2) Client-side model training: Assume there are a total of T global communication rounds. In each global communication round t = 1, ..., T, the client performs τ local update rounds, training the personalized model and the tensor local model sequentially. In each local update round t′ = 1, ..., τ, each client first uses... Used as a reference center to train its personalized model Then, the client uses the obtained approximate solution. To train a tensor local model This phase is repeated for T rounds until the iteration is complete;
[0042] 3) Model aggregation and update: Under the federated learning framework, considering the instability of network connections, servers typically set a fixed subset C of clients of size C. t Model aggregation is performed, and each selected client updates its local model. k∈C t The data is sent to the server, where the integrated tensor aggregation strategy is used to update the global model for ultra-short-term load forecasting. The updated global model for ultra-short-term load forecasting is then broadcast to the client, and this process continues until the iteration ends.
[0043] Moreover, S6 specifically refers to:
[0044] Design an integrated tensor aggregation strategy to reconstruct the factor matrix of the client-side tensor local model into complete tensor parameters:
[0045]
[0046] in: Let k be the approximate tensor obtained by recombining the factor matrix after the t-th round of global communication and the τ-th local update for client k.
[0047] The server calculates the tensor based on the weight of each client's data volume. The average value is used to control the aggregation direction of the global model for ultra-short-term load forecasting via β:
[0048]
[0049] Where: C t This refers to the collection of clients selected during the aggregation process.
[0050] The aggregated global model for ultra-short-term load forecasting is expressed as follows: and These correspond to the parameters of the full-version model and the parameters of the tensorized model, respectively:
[0051]
[0052] An ultra-short-term load forecasting system based on tensor decomposition and personalized federated learning includes:
[0053] The ultra-short-term load data analysis module is used to mine the temporal correlation and spatial distribution characteristics in ultra-short-term load data;
[0054] The CP tensor decomposition module is used to extract and compress features from the parameters of the personalized model for ultra-short-term load forecasting, thereby obtaining a local tensor model.
[0055] The model training objective design module is used to analyze the dual decoupling characteristics of the tensor decomposition personalized federated learning ultra-short-term load prediction model, so as to design the objective function for training the tensor decomposition personalized federated learning ultra-short-term load prediction model.
[0056] The tensor aggregation strategy design module is used to analyze the parameter tensor characteristics of the tensor decomposition personalized federated learning ultra-short-term load prediction model in order to design an integrated tensor aggregation strategy.
[0057] The ultra-short-term load forecasting optimization module is used to design training strategies for the tensor decomposition personalized federated learning ultra-short-term load forecasting model, and to update the global ultra-short-term load forecasting model by integrating tensor aggregation strategies to achieve ultra-short-term load forecasting.
[0058] A computer storage medium storing a computer program that, when run, implements the prediction method.
[0059] The advantages and beneficial effects of this invention are as follows:
[0060] 1. The prediction method of this invention enables personalized model training on the client using local data, and reduces the dimensionality of the model parameter factor matrix through CP tensor decomposition, thereby designing a tensorized local model, improving model training speed, and reducing communication overhead.
[0061] 2. This invention designs a two-layer loss function and introduces a regularization term to balance the distance between the personalized model and the tensor local model, thereby improving the model's generalization ability; it adopts an ensemble tensor aggregation strategy to improve training speed and robustness; and it proves through theoretical analysis that it has linearly accelerated frontier convergence. Attached Figure Description
[0062] Figure 1 This is a flowchart of the prediction method of the present invention;
[0063] Figure 2 This is a schematic diagram of the ultra-short-term load prediction model architecture based on tensor decomposition personalized federated learning of the present invention.
[0064] Figure 3This is a flowchart of the training strategy for the ultra-short-term load prediction model based on tensor decomposition personalized federated learning in this invention.
[0065] Figure 4 This is a comparison of the prediction performance of the ultra-short-term load prediction model based on tensor decomposition personalized federated learning of this invention with traditional methods.
[0066] Figure 5 This is a comparison of the convergence performance of the ultra-short-term load prediction model based on tensor decomposition personalized federated learning in this invention with traditional methods. Detailed Implementation
[0067] The present invention will be further described in detail below through specific embodiments. The following embodiments are merely descriptive and not limiting, and should not be used to limit the scope of protection of the present invention.
[0068] like Figure 1 As shown, this embodiment of the invention provides an ultra-short-term load forecasting method based on tensor decomposition and personalized federated learning. Its innovation lies in the following steps:
[0069] S1. Based on the spatiotemporal coupling characteristics of ultra-short-term load data, we will delve into the temporal correlation and spatial distribution features of the data, design a deep neural network on the server side, and construct a global model for ultra-short-term load prediction.
[0070] The input feature, i.e., the historical load sequence x, is first mapped to the intermediate feature space through the first fully connected layer. The calculation process is as follows:
[0071] h1 = RELU(W1x + b1);
[0072] in: This is the weight matrix; It is the bias vector; For the real number space; d mid d represents the dimension of the intermediate feature space. in is the dimension of the input features; x is the historical load sequence; ReLU is the activation function used to introduce nonlinearity and enhance the model's ability to represent the complex semantic relationships between data caused by high load volatility;
[0073] The output h1 of the intermediate layer is mapped to the output layer through the second fully connected layer. The calculation formula is as follows:
[0074] y = W2h1 + b2;
[0075] Where: y is the load value predicted by the model; This is the weight matrix for the second layer; This is the corresponding bias vector; For the real number space; d outd represents the dimension of the output feature. mid Let be the dimension of the intermediate feature space.
[0076] S2. Broadcast the constructed ultra-short-term load forecast global model to the clients as their personalized models.
[0077] S3. Use CP tensor decomposition to extract and compress features from the personalized model parameters of the client, and use the decomposed model as the tensor local model of each client.
[0078] The weights of the fully connected layers of the prediction model are decomposed using CP tensors. The decomposed prediction model weights are represented as the product of multiple low-rank tensor factors:
[0079]
[0080] Where: W is the weight matrix; A (1) and A (2) For factor matrix; α r (1) and ɑ r (2) A respectively (1) and A (2) The r-th column vector; R is the rank of the weight matrix;
[0081] Each element w in the weight matrix W i,j All can be obtained through factor matrix A (1) and A (2) The sum of the products of the corresponding column vector elements is represented as:
[0082]
[0083] The fully connected layer after tensor decomposition is represented as:
[0084] y′≈A (1) (A (2) ) T x+b;
[0085] Where: y′ is the predicted load value of the fully connected layer; A (1) and A (2) is the factor matrix; x is the historical load sequence; b is the bias vector of the fully connected layer.
[0086] S4. Design the objective function of the tensor decomposition personalized federated learning ultra-short-term load prediction model, and introduce a regularization term to control the Euclidean distance between the personalized model and the tensor local model, thereby improving the generalization ability of the global model for ultra-short-term load prediction.
[0087] The original high-dimensional weight parameters Let denot be an N-order tensor, and convert it into a CP tensor decomposition of rank R, denoted as .
[0088] Where: factor matrix The r-th column vector is ɑ r (n) , n=1,…,N, and A (n) =[ɑ1 (n) ,…,ɑ R (n) ]; For the real number space; I1~I N R represents the length of each dimension; R is the rank of the factor matrix.
[0089] The objective function is defined as:
[0090]
[0091] in: It is the local dataset of the kth client; Indicates the number of local samples; x i It is the input sequence of the sample; y i It is the target value; F k The local objective function of the k-th client is defined as:
[0092]
[0093] Where: θ k It is the personalized model for the kth client, f k (θ k ) is the expected value of the loss prediction of the personalized model on the local data distribution of client k. It is a tensor-local model; F k (·) includes an l2 norm regularization term, which controls the distance between the personalized model and the tensor local model, and λ controls the degree of regularization.
[0094] S5. Design a training strategy for a tensor decomposition personalized federated learning ultra-short-term load prediction model. Minimize the objective function to achieve personalized model optimization. At the same time, decouple the local tensor model and update the global ultra-short-term load prediction model through an integrated tensor aggregation strategy to complete the ultra-short-term load prediction.
[0095] On the client side, the tensor local model is first used as the reference center point, and the objective function F is then used to... kThe personalized model is trained by minimizing the input; then, based on the optimal personalized model, the tensor local model is decoupled, and the factor matrix of the tensor local model is uploaded to the server; after the server aggregates the factor matrix, it broadcasts the aggregated tensor and factor matrix to the client respectively, as the personalized and tensor local models, until the iteration is complete.
[0096] 1) Initialization: The server first initializes a global model for ultra-short-term load forecasting. And broadcast it to the client;
[0097] 2) Client-side model training: Assume there are a total of T global communication rounds. In each global communication round t = 1, ..., T, the client performs τ local update rounds, training the personalized model and the tensor local model sequentially. In each local update round t′ = 1, ..., τ, each client first uses... Used as a reference center to train its personalized model Then, the client uses the obtained approximate solution. To train a tensor local model This phase is repeated for T rounds until the iteration is complete;
[0098] 3) Model aggregation and update: Under the federated learning framework, considering the instability of network connections, servers typically set a fixed subset C of clients of size C. t Model aggregation is performed, and each selected client updates its local tensor model. k∈C t Send it to the server, update the global model of ultra-short-term load forecasting on the server side using the integrated tensor aggregation strategy, and broadcast the updated global model of ultra-short-term load forecasting to the client, and continue until the end of the iteration;
[0099] S6. Based on the characteristics of the tensor decomposition personalized federated learning ultra-short-term load forecasting model, an integrated tensor aggregation strategy is designed. Each client transmits the updated tensor local model parameters to the server, realizing tensor parameter aggregation, completing the global model parameter update for ultra-short-term load forecasting, and achieving ultra-short-term load forecasting.
[0100] Design an integrated tensor aggregation strategy to reconstruct the factor matrix of the client-side tensor local model into complete tensor parameters:
[0101]
[0102] in: Let k be the approximate tensor obtained by recombining the factor matrix after the t-th round of global communication and the τ-th local update for client k.
[0103] The server calculates the tensor based on the weight of each client's data volume. The average value is used to control the aggregation direction of the global model for ultra-short-term load forecasting via β:
[0104]
[0105] Where: C t This refers to the collection of clients selected during the aggregation process.
[0106] The aggregated global model for ultra-short-term load forecasting is expressed as follows: and These correspond to the parameters of the full-version model and the parameters of the tensorized model, respectively:
[0107]
[0108] This invention provides an ultra-short-term load forecasting system based on tensor decomposition and personalized federated learning, comprising:
[0109] The ultra-short-term load data analysis module is used to mine the temporal correlation and spatial distribution characteristics in ultra-short-term load data;
[0110] The CP tensor decomposition module is used to extract and compress features from the parameters of the personalized model for ultra-short-term load forecasting, thereby obtaining a local tensor model.
[0111] The model training objective design module is used to analyze the dual decoupling characteristics of the tensor decomposition personalized federated learning ultra-short-term load prediction model, so as to design the objective function for training the tensor decomposition personalized federated learning ultra-short-term load prediction model.
[0112] The tensor aggregation strategy design module is used to analyze the parameter tensor characteristics of the tensor decomposition personalized federated learning ultra-short-term load prediction model in order to design an integrated tensor aggregation strategy.
[0113] The ultra-short-term load forecasting optimization module is used to design training strategies for the tensor decomposition personalized federated learning ultra-short-term load forecasting model, and to update the global ultra-short-term load forecasting model by integrating tensor aggregation strategies to achieve ultra-short-term load forecasting.
[0114] This invention provides a computer storage medium storing a computer program, which, when run, implements the prediction method.
[0115] The architecture of the ultra-short-term load prediction model based on tensor decomposition and personalized federated learning designed in this invention is as follows: Figure 2 As shown. Based on the model's dual decoupling and tensor aggregation characteristics, a training strategy for a tensor decomposition personalized federated learning ultra-short-term load prediction model was designed. The training process is as follows: Figure 3As shown in the figure. The example uses electricity load data provided by the Independent System Operator of New England (ISO-NewEngland), sampled every 5 minutes. To improve the reliability of the experimental results, the dataset was first denoised, including filling missing values with local linear interpolation and removing duplicate data. Secondly, to test the model's generalization ability and robustness, the data was downsampled to generate 1-hour datasets (HD), 30-minute datasets (Semi-Hourly Dataset, SHD), and 15-minute datasets (Quarter-Hourly Dataset, QD).
[0116] During the verification process, the proposed method (FedACT) was compared with two benchmark frameworks, pFedMe and FedAvg. Figure 4 (a) shows the performance of each model on the RD dataset. The blue pentagram, gray triangle, and orange sphere represent the error metrics (MAXAE, MAE, RMSE) of the tensor decomposition-based personalized federated learning ultra-short-term load prediction model (FedACT), pFedMe, and FedAvg, respectively, on the RD dataset. Generally, models closer to the bottom right corner of the graph tend to have better performance. Figure 4 (a) It can be seen that FedACT performs best across all metrics, especially in improving forecast accuracy and reducing extreme errors. Figure 4 (b) Visually illustrates the differences between the predicted and actual values of each model on the RD dataset. The black, green, and orange solid lines represent the load predictions of FedACT, FedAvg, and pFedMe on the RD dataset, respectively. Figure 4 (b) It can be seen that the load curve predicted by FedACT best matches the actual load curve, capturing the spatiotemporal variability of power load data; followed by FedAvg, which also explains the variation pattern of power load well; while pFedMe is slightly inferior, with its predicted load curve showing some discrepancies with the actual load curve in some intervals. In summary, the ultra-short-term load forecasting method based on tensor decomposition personalized federated learning proposed in this patent significantly outperforms traditional methods in fitting the load curve.
[0117] also, Figure 5 The four graphs illustrate the convergence performance of three benchmark frameworks—FedACT, PedAvg, and pFedMe—on the RD, HD, SHD, and QD datasets, respectively. The purple, pink, and blue solid lines represent the loss function values of FedACT, PedAvg, and pFedMe at each communication round, respectively. Figure 5 (a) As can be seen, on the RD dataset, FedACT achieved the lowest loss value and demonstrated the highest accuracy in ultra-short-term power load forecasting. Although FedAvg had a similar convergence speed, it exhibited overfitting in the later stages of training; while pFedMe showed a rapid decrease in loss value in the initial stage, but subsequently oscillated and failed to converge effectively. Figure 5 (b) It can be seen that on the QD dataset, FedACT outperforms the benchmark methods in both convergence speed and final loss value. Figure 5 As shown in (c) and (d), FedACT maintains the fastest convergence speed on the SHD and HD datasets. Although its loss value is slightly higher than FedAvg, this slight difference has a negligible impact on prediction accuracy. pFedMe, on the other hand, consistently exhibits a higher loss value, with both prediction accuracy and convergence speed lower than FedACT. In summary, FedACT consistently demonstrates the highest stability, achieving fast convergence with the fewest communication rounds and achieving low convergence loss values across various datasets.
[0118] Although embodiments and drawings of the present invention have been disclosed for illustrative purposes, those skilled in the art will understand that various substitutions, variations and modifications are possible without departing from the spirit and scope of the present invention and the appended claims. Therefore, the scope of the present invention is not limited to the contents disclosed in the embodiments and drawings.
Claims
1. A method for ultra-short-term load forecasting based on tensor decomposition and personalized federated learning, characterized in that: The steps of the method are as follows: S1. Based on the spatiotemporal coupling characteristics of ultra-short-term load data, we will delve into the temporal correlation and spatial distribution features of the data, design a deep neural network on the server side, and construct a global model for ultra-short-term load prediction. S2. Broadcast the constructed ultra-short-term load forecast global model to the clients as their personalized models. S3. Use CP tensor decomposition to extract and compress features from the personalized model parameters, and use the decomposed model as the tensor local model for each client. S4. Design the objective function of the tensor decomposition personalized federated learning ultra-short-term load prediction model, and introduce a regularization term to control the Euclidean distance between the personalized model and the tensor local model. S5. Design a training strategy for a tensor decomposition personalized federated learning ultra-short-term load prediction model. Train the personalized model on the client side, update the personalized model parameters by minimizing the objective function, and decouple the tensor local model to complete the tensor local model parameter update with the trained personalized model as the target. S6. Based on the characteristics of the personalized federated learning ultra-short-term load prediction model of tensor decomposition, an integrated tensor aggregation strategy is designed. Each client transmits the trained and updated tensor local model parameters to the server to realize tensor parameter aggregation, complete the global model parameter update for ultra-short-term load prediction, and realize ultra-short-term load prediction.
2. The ultra-short-term load forecasting method based on tensor decomposition and personalized federated learning according to claim 1, characterized in that: Specifically, S1 is: The input feature, i.e., the historical load sequence x, is first mapped to the intermediate feature space through the first fully connected layer. The calculation process is as follows: h1 = RELU(W1x + b1); in: This is the weight matrix; It is the bias vector; For the real number space; d mid d represents the dimension of the intermediate feature space. in is the dimension of the input features; x is the historical load sequence; ReLU is the activation function used to introduce nonlinearity and enhance the model's ability to represent the complex semantic relationships between data caused by high load volatility; The output h1 of the intermediate layer is mapped to the output layer through the second fully connected layer. The calculation formula is as follows: y = W2h1 + b2; Where: y is the load value predicted by the model; This is the weight matrix for the second layer; This is the corresponding bias vector; For the real number space; d out d represents the dimension of the output feature. mid Let be the dimension of the intermediate feature space.
3. The ultra-short-term load forecasting method based on tensor decomposition and personalized federated learning according to claim 1, characterized in that: Specifically, S3 is: The weights of the fully connected layers in each client-specific personalized model are decomposed using CP tensors. The resulting prediction model weights are represented as the product of multiple low-rank tensor factors. Where: W is the weight matrix; A (1) and A (2) For factor matrix; α r (1) and ɑ r (2) A respectively (1) and A (2) The r-th column vector; R is the rank of the weight matrix; Each element w in the weight matrix W i,j All can be obtained through factor matrix A (1) and A (2) The sum of the products of the corresponding column vector elements is represented as: The fully connected layer after tensor decomposition is represented as: y′≈A (1) (A (2) ) T x+b; Where: y′ is the predicted load value of the fully connected layer; A (1) and A (2) is the factor matrix; x is the historical load sequence; b is the bias vector of the fully connected layer.
4. The ultra-short-term load forecasting method based on tensor decomposition and personalized federated learning according to claim 1, characterized in that: Specifically, S4 is: The original high-dimensional weight parameters Let denot be an N-order tensor, and convert it into a CP tensor decomposition of rank R, denoted as . Where: factor matrix The r-th column vector is ɑ r (n) , n=1,...,N, and A (n) =[ɑ1 (n) ,…,ɑ R (n) ]; For the real number space; I1~I N R represents the length of each dimension; R is the rank of the factor matrix. The objective function is defined as: in: It is the local dataset of the kth client; x represents the number of local samples for the k-th client; i It is the input sequence of the sample; y i It is the target value; F k The local objective function of the k-th client is defined as: Where: θ k It is the personalized model for the kth client; f k (θ k ) is the expected value of the loss prediction of the personalized model on the local data distribution of client k; It is a local model of tensors; F k (·) contains the l2 norm regularization term. The distance between the personalized model and the tensor local model is controlled by λ, and the degree of regularization is controlled by λ.
5. The ultra-short-term load forecasting method based on tensor decomposition and personalized federated learning according to claim 1, characterized in that: Specifically, S5 is: On the client side, the tensor local model is first used as the reference center point, and the objective function F is then used to... k The personalized model is trained by minimizing the input; then, based on the optimal personalized model, the tensor local model is decoupled, and the factor matrix of the tensor local model is uploaded to the server. After the server aggregates the factor matrix, it broadcasts the aggregated tensor and factor matrix to the client respectively, as the personalized and tensor local models, until the iteration is complete; 1) Initialization: The server first initializes a global model for ultra-short-term load forecasting. And broadcast it to the client; 2) Client-side model training: Assume there are a total of T global communication rounds. In each global communication round t = 1, ..., T, the client performs τ local update rounds, training the personalized model and the tensor local model sequentially. In each local update round t′ = 1, ..., τ, each client first uses... Used as a reference center to train its personalized model Then, the client uses the obtained approximate solution. To train a tensor local model This phase is repeated for T rounds until the iteration is complete; 3) Model aggregation and update: Under the federated learning framework, considering the instability of network connections, the server usually sets a fixed subset C of clients of size C. t Model aggregation is performed, and each selected client updates its local model. k∈C t The data is sent to the server, where the integrated tensor aggregation strategy is used to update the global model for ultra-short-term load forecasting. The updated global model for ultra-short-term load forecasting is then broadcast to the client, and this process continues until the iteration ends.
6. The ultra-short-term load forecasting method based on tensor decomposition and personalized federated learning according to claim 1, characterized in that: Specifically, S6 is: Design an integrated tensor aggregation strategy to reconstruct the factor matrix of the client-side tensor local model into complete tensor parameters: in: The approximate model parameters are obtained by recombining the factor matrix after the t-th round of global communication and the τ-th local update for client k. The server calculates the tensor based on the weight of each client's data volume. The average value is used to control the aggregation direction of the global model for ultra-short-term load forecasting via β: Where: C t This refers to the collection of clients selected during the aggregation process. The aggregated global model for ultra-short-term load forecasting is expressed as follows: and These correspond to the parameters of the full-version model and the parameters of the tensorized model, respectively:
7. An ultra-short-term load forecasting system utilizing the ultra-short-term load forecasting method based on tensor decomposition personalized federated learning as described in any one of claims 1 to 6, characterized in that: include: The ultra-short-term load data analysis module is used to mine the temporal correlation and spatial distribution characteristics in ultra-short-term load data; The CP tensor decomposition module is used to extract and compress features from the parameters of the personalized model for ultra-short-term load forecasting, thereby obtaining a local tensor model. The model training objective design module is used to analyze the dual decoupling characteristics of the tensor decomposition personalized federated learning ultra-short-term load prediction model, so as to design the objective function for training the tensor decomposition personalized federated learning ultra-short-term load prediction model. The tensor aggregation strategy design module is used to analyze the tensor quantization characteristics of parameters in the tensor decomposition personalized federated learning ultra-short-term load prediction model in order to design an integrated tensor aggregation strategy. The ultra-short-term load forecasting optimization module is used to design training strategies for tensor decomposition personalized federated learning ultra-short-term load forecasting models, and to update the global model parameters of ultra-short-term load forecasting through integrated tensor aggregation strategies to achieve ultra-short-term load forecasting.
8. A computer storage medium, characterized in that: The computer storage medium stores a computer program, which, when run, implements the prediction method as described in any one of claims 1 to 6.