A short-term residential load forecasting approach based on federated learning for socially aware analytics

Abstract

The present invention proposes a short-term residential load forecasting method based on social consciousness analysis federated learning, which belongs to the field of federated learning. It includes: constructing a local SGTSMixer model, inputting the historical load data of smart meters and the social demographic characteristics of different families in the region into the SGTSMixer model, outputting the predicted load data, and obtaining the local social consciousness analysis results; aggregating the global model, sending the local model parameters and social consciousness analysis results of all regions to the central server for aggregation, and generating representative global personalized models for each region; updating the local model, constructing a unified loss function, aligning the local model of each region with the global personalized model of the central server, updating the local model of each region according to the loss function value, and iteratively calculating the new local model and the global personalized model. While protecting the privacy of regional data, the present invention solves the problem that the social demographic characteristics of families in different regions show different distributions, and each region can also collaboratively improve the model.

Description

Technical Field

[0001] The present invention belongs to the field of federated learning, and in particular relates to a short-term residential load forecasting method based on social awareness analysis federated learning. Background Art

[0002] Residential short-term load forecasting (STLF) plays a key role in ensuring efficient operation and management of the power grid by accurately predicting the household's electricity consumption pattern over a short period of time (usually a few hours to a few days). Closely related to other smart grid initiatives such as demand response and distribution systems, existing research and utility practices have shown that accurate short-term forecasting enables utility companies to address various challenges brought about by the dynamic characteristics of residential energy consumption, optimize resource allocation, ensure grid stability, facilitate demand-side management, and improve customer satisfaction.

[0003] To perform short-term load forecasting in residential areas, traditional load forecasting techniques mainly rely on historical load data collected through advanced metering infrastructure (AMI) at the household level. This data is then transmitted to a central server through a communication network for forecast-related analysis. However, people are increasingly aware of the privacy issues in this process, which mainly stems from two aspects: 1) Centralized analysis requires sufficient communication and computing resources to collect, host, and process the collected user data, which makes the computational cost expensive when the problem scale is large; 2) Smart meter data can often be easily linked to specific individuals, which makes its transmission and storage pose a risk of privacy infringement by leaking information such as appliance usage patterns and user lifestyle details. Therefore, it is generally agreed that traditional centralized load forecasting is no longer considered a viable approach.

[0004] To address the above two major issues associated with smart meter data, federated analytics and federated learning are considered promising solutions, i.e., sharing locally trained models instead of original training data. However, most existing load forecasting federation methods are based only on geographic location and regional scale. When deploying federated learning models for cross-regional load forecasting, the different socio-demographic characteristics of consumers in the customer base may affect the accuracy and efficiency of the forecasting model. Summary of the invention

[0005] The existing short-term load forecasting methods do not consider the different distribution of socio-demographic characteristics among different customer groups, nor do they build personalized forecasting models suitable for specific regions. The purpose of the present invention is to provide a short-term residential load forecasting method based on social awareness analysis federated learning.

[0006] To solve the above technical problems, the short-term residential load forecasting method based on social awareness analysis federated learning of the present invention comprises the following steps:

[0007] Step S1: construct a local SGTSMixer model, input the historical load data of smart meters and the socio-demographic characteristics of different households in the region as input data into the SGTSMixer model, output the predicted load data, and obtain the local social awareness analysis results;

[0008] Step S2: Aggregate the global model, send the local model parameters and social awareness analysis results of all regions to the central server for aggregation, and generate representative global personalized models for each region;

[0009] Step S3: Update the local model, build a unified loss function, align the local model of each region with the global personalized model of each region on the central server, update the local model of each region according to the loss function value, and iteratively calculate the new local model parameters of each region and the global personalized model of each region.

[0010] Furthermore, the local SGTSMixer model consists of two parts, namely, 4 hybrid MLPs and a temporal projection mechanism; each hybrid MLP consists of a self-gated temporal hybrid MLP and a self-gated feature hybrid MLP;

[0011] The self-gated temporal mixing MLP consists of three parts; the first part consists of the first fully connected layer, the first ReLU activation function layer, and the first The first part is composed of the first self-gating layer and the first Sigmoid activation function layer; the third part is the residual connection;

[0012] The self-gated feature hybrid MLP consists of three parts; the first part consists of the second fully connected layer, the second ReLU activation function layer, the second Dropout layer, the third fully connected layer and the third Dropout layer; the second part consists of the second self-gating layer and the second Sigmoid activation function layer; the third part is the residual connection.

[0013] Furthermore, the input data is obtained as follows:

[0014] The first Specific socio-demographic characteristics of users in each region Get a vector by embedding ,vector For one A real matrix of size , where Indicates the original input length; Represents the quantity of a specific socio-demographic characteristic. With historical load data Splicing to get input data , input data For one A real matrix of size , where Represents the number of variables contained in the multivariate time series; expressed as:

[0015]

[0016]

[0017] in, represents embedding, Indicates a A real matrix of size , Indicates splicing, Indicates a A real matrix of size .

[0018] Furthermore, the whole process in the first hybrid MLP is expressed as:

[0019]

[0020] The output of one hybrid MLP is used as the input of the next hybrid MLP, expressed as:

[0021]

[0022]

[0023]

[0024]

[0025]

[0026] Represents input data After the first hybrid MLP processing, we get , represents the first hybrid MLP processing operation; , indicating that Send it to the second hybrid MLP for processing and get , represents the second hybrid MLP processing operation; Indicates that Send it to the third hybrid MLP for processing and get , represents the third hybrid MLP processing operation; Indicates that Send it to the fourth hybrid MLP for processing and get , Represents the fourth hybrid MLP processing operation.

[0027] In general, the input data After processing by 4 hybrid MLPs, the output is , using the formula To express, represents 4 mixed MLP processing operations, is a A real matrix of size .

[0028] Furthermore, for the Regions have specific socio-demographic characteristics family, define the data set for ,in Indicates Data for each region, Indicates The historical load data of the ith household in the region, Indicates The actual load value of the ith household in the region, Indicates The socio-demographic characteristics of the ith household in the region are Features of; Dataset Contains historical load data, actual load values ​​and specific socio-demographic characteristics of different households , No. Regions have specific socio-demographic characteristics The local social awareness analysis results for all households are calculated as follows:

[0029]

[0030] Express Data for all households in the dataset To splice, It means that after splicing, 4 mixed MLPs are used and the average value is taken to calculate the first Regions have specific socio-demographic characteristics Results of the local social awareness analysis of all households .

[0031] Further, step S2 is obtained by:

[0032] First, the similarity matrix is ​​calculated for the specific socio-demographic characteristics of users in all regions. , similarity matrix The calculation is as follows:

[0033] S I M s = [ s i m ( a 1 , s , a 1 , s ) ⋯ s i m ( a 1 , s , a t , s ) ⋯ s i m ( a 1 , s , a R , s ) ⋮ ⋮ ⋮ ⋮ ⋮ s i m ( a r , s , a 1 , s ) ⋯ s i m ( a r , s , a t , s ) ⋯ s i m ( a r , s , a R , s ) ⋮ ⋮ ⋮ ⋮ ⋮ s i m ( a R , s , a 1 , s ) ⋯ s i m ( a R , s , a t , s ) ⋯ s i m ( a R , s , a R , s ) ]

[0034] in , indicating the Regions have specific socio-demographic characteristics Results of the local social awareness analysis of all households With Regions have specific socio-demographic characteristics Results of the local social awareness analysis of all households The similarity between express and The dot product of yes The L2 norm of yes The L2 norm of , indicating the Regions, , indicating the Regions, Represents the total number of regions, that is, the number of all regions involved in the calculation.

[0035] Secondly, according to the similarity matrix Middle All similarities of the rows are calculated Specific socio-demographic characteristics of users in each region Global Personalization Model , expressed as:

[0036]

[0037] in, Indicates The local model parameters for each region. Regions and all regions (including The local model parameters of different regions are weighted according to the similarity. Specifically, first The similarity between each region Multiply the model parameters for the region , and then sum the results of all regions. Finally, the weighted sum is normalized using the sum of similarities of all regions to ensure that the output model parameters It is the result based on similarity normalization.

[0038] By aggregating A global personalized model for all socio-demographic characteristics of users in a region , get the Global personalized model for all socio-demographic characteristics of users in regions , No. Global personalized model for all socio-demographic characteristics of users in regions Defined as:

[0039]

[0040] in, Indicates All socio-demographic characteristics of the region.

[0041] Furthermore, the unified loss function in step S3 is Calculated as follows:

[0042]

[0043]

[0044]

[0045] in, represents a unified loss function, represents the local load forecasting loss of the local SGTSMixer model; represents personalized learning loss; is a hyperparameter used to balance the three objectives.

[0046] middle Indicates Data for each region, Indicates Regions Historical load data of households, Indicates Regions The actual load value of a household, Indicates Regions The socio-demographic characteristics of the families are characteristics. Indicates Local models for each region, Indicates that and Send to Local models for each region To make predictions, use To indicate the prediction result obtained through the local model, Represents the square of the L2 norm. Represents the dataset For each sample in the model, that is, the error between the predicted value and the actual load value of each household is calculated to obtain the total loss.

[0047] middle, It is Data for each region A subset of Indicates The specific socio-demographic characteristics of each region are data; Indicates Local models for each region, Indicates Regions have specific socio-demographic characteristics The global personalized model will and Send to The local model of the region and Regions have specific socio-demographic characteristics The global personalized model is then used to calculate the difference between the two. Represents the square of the L2 norm.

[0048] Furthermore, updating the local model in step S3 is achieved by:

[0049] Minimize the uniform loss function through the optimization process To update The local model for each region, usually using a variant of stochastic gradient descent, is defined as:

[0050]

[0051] It is The local model parameters for each region are represents the learning rate, is the loss function Relative to The gradient of , which represents the rate of change of the loss function under the current parameter value. This formula represents the model parameters Updated by gradient descent, at each iteration, the parameters Subtract the learning rate Multiply by the gradient To reduce the value of the loss function. This process is repeated until the model parameters converge, that is, the loss function reaches a minimum. After the local model is updated, each region calculates its modified local model parameters and transmits these updates to the central server to update the global personalized model for each region.

[0052] The short-term residential load forecasting method based on social awareness analysis federated learning of the present invention has the following beneficial technical effects:

[0053] 1) Construct the SocioFA framework to process the load data of specific regions. It adopts the federated learning framework. The central server and regions can communicate through model parameters. The data storage and model training process are implemented in a decentralized manner based on the federated learning framework, which protects the data privacy of the region. While protecting the data privacy of the region, it solves the problem that the socio-demographic characteristics of households in different regions show different distributions. The regions can also work together to improve the model.

[0054] 2) A novel model structure of SGTSMixer is constructed, which can skillfully extract the temporal patterns present in time series data and utilize the stacked MLP and self-gating mechanism to exploit the cross-variable information, thereby improving the efficiency and effectiveness and enhancing the accuracy of short-term residential load forecasting. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0056] Figure 1 It is a schematic diagram of the SGTSMixer local model structure of the present invention.

[0057] Figure 2 It is a schematic diagram of the aggregation structure of the global personalized models for each region of the present invention.

[0058] Figure 3 It is a schematic diagram of the local model update framework of the present invention.

[0059] Figure 4a Schematic diagram of short-term load forecasting (STLF) training loss under different frameworks.

[0060] Figure 4b Schematic diagram of MSE estimation for the short-term load forecasting (STLF) test set under different frameworks.

[0061] Figure 4c Schematic diagram of MAE evaluation on the short-term load forecasting (STLF) test set under different frameworks.

[0062] Figure 4d A test set for short-term load forecasting (STLF) under different frameworks Evaluation diagram.

[0063] Figure 5a Schematic diagram of the MSE evaluation of the test set for short-term load forecasting (STLF) under different model structures.

[0064] Figure 5b Schematic diagram of MAE evaluation of the test set for short-term load forecasting (STLF) under different model structures.

[0065] Figure 5c The test set for short-term load forecasting (STLF) under different model structures Evaluation diagram. DETAILED DESCRIPTION

[0066] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0067] When deploying a federated learning model of smart meter data for cross-regional load forecasting, the different socio-demographic characteristics of consumers in the customer base will affect the accuracy and efficiency of the forecasting model. This paper studies the heterogeneity of location-based characteristics of residential consumers, such as employment status, income, age, and family size, to provide important insights for predicting consumers' electricity consumption patterns. This paper specifically considers the different socio-demographic characteristics of consumers in customer bases in different regions and uses a federated learning framework to share model parameters.

[0068] The proposed SocioFA (Socio-aware Federated Analytics) framework for social-aware analytics federated learning is designed to address the challenges of heterogeneous data and models. The framework combines local social-aware analysis of socio-demographic characteristics and promotes the construction of local forecasting models suitable for each region to improve the accuracy of short-term residential load forecasting. The overall approach is:

[0069] 1) Initial round: Each region (denoted as ) Using historical load data from different households and specific socio-demographic characteristics To train local models and generate local prediction data and local social awareness analysis results After conducting individual training for each region, social awareness analysis results and local model parameters Will be transmitted to a central server.

[0070] 2) Subsequent training rounds: The central server evaluates the similarities between local social awareness analysis results and coordinates the aggregation of global personalized models for each region. The final result of the aggregation process is to send the global personalized models of each region back to each region, thus starting a new round of local model updates and global aggregation. This process is repeated until training is completed.

[0071] 3) When updating the local model, the local model generates local forecast data and local social awareness analysis results, and the load forecast loss is calculated using the actual value and the local forecast data. At the same time, the same data is input into the global personalized model that predicts future load data. The personalized learning loss comes from the difference between the local forecast and the global forecast. The local model of the region is then improved by minimizing these losses, so that each region can obtain a global personalized model while improving the accuracy of load forecasting.

[0072] In order to achieve the above objectives, the present invention proposes a short-term residential load forecasting method based on social awareness analysis federated learning, namely the SocioFA framework, which includes the following steps:

[0073] Step S1: Construct a local SGTSMixer (Self-Gating TSMixer) model. The historical load data of smart meters and the socio-demographic characteristics of different households in the region are input into the local SGTSMixer model, which outputs the predicted load data and obtains the local social awareness analysis results.

[0074] Specifically, Figure 1 As shown, step S1 includes the following steps:

[0075] The local SGTSMixer model is represented as , representing The local model of each region has model parameters: , which is carefully designed for load forecasting and providing analytical insights using meter data. The model consists of two main parts: 4 hybrid MLPs (Multi-Layer Perceptrons) and a time projection mechanism; each hybrid MLP consists of a self-gated time hybrid MLP and a self-gated feature hybrid MLP.

[0076] The self-gated temporal mixture MLP is designed to capture temporal patterns in time series data. The self-gated temporal mixture MLP consists of three parts. The first part consists of the first fully connected layer, the first ReLU activation function layer, and the first layers, which transform the input and apply a fully connected layer in the time dimension. The second part consists of the first self-gating layer (Gate-FC) and the first The third part is the residual connection, which improves learning by connecting the input data with the output of the first and second parts.

[0077] The self-gated feature mixture MLP is shared between different time steps and used to exploit covariate information. The present invention utilizes two-layer MLP to facilitate the learning transformation of complex features. The self-gated feature mixture MLP includes three parts; the first part consists of the second fully connected layer, the second ReLU activation function layer, the second Dropout layer, the third fully connected layer and the third Dropout layer, and the second part consists of the second self-gated layer (Gate-FC) and the second Sigmoid activation function layer; the third part is a residual connection, which improves learning by connecting the input data and the output of the first and second parts.

[0078] The temporal projection mechanism involves a fully connected layer applied to the time domain. This not only enables the model to capture temporal patterns, but also transforms the time series data from its original input length Mapping to the desired prediction length .

[0079] No. The historical load data of users in each region is used Said that Specific socio-demographic characteristics of users in different regions Said that Historical load data and specific socio-demographic characteristics of different household users in different regions Represented as a local SGTSMixer model of input, and ,in Indicates Data for each region, Indicates Regions Historical load data of households, Indicates Regions The actual load value of a household, Indicates Regions Sociodemographic characteristics of the family.

[0080] In order to obtain the input data of the local SGTSMixer model, Specific socio-demographic characteristics of users in each region Get a vector by embedding ,vector For one A real matrix of size , where represents the original input length (the number of historical data points used for prediction), Represents the quantity of a specific socio-demographic characteristic. With historical load data Splicing to get input data , input data For one A real matrix of size , where Represents the number of variables contained in the multivariate time series; expressed as:

[0081]

[0082]

[0083] in, represents embedding, Indicates a A real matrix of size , Indicates splicing, Indicates a A real matrix of size .

[0084] Enter the data The four hybrid MLPs are used to process the data in sequence. For the first hybrid MLP, the processing operation is expressed as , input data It is fed into the first part of the self-gated time mixture MLP, and is processed by the first fully connected layer, the first ReLU activation function layer, and the first Dropout layer to obtain the output of the first part of the self-gated time mixture MLP. , which can be expressed as:

[0085] X t = D r o p o u t [ R e L U ( W t r X e ⊕ b t r ) ]

[0086] in, represents the operation of adding the elements of each column; , are the weights and biases of the first fully connected layer of the self-gated temporal mixture MLP, For one A real matrix of size , For one A real matrix of size ; express The activation function is expressed as , nonlinearity is incorporated into the model; in addition, express Operation is a regularization technique that aims to reduce model overfitting. It is used to randomly deactivate neurons during training to alleviate overfitting in neural networks.

[0087] In addition, input data It is also fed into the second part of the self-gated temporal mixing MLP, passing through the first self-gating layer and the first The activation function layer is processed to obtain the output of the second part of the self-gated time mixing MLP , which can be expressed as:

[0088]

[0089] in, , are the weights and biases of the first self-gated layer of the self-gated temporal mixture MLP, For one A real matrix of size , For one A real matrix of size .

[0090] express The function, which is crucial for introducing nonlinearity, is defined as:

[0091]

[0092] The output of the first part of the self-gated temporal mixing MLP And the output of the second part End with element-wise multiplication to get the output of the first and second parts of the self-gated time-mixing MLP , which is expressed as:

[0093]

[0094] in represents element-wise multiplication, is a A real matrix of size .

[0095] An important aspect of the present invention is to combine the outputs of the first and second parts of the self-gated time mixing MLP With input data The output of the self-gated temporal mixing MLP is obtained by combining the residual connections , as shown below:

[0096]

[0097] Residual connections are crucial in enabling the model to effectively learn more complex architectures and selectively avoid unnecessary temporal mixing operations.

[0098] Then, the output of the self-gated temporal mixing MLP is Transpose As the input of the self-gated feature mixture MLP, is a A real matrix of size .

[0099] Input to the self-gated feature mixture MLP It is fed into the first part of the self-gated feature mixture MLP, and the output is obtained through the second fully connected layer, the second ReLU activation function layer, and the second Dropout layer. , output After processing by the third fully connected layer and the third Dropout layer, the output of the first part of the self-gated feature hybrid MLP is obtained , which can be expressed as:

[0100] X f 1 T = D r o p o u t [ R e L U ( W f 1 r X t o r T ⊕ b f 1 r ) ]

[0101]

[0102] in, , represents the weights and biases of the second fully connected layer of the self-gated feature mixture MLP, is a A real matrix of size , is a A real matrix of size ; , represents the weights and biases of the third fully connected layer of the self-gated feature mixture MLP, is a A real matrix of size , is a A real matrix of size .

[0103] In addition, the input of the self-gated feature mixing MLP It is also fed into the second part of the self-gated feature mixture MLP, through the second self-gating layer and the second The activation function layer is processed to obtain the output of the second part of the self-gated feature mixing MLP , which can be expressed as:

[0104]

[0105] in, , are the weights and biases of the second self-gating layer of the self-gated feature mixing MLP, For one A real matrix of size , For one A real matrix of size .

[0106] Output of the first part of the self-gated feature mixture MLP And the output of the second part End with element-wise multiplication to get the output of the first and second parts of the self-gated feature mixture MLP , which is expressed as:

[0107]

[0108] is a A real matrix of size .

[0109] It is worth noting that the self-gated feature mixing MLP output also includes a residual connection, which is defined as: the output of the first and second parts of the self-gated feature mixing MLP Mixing the input of the MLP with the self-gated features The output of the self-gated feature hybrid MLP is obtained by combining the residual connections , as shown below:

[0110]

[0111] Will After transposing, we get , is a Therefore, the entire process in the first hybrid MLP can be expressed as:

[0112]

[0113] The present invention has 4 hybrid MLPs, each of which has a unified structural design but different parameters. The output of one hybrid MLP is used as the input of the next hybrid MLP, which can be expressed as:

[0114]

[0115]

[0116]

[0117]

[0118]

[0119] Represents input data After the first hybrid MLP processing, we get , represents the first hybrid MLP processing operation; , indicating that Send it to the second hybrid MLP for processing and get , represents the second hybrid MLP processing operation; Indicates that Send it to the third hybrid MLP for processing and get , It means that it has been processed by three hybrid MLPs; Indicates that Send it to the fourth hybrid MLP for processing and get , Represents the fourth hybrid MLP processing operation.

[0120] In general, the input data After processing by 4 hybrid MLPs, the output is , using the formula To express, represents 4 mixed MLP processing operations, is a This output is then fed to the temporal projection layer of the temporal projection mechanism, which is projected by the fourth fully connected layer. , expressed as:

[0121]

[0122] in , represents the weights and biases of the fourth fully connected layer, is a A real matrix of size , is a A real matrix of size , is the required prediction length; is a A real matrix of size ; then the resulting projection For load forecasting, focus on The first line of the local load forecasting result data is obtained, which is expressed as .

[0123] The local model load forecast is analyzed simultaneously. Regions have specific socio-demographic characteristics family, define the data set for , where the dataset Contains historical load data for different households , actual load value and sociodemographic characteristics , No. Regions have specific socio-demographic characteristics The local social awareness analysis results for all households are calculated as follows:

[0124]

[0125] Express Data for all households in the dataset To splice, It means that after splicing, four mixed MLP operations are performed and then the average value is taken to calculate the first Regions have specific socio-demographic characteristics Results of the local social awareness analysis of all households .

[0126] gather Indicates that for Socio-demographic characteristics of users in the region , Indicates All socio-demographic characteristics of the region, It is Results of social awareness analysis for all households in all regions with all socio-demographic characteristics The social consciousness analysis results of all households with all socio-demographic characteristics after aggregation are compared with the local model parameters , transmitted together to the central server.

[0127] Step S2: Aggregate the global model, send the local model parameters and social awareness analysis results of all regions to the central server for aggregation, and generate a global personalized model representative of each region.

[0128] Specifically, Figure 2 As shown, step S2 includes the following steps:

[0129] On the central server side, local model parameters and local socio-demographic analysis results for all regions are uploaded. In order to obtain a global personalized model for each region, the similarity matrix is ​​first calculated. Specifically, for the specific socio-demographic characteristics of users in all regions , similarity matrix The calculation is as follows:

[0130] S I M s = [ s i m ( a 1 , s , a 1 , s ) ⋯ s i m ( a 1 , s , a t , s ) ⋯ s i m ( a 1 , s , a R , s ) ⋮ ⋮ ⋮ ⋮ ⋮ s i m ( a r , s , a 1 , s ) ⋯ s i m ( a r , s , a t , s ) ⋯ s i m ( a r , s , a R , s ) ⋮ ⋮ ⋮ ⋮ ⋮ s i m ( a R , s , a 1 , s ) ⋯ s i m ( a R , s , a t , s ) ⋯ s i m ( a R , s , a R , s ) ]

[0131] in , indicating the Regions have specific socio-demographic characteristics Results of the local social awareness analysis of all households With Regions have specific socio-demographic characteristics Results of the local social awareness analysis of all households The similarity between express and The dot product of yes The L2 norm of yes The L2 norm of Indicates Regions, Indicates Regions, Represents the total number of regions, that is, the number of all regions involved in the calculation.

[0132] Next, according to the similarity matrix To calculate the global personalized model for each region, specifically, according to the similarity matrix Middle All similarities of the rows are calculated Socio-demographic characteristics of users in each region Global Personalization Model , expressed as:

[0133]

[0134] in, Indicates The local model parameters of the region are calculated by the formula Regions and all regions (including The local model parameters of different regions are weighted according to the similarity. Specifically, first The similarity between each region Multiply the model parameters for the region , and then sum the results of all regions. Finally, the weighted sum is normalized using the sum of similarities of all regions to ensure that the output model parameters It is the result based on similarity normalization.

[0135] By aggregating A global personalized model for all socio-demographic characteristics of users in a region , get the Global personalized model for all socio-demographic characteristics of users in regions , No. Global personalized model for all socio-demographic characteristics of users in regions Defined as:

[0136]

[0137] in, Indicates All socio-demographic characteristics of the region.

[0138] Step S3: Update the local model, build a unified loss function, align the local model of each region with the global personalized model of each region on the central server, update the local model of each region according to the loss function value, and iteratively calculate the new local model parameters of each region and the global personalized model of each region.

[0139] Specifically, Figure 3 As shown, step S3 includes the following steps:

[0140] First, the historical load data and socio-demographic characteristics of different households in each region are processed through the local model. This processing produces local forecast data results and local social awareness analysis results. The load forecast loss is calculated using the actual values ​​and the local forecast data results. In order to update the The local model of each region, the unified loss function consists of two parts, expressed as , a unified loss function Calculated as follows:

[0141]

[0142] in, represents a unified loss function, represents the local load forecasting loss of the local SGTSMixer model; represents personalized learning loss; is a hyperparameter used to balance the three objectives.

[0143] It is expressed as the mean square loss (MSE) loss calculated based on the actual load value and the predicted load value.

[0144]

[0145] in, Indicates Data for each region, Indicates Regions Historical load data of households, Indicates Regions The actual load value of a household, Indicates Regions Sociodemographic characteristics of the family. Indicates Local models for each region, Indicates that and Feed into the local model of the rth region To make predictions, use To indicate the prediction result obtained through the local model, Represents the square of the L2 norm. Represents the dataset For each sample in the model, that is, the error between the predicted value and the actual load value of each household is calculated to obtain the total loss.

[0146] At the same time, the same data is input into the next The personalized learning loss is derived from the difference between the local prediction and the global prediction. represents the personalized learning loss, which aims to minimize the difference between the load forecast of the same socio-demographic characteristics and the load forecast of the global personalized model. Specifically, It can be expressed as:

[0147]

[0148] in, It is Data for each region A subset of Indicates The specific socio-demographic characteristics of each region are data; Indicates Local models for each region, Indicates Regions have specific socio-demographic characteristics The global personalized model will and Send to The local model of the region and Regions have specific socio-demographic characteristics The global personalized model is then used to calculate the difference between the two.

[0149] Then the unified loss function is minimized through the optimization process To update The local model for each region, usually using a variant of stochastic gradient descent, is defined as:

[0150]

[0151] It is The local model parameters for each region are represents the learning rate, is the loss function Relative to The gradient of , which represents the rate of change of the loss function under the current parameter value. This formula represents the model parameters Updated by gradient descent, at each iteration, the parameters Subtract the learning rate Multiply by the gradient To reduce the value of the loss function. This process is repeated until the model parameters converge, that is, the loss function reaches a minimum. After the local model is updated, each region calculates its modified local model parameters and transmits these updates to the central server to update the global personalized model for each region.

[0152] The present invention verifies the SocioFA framework by using the Irish Energy Regulator (CER) dataset, which was obtained during a smart metering electricity user behavior trial from July 14, 2009 to December 31, 2010. The dataset includes 30-minute electricity consumption records for 18 months, collected from 4232 Irish residential customers.

[0153] Information collected from the questionnaire survey revealed the social class of Irish resident clients, as shown in Table 1:

[0154] Table 1 Label table of family socio-demographic characteristics

[0155]

[0156] In order to analyze the impact of incorporating socio-demographic characteristics into the prediction model, the present invention gradually expanded the feature set, as shown in Table 2:

[0157] Table 2 Expanded family socio-demographic characteristics label table

[0158]

[0159] consider For each region, the data is divided into two parts: 80% of the data is used to train the model, and the remaining 20% ​​of the data is used to test the performance of the model. Set to 1, the learning rate Set to 0.0001, the original input length Set to 336, predicted length Set to 96, Set to 3, which contains the variables: load data, time data in hours, and time data in weeks.

[0160] The present invention employs a wide range of evaluation metrics to rigorously analyze the prediction accuracy of the model, especially the prediction results for a specific user group in context. After generating the predictions for that group, these metrics help compare the predicted values ​​with the actual values. This comparative analysis is critical to understanding the effectiveness and reliability of the model.

[0161] Mean Squared Loss (MSE) is a key metric that measures the average of the squared losses and the mean squared difference between the estimated and actual values. Its mathematical representation is as follows:

[0162]

[0163] in Indicates the total number of instances, represents the actual observed value, The model is The predicted value of instances. The average performance across all regions is measured.

[0164] Additionally, Mean Absolute Loss (MAE) is another important metric used in the analysis. It is the average of the absolute losses between the predicted and actual values, thus providing a linear score that does not overly penalize large deviations. MAE is given by:

[0165]

[0166] in , and Same as the previous definition.

[0167] at last, (i.e., coefficient of determination) is included in the evaluation metrics. This statistic measures the proportion of the variance in the dependent variable that can be predicted from the independent variables. It is a key indicator of the explanatory power of the model and is calculated as:

[0168]

[0169] in Represents the number of instances, and retain the meaning of actual and predicted values ​​separately, Represents the mean of the observations.

[0170] Each of these metrics provides a unique and important perspective on the model's ability to make accurate predictions, forming a comprehensive evaluation framework for the present study.

[0171] The effectiveness of the SocioFA framework of the present invention was evaluated and compared with three algorithms:

[0172] ①Centralized: In this approach, a central server trains a global model using all available data without considering privacy issues. This approach is expected to produce the highest performance benchmark.

[0173] ②FedAvg: A popular federated learning algorithm that utilizes a gradient-based mechanism and an averaging method to aggregate models across local updates.

[0174] ③Decentralized: Each region uses its own dataset to independently train the model without sharing personal data or model parameters, thereby protecting privacy.

[0175] MSE, MAE, The indicators are shown in Table 3:

[0176] Table 3 Comparison of different frameworks

[0177]

[0178] Table 3 lists the average MSE, MAE and The experimental design replicates the real-world situation, which is characterized by significant social class differences across regions. The results show that SocioFA outperforms FedAvg and Decentralized methods and approaches the benchmark set by Centralized methods.

[0179] The training loss of several algorithms with the communication rounds, MSE, MAE, Indicators such as Figures 4a to 4d As shown in Figure 2, the decrease in training loss and the MSE, MAE and The improvement of , highlights the ability of SocioFA in effective learning and generalization. It is worth noting that Figure 4a It is shown that SocioFA achieves faster convergence compared to other frameworks. Figure 4b , Figure 4c and Figure 4d It shows that the test evaluation stabilizes at the 200th communication round, after which additional training is stopped to prevent overfitting. Importantly, the test evaluation curves in the first 50 communication rounds show that SocioFA has a more significant decrease in MSE and MAE indicators than FedAvg. This indicates that SocioFA is able to absorb personalized information more quickly compared to FedAvg. These findings highlight the potential of SocioFA in integrating sociodemographic parameters to not only improve the accuracy of load forecasting, but also protect user privacy.

[0180] 1) Verify the effectiveness of the proposed local model SGTSMixer. Compare it with three different models:

[0181] ①Feedforward Neural Network (FNN): A traditional simple MLP model that can be applied to regression tasks in STLF.

[0182] ②LSTM: Long Short-Term Memory Network, a recurrent neural network known for its ability to capture temporal dependencies in time series data.

[0183] ③TSMixer: A variant of the mixed model that operates on time series data. Compared with SGTSMixer, it has no self-gating mechanism and cannot utilize socio-demographic characteristics.

[0184] Several models have different MSE, MAE, The indicators are as follows Figure 5a to Figure 5c As shown, a comparative analysis of the above models on various indicators is provided. Figure 5a The MSE on the test set at different communication rounds is shown. It can be observed that the MSE of all models shows a downward trend with the increase of communication rounds, which indicates that the prediction accuracy is improving over time. It is worth noting that SGTSMixer always outperforms other models, indicating its superior ability in minimizing prediction errors. Figure 5b The MAE on the test set is shown. Similar to the MSE, the MAE of all models also shows a downward trend. This metric further consolidates the effectiveness of SGTSMixer in producing predictions that are closer to the actual values, as it has consistently lower MAE values ​​during the communication rounds. The performance of LSTM on MAE is more unstable than MSE. This is because MAE is generally more sensitive to outliers than MSE. In the presence of outliers or noise that are common in load forecasting, LSTM is particularly susceptible due to its ability to capture complex temporal patterns. Figure 5c Depicts the test set , which is a measure of the proportion of the variance of the dependent variable explained by the independent variables in the model. The value tends to converge with the increase of communication rounds, indicating that the learning gradually stabilizes, but SGTSMixer reaches a high value, highlighting its efficient learning ability.

[0185] As the communication rounds progressed, the convergence of model performance metrics showed that all models benefited from more training iterations. However, SGTSMixer consistently outperformed all metrics, as evidenced by lower MSE and MAE and higher , emphasizing its role as a The robustness and potential of the STLF as a reliable tool are crucial for the accuracy and reliability of load prediction and are particularly critical in practical applications.

[0186] It is to be understood that the present invention is described by some embodiments, and it is known to those skilled in the art that various changes or equivalent substitutions may be made to these features and embodiments without departing from the spirit and scope of the present invention. In addition, under the teachings of the present invention, these features and embodiments may be modified to adapt to specific circumstances and materials without departing from the spirit and scope of the present invention. Therefore, the present invention is not limited by the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims of this application are within the scope of protection of the present invention.

Claims

1. A short-term residential load forecasting method based on social awareness analysis federated learning, characterized in that: The steps include: Step S1: construct a local SGTSMixer model, input the historical load data of smart meters and the socio-demographic characteristics of different households in the region as input data into the SGTSMixer model, output the predicted load data, and obtain the local social awareness analysis results; Step S2: Aggregate the global model, send the local model parameters and social awareness analysis results of all regions to the central server for aggregation, and generate representative global personalized models for each region; Step S3: Update the local model, construct a unified loss function, align the local model of each region with the global personalized model of each region on the central server, update the local model of each region according to the loss function value, and iteratively calculate the new local model parameters of each region and the global personalized model of each region; The local SGTSMixer model consists of two parts, namely, 4 hybrid MLPs and a temporal projection mechanism; each hybrid MLP consists of a self-gated temporal hybrid MLP and a self-gated feature hybrid MLP; The self-gated temporal mixing MLP consists of three parts; the first part consists of the first fully connected layer, the first ReLU activation function layer, and the first The first part is composed of the first self-gating layer and the first Sigmoid activation function layer; the third part is the residual connection; The self-gated feature hybrid MLP consists of three parts; the first part consists of the second fully connected layer, the second ReLU activation function layer, the second Dropout layer, the third fully connected layer and the third Dropout layer; the second part consists of the second self-gating layer and the second Sigmoid activation function layer; the third part is the residual connection; For Regions have specific socio-demographic characteristics family, define the data set for ,in Indicates Data for each region, Indicates The historical load data of the ith household in the region, Indicates The actual load value of the ith household in the region, Indicates The socio-demographic characteristics of the ith household in a region are Features; Dataset Contains historical load data, actual load values ​​and specific socio-demographic characteristics of different households , No. Regions have specific socio-demographic characteristics The local social awareness analysis results for all households are calculated as follows: Express Data for all households in the dataset To splice, It means that after splicing, four mixed MLP processing operations are performed, and then the average value is taken to calculate the first Regions have specific socio-demographic characteristics Results of the local social awareness analysis of all households .

2. The short-term residential load forecasting method based on social awareness analysis federated learning according to claim 1 is characterized in that: The input data is obtained as follows: The first Specific socio-demographic characteristics of users in each region Get a vector by embedding ,vector For one A real matrix of size , where Indicates the original input length; Indicates the quantity of a particular socio-demographic characteristic; Vector With historical load data Splicing to get input data , input data For one A real matrix of size , where Represents the number of variables contained in the multivariate time series; expressed as: in, represents embedding, Indicates a A real matrix of size , Indicates splicing, Indicates a A real matrix of size .

3. The short-term residential load forecasting method based on social awareness analysis federated learning according to claim 2 is characterized in that: The whole process in the first hybrid MLP is expressed as: The output of one hybrid MLP is used as the input of the next hybrid MLP, expressed as: Represents input data After the first hybrid MLP processing, we get , represents the first hybrid MLP processing operation; , indicating that Send it to the second hybrid MLP for processing and get , represents the second hybrid MLP processing operation; Indicates that Send it to the third hybrid MLP for processing and get , represents the third hybrid MLP processing operation; Indicates that Send it to the fourth hybrid MLP for processing and get , Represents the fourth hybrid MLP processing operation; input data After processing by 4 hybrid MLPs, the output is , using the formula To express, represents 4 mixed MLP processing operations, is a A real matrix of size ; This output is then fed into the temporal projection layer of the temporal projection mechanism, which is projected by the fourth fully connected layer. , expressed as: in , represents the weights and biases of the fourth fully connected layer, is a A real matrix of size , is a A real matrix of size , is the required prediction length; is a A real matrix of size ; then the resulting projection For load forecasting, focus on The first line of the local load forecasting result data is obtained, which is expressed as .

4. The short-term residential load forecasting method based on social awareness analysis federated learning according to claim 1 is characterized in that: Step S2 is obtained by: First, the similarity matrix is ​​calculated for all regions with specific socio-demographic characteristics of users , similarity matrix The calculation is as follows: in , indicating the Regions have specific socio-demographic characteristics Results of the local social awareness analysis of all households With Regions have specific socio-demographic characteristics Results of the local social awareness analysis of all households The similarity between express and The dot product of yes The L2 norm of yes The L2 norm of , indicating the Regions, , indicating the Regions, Indicates the total number of regions, that is, the number of all regions involved in the calculation; Secondly, according to the similarity matrix Middle All similarities of the rows are calculated Specific socio-demographic characteristics of users in each region Global Personalization Model , expressed as: in, Indicates Local model parameters for each region; By aggregating A global personalized model for all socio-demographic characteristics of users in a region , get the Global personalized model for all socio-demographic characteristics of users in regions , No. Global personalized model for all socio-demographic characteristics of users in regions Defined as: in, Indicates All socio-demographic characteristics of the region.

5. The short-term residential load forecasting method based on social awareness analysis federated learning according to claim 4 is characterized in that: The unified loss function in step S3 Calculated as follows: in, represents a unified loss function, represents the local load forecasting loss of the local SGTSMixer model; represents personalized learning loss; is a hyperparameter used to balance the three objectives; middle Indicates Data for each region, Indicates Regions Historical load data of households, Indicates Regions The actual load value of a household, Indicates Regions The socio-demographic characteristics of the families are Features; Indicates Local models for each region, Indicates that and Send to Local models for each region To make predictions, use To indicate the prediction result obtained through the local model, represents the square of L2 norm; middle, It is Data for each region A subset of Indicates The specific socio-demographic characteristics of each region are data; Indicates Regions have specific socio-demographic characteristics The global personalized model will and Send to The local model of the region and Regions have specific socio-demographic characteristics The global personalized model is then used to calculate the difference between the two.

6. The short-term residential load forecasting method based on social awareness analysis federated learning according to claim 5 is characterized in that: In step S3, updating the local model is achieved by: Minimize the uniform loss function through the optimization process To update The local model for each region, usually using a variant of stochastic gradient descent, is defined as: It is The local model parameters for each region are represents the learning rate, is the loss function Relative to The gradient of represents the rate of change of the loss function under the current parameter value.

Images