Multi-field oriented power energy prediction method and system based on attention mechanism

Abstract

The application provides a multi-field attention mechanism-based electric energy prediction method and system, wherein the method comprises the following steps: acquiring electric energy data and corresponding general features and field features; using a continuous feature encoder and a discrete feature encoder to encode continuous features and discrete features respectively; performing feature fusion on various encoded feature representations, taking the field feature representation as a query in the attention mechanism, and taking the general features as key values and key values in the attention mechanism; calculating the similarity between the query and the key value as the attention weight of the corresponding key value; performing weighted summation on the general features according to the attention weight, splicing the field features, and obtaining mixed features; and using a feedforward neural network to map the mixed features to the electric energy data to obtain an electric energy prediction value. The application can predict the electric energy of various cities and power consumption industries based on a single model, and can fuse electric energy data in different fields to realize accurate electric energy prediction.

Description

Technical Field

[0001] This invention belongs to the field of power prediction technology, and in particular relates to a power prediction method and system based on an attention mechanism for multiple fields. Background Technology

[0002] The task of electricity prediction aims to predict corresponding electricity values ​​based on given features (such as date, temperature, etc.). Electricity prediction techniques can be broadly categorized into time series-based methods and machine learning-based methods. Deep learning methods within machine learning have demonstrated significant advantages in electricity prediction. For example, one type of method models electricity prediction as a sequence prediction task, using sequence-based neural networks, such as Long Short-Term Memory networks or Transformer neural networks, to model the periodic variations in electricity sequences. Another type of method models electricity prediction as a regression task, using multilayer perceptrons or convolutional neural networks to model the complex mapping relationships between various input features and electricity values.

[0003] However, the mapping relationship between predicted features and electrical energy values ​​often differs across different domains (such as cities and industries). Traditional deep learning-based electrical energy prediction methods tend to construct different models for different domain-specific electrical energy prediction tasks. Deep learning-based electrical energy prediction methods require large amounts of data, and model training and prediction speeds are slow. Traditional single-domain prediction methods, which construct different prediction models for different domains, not only complicate the electrical energy prediction process but also fail to effectively utilize the shared information among electrical energy data from different domains. Summary of the Invention

[0004] This invention addresses the shortcomings of existing technologies by providing an attention-based method and system for predicting electrical energy across multiple fields.

[0005] In a first aspect, the present invention provides an attention-based method for predicting electrical energy across multiple domains, comprising:

[0006] Acquire electrical energy data and corresponding routine and domain features; routine features include date features, temperature features, and holiday features; domain features include city tags and industry tags;

[0007] Continuous and discrete features are encoded using a continuous feature encoder and a discrete feature encoder, respectively. For discrete features, each different feature is mapped to a corresponding multidimensional feature representation. For continuous features, features with different values ​​are mapped to different multidimensional feature representations through a feedforward neural network.

[0008] An attention mechanism is used to fuse features of various encoded feature representations. Domain feature representations are used as queries in the attention mechanism, and regular features are used as key values ​​and critical values ​​in the attention mechanism.

[0009] Calculate the similarity between the query and the key value as the attention weight for the corresponding key value;

[0010] After weighting and summing the regular features according to the attention weights, the mixture is concatenated with the domain features to obtain the hybrid features;

[0011] A feedforward neural network is used to map the mixed features onto the electrical energy data to obtain the predicted electrical energy value.

[0012] Furthermore, the method of encoding continuous features and discrete features using a continuous feature encoder and a discrete feature encoder respectively includes:

[0013] For continuous features, a multilayer perceptron is used as a continuous feature encoder to map continuous features with different values ​​into different multidimensional vector feature representations:

[0014] ;

[0015] in, MLP (·) represents an input dimension of 1 and an output dimension of 1. d Multilayer perceptron; y , m , d , w , h , l ∈ R d These represent continuous characteristics such as year, month, day, week, high temperature, and low temperature, respectively. 3D vector representation;

[0016] For discrete features, a tensor network is used as a discrete feature encoder to store a multi-dimensional vector feature representation for each feature:

[0017] ;

[0018] in, Encoder (·) represents the input dimension. k × d Tensor networks; k The number of discrete features; y' , m' , d' , w' , f , e , c and t ∈ R dThese represent discrete features: year, month, day, weekday, holidays, special events, city tags, and industry tags, respectively. d 3D vector representation.

[0019] Furthermore, the calculation of the similarity between the query and the key value as the attention weight of the corresponding key value includes:

[0020] Based on the attention mechanism, the similarity between the query and the key value is calculated as the attention weight of the corresponding key value according to the following formula:

[0021] ;

[0022] in, The cosine similarity function; q For query vector; k This is a key value vector; k i For the number to be calculated i One key value; Key value k i The corresponding attention weights; | k | represents the number of elements in the set; k j For calculation All required key value vectors; j For the corresponding serial number subscript.

[0023] Furthermore, the weighted summation of regular features based on attention weights, followed by concatenation with domain features, yields a hybrid feature, including:

[0024] Constructing a feature representation weighted by an attention mechanism:

[0025] ;

[0026] in, date When dates are continuous features, the feature representation is weighted by an attention mechanism; date When the date is a discrete feature, the feature representation is weighted by an attention mechanism; For the input of the first i Each feature attention weight; For the input of the first i A series of date features; For the input of the first i A discrete date feature representation; holi This represents the features of holidays after being weighted by the attention mechanism. For the input of the first i Characteristics of each holiday period; tempThis represents the feature representation of temperature after being weighted by an attention mechanism. For the input of the first i Each temperature feature is represented; |·| is the modulus of the set;

[0027] Constructing a hybrid feature vector r expression:

[0028] .

[0029] Furthermore, the step of mapping the mixed features to the electrical energy data using a feedforward neural network to obtain the predicted electrical energy value includes:

[0030] Calculate the predicted electrical energy value using the following formula:

[0031] ;

[0032] in, h (α) For the first α The output representation of a layered feedforward neural network, α =1, 2, 3; b (α) For the first α Bias of layered feedforward neural networks; W (α) For the first α The weight matrix of the layered feedforward neural network; ReLU(·) is the linear rectified function; This is the predicted value for electrical energy.

[0033] Secondly, the present invention provides an attention-based electrical energy prediction system for multiple domains, comprising:

[0034] The acquisition module is used to acquire electrical energy data and corresponding routine and domain features; routine features include date features, temperature features, and holiday features; domain features include city tags and industry tags.

[0035] The encoding module is used to encode continuous features and discrete features using a continuous feature encoder and a discrete feature encoder, respectively. For discrete features, each different feature is mapped to a corresponding multidimensional feature representation. For continuous features, features with different values ​​are mapped to different multidimensional feature representations through a feedforward neural network.

[0036] The feature fusion module is used to perform feature fusion on various encoded feature representations using an attention mechanism, taking the domain feature representation as the query in the attention mechanism, and taking the regular features as the key value and key value in the attention mechanism;

[0037] The calculation module is used to calculate the similarity between the query and the key value as the attention weight of the corresponding key value;

[0038] The feature concatenation module is used to weight and sum the regular features according to the attention weights, and then concatenate them with the domain features to obtain the hybrid features;

[0039] The power energy prediction module is used to map mixed features to power data using a feedforward neural network to obtain power energy prediction values.

[0040] Furthermore, the encoding module includes:

[0041] The continuous feature mapping unit is used to map continuous features, with the multilayer perceptron acting as a continuous feature encoder, to different multidimensional vector feature representations:

[0042] ;

[0043] in, MLP (·) represents an input dimension of 1 and an output dimension of 1. d Multilayer perceptron; y , m , d , w , h , l ∈ R d These represent continuous characteristics such as year, month, day, week, high temperature, and low temperature, respectively. 3D vector representation;

[0044] The discrete feature storage unit is used to store a multi-dimensional vector feature representation for each discrete feature, using a tensor network as a discrete feature encoder.

[0045] ;

[0046] in, Encoder (·) represents the input dimension. k × d Tensor networks; k The number of discrete features; y' , m' , d' , w' , f , e , c and t ∈ R d These represent discrete features: year, month, day, weekday, holidays, special events, city tags, and industry tags, respectively. d 3D vector representation.

[0047] Furthermore, the computing module includes:

[0048] The first computational unit is used to calculate the similarity between the query and the key value as the attention weight of the corresponding key value based on the attention mechanism and according to the following formula:

[0049] ;

[0050] in, The cosine similarity function; q For query vector; k This is a key value vector; k i For the number to be calculated i One key value; Key value k i The corresponding attention weights; | k | represents the number of elements in the set; k j For calculation All required key value vectors; j For the corresponding serial number subscript.

[0051] Furthermore, the feature splicing module includes:

[0052] The first building unit is used to construct the feature representation after attention mechanism weighting:

[0053] ;

[0054] in, date When dates are continuous features, the feature representation is weighted by an attention mechanism; date When the date is a discrete feature, the feature representation is weighted by an attention mechanism; For the input of the first i Each feature attention weight; For the input of the first i A series of date features; For the input of the first i A discrete date feature representation; holi This represents the features of holidays after being weighted by the attention mechanism. For the input of the first i Characteristics of each holiday period; temp This represents the feature representation of temperature after being weighted by an attention mechanism. For the input of the first i Each temperature feature is represented; |·| is the modulus of the set;

[0055] The second building unit is used to construct the hybrid feature vector. r expression:

[0056] .

[0057] Furthermore, the electrical energy prediction module includes:

[0058] The second calculation unit is used to calculate the predicted electrical energy value according to the following formula:

[0059] ;

[0060] in, h (α) For the first α The output representation of a layered feedforward neural network, α =1, 2, 3; b (α) For the first α Bias of layered feedforward neural networks; W (α) For the first α The weight matrix of the layered feedforward neural network; ReLU(·) is the linear rectified function; This is the predicted value for electrical energy.

[0061] This invention provides a multi-domain-based attention-based method and system for predicting electricity consumption. The method involves obtaining feature representations of continuous and discrete features using two feature encoders; fusing the obtained conventional and domain feature representations using an attention mechanism; and mapping the fused features using a deep neural network to obtain the predicted electricity consumption. This invention can predict electricity consumption for various cities and electricity-consuming industries based on a single model. Through domain information learning, it effectively fuses electricity consumption data from different domains, resulting in more accurate electricity consumption predictions. Attached Figure Description

[0062] To more clearly illustrate the technical solution of the present invention, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0063] Figure 1 A flowchart illustrating an attention-based energy prediction method for multiple domains, provided as an embodiment of the present invention;

[0064] Figure 2 A structural diagram of an attention-based energy prediction method for multiple domains provided in an embodiment of the present invention;

[0065] Figure 3 A computational schematic diagram of the attention mechanism provided in an embodiment of the present invention;

[0066] Figure 4 This is a structural diagram of an attention-based electrical energy prediction system for multiple domains, provided as an embodiment of the present invention. Detailed Implementation

[0067] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0068] For example, the embodiments of the present invention set an application scenario: electrical energy data within a certain time range is known, including the predicted target electrical energy value. y It includes the following common features: date feature (year) ,moon ,day and week ), Holiday characteristics (whether it is a holiday) Are there any special events? Temperature characteristics (high temperature) Low temperature It also includes domain features (city tags) Industry tags The task of electrical energy prediction is to predict the corresponding electrical energy value based on the input characteristics at future times. y .

[0069] In one embodiment, such as Figure 1 and Figure 2 As shown, this embodiment of the invention provides an attention-based method for predicting electrical energy across multiple domains, comprising:

[0070] Step 101: Obtain electrical energy data and corresponding routine and domain features; routine features include date features, temperature features, and holiday features; domain features include city tags and industry tags.

[0071] The specific details are shown in Table 1.

[0072] Table 1 Feature Classification

[0073]

[0074] Step 102: Encode continuous features and discrete features using a continuous feature encoder and a discrete feature encoder, respectively; for discrete features, map each different feature to the corresponding multidimensional feature representation; for continuous features, map features with different values ​​to different multidimensional feature representations using a feedforward neural network.

[0075] The data is divided into two categories: continuous features (temperature features and date features) and discrete features (date features, holiday features, and domain features). Two deep neural network encoders with different structures are used to encode the discrete features and the continuous features respectively.

[0076] Since temperature and date features are recorded as continuous values, a continuous feature encoder is designed to encode them. Holiday and domain features are recorded as discrete values, and dates can also be represented as discrete values; therefore, a discrete feature encoder is designed for these three sets of features. The data is based on domain division, and the goal is to predict the corresponding electricity consumption within each domain. Therefore, after obtaining the encoded representations of all features, the fused representations of other features and the domain features are calculated based on the domain features. The fused representations of each part are concatenated with domain features and input into three layers to obtain the domain-specific electricity prediction value. It's important to note that because date features are relatively important, two encoders are used simultaneously to encode them, enriching the feature information.

[0077] For continuous date features , , , and temperature characteristics , By employing a multilayer perceptron as the encoder, different numerical values ​​of continuous features are mapped into multidimensional vector feature representations, enabling better representation and processing of these features. Through the deep neural network structure of the multilayer perceptron, complex nonlinear relationships in the data can be learned, thus achieving more refined feature extraction and encoding.

[0078] For example, for continuous features, a multilayer perceptron is used as a continuous feature encoder to map continuous features with different values ​​into different multidimensional vector feature representations:

[0079] .

[0080] in, MLP (·) represents an input dimension of 1 and an output dimension of 1. d Multilayer perceptron; y , m , d , w , h , l ∈ R d These represent continuous characteristics such as year, month, day, week, high temperature, and low temperature, respectively. 3D vector representation;

[0081] For discrete features, date features , , , Holiday characteristics , and domain characteristics , By using a tensor network as an encoder, each feature is mapped to a multi-dimensional vector feature representation, thus each discrete feature corresponds to a specified feature space. This encoding method not only effectively preserves the information of discrete features but also enables feature extraction at different scales to obtain richer and more effective feature representations.

[0082] .

[0083] in, Encoder (·) represents the input dimension. k × d Tensor networks; k The number of discrete features; y' , m' , d' , w' , f , e , c and t ∈ R d These represent discrete features: year, month, day, weekday, holidays, special events, city tags, and industry tags, respectively. d 3D vector representation.

[0084] Step 103: Use an attention mechanism to fuse features of various encoded feature representations, use the domain feature representation as the query in the attention mechanism, and use the regular features as the key value and key value in the attention mechanism.

[0085] Based on domain features, feature fusion is performed on other groups of features. This is represented by domain features (…). For the query vector, the subtype feature is date (continuous). Date (discrete) Holidays Temperature As key and critical values, an attention mechanism is designed to achieve feature representation fusion, such as... Figure 3 As shown.

[0086] Step 104: Calculate the similarity between the query and the key value as the attention weight of the corresponding key value.

[0087] By focusing attention on the most important features, better feature representation and prediction results can be achieved. This method can effectively handle different relationships and varying importance among input features and extract richer information from the original features.

[0088] Firstly, to calculate the relationship between subtype features and domain features, a dot product attention algorithm was used, directly applying the query vector... q and key value vector k i Perform a dot product and then divide by a scaling factor to obtain the attention score. Based on the attention mechanism, the similarity between the query and the key value is calculated as the attention weight for the corresponding key value according to the following formula:

[0089] .

[0090] in, The cosine similarity function; q For query vector; k This is a key value vector; k i For the number to be calculated i One key value; Key value k i The corresponding attention weights; | k | represents the number of elements in the set; k j For calculation All required key value vectors; j For the corresponding serial number subscript.

[0091] When calculating the relationship between each subtype feature and domain feature... Each subtype feature is represented separately. 、( 、( and( , Preserve as a domain feature representation ( .

[0092] Step 105: After weighting and summing the regular features according to the attention weights, concatenate them with the domain features to obtain the hybrid features.

[0093] The key values ​​are weighted and summed to obtain a feature representation that is more favorable for specific predictions. The weights obtained above are calculated using attention scores. This is used to perform a weighted summation of the key values ​​to produce a fused feature representation. Since there are four sub-type features, four weighting operations are performed separately. The attention-weighted feature representation is then constructed:

[0094] .

[0095] in, date When dates are continuous features, the feature representation is weighted by an attention mechanism; date When the date is a discrete feature, the feature representation is weighted by an attention mechanism; For the input of the first i Each feature attention weight; For the input of the first i A series of date features; For the input of the first i A discrete date feature representation; holi This represents the features of holidays after being weighted by the attention mechanism. For the input of the first i Characteristics of each holiday period; temp This represents the feature representation of temperature after being weighted by an attention mechanism. For the input of the first i Each temperature feature is represented; |·| is the modulus of the set;

[0096] The weighted regular features and domain features are concatenated to obtain a final mixed feature vector. This method combines different types of feature information to improve prediction accuracy. Constructing the mixed feature vector... r expression:

[0097] .

[0098] Step 106: Use a feedforward neural network to map the mixed features to the power data to obtain the predicted power value.

[0099] Mixed features through a feedforward neural network r Mapped to power data This feedforward neural network employs a multilayer perceptron structure to capture complex nonlinear relationships in mixed features and convert them into an output capable of predicting electricity consumption data. Specifically, the process of mapping mixed features to electricity consumption data through this feedforward neural network involves three layers, each containing multiple neurons. Each neuron converts the input signal into an output through a nonlinear activation function.

[0100] Calculate the predicted electrical energy value using the following formula:

[0101] .

[0102] in, h (α) For the first α The output representation of a layered feedforward neural network, α =1, 2, 3;b (α) For the first α Bias of layered feedforward neural networks; W (α) For the first α The weight matrix of the layered feedforward neural network; ReLU(·) is the linear rectified function; This is the predicted value for electrical energy.

[0103] For traditional generalized electricity prediction models, we used mean squared error as the loss function:

[0104] .

[0105] in These are network parameters. Regularization term, It is the sample size. It is the actual electrical energy value of the sample. It is the electrical energy value predicted by the model.

[0106] In the experiments conducted in this embodiment of the invention, the experimental parameters were set as follows: the representation dimension of all features was 300; the dropout rate was 0.5; the learning rate was 0.001; the weight of the L2 loss was 0.01; and the network was optimized using Adam. The model was tested on the public electricity consumption prediction dataset ECL (Electricity Consuming Load). The model input was modified to use the meteorological data provided in the ECL dataset, treating the electricity consumption data of different users in the dataset as different domains. The method provided in this embodiment of the invention (denoted as MDEPMA, i.e., Multi-Domain Electric Prediction Model with Attention mechanism) was trained using all domains and tested on each domain. The remaining models were trained and tested separately on each domain. The final average MSE predicted across all domains was 0.312, and the average MAE was 0.403. The results show that the prediction performance of the method of this invention is superior to some existing electricity consumption prediction methods. Specific experimental results are shown in Table 2.

[0107] Table 2 Prediction Results

[0108]

[0109] This invention proposes a multi-domain-oriented attention-based power energy prediction method, focusing on prediction across multiple domains. The method models information from different domains and utilizes an attention mechanism to fuse domain features with conventional features, achieving multi-domain power energy prediction based on a single model. Compared to existing power energy prediction methods, this invention can more effectively utilize power energy data from different domains, improve the prediction accuracy in each domain, and simplify the prediction process across different domains.

[0110] Based on the same inventive concept, this invention also provides an attention-based power prediction system for multiple domains. Since the principle of this system in solving the problem is similar to that of the attention-based power prediction method for multiple domains, the implementation of this system can refer to the implementation of the attention-based power prediction method for multiple domains, and the repeated parts will not be described again.

[0111] In another embodiment, the attention-based energy prediction system for multiple domains provided by this invention, such as... Figure 4 As shown, it includes:

[0112] The acquisition module 10 is used to acquire electrical energy data and corresponding routine features and domain features; routine features include date features, temperature features and holiday features; domain features include city tags and industry tags.

[0113] The encoding module 20 is used to encode continuous features and discrete features using a continuous feature encoder and a discrete feature encoder, respectively; for discrete features, each different feature is mapped to a corresponding multidimensional feature representation; for continuous features, features with different values ​​are mapped to different multidimensional feature representations through a feedforward neural network.

[0114] The feature fusion module 30 is used to perform feature fusion on various encoded feature representations using an attention mechanism, using the domain feature representation as the query in the attention mechanism, and using the regular features as the key value and key value in the attention mechanism.

[0115] The calculation module 40 is used to calculate the similarity between the query and the key value as the attention weight of the corresponding key value.

[0116] The feature concatenation module 50 is used to weight and sum the regular features according to the attention weights, and then concatenate them with the domain features to obtain the hybrid features.

[0117] The power prediction module 60 is used to map mixed features to power data using a feedforward neural network to obtain power prediction values.

[0118] For example, the encoding module includes:

[0119] The continuous feature mapping unit is used to map continuous features, with the multilayer perceptron acting as a continuous feature encoder, to different multidimensional vector feature representations:

[0120] .

[0121] in, MLP (·) represents an input dimension of 1 and an output dimension of 1. d Multilayer perceptron; y , m , d , w , h , l ∈ R d These represent continuous characteristics such as year, month, day, week, high temperature, and low temperature, respectively. 3D vector representation.

[0122] The discrete feature storage unit is used to store a multi-dimensional vector feature representation for each discrete feature, using a tensor network as a discrete feature encoder.

[0123] .

[0124] in, Encoder (·) represents the input dimension. k × d Tensor networks; k The number of discrete features; y' , m' , d' , w' , f , e , c and t ∈ R d These represent discrete features: year, month, day, weekday, holidays, special events, city tags, and industry tags, respectively. d 3D vector representation.

[0125] For example, the computing module includes:

[0126] The first computational unit is used to calculate the similarity between the query and the key value as the attention weight of the corresponding key value based on the attention mechanism and according to the following formula:

[0127] .

[0128] in, The cosine similarity function; q For query vector; kThis is a key value vector; k i For the number to be calculated i One key value; Key value k i The corresponding attention weights; | k | represents the number of elements in the set; k j For calculation All required key value vectors; j For the corresponding serial number subscript.

[0129] For example, the feature splicing module includes:

[0130] The first building unit is used to construct the feature representation after attention mechanism weighting:

[0131] .

[0132] in, date When dates are continuous features, the feature representation is weighted by an attention mechanism; date When the date is a discrete feature, the feature representation is weighted by an attention mechanism; For the input of the first i Each feature attention weight; For the input of the first i A series of date features; For the input of the first i A discrete date feature representation; holi This represents the features of holidays after being weighted by the attention mechanism. For the input of the first i Characteristics of each holiday period; temp This represents the feature representation of temperature after being weighted by an attention mechanism. For the input of the first i A temperature feature is represented; |·| is the modulus of the set.

[0133] The second building unit is used to construct the hybrid feature vector. r expression:

[0134] .

[0135] For example, the electrical energy prediction module includes:

[0136] The second calculation unit is used to calculate the predicted electrical energy value according to the following formula:

[0137] .

[0138] in, h(α) For the first α The output representation of a layered feedforward neural network, α =1, 2, 3; b (α) For the first α Bias of layered feedforward neural networks; W (α) For the first α The weight matrix of the layered feedforward neural network; ReLU(·) is the linear rectified function; This is the predicted value for electrical energy.

[0139] For more detailed information on the working process of each of the above modules, please refer to the relevant content disclosed in the foregoing embodiments, which will not be repeated here.

[0140] In another embodiment, the present invention provides a computer device including a processor and a memory; wherein the processor executes a computer program stored in the memory to implement the steps of the above-described attention-based energy prediction method for multiple domains.

[0141] For more detailed information on the above methods, please refer to the relevant content disclosed in the foregoing embodiments, which will not be repeated here.

[0142] In another embodiment, the present invention provides a computer-readable storage medium for storing a computer program; when the computer program is executed by a processor, it implements the steps of the above-described attention-based energy prediction method for multiple domains.

[0143] For more detailed information on the above methods, please refer to the relevant content disclosed in the foregoing embodiments, which will not be repeated here.

[0144] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. The systems, devices, and storage media disclosed in the embodiments are described simply because they correspond to the methods disclosed in the embodiments; relevant details can be found in the method section.

[0145] Those skilled in the art will clearly understand that the techniques in the embodiments of the present invention can be implemented using software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solutions in the embodiments of the present invention, or the parts that contribute to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments or certain parts of the embodiments of the present invention.

[0146] The present invention has been described in detail above with reference to specific embodiments and exemplary examples; however, these descriptions should not be construed as limiting the present invention. Those skilled in the art will understand that various equivalent substitutions, modifications, or improvements can be made to the technical solutions and embodiments of the present invention without departing from the spirit and scope of the invention, and all such modifications and improvements fall within the scope of the present invention. The scope of protection of the present invention is defined by the appended claims.

Claims

1. A multi-domain-oriented attention-based method for predicting electrical energy, characterized in that, include: Acquire electrical energy data and corresponding conventional and domain-specific characteristics; Regular features include date features, temperature features, and holiday features. Date features include year, month, day, and weekday. Temperature features include high temperature and low temperature. Holiday features include holidays (festival) and special events (event). Domain features include city tags and industry tags. Encoding continuous features and discrete features using a continuous feature encoder and a discrete feature encoder respectively, specifically including: For continuous features, a multilayer perceptron is used as a continuous feature encoder to map continuous features with different values ​​into different multidimensional vector feature representations: ； in, MLP (·) represents an input dimension of 1 and an output dimension of 1. D Multilayer perceptron; y , m , d , w , h , l ∈ R D These represent continuous characteristics such as year, month, day, week, high temperature, and low temperature, respectively. D 3D vector representation; For discrete features, a tensor network is used as a discrete feature encoder to store a multi-dimensional vector feature representation for each feature: ； in, Encoder (·) represents the input dimension. k × D Tensor networks; k The number of discrete features; y' , m' , d' , w' , f , e , c and t ∈ R D These represent discrete features: year, month, day, weekday, holiday, special event, city tag, and industry tag, respectively. D 3D vector representation; For discrete features, each distinct feature is mapped to a corresponding multidimensional feature representation; for continuous features, features with different values ​​are mapped to different multidimensional feature representations through a feedforward neural network. An attention mechanism is used to fuse features of various encoded feature representations. Domain feature representations are used as queries in the attention mechanism, and regular features are used as key values ​​and critical values ​​in the attention mechanism. Calculate the similarity between the query and the key value as the attention weight for the corresponding key value; After weighting and summing the regular features according to the attention weights, the results are concatenated with the domain features to obtain the hybrid features. A feedforward neural network is used to map the mixed features onto the electrical energy data to obtain the predicted electrical energy value.

2. The attention-based electrical energy prediction method for multiple domains according to claim 1, characterized in that, The calculation of the similarity between the query and the key value as the attention weight for the corresponding key value includes: Based on the attention mechanism, the similarity between the query and the key value is calculated as the attention weight of the corresponding key value according to the following formula: ； in, The cosine similarity function; q For query vector; k This is a key value vector; k i For the number to be calculated i One key value; Key value k i The corresponding attention weights; | k | represents the number of elements in the set; k j For calculation All required key value vectors; j For the corresponding serial number subscript.

3. The attention-based electrical energy prediction method for multiple domains according to claim 1, characterized in that, The hybrid features obtained by weighting and summing the regular features according to the attention weights and then concatenating them with the domain features include: Constructing a feature representation weighted by an attention mechanism: ； in, date When dates are continuous features, the feature representation is weighted by an attention mechanism; date When the date is a discrete feature, the feature representation is weighted by an attention mechanism; For the input of the first i Each feature attention weight; For the input of the first i A series of date features; For the input of the first i A discrete date feature representation; holi This is a feature representation of holiday characteristics after being weighted by an attention mechanism; For the input of the first i Characteristics of each holiday period; temp This represents the feature representation of temperature after being weighted by an attention mechanism. For the input of the first i Each temperature feature is represented; |·| is the modulus of the set; Constructing a hybrid feature vector r expression: 。 4. The attention-based electrical energy prediction method for multiple domains according to claim 1, characterized in that, The method of mapping mixed features to electrical energy data using a feedforward neural network to obtain predicted electrical energy values ​​includes: Calculate the predicted electrical energy value using the following formula: ； in, h (α) For the first α The output representation of a layered feedforward neural network, α =1, 2, 3; b (α) For the first α Bias of layered feedforward neural networks; W (α) For the first α The weight matrix of the layered feedforward neural network; ReLU(·) is the linear rectified function; This is the predicted value for electrical energy.

5. An attention-based electrical energy prediction system for multiple domains, characterized in that, include: The acquisition module is used to acquire electrical energy data, as well as corresponding conventional and domain characteristics. Regular features include date features, temperature features, and holiday features. Date features include year, month, day, and weekday; temperature features include high temperature and low temperature; and holiday features include holidays (festival) and special events (event). Domain features include city tags and industry tags. The encoding module is used to encode continuous features and discrete features using a continuous feature encoder and a discrete feature encoder, respectively; for Discrete features map each distinct feature to a corresponding multidimensional feature representation; for Continuous features are mapped to different multidimensional feature representations by a feedforward neural network, which maps features with different values ​​to different multidimensional feature representations. The encoding module includes: The continuous feature mapping unit is used to map continuous features, with the multilayer perceptron acting as a continuous feature encoder, to different multidimensional vector feature representations: ； in, MLP (·) represents an input dimension of 1 and an output dimension of 1. D Multilayer perceptron; y , m , d , w , h , l ∈ R D These represent continuous characteristics such as year, month, day, week, high temperature, and low temperature, respectively. D 3D vector representation; The discrete feature storage unit is used to store a multi-dimensional vector feature representation for each discrete feature, using a tensor network as a discrete feature encoder. ； in, Encoder (·) represents the input dimension. k × D Tensor networks; k The number of discrete features; y' , m' , d' , w' , f , e , c and t ∈ R D These represent discrete features: year, month, day, weekday, holidays, special events, city tags, and industry tags, respectively. D 3D vector representation; The feature fusion module is used to perform feature fusion on various encoded feature representations using an attention mechanism, taking the domain feature representation as the query in the attention mechanism, and taking the regular features as the key value and key value in the attention mechanism; The calculation module is used to calculate the similarity between the query and the key value as the attention weight of the corresponding key value; The feature concatenation module is used to weight and sum the regular features according to the attention weights, and then concatenate them with the domain features to obtain the hybrid features; The power energy prediction module is used to map mixed features to power data using a feedforward neural network to obtain power energy prediction values.

6. The attention-based electrical energy prediction system for multiple domains according to claim 5, characterized in that, The computing module includes: The first computational unit is used to calculate the similarity between the query and the key value as the attention weight of the corresponding key value based on the attention mechanism and according to the following formula: ； in, The cosine similarity function; q For query vector; k This is a key value vector; k i For the number to be calculated i One key value; Key value k i The corresponding attention weights; | k | represents the number of elements in the set; k j For calculation All required key value vectors; j For the corresponding serial number subscript.

7. The attention-based electrical energy prediction system for multiple domains according to claim 5, characterized in that, The feature splicing module includes: The first building unit is used to construct the feature representation after attention mechanism weighting: ； in, date When dates are continuous features, the feature representation is weighted by an attention mechanism; date When the date is a discrete feature, the feature representation is weighted by an attention mechanism; For the input of the first i Each feature attention weight; For the input of the first i A series of date features; For the input of the first i A discrete date feature representation; holi This is a feature representation of holiday characteristics after being weighted by an attention mechanism; For the input of the first i Characteristics of each holiday period; temp This represents the feature representation of temperature after being weighted by an attention mechanism. For the input of the first i Each temperature feature is represented; |·| is the modulus of the set; The second building unit is used to construct the hybrid feature vector. r expression: 。 8. The attention-based electrical energy prediction system for multiple domains according to claim 5, characterized in that, The electrical energy prediction module includes: The second calculation unit is used to calculate the predicted electrical energy value according to the following formula: ； in, h (α) For the first α The output representation of a layered feedforward neural network, α =1, 2, 3; b (α) For the first α Bias of layered feedforward neural networks; W (α) For the first α The weight matrix of the layered feedforward neural network; ReLU(·) is the linear rectified function; This is the predicted value for electrical energy.

Images