Multi-industry power load data prediction method, equipment, medium and product

Abstract

The invention discloses a multi-industry power load data prediction method and device, a medium and a product. The method comprises the following steps: preprocessing power load data of multiple industries in a power table to obtain complete power table data; through a TabTransform model, category type data in the complete power table data are converted into context embedding features, and numerical data in the complete power load data are converted into a high-dimensional continuous feature sequence; a dependency relationship among category embedding features is learned through a plurality of context modeling layers in a TabTransform model, context embedding and high-dimensional continuous features are fused, a target feature sequence used for prediction is constructed, each TabTransform layer comprises a multi-head self-attention mechanism, and the multi-head self-attention mechanism comprises a multi-head self-attention mechanism; and splicing the context-enhanced embedded features and the numerical features, inputting the spliced features into a multi-layer perceptron, and outputting a prediction result of the multi-industry power load data. According to the method, the context modeling capability is enhanced through a multi-head self-attention mechanism, and the prediction accuracy of the model on the power load data is effectively improved.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of power forecasting technology, and in particular to a method, device, medium, and product for predicting power load data in multiple industries. Background Art

[0002] With the increasing complexity of the power market and the diversification of power data sources, power load data has shown strong heterogeneity and complexity, covering many different types of characteristics such as time dimension, weather factors, historical load data, etc.

[0003] Electricity consumption characteristics vary significantly across industries. For example, in the industrial sector, power load is influenced by multiple factors, including production equipment, machine operation, and production schedules, and exhibits complex cyclical fluctuations. In the commercial sector, electricity demand in shopping malls and office buildings is significantly impacted by external factors such as holidays and weather. In residential electricity consumption, seasonal variations, electricity usage habits, and household size directly influence load fluctuations. Furthermore, with the increasing adoption of electric vehicles, electricity demand in the transportation sector is rapidly increasing. The power demand characteristics of these emerging sectors are even more complex. Accurately forecasting loads using this heterogeneous power load data is a pressing technical challenge.

[0004] Traditional machine learning models often fail to fully capture the characteristic differences of power load data in various industries when processing them, resulting in low model prediction accuracy. Summary of the Invention

[0005] The present invention provides a multi-industry power load data prediction method, equipment, medium and product to solve the problem that traditional machine learning models cannot fully capture the characteristic differences of power load data in various industries when processing power load data in various industries.

[0006] According to one aspect of the present invention, a method for predicting power load data for multiple industries is provided, comprising:

[0007] Preprocess the power load data of multiple industries in the power table to obtain complete power table data;

[0008] Inputting the complete power table data into a TabTransformer model, converting the categorical data in the complete power table data into contextual embedding features through the TabTransformer model, and converting the numerical data in the complete power load data into a high-dimensional continuous feature sequence through the TabTransformer model;

[0009] Learning long-range dependencies between the context embedding features through multiple Transformer layers in the Tab Transformer model to obtain context embeddings, and generating a target feature sequence based on the context embeddings and the high-dimensional continuous feature sequence, wherein each Transformer layer includes a multi-head self-attention mechanism;

[0010] The target feature sequence is input into the multi-layer perceptron in the TabTransformer model to output the prediction results of the power load data of multiple industries.

[0011] According to another aspect of the present invention, there is provided a device for predicting power load data, comprising:

[0012] A preprocessing module is used to preprocess the power load data of multiple industries in the power table to obtain complete power table data;

[0013] a conversion module, configured to input the complete power table data into a TabTransformer model, convert the categorical data in the complete power table data into contextual embedding features through the TabTransformer model, and convert the numerical data in the complete power load data into a high-dimensional continuous feature sequence through the TabTransformer model;

[0014] A learning module, configured to learn the long-range dependencies between the context embedding features through multiple Transformer layers in the TabTransformer model to obtain context embeddings, and generate a target feature sequence based on the context embeddings and the high-dimensional continuous feature sequence, wherein each Transformer layer includes a multi-head self-attention mechanism;

[0015] The output module is used to input the target feature sequence into the multi-layer perceptron in the TabTransformer model and output the prediction results of the power load data of multiple industries.

[0016] According to another aspect of the present invention, there is provided an electronic device, the electronic device comprising: at least one processor;

[0017] and a memory communicatively coupled to the at least one processor;

[0018] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the multi-industry power load data prediction method described in any embodiment of the present invention.

[0019] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the multi-industry power load data prediction method described in any embodiment of the present invention when executed.

[0020] According to another aspect of the present invention, a computer program product is provided, which includes a computer program. When the computer program is executed by a processor, it implements the multi-industry power load data prediction method described in any embodiment of the present invention.

[0021] The technical solution of the embodiment of the present invention enables the TabTransformer model to fully learn the interactive relationship between different categorical data and numerical data through the multi-head self-attention mechanism in the Transformer layer, solving the problem that traditional machine learning models cannot fully capture the characteristic differences of power load data in various industries, and achieving the beneficial effect of improving the model's prediction accuracy for power load data.

[0022] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present invention, nor is it intended to limit the scope of the present invention. Other features of the present invention will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0024] Figure 1 A schematic diagram of a flow chart of a multi-industry power load data prediction method provided in the first embodiment of the present invention;

[0025] Figure 2 A schematic flow chart of a multi-industry power load data prediction method provided in the second embodiment of the present invention;

[0026] Figure 3 A schematic flow chart of a method for predicting power load data for multiple industries provided in the third embodiment of the present invention;

[0027] Figure 4 A schematic diagram of the structure of a multi-industry power load data prediction device provided by the fourth embodiment of the present invention;

[0028] Figure 5 This is a structural diagram of an electronic device for a multi-industry power load data prediction method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0029] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only embodiments of a part of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work should fall within the scope of protection of the present invention. It should be understood that the various steps described in the method implementation mode of the present invention can be performed in different orders and / or in parallel. In addition, the method implementation mode may include additional steps and / or omit the steps shown. The scope of the present invention is not limited in this respect.

[0030] As used herein, the term "including" and its variations are open-ended, i.e., "including but not limited to." The term "based on" means "based, at least in part, on." The term "one embodiment" means "at least one embodiment," the term "another embodiment" means "at least one additional embodiment," and the term "some embodiments" means "at least some embodiments." Other terms are defined in the following description.

[0031] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0032] It should be noted that the modifications of "one" and "multiple" mentioned in the present invention are illustrative rather than restrictive. Those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0033] The names of the messages or information exchanged between multiple devices in the embodiments of the present invention are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0034] Example 1

[0035] Figure 1This is a flow chart of a multi-industry power load data forecasting method provided in Example 1 of the present invention. This method is applicable to the accurate forecasting and task analysis of multi-source heterogeneous power load data, particularly in scenarios facing the ever-changing power market and the diverse types of power load data. The method can be performed by a power load data forecasting device, which can be implemented by software and / or hardware and is generally integrated into an electronic device. In this embodiment, the electronic device includes but is not limited to a computer device.

[0036] like Figure 1 As shown, a multi-industry power load data prediction method provided by the first embodiment of the present invention includes the following steps:

[0037] S110 , pre-processing the power load data of multiple industries in the power table to obtain complete power table data.

[0038] One electricity table may include electricity data for multiple industries, including industry, commerce, residential, and transportation.

[0039] Among them, the power load data of each industry in the original power table can include numerical data and categorical data. Numerical data can be understood as specific numerical values, and numerical data is continuous data; categorical data can be understood as different classification data, for example, weather types include sunny, rainy, snowy and other categories, and categorical data is discrete data.

[0040] In this embodiment, preprocessing can be performed to fill in missing data in the original electricity tables of multiple industries to obtain complete electricity table data. Specifically, if the missing data is numerical data, the average value or median value can be used to fill in the missing data. For example, if the missing data is the electricity consumption of a certain day in the industrial power load data, the average value of the electricity consumption of that month in the industrial power load data or the median of the electricity consumption of that month can be used as the missing value. If the missing data is categorical data, the most frequent value can be used to fill in the missing data. For example, if the missing data is the weather in the residential power load data, the weather that appears most frequently in that month in the residential power load data can be used as the missing weather data.

[0041] S120. Input the complete power table data into a TabTransformer model, convert the categorical data in the complete power table data into contextual embedded features through the TabTransformer model, and convert the numerical data in the complete power load data into a high-dimensional continuous feature sequence through the TabTransformer model.

[0042] Among them, the TabTransformer model is a deep learning architecture based on the self-attention mechanism.

[0043] In this embodiment, the complete electricity table data is input into the TabTransformer model. The TabTransformer model performs specific feature processing on the categorical data (i.e., classification features) within the complete electricity table data and converts it into contextually embedded features. Different industries process categorical data differently, and different feature processing methods are applied to specific industry features. Contextually embedded features refer to word representation methods that can dynamically adjust based on the specific context of the word in a sentence or text.

[0044] In this embodiment, the TabTransformer model performs standardization processing and linear transformation on the numerical data (i.e., continuous features) in the complete power table data to obtain a high-dimensional continuous feature sequence. Among them, the high-dimensional continuous feature sequence has richer semantics than the continuous feature. For example, the power data is 2000V, and time information and location information are added to it to obtain a high-dimensional continuous feature sequence. It should be noted that for heterogeneous data in multiple industries, specific feature processing is performed on the categorical data of different industries according to the characteristics of different industries, and the numerical data of different industries are standardized to achieve data normalization, so as to ensure that all types of data can be efficiently integrated in the TabTransformer model.

[0045] S130. Learning the long-distance dependency between the context embedding features through multiple Transformer layers in the TabTransformer model to obtain context embedding, and generating a target feature sequence based on the context embedding and the high-dimensional continuous feature sequence, wherein each Transformer layer includes a multi-head self-attention mechanism.

[0046] In this embodiment, the TabTransformer model may include multiple Transformer layers and multi-layer perceptrons. Each Transformer layer may include a multi-head attention mechanism and a feedforward neural network.

[0047] The multi-head attention mechanism can capture long-range dependencies between contextual embedding features to obtain contextual embeddings. This is particularly effective in identifying long-term dependencies and cross-industry interactions between features when dealing with cyclically changing power load data. A feedforward neural network can perform nonlinear transformations on long contextual embeddings.

[0048] In this embodiment, the context embedding and the high-dimensional continuous feature sequence are fused and then nonlinearly transformed to generate a target feature sequence, which has richer feature representation.

[0049] S140: Input the target feature sequence into the multi-layer perceptron in the TabTransformer model, and output the prediction results of the power load data of multiple industries.

[0050] Among them, the prediction results are the predicted values ​​of power load data in multiple industries.

[0051] In this embodiment, the multilayer perceptron performs nonlinear mapping on the target feature sequence and outputs the predicted value of the power load data.

[0052] Among them, the nonlinear activation function of the multilayer perceptron adopts ReLU, and the TabTransformer model is optimized by using cross entropy or mean square error as the loss function.

[0053] A multi-industry power load data prediction method provided in a first embodiment of the present invention first pre-processes the power load data of multiple industries in a power table to obtain complete power table data; then inputs the complete power table data into a TabTransformer model, and converts the categorical data in the complete power table data into context embedding features through the TabTransformer model, and converts the numerical data in the complete power load data into a high-dimensional continuous feature sequence through the TabTransformer model; then, the long-distance dependency relationship between the context embedding features is learned through multiple Transformer layers in the TabTransformer model to obtain context embedding, and a target feature sequence is generated based on the context embedding and the high-dimensional continuous feature sequence, wherein each Transformer layer includes a multi-head self-attention mechanism; finally, the target feature sequence is input into the multi-layer perceptron in the TabTransformer model to output the prediction results of the multi-industry power load data. The above method uses the multi-head self-attention mechanism in the TabTransformer layer to enable the TabTransformer model to fully learn the interactive relationship between different categorical data and numerical data, solving the problem that traditional machine learning models cannot fully capture the characteristic differences of power load data in various industries, and can improve the model's prediction accuracy for power load data.

[0054] Based on the above embodiment, a modified embodiment of the above embodiment is proposed. It should be noted that, in order to simplify the description, only the differences from the above embodiment are described in the modified embodiment.

[0055] In one embodiment, the categorical data in the complete power load data is converted into contextual embedded features through the TabTransformer model, including: encoding the categorical data in the complete power load data into a categorical feature vector of fixed dimension through the embedding layer in the TabTransformer model; and contextually embedding the categorical feature vector through the context modeling layer in the TabTransformer model to obtain contextual embedded features.

[0056] Among them, for each category feature in the classification feature vector, the embedding formula is:

[0057] e cat =W emb ·x cat

[0058] In the above formula, W emb represents the embedding matrix, x cat represents the category feature, e cat Represents contextual embedding features.

[0059] It should be noted that the embedding dimension can be adaptively adjusted according to the load patterns of different industries to improve the adaptability of the TabTransformer model to the data.

[0060] In one embodiment, the numerical data in the complete power load data is converted into a high-dimensional continuous feature sequence through the TabTransformer model, including: standardizing the numerical data in the complete power load data to obtain normalized continuous features; and mapping the normalized continuous features into a high-dimensional continuous feature sequence through linear transformation.

[0061] Among them, the mapping of continuous features is performed through linear transformation, and the corresponding formula is as follows:

[0062] e cont =W cont ·x cont

[0063] In the above formula, x cont represents continuous features, W cont represents a linear matrix, e cont Represents a high-dimensional continuous feature sequence.

[0064] Furthermore, the method also includes: in the process of obtaining context embedding by learning the long-distance dependency between the context embedding features through multiple Transformer layers in the TabTransformer model, the weights of the categorical data and numerical data that are less than a preset number in the complete power load data are increased, and the weights of the categorical data and numerical data that are greater than a preset number in the complete power load data are reduced.

[0065] Among them, a category weight adjustment mechanism is introduced for category imbalanced data to enhance the learning ability of the TabTransformer model for minority category data.

[0066] Example 2

[0067] Figure 2 This is a flow chart of a multi-industry power load data prediction method provided by the second embodiment of the present invention. This second embodiment is optimized based on the above embodiments. For details not yet fully described in this embodiment, please refer to the first embodiment.

[0068] like Figure 2 As shown, a multi-industry power load data prediction method provided by the second embodiment of the present invention includes the following steps:

[0069] S210 , pre-processing the power load data of multiple industries in the power table to obtain complete power table data.

[0070] S220. Input the complete power table data into a TabTransformer model, convert the categorical data in the complete power table data into contextual embedded features through the TabTransformer model, and convert the numerical data in the complete power load data into a high-dimensional continuous feature sequence through the TabTransformer model.

[0071] S230. For each Transformer layer in the TabTransformer model, the long-distance dependency between the context embedding features is obtained through a multi-head self-attention mechanism to obtain context embedding, and the context embedding is nonlinearly transformed through a feedforward neural network.

[0072] Among them, through parallel calculation of multiple attention heads, different levels of relationships between contextual embedding features can be captured.

[0073] The calculation formula of the self-attention mechanism of each Transformer layer is:

[0074]

[0075] In the above formula, Q, K, V represent query, key, and value matrices respectively. Q, K, V are obtained by linear transformation of context embedding features. k Represents the dimension of the attention head.

[0076] S240: Concatenate the context embedding generated by each Transformer layer and the high-dimensional continuous feature sequence into a fused feature sequence.

[0077] S250: Performing a nonlinear transformation on the fused feature sequence output by each Transformer layer through the feedforward neural network to generate a target feature sequence.

[0078] S260: Input the target feature sequence into the multi-layer perceptron in the TabTransformer model to output prediction results of multi-industry power load data.

[0079] A multi-industry power load data prediction method is provided in the second embodiment of the present invention. The multi-head attention mechanism of the TabTransformer model provides a specific weighted processing method for the power load data prediction of each industry, so that the interaction relationship between different category features and continuous features can be fully explored, thereby improving the accuracy of the prediction.

[0080] Example 3

[0081] Figure 3 This is a flow chart of a method for predicting power load data for multiple industries provided by the third embodiment of the present invention. This third embodiment is optimized based on the above embodiments. For details not yet fully described in this embodiment, please refer to the first and second embodiments.

[0082] like Figure 3 As shown, a multi-industry power load data prediction method provided by the third embodiment of the present invention includes the following steps:

[0083] S310 , pre-processing the power load data of multiple industries in the power table to obtain complete power table data.

[0084] S320: Input the complete power table data into a TabTransformer model, convert the categorical data in the complete power table data into contextual embedded features through the TabTransformer model, and convert the numerical data in the complete power load data into a high-dimensional continuous feature sequence through the TabTransformer model.

[0085] S330. Learning the long-distance dependencies between the context embedding features through multiple Transformer layers in the TabTransformer model to obtain context embedding, and generating a target feature sequence based on the context embedding and the high-dimensional continuous feature sequence, wherein each Transformer layer includes a multi-head self-attention mechanism.

[0086] S340: Input the target feature sequence into the multilayer perceptron in the TabTransformer model.

[0087] S350. Perform a nonlinear transformation on the target feature sequence through the nonlinear activation function of each layer in the multi-layer perceptron, and output prediction results of multi-industry power load data.

[0088] Each layer of the multilayer perceptron MLP is transformed nonlinearly through the ReLU activation function, and the final output of the TabTransformer model is the prediction result of the power load data. The forward propagation formula of the multilayer perceptron is:

[0089] y=ReLU(W MLP ·e final +b MLP )

[0090] In the above formula, e final represents the target feature sequence, W MLP and b MLP They represent the weights and biases of the multilayer perceptron, and ReLU() represents the activation function.

[0091] S360: Optimize the TabTransformer model using the average error as a loss function.

[0092] Among them, in order to improve the performance of the TabTransformer model, the average error is used as the loss function to optimize the TabTransformer model. The corresponding formula is as follows:

[0093]

[0094] In the above formula, and They represent the predicted value and true value of the i-th power load data respectively, and N represents the total number of power load data.

[0095] S370. Perform a visual analysis on the attention weights of each Transformer layer in the TabTransformer model.

[0096] Among them, by visualizing the attention weights of each layer of Transformer, an interpretable analysis of the TabTransformer model is provided to help researchers understand the importance of features in load forecasting.

[0097] In this example, by visualizing the attention weights of each Transformer layer, we can clearly see the importance of each feature in different prediction tasks. This mechanism is particularly suitable for power load data forecasting tasks, helping researchers understand how the model uses industry characteristics to predict load, thereby improving the model's transparency and operability.

[0098] Example 4

[0099] Figure 4 This is a structural schematic diagram of a multi-industry power load data prediction device provided in Example 4 of the present invention. The device can be used for accurate prediction and task analysis of multi-source heterogeneous power load data, wherein the device can be implemented by software and / or hardware and is generally integrated on an electronic device.

[0100] like Figure 4 As shown, the device includes: a preprocessing module 110, a conversion module 120, a learning module 130 and an output module 140.

[0101] A preprocessing module 110 is used to preprocess the power load data of multiple industries in the power table to obtain complete power table data;

[0102] a conversion module 120 for inputting the complete power table data into a TabTransformer model, converting the categorical data in the complete power table data into contextual embedding features through the TabTransformer model, and converting the numerical data in the complete power load data into a high-dimensional continuous feature sequence through the TabTransformer model;

[0103] A learning module 130 is configured to learn the long-range dependencies between the context embedding features through multiple Transformer layers in the TabTransformer model to obtain context embeddings, and generate a target feature sequence based on the context embeddings and the high-dimensional continuous feature sequence, wherein each Transformer layer includes a multi-head self-attention mechanism;

[0104] The output module 140 is used to input the target feature sequence into the multi-layer perceptron in the TabTransformer model to output the prediction results of the power load data of multiple industries.

[0105] In this embodiment, the device first preprocesses the power load data of multiple industries in the power table through the preprocessing module 110 to obtain complete power table data; then, the complete power table data is input into the TabTransformer model through the conversion module 120, and the categorical data in the complete power table data is converted into context embedding features through the TabTransformer model, and the numerical data in the complete power load data is converted into a high-dimensional continuous feature sequence through the TabTransformer model; then, the long-distance dependency between the context embedding features is learned through multiple Transformer layers in the TabTransformer model through the learning module 130 to obtain context embedding, and a target feature sequence is generated based on the context embedding and the high-dimensional continuous feature sequence, wherein each Transformer layer includes a multi-head self-attention mechanism; finally, the target feature sequence is input into the multi-layer perceptron in the TabTransformer model through the output module 140 to output the prediction results of the power load data of multiple industries.

[0106] This embodiment provides a power load data prediction device, which can improve the prediction accuracy of the model for power load data.

[0107] Furthermore, the conversion module 120 includes a first conversion submodule and a second conversion submodule;

[0108] The first conversion submodule includes:

[0109] an encoding unit, configured to encode the categorical data in the complete power load data into a categorical feature vector of fixed dimension through an embedding layer in the TabTransformer model;

[0110] A computing unit is configured to perform context embedding on the category feature vector through a context modeling layer in the TabTransformer model to obtain a context embedding feature.

[0111] The second conversion submodule includes:

[0112] a processing unit, configured to normalize the numerical data in the complete power load data by using the TabTransformer model to obtain normalized continuous features;

[0113] A mapping unit is used to map the normalized continuous features into a high-dimensional continuous feature sequence through linear transformation using the TabTransformer model.

[0114] Furthermore, each Transformer layer in the TabTransformer model also includes a feedforward neural network. Accordingly, the learning module 130 includes:

[0115] A generation unit is configured to obtain, for each Transformer layer, long-range dependencies between the context embedding features through the multi-head self-attention mechanism to obtain context embeddings, and perform nonlinear transformation on the context embeddings through the feedforward neural network;

[0116] A fusion unit, configured to concatenate the context embedding generated by each Transformer layer with the high-dimensional continuous feature sequence into a fused feature sequence;

[0117] The transformation unit is used to perform nonlinear transformation on the fused feature sequence output by each Transformer layer through the feedforward neural network to generate a target feature sequence.

[0118] Furthermore, the output module 140 includes:

[0119] An input unit, configured to input the target feature sequence into the multilayer perceptron in the TabTransformer model;

[0120] The output unit is used to perform nonlinear transformation on the target feature sequence through the nonlinear activation function of each layer in the multilayer perceptron, and output the prediction result of the power load data.

[0121] Furthermore, the device also includes an adjustment module for increasing the weights of categorical data and numerical data that are less than a preset number in the complete power load data, and reducing the weights of categorical data and numerical data that are greater than a preset number in the complete power load data, in the process of obtaining context embedding by learning the long-distance dependencies between the context embedding features through multiple Transformer layers in the TabTransformer model.

[0122] Furthermore, the device also includes an optimization module for optimizing the TabTransformer model by using the average error as a loss function.

[0123] Furthermore, the device also includes an analysis module for visually analyzing the attention weights of each Transformer layer in the TabTransformer model.

[0124] The above-mentioned power load data prediction device can execute the power load data prediction method provided by any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0125] Example 5

[0126] Figure 5 A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or claimed herein.

[0127] like Figure 5 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., which is communicatively connected to the at least one processor 11. The memory stores a computer program that can be executed by the at least one processor. The processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 into the random access memory (RAM) 13. Various programs and data required for the operation of the electronic device 10 can also be stored in the RAM 13. The processor 11, ROM 12, and RAM 13 are connected to each other via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0128] Multiple components in the electronic device 10 are connected to the I / O interface 15, including an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a magnetic disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0129] The processor 11 can be any general-purpose and / or specialized processing component with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any other suitable processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as the multi-industry power load data forecasting method.

[0130] In some embodiments, the multi-industry power load data forecasting method can be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps in the multi-industry power load data forecasting method described above can be performed. Alternatively, in other embodiments, the processor 11 can be configured to execute the multi-industry power load data forecasting method in any other appropriate manner (for example, by means of firmware).

[0131] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0132] In some embodiments, the multi-industry power load data prediction method can be implemented as a computer program, which is invisibly included in a computer program product. When the computer program is executed by a processor, the multi-industry power load data prediction method of the present invention is implemented. The computer program product can be understood as a software product that mainly implements its solution through a computer program. The computer program for implementing the method of the present invention can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flowchart and / or block diagram are implemented. The computer program can be executed entirely on the machine, partially on the machine, partially on the machine as a stand-alone software package and partially on a remote machine, or entirely on a remote machine or server.

[0133] In the context of the present invention, computer-readable storage media can be tangible media that can contain or store a computer program for use with an instruction execution system, device or equipment or used in combination with an instruction execution system, device or equipment. Computer-readable storage media can include but are not limited to electronic, magnetic, optical, electromagnetic, infrared or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, computer-readable storage media can be machine-readable signal media. More specific examples of machine-readable storage media can include electrical connections based on one or more lines, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0134] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0135] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0136] A computing system may include clients and servers. The clients and servers are typically remote from each other and typically interact via a communication network. This client-server relationship arises through computer programs running on the respective computers, creating a client-server relationship. The server may be a cloud server, also known as a cloud computing server or cloud host. This server is a hosting product within the cloud computing service ecosystem that addresses the management difficulties and limited scalability of traditional physical hosting and VPS services.

[0137] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in the present invention can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution of the present invention can be achieved. This is not limited herein.

[0138] The above specific embodiments do not limit the scope of protection of the present invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention are intended to be included within the scope of protection of the present invention.

Claims

1. A multi-industry power load data prediction method, characterized in that: The method comprises: Preprocess the power load data of multiple industries in the power table to obtain complete power table data; Inputting the complete power table data into a TabTransformer model, converting the categorical data in the complete power table data into contextual embedding features through the TabTransformer model, and converting the numerical data in the complete power load data into a high-dimensional continuous feature sequence through the TabTransformer model; Learning long-range dependencies between the context embedding features through multiple Transformer layers in the TabTransformer model to obtain context embeddings, and generating a target feature sequence based on the context embeddings and the high-dimensional continuous feature sequence, wherein each Transformer layer includes a multi-head self-attention mechanism; The target feature sequence is input into the multi-layer perceptron in the TabTransformer model to output the prediction results of the power load data of multiple industries.

2. The method according to claim 1, characterized in that The TabTransformer model is used to convert the categorical data in the complete power load data into contextual embedding features, including: Encoding the categorical data in the complete power load data into a categorical feature vector of fixed dimension through an embedding layer in the TabTransformer model; The category feature vector is context-embedded through the context modeling layer in the TabTransformer model to obtain context-embedded features.

3. The method according to claim 1 or 2, characterized in that Converting the numerical data in the complete power load data into a high-dimensional continuous feature sequence includes: The TabTransformer model is used to normalize the numerical data in the complete power load data to obtain normalized continuous features; The normalized continuous features are mapped into a high-dimensional continuous feature sequence through linear transformation.

4. The method according to claim 1, wherein Each Transformer layer in the TabTransformer model further includes a feedforward neural network. Accordingly, a context embedding is obtained by learning the long-range dependency between the context embedding features through multiple Transformer layers in the TabTransformer model, and a target feature sequence is generated based on the context embedding and the high-dimensional continuous feature sequence, including: For each Transformer layer, the long-range dependency between the context embedding features is obtained through the multi-head self-attention mechanism to obtain the context embedding, and the context embedding is nonlinearly transformed through the feedforward neural network; Concatenate the context embedding generated by each Transformer layer and the high-dimensional continuous feature sequence into a fused feature sequence; The fused feature sequence output by each Transformer layer is subjected to nonlinear transformation through the feedforward neural network to generate a target feature sequence.

5. The method according to claim 1 or 4, characterized in that The step of inputting the target feature sequence into the multilayer perceptron in the TabTransformer model and outputting the prediction result of the power load data includes: Inputting the target feature sequence into the multilayer perceptron in the TabTransformer model; The target feature sequence is nonlinearly transformed by the nonlinear activation function of each layer in the multilayer perceptron, and a prediction result of the power load data is output.

6. The method according to claim 1, characterized in that The method further comprises: In the process of obtaining context embedding by learning the long-distance dependency between the context embedding features through multiple Transformer layers in the TabTransformer model, the weights of the categorical data and numerical data that are less than a preset number in the complete power load data are increased, and the weights of the categorical data and numerical data that are greater than a preset number in the complete power load data are reduced.

7. The method according to claim 1 or 6, characterized in that The method further comprises: The TabTransformer model is optimized using the average error as the loss function; Visualize the attention weights of each Transformer layer in the TabTransformer model.

8. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to execute the multi-industry power load data prediction method according to any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the multi-industry power load data prediction method according to any one of claims 1 to 7 when executed.

10. A computer program product, characterized in that The computer program product comprises a computer program, which, when executed by a processor, implements the multi-industry power load data prediction method according to any one of claims 1 to 7.

Images