Middle and low voltage load time series data prediction method and device based on natural language processing

Abstract

The invention discloses a medium and low voltage load time series data prediction method and device based on natural language processing, and belongs to the field of data prediction, and the method comprises the steps: recognizing a load prediction intention in user input through a natural language processing technology; obtaining a medium and low voltage load data set according to the load prediction intention; respectively inputting the medium and low voltage load data set into a preset BERT model and a preset CTRL model to carry out medium and low voltage load prediction processing so as to obtain a first prediction result and a second prediction result; and integrating the first prediction result and the second prediction result to obtain medium and low voltage load time sequence data. According to the method, prediction and integration are carried out at the same time by calling the two models, on one hand, massive increasing time sequence load data of the medium and low voltage power grid can be adapted, on the other hand, prediction precision can be improved, prediction errors can be reduced, and therefore actual prediction requirements are met.

Description

Technical Field

[0001] This invention relates to the technical field of load time series data prediction, and in particular to a method and apparatus for predicting medium and low voltage load time series data based on natural language processing. Background Technology

[0002] With the rapid development of new power systems and digital grids, medium and low voltage load data, as key data for power system operation and dispatch, are of great significance to grid security, economic dispatch, and the consumption of new energy sources due to their accuracy and real-time performance.

[0003] To determine the time series data of medium and low voltage loads in the power system, the commonly used method is to collect historical load data of the power system, environmental parameters (meteorological factors) of the power system, and electricity consumption behavior data of users. Then, machine learning methods are used to combine the above parameters and data to build a model, and finally, the trained model is used for prediction.

[0004] However, the commonly used methods have the following technical problems: With the popularization of smart meters, IoT devices and distributed energy, the time-series load data of medium and low voltage power grids are showing characteristics of massive growth, high dimensionality and strong nonlinearity. Existing methods process them with a single model, and their prediction accuracy and computational efficiency are difficult to meet the actual needs. Summary of the Invention

[0005] This invention provides a method and apparatus for predicting medium and low voltage load time series data based on natural language processing, which can solve the technical problem that the prediction accuracy and computational efficiency of existing technologies are difficult to meet practical needs.

[0006] A first aspect of this invention provides a method for predicting medium- and low-voltage load time-series data based on natural language processing, the method comprising:

[0007] Identify the load prediction intent in user input using natural language processing technology;

[0008] Obtain a dataset of medium and low voltage loads based on the load forecasting intent;

[0009] The low- and medium-pressure load datasets are input into the preset BERT model and the preset CTRL model respectively for low- and medium-pressure load prediction processing to obtain the first prediction result and the second prediction result.

[0010] By integrating the first and second prediction results, the time series data of medium and low voltage loads are obtained.

[0011] This invention uses two models to simultaneously predict and integrate data. On the one hand, it can adapt to the massive increase in time-series load data of medium and low voltage power grids. On the other hand, it can improve the accuracy of prediction and reduce prediction errors, thereby meeting actual prediction needs.

[0012] In conjunction with the first aspect, in one implementation, the step of identifying the load prediction intent in the user input using natural language processing technology includes:

[0013] A bidirectional recurrent neural network (Bi-RNN) is used to encode the contextual semantic information of the user input;

[0014] Key features are extracted using 1D convolutional layers of a convolutional neural network (CNN).

[0015] The probability of a user choosing to perform load time series forecasting is output based on the Softmax classifier, and the load forecasting intention is determined based on the probability of the load time series forecasting.

[0016] In conjunction with the first aspect, in one implementation, the integration of the first prediction result and the second prediction result to obtain medium- and low-voltage load time-series data includes:

[0017] The first prediction result and the second prediction result are combined to obtain the merged result;

[0018] The merged result is input into a preset fully connected layer so that the preset fully connected layer integrates the first prediction result and the second prediction result to obtain medium and low voltage load time series data.

[0019] In conjunction with the first aspect, in one implementation, the preset fully connected layer integrates the first prediction result and the second prediction result to obtain medium- and low-voltage load time-series data, including:

[0020] The following formula is used to integrate and obtain the time-series data of medium and low voltage loads:

[0021]

[0022] In the above formula, h1 is the feature vector of the first prediction result, h2 is the feature vector of the second prediction result, and h... concat y is the feature vector of the merged result, W is the weight matrix, b is the bias vector, and y is the time series data of medium and low voltage loads.

[0023] In conjunction with the first aspect, in one implementation, the preset BERT model includes: a BERT input layer, several Transformer Encoder layers, and a BERT output layer connected in sequence;

[0024] The BERT input layer includes: a BERT word embedding layer, a BERT segment embedding layer, and a BERT position embedding layer;

[0025] Each Transformer Encoder layer comprises: a BERT self-attention layer, a BERT feedforward network, and a BERT residual and normalization layer connected in sequence.

[0026] In conjunction with the first aspect, in one implementation, the first prediction result output by the BERT output layer is as follows:

[0027]

[0028] in, For the first prediction of the next τ steps, W y For predicting the head weight matrix, H t L For the output of the last Transformer Encoder layer, b y To predict head bias.

[0029] In conjunction with the first aspect, in one implementation, the preset CTRL model includes: a CTRL input layer, a Transformer layer, and a CTRL output layer;

[0030] The CTRL input layer includes: a CTRL word embedding layer, a CTRL control code embedding layer, and a CTRL position encoding layer;

[0031] The Transformer layer includes: CTRL multi-head attention layer, CTRL feedforward network, and CTRLBERT residual and normalization.

[0032] In conjunction with the first aspect, in one implementation, the CTRL output layer includes:

[0033]

[0034] Among them, O t G represents the unnormalized vocabulary score. t L Let J be the final hidden state of the t-th token. T e For the word embedding matrix shared with the input, P(u t+1 Let ||·) be a conditional probability distribution, and max(·) be the maximum value function. t+1 This is the second prediction result.

[0035] In conjunction with the first aspect, in one implementation, after the step of obtaining the medium and low voltage load time-series data, the method further includes:

[0036] Calculate the mean absolute error, mean square error, and root mean square error of the medium and low voltage load time series data respectively;

[0037] The prediction accuracy is determined based on the mean absolute error, the mean square error, and the root mean square error.

[0038] A second aspect of the present invention provides a device for predicting medium- and low-voltage load time-series data based on natural language processing, the device comprising:

[0039] The intent recognition module is used to determine the user's intent for load prediction through a semantic recognition model;

[0040] The acquisition module is used to acquire a dataset of medium and low voltage loads according to the load forecasting intent;

[0041] The prediction module is used to input the medium and low voltage load dataset into a preset BERT model and a preset CTRL model respectively for medium and low voltage load prediction processing, and obtain a first prediction result and a second prediction result.

[0042] An integration module is used to integrate the first prediction result and the second prediction result to obtain medium and low voltage load time series data.

[0043] Compared to existing technologies, the present invention provides a method and apparatus for predicting medium- and low-voltage load time-series data based on natural language processing. The advantages of this method are as follows: The invention identifies the load prediction intent in the user's input using natural language processing technology; after obtaining a medium- and low-voltage load dataset based on the load prediction intent, it calls a preset BERT model and a preset CTRL model respectively to perform medium- and low-voltage load prediction processing based on the dataset; finally, it integrates the prediction results of the two models to obtain the predicted medium- and low-voltage load time-series data. By calling two models simultaneously for prediction and integration, it can adapt to the massively increasing time-series load data of medium- and low-voltage power grids, and improve prediction accuracy while reducing prediction errors, thereby meeting actual prediction needs. Attached Figure Description

[0044] Figure 1 This is a flowchart illustrating a method for predicting medium and low voltage load time series data based on natural language processing, according to an embodiment of the present invention.

[0045] Figure 2 This is a schematic diagram of the structure of a BERT model provided in an embodiment of the present invention;

[0046] Figure 3 This is a schematic diagram of the structure of a CTRL model provided in an embodiment of the present invention;

[0047] Figure 4 This is an operation flowchart of a method for predicting medium and low voltage load time series data based on natural language processing, provided by an embodiment of the present invention.

[0048] Figure 5 This is a schematic diagram of a device for predicting medium and low voltage load time series data based on natural language processing, provided in an embodiment of the present invention. Detailed Implementation

[0049] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0050] With the rapid development of new power systems and digital grids, medium and low voltage load data, as key data for power system operation and dispatch, are of great significance to grid security, economic dispatch, and the consumption of new energy sources due to their accuracy and real-time performance.

[0051] To determine the time series data of medium and low voltage loads in the power system, the commonly used method is to collect historical load data of the power system, environmental parameters (meteorological factors) of the power system, and electricity consumption behavior data of users. Then, machine learning methods are used to combine the above parameters and data to build a model, and finally, the trained model is used for prediction.

[0052] However, the commonly used methods have the following technical problems: With the popularization of smart meters, IoT devices and distributed energy, the time-series load data of medium and low voltage power grids are showing characteristics of massive growth, high dimensionality and strong nonlinearity. Existing methods process them with a single model, and their prediction accuracy and computational efficiency are difficult to meet the actual needs.

[0053] To address the aforementioned issues, the following specific embodiments will provide a detailed description and explanation of a method and apparatus for predicting medium and low voltage load time series data based on natural language processing, as provided in this application.

[0054] To address the technical problem that existing technologies cannot meet practical needs in terms of both prediction accuracy and computational efficiency, referencing Figure 1 The diagram shows a flowchart of a method for predicting medium and low voltage load time series data based on natural language processing, according to an embodiment of the present invention.

[0055] As an example, the method for predicting medium and low voltage load time series data based on natural language processing may include:

[0056] S11. Identify the load prediction intent in the user input using natural language processing technology.

[0057] In one embodiment, a bidirectional recurrent neural network (Bi-RNN) based on natural language processing techniques can be used to encode the contextual semantic information of the user input, wherein the operation includes the following steps:

[0058] S111. Use a bidirectional recurrent neural network to encode the contextual semantic information of the user input.

[0059] S112. Extract key features through the 1D convolutional layer of a convolutional neural network.

[0060] S113. Based on the Softmax classifier, output the probability of the user selecting to perform load time series prediction, and determine the load prediction intention based on the probability of the load time series prediction.

[0061] The text sequence input by the user is first processed through word segmentation to obtain a word sequence X = {x1, x2, ..., x...} n}, where n is the number of words after word segmentation. Each word x t Convert into initial word vectors through a pre-trained word embedding model. d emb This represents the word vector dimension, with a default value of 300.

[0062] Two stacked LSTM layers are used as the basic unit of Bi-RNN to replace the traditional RNN in order to solve the gradient vanishing problem. The forward LSTM processes the sequence from left to right:

[0063]

[0064] Backward LSTM processes data in reverse from right to left:

[0065]

[0066] The hidden layer dimension was set to 256, and a dropout layer was used with a dropout rate of 0.3 to suppress overfitting. The forward and backward hidden states were concatenated and fused with the original word vectors.

[0067]

[0068] This fusion vector contains both the contextual semantic information of the words and retains the basic meaning of the original word embeddings, laying the foundation for subsequent feature extraction.

[0069] In one embodiment, key features are extracted through a 1D convolutional layer of a convolutional neural network (CNN). The aim is to extract key features that are strongly related to the "load prediction" intent from the encoded sequence vector, thereby achieving semantic focus through the local feature extraction capability of CNN.

[0070] The sequence vector {h1,h2,...,h} generated during the semantic encoding stage will be used to...n Stacked in chronological order, they form a two-dimensional matrix H∈R n×812 (812 = 2 × 256 + 300), each row corresponds to the fusion vector of a word.

[0071] Using 16 convolutional kernels with a filter width of 1 c ∈R 1×812×16 With a stride of 1, perform row-by-row convolution on matrix H:

[0072]

[0073] in, The ReLU activation function is used to introduce non-linear features, which is the bias of the k-th convolutional kernel.

[0074] The feature sequence output by each convolution kernel Perform global max pooling:

[0075]

[0076] The final concatenation yields a 16-dimensional feature vector. This vector encapsulates the 16 most significant predictive-related features from the input sequence.

[0077] In one embodiment, based on the probability of a user choosing to perform load time series prediction output by the Softmax classifier, a two-layer fully connected network is mainly used as the basic structure of the Softmax classifier. The first layer maps the 16-dimensional feature vector to 64 dimensions:

[0078] f = Tanh(W1·v) c +b1)(W1∈R 64×16 ,b1∈R 64 ).

[0079] The second layer outputs two types of probabilities:

[0080] p = Softmax(W²·f + b²) (W²∈R) 2×64 b2∈R 2 ).

[0081] S12. Obtain the low- and medium-voltage load dataset according to the load forecasting intent.

[0082] In one embodiment, the load forecasting intention can be determined based on the probability of load timing prediction, and then the determination of whether to acquire the medium and low voltage load dataset can be made based on the load forecasting intention.

[0083] For example, if the probability of load time-series forecasting is greater than or equal to a preset threshold, the load forecasting intention is determined to be yes; conversely, if the probability of load time-series forecasting is less than the preset threshold, the load forecasting intention is determined to be no. If the load forecasting intention is yes, it is confirmed that the user has a forecasting intention, and at this time, the low- and medium-voltage load dataset can be obtained.

[0084] The preset threshold θ is a baseline value of 0.8. When p > θ, a trigger signal is generated to the acquisition module to acquire the real medium and low voltage load data and form a dataset, thus obtaining the medium and low voltage load dataset; otherwise, a "non-predictive intent" flag is output, and the subsequent process is terminated.

[0085] S13. Input the medium and low pressure load dataset into the preset BERT model and the preset CTRL model respectively to perform medium and low pressure load prediction processing to obtain the first prediction result and the second prediction result.

[0086] After obtaining the low- and medium-voltage load dataset, the low- and medium-voltage load dataset can be input into the trained BERT model and CTRL model respectively to perform low- and medium-voltage load prediction processing, and the first prediction result and the second prediction result can be obtained respectively.

[0087] This invention selects BERT and CTRL models for medium- and low-voltage load forecasting. The BERT model, with its bidirectional Transformer architecture and masked pre-training mechanism, can effectively capture long-term dependencies and complex nonlinear patterns in load data. The CTRL model, due to its controllable generation capabilities, can flexibly introduce external conditional variables for conditional load forecasting, enhancing the model's ability to fuse multi-source heterogeneous data. The combination of these two models improves the accuracy and robustness of time-series forecasts.

[0088] It should be noted that the data does not need to be processed before being input into the model for training. Some processing, such as normalization, can be implemented through code, so the data does not need to be processed before being input into the model.

[0089] In one embodiment, to distinguish the datasets input to the model, the low- and medium-voltage load datasets input to the preset BERT model are x. t Input the preset CTRL model's low- and medium-pressure load dataset as k t The two datasets are referred to by different names because they are input into two different models, hence the use of different notations.

[0090] Reference Figure 2 The diagram shows a structural schematic of the BERT model provided in an embodiment of the present invention.

[0091] In an optional embodiment, the preset BERT model includes: a BERT input layer, several Transformer Encoder layers, and a BERT output layer connected in sequence;

[0092] The BERT input layer includes: BERT Token Embedding, BERT Segment Embedding, and BERT Position Embedding.

[0093] Specifically, the BERT input layer can be represented as follows:

[0094] E t =W v x t +W p p t +W s s t ;

[0095] Where, x t Let W be the observation value at time t. v For the value embedding matrix, W p For the learnable positional encoding matrix, p t W is the one-hot encoding of position t. s For the piecewise embedding matrix, s t E is the one-hot encoding of the segment ID. t Let be the final embedding representation at time t.

[0096] Each Transformer Encoder layer comprises: a BERT self-attention layer, a BERT feedforward network, and a BERT residual and normalization layer connected in sequence.

[0097] The BERT self-attention layer consists of multi-head self-attention, BERT residuals, and normalized residual connections followed by layer normalization. In one embodiment, the Transformer Encoder layer formula is as follows:

[0098]

[0099] Where E1 is the final embedding at time 1, E2 is the final embedding at time 2, and E... T Let E be the final embedding at time T, and E be the output of the input representation layer. 0 As the input to the first layer of the Transformer Encoder, Q l h Let H be the query matrix of the h-th head. l-1W is the output of the (l-1)th Transformer Encoder layer. h Q Let K be the Query projection matrix of the h-th attention head. l h Let W be the key matrix of the h-th head. h K V is the Key projection matrix of the h-th attention head. l h Let W be the Value matrix of the h-th head. h V Let h be the Value projection matrix of the h-th attention head, and let Attention be the value projection matrix of the h-th attention head. h Let h be the attention output of the h-th head, and softmax(·) be the normalization function. k For each attention head, M is the dimension, where M is the attention mask matrix, and Z is the dimension of the attention head. l The intermediate representation after residual connection is defined by LayerNorm(·), which is the intra-layer feature normalization function, Concat(·), which is the tensor concatenation function, FFN(·), which is the feature nonlinear transformation function, GELU(·), which is the smoothing activation function, and W. O This is the multi-head attention merging matrix, where W1 is the weight of the first layer of the FFN, b1 is the bias of the first layer of the FFN, W2 is the weight of the second layer of the FFN, b2 is the bias of the second layer of the FFN, and H... t L This is the output of the last layer of the Encoder.

[0100] In one embodiment, the first prediction result output by the BERT output layer is shown in the following formula:

[0101]

[0102] in, For the first prediction of the next τ steps, W y For predicting the head weight matrix, H t L For the output of the last Transformer Encoder layer, b y To predict head bias.

[0103] The BERT model exhibits strong adaptability in medium- and low-voltage load forecasting, accurately capturing the spatiotemporal correlation characteristics, temporal dependencies, and contextual semantics of medium- and low-voltage loads. Its bidirectional encoding architecture can simultaneously learn the forward and backward dependencies of historical medium- and low-voltage load data, effectively modeling the periodicity and suddenness of medium- and low-voltage load fluctuations. Through transfer learning, the pre-trained BERT model can quickly adapt to the medium- and low-voltage load characteristics of different regional power grids, and its multi-layer Transformer structure can automatically extract short-term fluctuations and long-term trends of medium- and low-voltage load data, demonstrating superior performance compared to traditional methods in nonlinear relationship modeling and feature interaction.

[0104] Reference Figure 3 The diagram shows a structural schematic of a CTRL model provided in an embodiment of the present invention.

[0105] In an optional embodiment, the preset CTRL model includes: a CTRL input layer, a Transformer layer, and a CTRL output layer;

[0106] The CTRL input layer includes: CTRL token embedding, CTRL control code embedding, and CTRL position encoding layer.

[0107] In one embodiment, the CTRL input layer can be as follows:

[0108] T t =G v k t +G c c+P;

[0109] Among them, T t G is the final input at time t. v Let Token be the embedding vector, and k t Let G be the integer ID of the t-th token. c The control code embedding matrix is ​​defined as follows: c is the integer ID of the control code, and P is the position encoding vector.

[0110] The Transformer layer includes: CTRL multi-head attention layer, CTRL feedforward network, and CTRLBERT residual and normalization.

[0111] Among them, the CTRL multi-head attention layer is a multi-head attention layer, and the sub-layers of the Transformer layer all contain residual connections and LayerNorm.

[0112] In one embodiment, the Transformer layer can be as follows:

[0113]

[0114] Where T1 is the final embedding at time 1, T2 is the final embedding at time 2, and T... T Let R be the final embedding at time T, R be the output of the input representation layer, and L be the final embedding. 0 As the input to the first layer of the Transformer, Q l h Let L be the query matrix with the h-th head. l-1 W is the output of the (l-1)th Transformer layer. h Q Let K be the Query projection matrix of the h-th attention head. l h Let W be the key matrix of the h-th head. h K V is the Key projection matrix of the h-th attention head. l h Let W be the Value matrix of the h-th head. h V Let h be the Value projection matrix of the h-th attention head, and let Attention be the value projection matrix of the h-th attention head. h For the attention output of the h-th head, h k For each attention head, M is the dimension, where M is the attention mask matrix, and F is the dimension of the attention head. l The intermediate representation after residual connection is defined by LayerNorm(·), which is the intra-layer feature normalization function, Concat(·), which is the tensor concatenation function, FFN(·), which is the feature nonlinear transformation function, Dropout(·), which is the normalization function, GELU(·), which is the smoothing activation function, and W. O The multi-head attention merging matrix is ​​defined as follows: W1 represents the weights of the first layer of the FFN, b1 represents the bias of the first layer of the FFN, W2 represents the weights of the second layer of the FFN, b2 represents the bias of the second layer of the FFN, and G represents the weights of the second layer of the FFN. t L This is the output of the last Transformer layer.

[0115] In one embodiment, the CTRL output layer has two parts: a language probability header and a probability generator, the formula of which is shown below:

[0116]

[0117] Among them, O t G represents the unnormalized vocabulary score. t L Let J be the final hidden state of the t-th token. T e For the word embedding matrix shared with the input, P(u t+1Let ||·) be a conditional probability distribution, and max(·) be the maximum value function. t+1 This is the second prediction result.

[0118] S14. Integrate the first prediction result and the second prediction result to obtain the medium and low voltage load time series data.

[0119] After obtaining the prediction results from the two models, the two prediction results can be aggregated to obtain the predicted medium and low pressure load time series data.

[0120] In one embodiment, integrating the first prediction result and the second prediction result to obtain the medium- and low-voltage load time series data may include the following sub-steps:

[0121] S141. The first prediction result and the second prediction result are combined to obtain the merged result.

[0122] S142. Input the merging result into a preset fully connected layer so that the preset fully connected layer integrates the first prediction result and the second prediction result to obtain medium and low voltage load time series data.

[0123] In one embodiment, the fully connected layer integrates the medium- and low-voltage load time-series data using the following formula:

[0124]

[0125] In the above formula, h1 is the feature vector of the first prediction result, h2 is the feature vector of the second prediction result, and h... concat y is the feature vector of the merged result, W is the weight matrix, b is the bias vector, and y is the time series data of medium and low voltage loads.

[0126] In one embodiment, to evaluate whether the predicted medium- and low-pressure load time-series data meets the requirements, the method may further include the following steps, as an example:

[0127] S15. Calculate the mean absolute error, mean square error, and root mean square error of the medium and low pressure load time series data respectively.

[0128] S16. Determine the prediction accuracy based on the mean absolute error, the mean square error, and the root mean square error.

[0129] Specifically, mean absolute error, mean square error, and root mean square error can be used as evaluation indicators, and their calculation formulas are as follows:

[0130]

[0131] Where MAE is the mean absolute error, MSE is the mean squared error, RMSE is the root mean squared error, n is the total number of predicted samples, and yi For the true value at the i-th time point, Let be the predicted value at time point i, and |·| be the absolute value operation. 2 For squaring operations, This is for square root operations.

[0132] Reference Figure 4 The diagram shows an operation flowchart of a method for predicting medium and low voltage load time series data based on natural language processing, according to an embodiment of the present invention.

[0133] Specifically, the operation process of the method for predicting medium and low voltage load time series data based on natural language processing may include the following steps:

[0134] The first step is to use natural language processing technology to identify the user's intent to predict the load.

[0135] The second step is to obtain the actual low- and medium-voltage load dataset (x) based on the stated load forecasting intent. t and k t ), where the two symbols are different names for the same dataset because they are input into two different models.

[0136] The third step is to use the medium and low voltage load dataset as input x. t and k t The BERT model and CTRL model were respectively input for training and prediction.

[0137] The fourth step involves integrating the results obtained from training and predicting the dataset using the BERT and CTRL models through a fully connected layer to obtain the final prediction result.

[0138] The fifth step is to calculate the evaluation indicators: mean absolute error, mean square error, and root mean square error. The larger the error, the lower the model's prediction accuracy, and the more necessary it is to adjust the model parameters. The smaller the error, the higher the model's prediction accuracy.

[0139] In this embodiment, the present invention provides a method for predicting medium- and low-voltage load time-series data based on natural language processing. Its advantages are as follows: The present invention identifies the load prediction intent in the user's input using natural language processing technology; after obtaining a medium- and low-voltage load dataset based on the load prediction intent, it respectively calls a preset BERT model and a preset CTRL model to perform medium- and low-voltage load prediction processing based on the medium- and low-voltage load dataset; finally, it integrates the prediction results of the two models to obtain the predicted medium- and low-voltage load time-series data. By calling two models simultaneously for prediction and integration, it can adapt to the massively increasing time-series load data of medium- and low-voltage power grids, and improve prediction accuracy and reduce prediction errors, thereby meeting actual prediction needs.

[0140] This invention also provides a device for predicting medium and low voltage load time series data based on natural language processing, see [link to documentation]. Figure 5 The diagram shows a schematic of the structure of a medium- and low-voltage load time-series data prediction device based on natural language processing provided in an embodiment of the present invention.

[0141] As an example, the device for predicting medium and low voltage load time series data based on natural language processing may include:

[0142] The intent recognition module 201 is used to determine the user's input intent for load prediction through a semantic recognition model;

[0143] The acquisition module 202 is used to acquire a dataset of medium and low voltage loads according to the load forecasting intent;

[0144] The prediction module 203 is used to input the medium and low pressure load dataset into a preset BERT model and a preset CTRL model respectively for medium and low pressure load prediction processing to obtain a first prediction result and a second prediction result.

[0145] The integration module 204 is used to integrate the first prediction result and the second prediction result to obtain medium and low voltage load time series data.

[0146] Optionally, the integration of the first prediction result and the second prediction result to obtain the medium and low voltage load time series data includes:

[0147] The first prediction result and the second prediction result are combined to obtain the merged result;

[0148] The merged result is input into a preset fully connected layer so that the preset fully connected layer integrates the first prediction result and the second prediction result to obtain medium and low voltage load time series data.

[0149] Optionally, the preset fully connected layer integrates the first prediction result and the second prediction result to obtain medium and low voltage load time series data, including:

[0150] The following formula is used to integrate and obtain the time-series data of medium and low voltage loads:

[0151]

[0152] In the above formula, h1 is the feature vector of the first prediction result, h2 is the feature vector of the second prediction result, and h... concat y is the feature vector of the merged result, W is the weight matrix, b is the bias vector, and y is the time series data of medium and low voltage loads.

[0153] Optionally, the preset BERT model includes: a BERT input layer, several TransformerEncoder layers, and a BERT output layer connected in sequence;

[0154] The BERT input layer includes: a BERT word embedding layer, a BERT segment embedding layer, and a BERT position embedding layer;

[0155] Each Transformer Encoder layer comprises: a BERT self-attention layer, a BERT feedforward network, and a BERT residual and normalization layer connected in sequence.

[0156] Optionally, the first prediction result output by the BERT output layer is shown in the following formula:

[0157]

[0158] in, For the first prediction of the next τ steps, W y For predicting the head weight matrix, H t L For the output of the last Transformer Encoder layer, b y To predict head bias.

[0159] Optionally, the preset CTRL model includes: a CTRL input layer, a Transformer layer, and a CTRL output layer;

[0160] The CTRL input layer includes: a CTRL word embedding layer, a CTRL control code embedding layer, and a CTRL position encoding layer;

[0161] The Transformer layer includes: CTRL multi-head attention layer, CTRL feedforward network, and CTRLBERT residual and normalization.

[0162] Optionally, the CTRL output layer includes:

[0163]

[0164] Among them, O t G represents the unnormalized vocabulary score. t L Let J be the final hidden state of the t-th token. T e For the word embedding matrix shared with the input, P(u t+1 Let ||·) be a conditional probability distribution, and max(·) be the maximum value function. t+1 This is the second prediction result.

[0165] Optionally, the device further includes:

[0166] The calculation module is used to calculate the mean absolute error, mean square error, and root mean square error of the medium and low voltage load time series data after the step of obtaining the medium and low voltage load time series data.

[0167] The determination module is used to determine the prediction accuracy based on the mean absolute error, the mean square error, and the root mean square error.

[0168] Those skilled in the art will understand that, for ease of description and brevity, the specific working process of the device described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0169] Furthermore, this application also provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the method for predicting medium and low voltage load time series data based on natural language processing as described in the above embodiments.

[0170] Furthermore, embodiments of this application also provide a computer-readable storage medium storing a computer-executable program for causing a computer to execute the natural language processing-based method for predicting medium and low voltage load time series data as described in the above embodiments.

[0171] In the description of the embodiments of the present invention, it should be noted that the terms "above," "below," etc., indicating the orientation or positional relationship are based on the orientation or positional relationship shown in the accompanying drawings, and are only for the convenience of describing the embodiments of the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the present invention. When an element such as a layer, region, or substrate is referred to as being "above" or "on top of" another element, it may be directly on the other element, or there may be an intermediate element. Conversely, when an element is referred to as being "directly on" or "above" another element, there is no intermediate element. It should also be understood that when an element is referred to as being "below" or "under" another element, it may be directly below or under the other element, or there may be an intermediate element. Conversely, when an element is referred to as being "directly below" or "under" another element, there is no intermediate element. Unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to fixed connections, detachable connections, or integral connections; they can refer to mechanical connections or electrical connections; they can refer to direct connections or indirect connections through an intermediate medium; and they can refer to the internal connection between two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0172] Those skilled in the art will understand that embodiments of this application may also include computer program products. Therefore, this application may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application may take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0173] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), devices, and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0174] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0175] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0176] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for predicting medium- and low-voltage load time-series data based on natural language processing, characterized in that, The method includes: Identify the load prediction intent in user input using natural language processing technology; Obtain a dataset of medium and low voltage loads based on the load forecasting intent; The low- and medium-pressure load datasets are input into the preset BERT model and the preset CTRL model respectively for low- and medium-pressure load prediction processing to obtain the first prediction result and the second prediction result. By integrating the first and second prediction results, the time series data of medium and low voltage loads are obtained.

2. The method for predicting medium and low voltage load time series data based on natural language processing according to claim 1, characterized in that, The process of identifying the load prediction intent in user input using natural language processing technology includes: A bidirectional recurrent neural network is used to encode the contextual semantic information of the user input; Key features are extracted using a 1D convolutional layer of a convolutional neural network; The probability of a user choosing to perform load time series forecasting is output based on the Softmax classifier, and the load forecasting intention is determined based on the probability of the load time series forecasting.

3. The method for predicting medium and low voltage load time series data based on natural language processing according to claim 1, characterized in that, The integration of the first and second prediction results to obtain medium- and low-voltage load time-series data includes: The first prediction result and the second prediction result are combined to obtain the merged result; The merged result is input into a preset fully connected layer so that the preset fully connected layer integrates the first prediction result and the second prediction result to obtain medium and low voltage load time series data.

4. The method for predicting medium and low voltage load time series data based on natural language processing according to claim 2, characterized in that, The preset fully connected layer integrates the first prediction result and the second prediction result to obtain medium and low voltage load time series data, including: The following formula is used to integrate and obtain the time-series data of medium and low voltage loads: In the above formula, h1 is the feature vector of the first prediction result, h2 is the feature vector of the second prediction result, and h... concat y is the feature vector of the merged result, W is the weight matrix, b is the bias vector, and y is the time series data of medium and low voltage loads.

5. The method for predicting medium and low voltage load time series data based on natural language processing according to claim 1, characterized in that, The preset BERT model includes: a BERT input layer, several TransformerEncoder layers and a BERT output layer connected in sequence; The BERT input layer includes: a BERT word embedding layer, a BERT segment embedding layer, and a BERT position embedding layer; Each Transformer Encoder layer comprises: a BERT self-attention layer, a BERT feedforward network, and a BERT residual and normalization layer connected in sequence.

6. The method for predicting medium and low voltage load time series data based on natural language processing according to claim 4, characterized in that, The first prediction result output by the BERT output layer is shown in the following formula: in, For the first prediction of the next τ steps, W y For predicting the head weight matrix, H t L For the output of the last Transformer Encoder layer, b y To predict head bias.

7. The method for predicting medium and low voltage load time series data based on natural language processing according to claim 1, characterized in that, The preset CTRL model includes: a CTRL input layer, a Transformer layer, and a CTRL output layer; The CTRL input layer includes: a CTRL word embedding layer, a CTRL control code embedding layer, and a CTRL position encoding layer; The Transformer layer includes: CTRL multi-head attention layer, CTRL feedforward network, and CTRLBERT residual and normalization.

8. The method for predicting medium and low voltage load time series data based on natural language processing according to claim 6, characterized in that, The CTRL output layer includes: Among them, O t G represents the unnormalized vocabulary score. t L Let J be the final hidden state of the t-th token. T e For the word embedding matrix shared with the input, P(u t+1 Let ||·) be a conditional probability distribution, and max(·) be the maximum value function. t+1 This is the second prediction result.

9. The method for predicting medium and low voltage load time series data based on natural language processing according to any one of claims 1-8, characterized in that, After the step of obtaining the medium and low voltage load time series data, the method further includes: Calculate the mean absolute error, mean square error, and root mean square error of the medium and low voltage load time series data respectively; The prediction accuracy is determined based on the mean absolute error, the mean square error, and the root mean square error.

10. A device for predicting medium- and low-voltage load time-series data based on natural language processing, characterized in that, The device includes: The intent recognition module is used to determine the user's intent for load prediction through a semantic recognition model; The acquisition module is used to acquire a dataset of medium and low voltage loads according to the load forecasting intent; The prediction module is used to input the medium and low voltage load dataset into a preset BERT model and a preset CTRL model respectively for medium and low voltage load prediction processing, and obtain a first prediction result and a second prediction result. An integration module is used to integrate the first prediction result and the second prediction result to obtain medium and low voltage load time series data.

Images