Power time sequence prediction method and system based on large-scale language model, terminal and storage medium thereof

Abstract

The invention relates to a power time sequence prediction method and system based on a large-scale language model, a terminal and a storage medium thereof, and the method is characterized in that the method comprises the following steps: collecting a power time sequence, and decomposing the power time sequence into a seasonal term sequence and a trend term sequence through an adaptive decomposition module; performing operation on the season item sequence and the trend item sequence by adopting instance regularization and fragmentation to obtain season data and trend data; and splicing and mapping the seasonal data and the trend data, inputting the spliced and mapped seasonal data and trend data into a pre-trained GPT model, and outputting a predicted quantity of the power time sequence. The invention designs a novel universal time sequence large-scale language model, which is effective and reliable, accurate in analysis result, and simple and feasible in parameter training.

Description

Technical Field

[0001] The present invention relates to the field of power systems, and more specifically, to a method and system for predicting power time series based on a large-scale language model. Background Art

[0002] Time series forecasting, that is, predicting future discrete time signals based on historical observations, is widely used in many fields such as medicine, finance, transportation, and especially electricity. In recent years, deep neural networks have been widely used in time series forecasting tasks, and have achieved significant performance improvements compared to traditional statistical learning methods and machine learning methods, such as convolutional neural networks (CNN), recurrent neural networks (RNN), graph neural networks (GNN), and Transformers.

[0003] With the improvement of hardware performance and the increase of data volume, the scale of deep learning models has gradually expanded. In the fields of computer vision and natural language processing, large-scale deep network models with pre-training-fine-tuning training paradigm have emerged. These large models attempt to outperform traditional deep learning models on downstream tasks with few or even no supervised samples by pre-training on tens of billions of data, such as CLIP and SAM in the field of computer vision and ChatGPT and LLaMA in the field of natural language processing. This generalization ability makes researchers naturally want to build a large-scale deep model in the field of time series analysis to solve data analysis and downstream tasks in unseen fields. However, there are still few studies on large-scale models in the field of time series analysis. First, there is a lack of sufficient data to train a basic large model in the field of time series analysis. The largest time series field data set is less than 10GB in size. Second, time series data is different from text data in the field of natural language processing. Even text data in different fields contain the same or similar semantic information, while time series data in different fields may have almost completely different patterns and features. These difficulties make it extremely difficult to train a large-scale model in the field of time series analysis from scratch, and it is difficult to have strong generalization properties.

[0004] Some research works have tried to use hint engineering and fine-tuning techniques to migrate large-scale language models from other fields to the field of time series analysis, and have achieved some results. However, these methods do not fully consider the characteristics of time series data, especially the characteristics of time series data in the power field, which makes them have no obvious advantages in comparison with the performance of traditional time series prediction models.

[0005] In response to the above problems, a new power time series prediction method and system based on large-scale language models is urgently needed. Summary of the invention

[0006] In order to address the deficiencies in the prior art, the present invention provides a method and system for power time series prediction based on a large-scale language model. Starting from the characteristics of time series data in the electrical field and the characteristics of time series prediction tasks, improvements are made on the existing large-scale language model GPT-2, and a new general-purpose time series large-scale language model is designed.

[0007] The present invention adopts the following technical solution.

[0008] The first aspect of the present invention relates to a method for predicting electric power time series based on a large-scale language model, the method comprising the following steps: collecting electric power time series, and decomposing the electric power time series into a seasonal item sequence and a trend item sequence using an adaptive decomposition module; using instance regularization and fragmentation to respectively operate the seasonal item sequence and the trend item sequence to obtain seasonal data and trend data; after splicing and mapping the seasonal data and the trend data, input them into a pre-trained GPT model, and output the predicted value of the electric power time series.

[0009] Preferably, the power time series is decomposed into a seasonal item sequence and a trend item sequence using an adaptive decomposition module, including: setting K average pooling filters with different kernel sizes; using each average pooling filter to perform a first average pooling operation on the power time series to obtain a first-order trend item; filling the first-order trend item and then performing a second average pooling operation to obtain a second-order trend item; performing weighted summation on the first-order trend item and the second-order trend item and inputting them into a neural network to obtain a trend item for each average pooling filter; after weighting the trend items of K average pooling filters, obtaining a trend item sequence of the power time series; subtracting the trend item sequence from the power time series to obtain a seasonal item sequence.

[0010] Preferably, instance regularization and sharding are used to respectively operate on the seasonal item sequence and the trend item sequence to obtain seasonal data and trend data, including: instance regularization is: , In the formula, and They are trend term series and seasonal term series respectively; and are expectation and variance operations respectively, and are respectively the learnable scaling parameters and offsets in the trainable affine parameter vector; is the error term.

[0011] Sharding into the seasonal term sequence after regularization of the instance , trend item sequence Segmentation is performed to obtain multiple sharded seasonal and trend data.

[0012] Preferably, after splicing and mapping the seasonal data and the trend data, they are input into the pre-trained GPT model to output the predicted value of the power time series, including: pre-training the GPT model, the pre-training is: the GPT model is trained with the goal of minimizing the L2 loss function, so as to fine-tune the parameters of the residual and the regularization layer through training; fine-tuning uses the gradient descent algorithm to iteratively update the values ​​of the first weight vector, the second weight vector and the learnable scaling parameter.

[0013] Preferably, the fine-tuning uses a gradient descent algorithm to iteratively update the first weight vector, the second weight vector and the learnable scaling parameter The value of includes: the first weight vector is , where and are the weights of the first-order trend term and the second-order trend term respectively; the second weight vector is , where are the weights of the trend terms of the K average pooling filters, respectively.

[0014] Preferably, the gradient descent algorithm comprises: assuming a differentiable parameter is the noise objective function The independent variable, noise objective function Values ​​and parameters of The value of is related; Initialize the learning rate , the exponential decay rate of the moving average moment vector estimate and ,parameter , first moment vector , the second moment vector , time step ,gradient ; Assume that at time step Lower differentiable parameter for ,and If it does not converge, then at a time step of When solving the noise objective function In the differentiable parameters Gradient in direction ; According to the gradient Solve the first moment vector and the second moment vector, as well as the deviation correction of the first moment vector and the second moment vector; according to the deviation correction of the first moment vector and the second moment vector and the learning rate, the differentiable parameter In time step The value of is updated until the differentiable parameter Until convergence; the differentiable parameters The corresponding parameters at convergence The value of is taken as the first weight vector after fine-tuning , the second weight vector With learnable scaling parameters The value of .

[0015] Preferably, according to the gradient Solving the first moment vector and the second moment vector, as well as the deviation correction amount of the first moment vector and the second moment vector, includes: The first moment vector is: , The second moment vector is: , In the formula, ; The deviation correction of the first moment vector is: , The deviation correction of the second moment vector is: , In the formula, and is the exponential moving average moment vector and In time step The moment vector estimate under .

[0016] The second aspect of the present invention relates to a power time series prediction system based on a large-scale language model using the method in the first aspect of the present invention, the system comprising a decomposition module, a calculation module and a prediction module; wherein the decomposition module is used to collect power time series, and decompose the power time series into seasonal item sequences and trend item sequences using an adaptive decomposition module; the calculation module is used to use instance regularization and fragmentation to perform operations on the seasonal item sequences and the trend item sequences respectively to obtain seasonal data and trend data; the prediction module is used to splice and map the seasonal data and the trend data, and then input them into the pre-trained GPT model to output the predicted value of the power time series.

[0017] Preferably, the decomposition module uses an adaptive decomposition module to decompose the power time series into a seasonal item sequence and a trend item sequence, including: setting K average pooling filters with different kernel sizes; using each average pooling filter to perform a first average pooling operation on the power time series to obtain a first-order trend item; filling the first-order trend item and then performing a second average pooling operation to obtain a second-order trend item; performing weighted summation on the first-order trend item and the second-order trend item and inputting them into the neural network to obtain a trend item for each average pooling filter; after weighting the trend items of K average pooling filters, obtaining a trend item sequence of the power time series; subtracting the trend item sequence from the power time series to obtain a seasonal item sequence.

[0018] Preferably, the operation module uses instance regularization and sharding to operate on the seasonal item sequence and the trend item sequence respectively to obtain seasonal data and trend data, including: instance regularization is: , In the formula, and They are trend term series and seasonal term series respectively; and are expectation and variance operations respectively, and are respectively the learnable scaling parameters and offsets in the trainable affine parameter vector; is the error term; Sharding into the seasonal term sequence after regularization of the instance , trend item sequence Segmentation is performed to obtain multiple sharded seasonal and trend data.

[0019] Preferably, the prediction module concatenates and maps the seasonal data and trend data, inputs them into the pre-trained GPT model, and outputs the predicted value of the power time series, including: pre-training the GPT model, the pre-training is: pre-training the GPT model, the pre-training is: the GPT model is trained with the goal of minimizing the L2 loss function, so as to fine-tune the parameters of the residual and regularization layer through training; fine-tuning, using the gradient descent algorithm to iteratively update the values ​​of the first weight vector, the second weight vector and the learnable scaling parameter.

[0020] Preferably, in the pre-training module, fine-tuning is performed by iteratively updating the first weight vector, the second weight vector and the learnable scaling parameter using a gradient descent algorithm. The value of includes: the first weight vector is , where and are the weights of the first-order trend term and the second-order trend term respectively; the second weight vector is , where are the weights of the trend terms of the K average pooling filters, respectively.

[0021] Preferably, in the pre-training module, the gradient descent algorithm includes: assuming a differentiable parameter is the noise objective function The independent variable, noise objective function Values ​​and parameters of The value of is related; Initialize the learning rate , the exponential decay rate of the moving average moment vector estimate and ,parameter , first moment vector , the second moment vector , time step ,gradient ; Assume that at time step Lower differentiable parameter for ,and If it does not converge, then at a time step of When solving the noise objective function In the differentiable parameters Gradient in direction ; According to the gradient Solve the first moment vector and the second moment vector, as well as the deviation correction of the first moment vector and the second moment vector; according to the deviation correction of the first moment vector and the second moment vector and the learning rate, the differentiable parameter In time step The value of is updated until the differentiable parameter Until convergence; the differentiable parameters The corresponding parameters at convergence The value of is taken as the first weight vector after fine-tuning , the second weight vector With learnable scaling parameters The value of .

[0022] Preferably, in the pre-training module, according to the gradient Solving the first moment vector and the second moment vector, and the deviation correction amount of the first moment vector and the second moment vector, including: the first moment vector is: , The second moment vector is: , In the formula, ; The deviation correction of the first moment vector is: . The deviation correction of the second moment vector is: , In the formula, and is the exponential moving average moment vector and In time step The moment vector estimate under .

[0023] A third aspect of the present invention relates to a terminal, comprising a processor and a storage medium; the storage medium is used to store instructions; the processor is used to operate according to the instructions to execute the steps of the method in the first aspect of the present invention.

[0024] A fourth aspect of the present invention relates to a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method in the first aspect of the present invention.

[0025] The beneficial effect of the present invention is that, compared with the prior art, the power time series prediction method and system based on the large-scale language model in the present invention is improved on the basis of the existing large-scale language model GPT-2 based on the characteristics of the time series data in the electrical field and the characteristics of the time series prediction task, and a new general-purpose time series large-scale language model is designed. The present invention is effective and reliable, the analysis results are accurate, and the parameter training is simple and easy.

[0026] The beneficial effects of the present invention also include: 1. The present invention fully extracts the multi-dimensional features in the power time series, performs pooling filtering on the time series through multiple cores of different sizes, and flexibly adjusts the weights between the various filtering results through training, ultimately achieving accurate extraction of seasonal items and trend items of the power time series, as well as future time series prediction based on seasonal items and trend items.

[0027] 2. The present invention adopts the gradient descent algorithm to train the weights of each filtering component to obtain the optimal solution, which ensures that the large-scale language model of the present invention is not only used in power time series, but also has universality for time series with different sources and different meanings. Without missing the multi-dimensional characteristics of the time series, it ensures that the prediction results are accurate and effective. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 The present invention is a flowchart of a method for predicting electric power time series based on a large-scale language model. DETAILED DESCRIPTION

[0029] In order to make the purpose, technical scheme and advantages of the present invention clearer, the technical scheme of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. The embodiments described in the present invention are only embodiments of a part of the present invention, not all embodiments. Based on the spirit of the present invention, all other embodiments not recorded in the present invention obtained by ordinary technicians in this field according to the embodiments recorded in the present invention without making creative work should belong to the protection scope of the present invention.

[0030] The present invention attempts to design a new general-purpose time series large-scale language model based on the characteristics of time series data in the electrical field and the characteristics of time series prediction tasks, based on the existing large-scale language model GPT-2, which is called ADGPT (Adaptive Decomposition Generative Pretrained Transformer) in the present invention.

[0031] In view of the characteristics of data in the power field, which has clear periodicity and changeable trends, the present invention designs a decomposition module that adds a learnable progressive decomposition module to gradually decompose the input data into seasonal components and trend components, solving the problem that the self-attention mechanism is difficult to decompose complex temporal associations, and realizing the decoupling of complex temporal associations. Subsequently, the sequence data of each component is instance regularized to reduce the impact of the time series distribution offset, and then each component is mapped, spliced ​​and tokenized to facilitate the GPT-2 model's understanding of the input data. Finally, the processed training data is input into the pre-trained GPT-2 model to freeze the attention layer and the feedforward layer for fine-tuning and inference prediction. The present invention designs a new fine-tuned large-scale language model with an added progressive decomposition module, which can be applied to real time series prediction including but not limited to the electrical field.

[0032] Figure 1 FIG. 1 is a flow chart of a method for predicting power time series based on a large-scale language model according to the present invention. Figure 1 As shown, the first aspect of the present invention relates to a method for predicting electric power time series based on a large-scale language model, and the method includes steps 1 to 3.

[0033] Step 1: Collect the power time series and use the adaptive decomposition module to decompose the power time series into seasonal item series and trend item series.

[0034] In order to improve the performance of large-scale language model prediction, the present invention first introduces a learnable time series decomposition module suitable for deep learning parallel training to help large models understand complex temporal associations that cannot be easily decoupled by the self-attention mechanism.

[0035] Various types of power data have the characteristics of time continuity during the operation of the power grid, such as the analog quantities of equipment and lines collected from the power grid, the power generation power of the power grid, the load changes of the power grid, etc., which are all power time series with time continuity. Reasonable prediction of such data can provide an important basis for the reasonable and effective operation of the power system and ensure the stability and safety of the power grid.

[0036] Preferably, an adaptive decomposition module is used to decompose the power time series into a seasonal item sequence and a trend item sequence, including: setting K average pooling filters with different kernel sizes; using each average pooling filter to perform a first average pooling operation on the power time series to obtain a first-order trend item; filling the first-order trend item and then performing a second average pooling operation to obtain a second-order trend item; performing weighted summation on the first-order trend item and the second-order trend item and inputting them into a neural network to obtain a trend item for each average pooling filter; after weighting the trend items of K average pooling filters, obtaining a trend item sequence of the power time series; subtracting the trend item sequence from the power time series to obtain a seasonal item sequence.

[0037] To achieve this goal, each decomposition module has the following operations: set K average pooling filters, the value of K is the same as the number of period types of the time series, and their kernel sizes are { }, the kernel size is the same as the length of each period of the time series. For the average pooling filters with different kernel sizes, the first average pooling operation and the second average pooling operation are performed in sequence.

[0038] Specifically, for each time series data , in filling After average pooling . This gives the current time The lower pooling kernel is The output of an average pooling filter . Continue with the second average pooling operation, then we have is the output of the second average pooling . Taking the weighted sum of the first-order trend term and the second-order trend term, we have , That is the trend term of each average pooling filter. and The weight of the two is weighted, and in the subsequent steps, the value is obtained through gradient descent algorithm training.

[0039] In addition, the trend terms of multiple average pooling filters are Weighted sum, where the weight is The K weights here are also mentioned in the description of the gradient descent algorithm later in the text. Multiply , Multiply , …, Multiply , weighted addition is used to calculate the final trend term, from the original power time series data Subtract the trend term from the seasonal term to obtain the seasonal term.

[0040] Step 2: Use instance regularization and sharding to operate on the seasonal item sequence and trend item sequence respectively to obtain seasonal data and trend data.

[0041] Preferably, instance regularization and sharding are used to respectively operate on the seasonal item sequence and the trend item sequence to obtain seasonal data and trend data, including: instance regularization is: , In the formula, and They are trend term series and seasonal term series respectively; and are expectation and variance operations respectively, and are respectively the learnable scaling parameters and offsets in the trainable affine parameter vector; is the error term.

[0042] Calculate the decomposed time series trend terms separately and seasonal items The mean and variance of the trend term and seasonal items Perform instance regularization and then learn the scaling parameters Multiply and add the offset , where the offset is to prevent the gradient vanishing problem caused by the regularized value being too small, and is generally set to 0.01. The adaptive learning rate gradient descent optimization algorithm is used to update the value of the parameter along the direction of its gradient descent.

[0043] Preferably, the sharding is the seasonal term sequence after the instance regularization , trend item sequence Segmentation is performed to obtain multiple sharded seasonal and trend data.

[0044] For each input time series trend term after regularization and seasonal items The data is split into patches, or slices. These patches can be overlapping or non-overlapping, and overlap when the sliding step is small. If the length of each patch is P, and the sliding step of each cutting window is S, the number of patches can be expressed as N=[(LP) / S]+2.

[0045] Step 3: After splicing and mapping the seasonal data and trend data, input them into the pre-trained GPT model to output the predicted value of the power time series.

[0046] The present invention uses the GPT-2 model for time series prediction. The entire model architecture and training process are as follows Figure 1 First, we set the hyperparameters, where the training batch size is 16, the early stopping method is set during training, the patience number is 3, and the maximum number of training rounds is 10.

[0047] Preferably, after splicing and mapping the seasonal data and the trend data, they are input into the pre-trained GPT model to output the predicted value of the power time series, including: pre-training the GPT model, wherein the pre-training is: the GPT model is trained with the goal of minimizing the L2 loss function, so as to fine-tune the parameters of the residual and the regularization layer through training; and the values ​​of the first weight vector, the second weight vector and the learnable scaling parameter are iteratively updated by the gradient descent algorithm.

[0048] The model includes a frozen decoder module and an output linear mapping layer in sequence. In the frozen decoder module, a multi-head attention layer, a first residual and regularization layer, a feedforward layer, and a second residual and regularization layer are included in sequence. The input of the model is simultaneously input to the multi-head attention layer and the first residual and regularization layer. The output data of the multi-head attention layer and the input data of the model are simultaneously processed by the residual and regularization layer and then synchronously output to the feedforward layer and the second residual and regularization layer. The feedforward layer provides the output to the second residual and regularization layer, and the second residual and regularization layer simultaneously operates the output of the feedforward layer and the output of the first residual and regularization layer to obtain the decoder output. After the tensor data output by the Transformer decoder is expanded, the linear mapping layer receives the output of the decoder, and the obtained time series patch is embedded through the linear mapping layer to obtain the final output information of the model. The linear mapping layer here is learnable and is used to map the intermediate latent variables output by the Transformer into the dimensions and length required for the predicted value, thereby obtaining the final prediction result of the model.

[0049] Using the above method, the position encoding of each time series patch is calculated, and in each training batch, its position encoding is concatenated with the embedded representation and input into the multi-layer frozen Transformer block to perform forward propagation calculation and obtain the predicted output of the model.

[0050] During the model training process, the parameters and weights of the attention layer and feedforward layer in the GPT-2 model are frozen, and only the parameters of the multi-head attention layer and feedforward layer are fine-tuned. During the fine-tuning process, the predicted output and the true future sequence value are input into the loss function to calculate the training loss. The fine-tuning training is based on the L2 loss function and the Adam adaptive learning rate optimizer. The L2 loss function is expressed as: , In the formula, Input for the GPT-2 model, is the model output during parameter fine-tuning, is the real future sequence value, is the number of the sequence value in the sequence, is the sequence length.

[0051] Preferably, the fine-tuning uses a gradient descent algorithm to iteratively update the first weight vector, the second weight vector and the learnable scaling parameter The value of includes: the first weight vector is , where and are the weights of the first-order trend term and the second-order trend term respectively; the second weight vector is , where are the weights of the trend terms of the K average pooling filters, respectively.

[0052] According to the total loss value, the model training and parameter fine-tuning are realized through the loss back propagation operation in the neural network. In the present invention, the loss back propagation formula is realized by gradient descent. Specifically, the present invention adopts the random batch gradient descent algorithm with adaptive learning rate to obtain the network weights and balance weight parameters of each part in the fine-tuning process. The value is updated along the direction of its gradient descent, and the update goal is to reduce the total L2 loss value, thereby achieving optimization. In this way, the model effectively learns the patterns and characteristics of the event sequence data in the power field, ensuring the effectiveness and accuracy of the analysis results.

[0053] Preferably, the gradient descent algorithm comprises: assuming a differentiable parameter is the noise objective function The independent variable, noise objective function Values ​​and parameters of The value of is related; Initialize the learning rate , the exponential decay rate of the moving average moment vector estimate and ,parameter , first moment vector , the second moment vector , time step ,gradient ; Assume that at time step Lower differentiable parameter for ,and If it does not converge, then at a time step of When solving the noise objective function In the differentiable parameters Gradient in direction ; According to the gradient Solve the first moment vector and the second moment vector, as well as the deviation correction of the first moment vector and the second moment vector; according to the deviation correction of the first moment vector and the second moment vector and the learning rate, the differentiable parameter In time step The value of is updated until the differentiable parameter Until convergence; the differentiable parameters The corresponding parameters at convergence The value of is taken as the first weight vector after fine-tuning , the second weight vector With learnable scaling parameters The value of .

[0054] The invention also discloses a differentiable parameter And the noise objective function Noise objective function is an arbitrary, unknown sequence of convex cost functions, for example, , as well as At each moment, in order to predict the parameters and a cost function unknown in advance When the natural characteristics of the sequence are not known in advance, it is necessary to evaluate the sum of the prediction errors of the algorithm at each moment. The prediction error here refers to the online prediction value The optimal fixed point of the sum function difference.

[0055] The sum of the prediction errors of the algorithm at each moment is: , Preferably, according to the gradient Solving the first moment vector and the second moment vector, as well as the deviation correction amount of the first moment vector and the second moment vector, includes: The first moment vector is: The second moment vector is: , In the formula, ; The deviation correction of the first moment vector is: , The deviation correction of the second moment vector is: , In the formula, and is the exponential moving average moment vector and In time step The moment vector estimate under .

[0056] The differentiable parameter The corresponding parameters at convergence The value of is taken as the first weight vector after fine-tuning , the second weight vector With the learnable scaling parameter The value of is as follows: , Repeat the above steps until the loss of the fine-tuned model on the training dataset no longer decreases or the fine-tuning training is terminated after the maximum training rounds are reached.

[0057] By adopting the above method, after the training is completed and the model file of the fine-tuned large-scale model is saved, the model can be called to predict the future values ​​of time series data in the power field of any length. Compared with the traditional method of using deep learning to predict time series in the power field, the present invention has lower training difficulty and more stable prediction performance.

[0058] The second aspect of the present invention relates to an electric power time series prediction system based on a large-scale language model using the method in the first aspect of the present invention, the system comprising a decomposition module, a calculation module and a prediction module; wherein the decomposition module is used to collect electric power time series, and decompose the electric power time series into a seasonal item sequence and a trend item sequence using an adaptive decomposition module; the calculation module is used to use instance regularization and fragmentation to respectively calculate the seasonal item sequence and the trend item sequence to obtain seasonal data and trend data; the prediction module is used to splice and map the seasonal data and the trend data, and then input them into a pre-trained GPT model to output the predicted value of the electric power time series.

[0059] Preferably, the decomposition module uses an adaptive decomposition module to decompose the power time series into a seasonal item sequence and a trend item sequence, including: setting K average pooling filters with different kernel sizes; using each average pooling filter to perform a first average pooling operation on the power time series to obtain a first-order trend item; filling the first-order trend item and then performing a second average pooling operation to obtain a second-order trend item; performing weighted summation on the first-order trend item and the second-order trend item and inputting them into the neural network to obtain a trend item for each average pooling filter; after weighting the trend items of K average pooling filters, obtaining a trend item sequence of the power time series; subtracting the trend item sequence from the power time series to obtain a seasonal item sequence.

[0060] Preferably, the operation module uses instance regularization and sharding to operate on the seasonal item sequence and the trend item sequence respectively to obtain seasonal data and trend data, including: instance regularization is: , In the formula, and They are trend term series and seasonal term series respectively; and are expectation and variance operations respectively, and are respectively the learnable scaling parameters and offsets in the trainable affine parameter vector; is the error term; Sharding into the seasonal term sequence after regularization of the instance , trend item sequence Segmentation is performed to obtain multiple sharded seasonal and trend data.

[0061] Preferably, the prediction module concatenates and maps the seasonal data and trend data, inputs them into the pre-trained GPT model, and outputs the predicted value of the power time series, including: pre-training the GPT model, the pre-training is: the GPT model is trained with the goal of minimizing the L2 loss function, so as to fine-tune the parameters of the residual and regularization layer through training; fine-tuning, using the gradient descent algorithm to iteratively update the values ​​of the first weight vector, the second weight vector and the learnable scaling parameter.

[0062] Preferably, in the pre-training module, fine-tuning is performed by iteratively updating the first weight vector, the second weight vector and the learnable scaling parameter using a gradient descent algorithm. The value of includes: the first weight vector is , where and are the weights of the first-order trend term and the second-order trend term respectively; the second weight vector is , where are the weights of the trend terms of the K average pooling filters, respectively.

[0063] Preferably, in the pre-training module, the gradient descent algorithm includes: assuming a differentiable parameter is the noise objective function The independent variable, noise objective function Values ​​and parameters of The value of is related; Initialize the learning rate , the exponential decay rate of the moving average moment vector estimate and ,parameter , first moment vector , the second moment vector , time step ,gradient ; Assume that at time step Lower differentiable parameter for ,and If it does not converge, then at a time step of When solving the noise objective function In the differentiable parameters Gradient in direction ; According to the gradient Solve the first moment vector and the second moment vector, as well as the deviation correction of the first moment vector and the second moment vector; according to the deviation correction of the first moment vector and the second moment vector and the learning rate, the differentiable parameter In time step The value of is updated until the differentiable parameter Until convergence; the differentiable parameters The corresponding parameters at convergence The value of is taken as the first weight vector after fine-tuning , the second weight vector With learnable scaling parameters The value of .

[0064] Preferably, in the pre-training module, according to the gradient Solving the first moment vector and the second moment vector, and the deviation correction amount of the first moment vector and the second moment vector, including: the first moment vector is: , The second moment vector is: , In the formula, ; The deviation correction of the first moment vector is: , The deviation correction of the second moment vector is: , In the formula, and is the exponential moving average moment vector and In time step The moment vector estimate under .

[0065] A third aspect of the present invention relates to a terminal, comprising a processor and a storage medium; the storage medium is used to store instructions; the processor is used to operate according to the instructions to execute the steps of the method according to the first aspect of the present invention.

[0066] A fourth aspect of the present invention relates to a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method in the first aspect of the present invention.

[0067] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents, and any modifications or equivalent replacements that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A method for predicting power time series based on a large-scale language model, characterized in that: The method comprises the following steps: Collecting power time series, and decomposing the power time series into a seasonal item series and a trend item series by using an adaptive decomposition module; Using instance regularization and sharding, respectively operate the seasonal item sequence and the trend item sequence to obtain seasonal data and trend data; After the seasonal data and trend data are spliced ​​and mapped, they are input into the pre-trained GPT model to output the predicted value of the power time series.

2. The method for predicting power time series based on a large-scale language model according to claim 1, characterized in that: The method of decomposing the power time series into a seasonal term sequence and a trend term sequence by using an adaptive decomposition module includes: Set K average pooling filters with different kernel sizes; Using each average pooling filter to perform a first average pooling operation on the power time series to obtain a first-order trend term; After filling the first-order trend term, a second average pooling operation is performed to obtain a second-order trend term; Performing weighted summation on the first-order trend term and the second-order trend term and inputting the resultant into a neural network to obtain a trend term for each average pooling filter; After weighting the trend items of the K average pooling filters, a trend item sequence of the power time series is obtained; The trend term series is subtracted from the power time series to obtain the season term series.

3. The method for predicting power time series based on a large-scale language model according to claim 2, characterized in that: The example regularization and sharding are used to respectively operate the seasonal item sequence and the trend item sequence to obtain seasonal data and trend data, including: The example is regularized as: ， In the formula, and They are trend term series and seasonal term series respectively; and are expectation and variance operations respectively, and are respectively the learnable scaling parameters and offsets in the trainable affine parameter vector; is the error term; The sharding is the seasonal term sequence after regularization of the instance , trend item sequence Segmentation is performed to obtain multiple sharded seasonal and trend data.

4. The method for predicting power time series based on a large-scale language model according to claim 3 is characterized by: After the seasonal data and trend data are spliced ​​and mapped, they are input into the pre-trained GPT model to output the predicted amount of the power time series, including: The GPT model is pre-trained, and the pre-training is: The GPT model is trained with the goal of minimizing the L2 loss function, so as to fine-tune the parameters of the residual and regularization layers through the training; The fine-tuning utilizes a gradient descent algorithm to iteratively update the values ​​of the first weight vector, the second weight vector, and the learnable scaling parameter.

5. The method for predicting power time series based on a large-scale language model according to claim 4, characterized in that: The fine-tuning uses a gradient descent algorithm to iteratively update the first weight vector, the second weight vector and the learnable scaling parameter The values ​​of include: The first weight vector is , where and are the weights of the first-order trend term and the second-order trend term respectively; The second weight vector is , where are the weights of the trend terms of the K average pooling filters respectively.

6. The method for predicting power time series based on a large-scale language model according to claim 5, characterized in that: The gradient descent algorithm comprises: Assuming differentiable parameters is the noise objective function The independent variable, the noise objective function Values ​​and parameters of The value of is related; Initialize learning rate , the exponential decay rate of the moving average moment vector estimate and ,parameter , first moment vector , the second moment vector , time step ,gradient ; Assume that at time step Lower differentiable parameter for ,and If it does not converge, then at a time step of When solving the noise objective function In the differentiable parameters Gradient in direction ; According to the gradient Solving the first moment vector and the second moment vector, and the deviation correction amount between the first moment vector and the second moment vector; The differentiable parameter is adjusted according to the deviation correction amount of the first moment vector and the second moment vector and the learning rate In time step The value of is updated until the differentiable parameter Until convergence; The differentiable parameter The corresponding parameters at convergence The value of is taken as the first weight vector after fine-tuning , the second weight vector With the learnable scaling parameter The value of .

7. The method for predicting power time series based on a large-scale language model according to claim 6, characterized in that: According to the gradient Solving the first moment vector and the second moment vector, as well as the deviation correction amount of the first moment vector and the second moment vector, includes: The first moment vector is: ， The second moment vector is: ， In the formula, ; The deviation correction of the first moment vector is: ， The deviation correction of the second moment vector is: ， In the formula, and is the exponential moving average moment vector and In time step The moment vector estimate under .

8. A power time series prediction system based on a large-scale language model using the method according to any one of claims 1 to 7, characterized in that: The system includes a decomposition module, a calculation module and a prediction module; wherein, The decomposition module is used to collect the power time series and decompose the power time series into a seasonal item series and a trend item series by using an adaptive decomposition module; The operation module is used to use instance regularization and sharding to respectively operate the seasonal item sequence and the trend item sequence to obtain seasonal data and trend data; The prediction module is used to splice and map the seasonal data and trend data, input them into the pre-trained GPT model, and output the predicted value of the power time series.

9. The power time series prediction system based on large-scale language model according to claim 8, characterized in that: In the decomposition module, the power time series is decomposed into a seasonal item sequence and a trend item sequence by using an adaptive decomposition module, including: Set K average pooling filters with different kernel sizes; Using each average pooling filter to perform a first average pooling operation on the power time series to obtain a first-order trend term; After filling the first-order trend term, a second average pooling operation is performed to obtain a second-order trend term; Performing weighted summation on the first-order trend term and the second-order trend term and inputting the resultant into a neural network to obtain a trend term for each average pooling filter; After weighting the trend items of the K average pooling filters, a trend item sequence of the power time series is obtained; The trend term series is subtracted from the power time series to obtain the season term series.

10. The power time series prediction system based on large-scale language model according to claim 9, characterized in that: In the operation module, instance regularization and sharding are used to operate the seasonal item sequence and the trend item sequence respectively to obtain seasonal data and trend data, including: The example is regularized as: ， In the formula, and They are trend term series and seasonal term series respectively; and are expectation and variance operations respectively, and are respectively the learnable scaling parameters and offsets in the trainable affine parameter vector; is the error term; The sharding is the seasonal term sequence after regularization of the instance , trend item sequence Segmentation is performed to obtain multiple sharded seasonal and trend data.

11. The power time series prediction system based on large-scale language model according to claim 10, characterized in that: In the prediction module, the seasonal data and trend data are concatenated and mapped, and then input into the pre-trained GPT model to output the predicted value of the power time series, including: The GPT model is pre-trained, and the pre-training is: The GPT model is trained with the goal of minimizing the L2 loss function, so as to fine-tune the parameters of the residual and regularization layers through the training; The fine-tuning utilizes a gradient descent algorithm to iteratively update the values ​​of the first weight vector, the second weight vector, and the learnable scaling parameter.

12. The power time series prediction system based on large-scale language model according to claim 11, characterized in that: In the prediction module, the fine-tuning uses a gradient descent algorithm to iteratively update the first weight vector, the second weight vector and the learnable scaling parameter The values ​​of include: The first weight vector is , where and are the weights of the first-order trend term and the second-order trend term respectively; The second weight vector is , where are the weights of the trend terms of the K average pooling filters respectively.

13. The power time series prediction system based on large-scale language model according to claim 12, characterized in that: In the prediction module, the gradient descent algorithm includes: Assuming differentiable parameters is the noise objective function The independent variable, the noise objective function Values ​​and parameters of The value of is related; Initialize learning rate , the exponential decay rate of the moving average moment vector estimate and ,parameter , first moment vector , the second moment vector , time step ,gradient ; Assume that at time step Lower differentiable parameter for ,and If it does not converge, then at a time step of When solving the noise objective function In the differentiable parameters Gradient in direction ; According to the gradient Solving the first moment vector and the second moment vector, and the deviation correction amount between the first moment vector and the second moment vector; The differentiable parameter is adjusted according to the deviation correction amount of the first moment vector and the second moment vector and the learning rate In time step The value of is updated until the differentiable parameter Until convergence; The differentiable parameter The corresponding parameters at convergence The value of is taken as the first weight vector after fine-tuning , the second weight vector With the learnable scaling parameter The value of .

14. The power time series prediction system based on large-scale language model according to claim 13, characterized in that: In the prediction module, according to the gradient Solving the first moment vector and the second moment vector, as well as the deviation correction amount of the first moment vector and the second moment vector, includes: The first moment vector is: ， The second moment vector is: ， In the formula, ; The deviation correction of the first moment vector is: , The deviation correction of the second moment vector is: , In the formula, and is the exponential moving average moment vector and In time step The moment vector estimate under .

15. A terminal comprising a processor and a storage medium; characterized in that: The storage medium is used to store instructions; The processor is configured to operate according to the instructions to execute the steps of the method according to any one of claims 1-7.

16. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images