Multivariable prediction method and device based on time sequence large model and electronic equipment

Abstract

The invention relates to the technical field of time sequence large models, and provides a multivariable prediction method and device based on a time sequence large model and electronic equipment, and the method comprises the steps: carrying out the mixed sampling of multivariable data, and obtaining univariate data; performing slice division on the univariate data according to a preset fixed time point to obtain a plurality of non-overlapping continuous fragments, and obtaining target data; the target data is used for training a time sequence large model, so that the model can learn related information among different variables through an attention mechanism; and inputting to-be-predicted data into the time sequence large model, gradually generating a future time sequence by the model through an autoregression mode, taking the prediction of each step as the input of the next step, and outputting a prediction result until the prediction of the whole time period is completed. According to the method, multiple variables are efficiently mixed into single variables in time sequence prediction, time sequence prediction of the multiple variables is achieved, and meanwhile the capacity of the model in processing complex time sequence data is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of large time series models, and in particular to a multivariate prediction method, device and electronic equipment based on large time series models. Background Art

[0002] The large language model uses large-scale self-supervised learning, pre-trains with a large amount of unlabeled data, and uses the Transformer structure and the core attention mechanism, which enables the model to effectively capture contextual information, process multimodal data, and perform well in generation tasks. Inspired by the large language model, more and more researchers have begun to explore large-scale pre-trained time series models, which improve the generalization ability and performance of the model by pre-training on large-scale datasets.

[0003] However, in the field of time series data processing, especially in scenarios involving multi-variable inputs, existing large models mostly rely on single-variable inputs, which limits the model's ability to process complex time series data. Summary of the invention

[0004] The present invention provides a multivariate prediction method, device and electronic device based on a large time series model, which not only realizes the time series prediction of multiple variables, but also improves the ability of the model in processing complex time series data.

[0005] The present invention provides a multivariate prediction method based on a time series large model, comprising: performing mixed sampling on multivariate data to obtain univariate data; slicing and dividing the univariate data according to preset fixed time points to obtain multiple non-overlapping continuous segments; vectorizing each segment and adding position coding, length coding and frequency coding information to obtain target data; the target data is used to train the time series large model so that the time series large model can learn the relevant information between different variables through an attention mechanism; the data to be predicted is input into the time series large model, and the time series large model gradually generates the future time series through an autoregressive method, and the prediction of each step is used as the input of the next step until the prediction of the entire time period is completed, and the prediction result is output.

[0006] According to a multivariate prediction method based on a time series large model provided by the present invention, multivariate data is mixed sampled to obtain univariate data, including: normalizing the time series data to obtain multivariate data; and mixing the multivariate data to obtain univariate data.

[0007] According to a multivariate prediction method based on a large time series model provided by the present invention, multivariate data is mixed and sampled to obtain single variable data, including: taking a time point of each variable in the multivariate data in turn, mixing the multivariate into a single variable; and merging all the single variables to obtain the single variable data.

[0008] According to a multivariate prediction method based on a large time series model provided by the present invention, each fragment is vectorized, including: converting each fragment into a vector of a fixed size for input into the large time series model; wherein each fragment outputs a vector through a multilayer perceptron, and the vector dimension matches the input dimension specified by the large time series model.

[0009] According to a multivariate prediction method based on a large time series model provided by the present invention, the large time series model includes a transformer network; position coding is used to provide the large time series model with information about the position of each time point in the sequence; length coding is used to enable the large time series model to adapt to different context lengths by randomly masking part or all of the input fragments; frequency coding is used to indicate high-frequency data, medium-frequency data and low-frequency data; high-frequency data represents data with a frequency of minutes, hours, and days; medium-frequency data represents data of weeks and months, and low-frequency data represents data of quarters and years.

[0010] According to a multivariate prediction method based on a large time series model provided by the present invention, single variable data is sliced ​​and divided according to preset fixed time points to obtain multiple non-overlapping continuous segments, including: dividing the single variable data into multiple non-overlapping initial continuous segments, determining the number of time points in each segment, and obtaining first intermediate data; traversing the first intermediate data, each time taking out continuous preset fixed time points as a segment, thereby obtaining multiple non-overlapping continuous segments.

[0011] The present invention also provides a multivariate prediction device based on a large time series model, comprising: a mixed sampling module, used for performing mixed sampling on multivariate data to obtain single variable data; a slicing division module, used for slicing and dividing the single variable data according to preset fixed time points to obtain multiple non-overlapping continuous segments; a vectorization module, used for vectorizing each segment and adding position coding, length coding and frequency coding information to obtain target data; the target data is used to train the large time series model so that the large time series model can learn the relevant information between different variables through an attention mechanism; a prediction module, used to input the data to be predicted into the large time series model, and the large time series model gradually generates the future time series through an autoregressive method, and the prediction of each step is used as the input of the next step until the prediction of the entire time period is completed, and the prediction result is output.

[0012] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, any of the multivariate prediction methods based on a large time series model described above is implemented.

[0013] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the multivariate prediction methods based on a large time series model as described above.

[0014] The present invention also provides a computer program product, including a computer program, which, when executed by a processor, implements any of the multivariate prediction methods based on a large time series model as described above.

[0015] The multivariate prediction method, device and electronic device based on the time series large model provided by the present invention include: mixing and sampling multivariate data to obtain single variable data; slicing and dividing the single variable data according to preset fixed time points to obtain multiple non-overlapping continuous segments; vectorizing each segment and adding position coding, length coding and frequency coding information to obtain target data; the target data is used to train the time series large model so that the time series large model can learn the relevant information between different variables through the attention mechanism; the data to be predicted is input into the time series large model, and the time series large model gradually generates the future time series by autoregression, and the prediction of each step is used as the input of the next step until the prediction of the entire time period is completed, and the prediction result is output. Through the above method, the present invention proposes a method for efficiently mixing multiple variables into single variables in time series prediction, realizes the time series prediction of multiple variables, and also improves the ability of the model in processing complex time series data. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0017] Figure 1 It is one of the flow charts of the multivariate prediction method based on the time series large model provided in the embodiment of the present invention.

[0018] Figure 2 This is the second flow chart of the multivariate prediction method based on the time series large model provided in the embodiment of the present invention.

[0019] Figure 3 It is a structural schematic diagram of a multivariate prediction device based on a large time series model provided in an embodiment of the present invention.

[0020] Figure 4 It is a schematic diagram of the physical structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0021] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0022] In the description of the embodiments of the present invention, it should be noted that the terms “first”, “second” and “third” are only used for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0023] In the description of the embodiments of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "connected" and "connection" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium. For ordinary technicians in this field, the specific meanings of the above terms in the embodiments of the present invention can be understood according to specific circumstances.

[0024] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the embodiment of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0025] In the related technologies, in the field of time series data processing, especially in scenarios involving multivariate input, some large language models rely on single variable input, which limits the model's ability to process complex time series data. In order to overcome this limitation, researchers began to explore methods for processing multivariate data, but most methods focus on feature fusion or parallel processing.

[0026] Based on this, the present invention provides a multivariate prediction method based on a large time series model, and proposes a method for efficiently mixing multivariate data into a single variable based on the large time series model. The mixed data is used to train or fine-tune the large time series model, so that the model can effectively capture and learn the relevant information between different variables through the attention mechanism, thereby realizing time series prediction of multiple variables.

[0027] See also Figure 1 , Figure 1 It is one of the flow charts of the multivariate prediction method based on the time series large model provided by an embodiment of the present invention. In this embodiment, the multivariate prediction method based on the time series large model may include steps S110 to S140, and each step is specifically as follows: S110: Mix and sample the multivariate data to obtain univariate data.

[0028] S120: Slice and divide the univariate data according to preset fixed time points to obtain a plurality of non-overlapping continuous segments.

[0029] S130: vectorize each segment and add position coding, length coding and frequency coding information to obtain target data.

[0030] S140: The data to be predicted is input into the time series large model. The time series large model gradually generates the future time series through autoregression. The prediction of each step is used as the input of the next step. After the prediction of the entire time period is completed, the prediction result is output.

[0031] Among them, the target data is used to train the time series large model so that the time series large model can learn the relevant information between different variables through the attention mechanism.

[0032] In this embodiment, multivariate data can be obtained from the time series data to be predicted. The multivariate data can be integrated into univariate data according to the time point, and then sliced ​​and divided according to the preset fixed time point to obtain multiple non-overlapping continuous fragments (patches). Vectorization is performed according to the fragments, and position coding, length coding and frequency coding information are added to obtain target data that can be applied to the time series large model for training. It should be noted that the newly added position coding, length coding and frequency coding are intended to better fit the training habits of the time series large model. After training the target data, the time series large model can learn the relevant information between different variables through the attention mechanism, and realize the time series prediction of multiple variables; at the same time, it also improves the ability of the model in processing complex time series data.

[0033] In the prediction phase, the input is put into the neural network for model training and prediction. The model is trained through the previous time segments, and the goal is to predict the output at the next time point.

[0034] Alternatively, a standard Transformer decoder structure can be used.

[0035] Specifically, the time series big model can gradually generate future time series through autoregression. The autoregression method uses past historical data to predict future data points, and each step of the prediction will be used as the input of the next step until the prediction of the entire time period is completed. Optionally, the time series big model can adopt an encoder-only architecture, a decoder-only architecture, or an encoder-decoder architecture.

[0036] Optionally, the time series model includes a transformer network. The transformer network is a deep learning model architecture that introduces a self-attention mechanism, which enables the model to capture long-range dependencies when processing sequence data.

[0037] Specifically, the transformer network consists of two main parts: the encoder and the decoder. Each part is composed of multiple identical layers stacked together.

[0038] The encoder is responsible for converting the input sequence into an intermediate representation. Each layer of the encoder consists of two main parts: a self-attention mechanism and a feedforward neural network. The self-attention mechanism helps capture the dependencies between different positions in the input sequence.

[0039] The decoder is responsible for generating the output sequence. In addition to the self-attention mechanism and feed-forward neural network, each decoder layer also includes an additional attention mechanism to focus on the output of the encoder. The self-attention mechanism of the decoder is similar to that of the encoder, but with the addition of a mask to ensure that the generation can only depend on the generated parts.

[0040] When a large temporal model with a Decoder-Only architecture is chosen, only the decoder part of the Transformer is used. In this architecture, there is only the decoder part, which is composed of multiple decoder layers stacked together. Each decoder layer includes a self-attention mechanism, an interactive attention mechanism with the encoder output (but this part is empty because there is no encoder part), and a feedforward neural network. The decoder's self-attention mechanism usually uses a mask to ensure that each generated position can only depend on the current and previous positions (for sequence generation tasks). It is suitable for tasks that require the generation of coherent text, such as language model generation, automatic text generation, etc. When the model needs to generate text based on certain conditions, only the decoder part is used to generate new text.

[0041] In some embodiments, the core module of the time series large model is Masked Multi-Head Self-Attention. In the Decoder-Only model, the attention mechanism is used to capture the dependencies between different time steps. Because it is an autoregressive task, the output of each time step can only depend on the current and previous time steps. Through the Masking method, the model can avoid "seeing" future information.

[0042] In some embodiments, the core module of the time series large model may include Feed-Forward Networks (FFN). Each decoder layer is followed by a feed-forward neural network to further process the features obtained from the attention mechanism. The loss function used is the mean square error (MSE).

[0043] It should be noted that the self-attention mechanism allows the model to capture long-range dependencies by paying attention to all other positions in the sequence while processing each position in the sequence. It generates a representation of each position by calculating the weighted sum of each position in the input sequence.

[0044] Position encoding is used to provide the time series model with information about the position of each time point in the sequence; length encoding is used to enable the time series model to adapt to different context lengths by randomly masking part or all of the input fragments; frequency encoding is used to indicate high-frequency data, medium-frequency data, and low-frequency data; high-frequency data represents data with a frequency of minutes, hours, and days; medium-frequency data represents data of weeks and months, and low-frequency data represents data of quarters and years.

[0045] Since the transformer network does not have a convolutional or cyclic structure and cannot directly process the position information in the sequence, this embodiment introduces position encoding to represent the information of each position in the sequence. The position encoding will be added to the embedded representation of the input to preserve the position information.

[0046] In some embodiments, the step of performing mixed sampling on multivariate data to obtain univariate data may further include: Normalize the time series data to obtain multivariate data; mix and sample the multivariate data to obtain univariate data.

[0047] In some embodiments, the step of performing mixed sampling on multivariate data to obtain univariate data may further include: Take one time point of each variable in the multivariate data in turn, mix the multivariate data into a single variable; merge all the single variables to obtain the single variable data.

[0048] In some embodiments, the step of vectorizing each fragment may further include: Each fragment is converted into a vector of a fixed size for input into the large temporal model; wherein each fragment outputs a vector through a multilayer perceptron, and the vector dimension matches the input dimension specified by the large temporal model.

[0049] Multilayer Perceptron (MLP) is a basic feedforward neural network consisting of multiple layers of neurons (or nodes).

[0050] Optionally, the MLP may include an input layer, a hidden layer, and an output layer.

[0051] The input layer is used to receive input data. Each node represents an input feature.

[0052] There are one or more hidden layers, and each hidden layer consists of multiple neurons. These neurons perform nonlinear transformations on the input through activation functions. The number of hidden layers and the number of neurons in each layer are hyperparameters of the model.

[0053] The output layer is used to generate the final prediction results. The structure of the output layer depends on the specific task. For example, classification tasks usually use the softmax activation function, and regression tasks usually use the linear activation function.

[0054] An example of the MLP workflow is as follows: Forward propagation: The input data is passed from the input layer to the hidden layer, then nonlinearly transformed by the activation function, and finally passed through the output layer to obtain the prediction result. Each neuron in each layer is connected to all neurons in the previous layer, and the input of each neuron is calculated by weighted sum.

[0055] Activation function: used to introduce nonlinearity so that the network can learn and represent complex functions. Common activation functions include ReLU (rectified linear unit), Sigmoid and Tanh.

[0056] Backpropagation: By calculating the gradient of the loss function and updating the network weights using an optimization algorithm such as gradient descent, backpropagation is used to train the model by gradually adjusting the network weights so that the predicted results are as close to the actual labels as possible.

[0057] In some embodiments, the step of slicing and dividing the univariate data according to preset fixed time points to obtain a plurality of non-overlapping continuous segments may further include: The univariate data is divided into a plurality of non-overlapping initial continuous segments, the number of time points in each segment is determined, and the first intermediate data is obtained; the first intermediate data is traversed, and continuous preset fixed time points are taken as a segment each time, so as to obtain a plurality of non-overlapping continuous segments.

[0058] See also Figure 2 , Figure 2 This is the second flow chart of the multivariate prediction method based on the time series large model provided in the embodiment of the present invention.

[0059] Based on the large time series model of the Decoder-Only architecture, this embodiment proposes a method for efficiently mixing multiple variables into a single variable in time series prediction. First, the time series data is normalized, the processed multivariate data is uniformly sampled and mixed into single variable data, the data is divided into patches according to fixed time points, and then each patch is vectorized, adding position coding, length coding, frequency coding and other information, and the vector is sent to the subsequent transformer network, and the model performs learning and prediction. The model effectively captures and learns the relevant information between different variables through the attention mechanism, and realizes the time series prediction of multiple variables.

[0060] The specific descriptions of each process are as follows: ① Data normalization: Time series data is normalized and scaled to a range with a mean of 0 and a variance of 1, so that the data distribution is close to a normal distribution.

[0061] ②Multivariate data mixing: Take one time point of each variable in turn and mix the multiple variables into a single variable.

[0062] ③Data patch division: Divide the one-dimensional time series in the previous step into multiple non-overlapping continuous segments (patches), determine the number of time points in each patch, traverse the entire time series, and take out a fixed number of continuous time points each time as a patch. For time series whose length is not a multiple of the fixed number of time points, add zero values ​​at the end to fill it to a multiple of the nearest fixed number of time points, or ignore the insufficient part during input.

[0063] ④Time series vectorization: Convert each patch into a fixed-size vector for input into the Transformer layer. Each patch outputs a vector through a multi-layer perceptron (MLP), whose dimension matches the input dimension (dim) specified by the model.

[0064] Time series vectorization can be seen as vectorization implemented by the Embedding layer, which is a layer in the deep learning model used to convert discrete categorical data into continuous dense vectors. The purpose of this conversion is to map the semantic information of the categorical data into a low-dimensional real number space, so that the model can process and learn this data more efficiently.

[0065] Position encoding, since the Transformer model itself does not have the ability to process sequence order, it is necessary to add position encoding to the vector of each patch. Position encoding can be a combination of sine and cosine functions to provide the model with information about the position of each time point in the sequence.

[0066] Length encoding enables the model to adapt to different context lengths by randomly masking part or all of the input patches. Specifically, in each training batch, a starting point r is randomly selected, and the value of r is less than the number of patches, which means that the part from this point to the end of the patch is masked. This masking strategy ensures that the model can see all possible context lengths during training. Create a binary mask vector with the same length as the input time series, where 1 indicates that the corresponding time point should be considered by the model, and 0 indicates that the corresponding time point should be ignored.

[0067] Frequency encoding, also known as resolution encoding, is set to 0, 1, and 2 to represent high-frequency data, medium-frequency data, and low-frequency data, respectively. High-frequency data represents data with a frequency of minutes, hours, and days, medium-frequency data represents data with a frequency of weeks and months, and low-frequency data represents data with a frequency of quarters and years. The resolution sequence of the patch length is randomly initialized, and the resolution encoding vector of dim dimension is output after passing through a multi-layer perceptron (MLP).

[0068] ⑤Transformer network training or fine-tuning: Input into the transformer network for model training and prediction.

[0069] In the training phase, the model needs to be trained from scratch, and the model learns how to extract features from the data and make predictions. The training is completed by repeating the process of forward propagation, loss calculation, back propagation, and parameter update until the model converges or reaches the predetermined training round.

[0070] Fine-tuning is a post-training phase in which the pre-trained model needs to be adjusted and optimized for a specific task or dataset. This is done by further training the model using a task-specific dataset; adjusting the learning rate and other hyperparameters to ensure that the model can adapt to the new task; evaluating the model's performance on the validation set to ensure that the fine-tuning process does not lead to overfitting, and making necessary adjustments to the model.

[0071] In the prediction phase, the trained model can be used to predict new data and generate the final output.

[0072] In summary, based on the time series big model, this embodiment uniformly samples the multivariate data and mixes them into univariate data, divides the data into patches according to fixed time points, then vectorizes each patch, and adds information such as position coding, length coding, and frequency coding. The data is input into the time series big model for training or fine-tuning, so that the model can effectively capture and learn the relevant information between different variables through the attention mechanism, and in the prediction stage, the model gradually generates the future time series through autoregression. The prediction of each step will be used as the input of the next step until the prediction of the entire time period is completed, and finally the time series prediction of multiple variables is realized.

[0073] The present invention also provides a multivariate prediction device based on a large time series model. The multivariate prediction device based on a large time series model provided by the present invention is described below. The multivariate prediction device based on a large time series model described below and the multivariate prediction method based on a large time series model described above can be referenced to each other.

[0074] See also Figure 3 , Figure 3 3 is a schematic diagram of the structure of a multivariate prediction device based on a time series large model provided by an embodiment of the present invention. In this embodiment, the multivariate prediction device based on a time series large model may include a mixed sampling module 310, a slice division module 320, a vectorization module 330 and a prediction module 340.

[0075] The mixed sampling module 310 is used to perform mixed sampling on multivariate data to obtain univariate data.

[0076] The slicing module 320 is used to slice and divide the univariate data according to preset fixed time points to obtain multiple non-overlapping continuous segments.

[0077] The vectorization module 330 is used to vectorize each segment and add position coding, length coding and frequency coding information to obtain target data.

[0078] Among them, the target data is used to train the time series large model so that the time series large model can learn the relevant information between different variables through the attention mechanism.

[0079] The prediction module 340 is used to input the data to be predicted into the time series large model. The time series large model gradually generates the future time series through autoregression. The prediction of each step is used as the input of the next step until the prediction of the entire time period is completed and the prediction result is output.

[0080] In some embodiments, the mixed sampling module 310 can be specifically used to: normalize the time series data to obtain multivariate data; and perform mixed sampling on the multivariate data to obtain univariate data.

[0081] In some embodiments, the mixed sampling module 310 can be specifically used to: sequentially take a time point of each variable in the multivariate data, mix the multivariate data into a single variable; merge all the single variables to obtain the single variable data.

[0082] In some embodiments, the vectorization module 330 can be specifically used to: convert each fragment into a vector of a fixed size for input into a large time series model; wherein each fragment outputs a vector through a multilayer perceptron, and the vector dimension matches the input dimension specified by the large time series model.

[0083] In some embodiments, the time series big model includes a transformer network; position encoding is used to provide the time series big model with information about the position of each time point in the sequence; length encoding is used to enable the time series big model to adapt to different context lengths by randomly masking part or all of the input fragments; frequency encoding is used to indicate high-frequency data, medium-frequency data and low-frequency data; high-frequency data represents data with a frequency of minutes, hours, and days; medium-frequency data represents data of weeks and months, and low-frequency data represents data of quarters and years.

[0084] In some embodiments, the slice division module 320 can be specifically used to: divide the univariate data into multiple non-overlapping initial continuous segments, determine the number of time points in each segment, and obtain first intermediate data; traverse the first intermediate data, and each time take out continuous preset fixed time points as a segment, thereby obtaining multiple non-overlapping continuous segments.

[0085] In summary, the embodiment of the present invention provides a multivariate prediction device based on a large time series model, including: a mixed sampling module, which is used to perform mixed sampling on multivariate data to obtain single variable data; a slicing division module, which is used to slice and divide the single variable data according to a preset fixed time point to obtain multiple non-overlapping continuous segments; a vectorization module, which is used to vectorize each segment and add position coding, length coding and frequency coding information to obtain target data; the target data is used to train the large time series model so that the large time series model can learn the relevant information between different variables through the attention mechanism; a prediction module, which is used to input the data to be predicted into the large time series model, and the large time series model gradually generates the future time series by autoregression. The prediction of each step is used as the input of the next step until the prediction of the entire time period is completed, and the prediction result is output. In the above manner, the present invention can efficiently mix multiple variables into single variables in time series prediction, realize the prediction of multivariate time series, and also improve the ability of the model in processing complex time series data.

[0086] On the other hand, an embodiment of the present invention further provides an electronic device, see Figure 4 , Figure 4is a schematic diagram of the physical structure of an electronic device provided by an embodiment of the present invention, such as Figure 4 As shown, the electronic device may include: the electronic device may include a memory 420, a processor 410, and a computer program stored in the memory 420 and executable on the processor 410. When the processor 410 executes the program, the multivariate prediction method based on the time series large model provided by the above methods is implemented.

[0087] Optionally, the electronic device may further include a communication bus 430 and a communication interface (CommunicationsInterface) 440, wherein the processor 410, the communication interface 440, and the memory 420 communicate with each other through the communication bus 430. The processor 410 may call a computer program in the memory 420 to execute a multivariate prediction method based on a time series large model, and the method may include: The multivariate data is mixed and sampled to obtain the univariate data; the univariate data is sliced ​​and divided according to the preset fixed time points to obtain multiple non-overlapping continuous segments; each segment is vectorized, and the position coding, length coding and frequency coding information are added to obtain the target data; the target data is used to train the time series large model so that the time series large model can learn the relevant information between different variables through the attention mechanism; the data to be predicted is input into the time series large model, and the time series large model gradually generates the future time series through autoregression. The prediction of each step is used as the input of the next step until the prediction of the entire time period is completed, and the prediction result is output.

[0088] In addition, the logic instructions in the above-mentioned memory 420 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0089] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the multivariate prediction method based on the large time series model provided by the above methods. Its steps and principles have been introduced in detail in the above methods and will not be repeated here.

[0090] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, it is implemented to execute the multivariate prediction method based on the time series large model provided by the above-mentioned methods. The steps and principles thereof have been introduced in detail in the above-mentioned methods and will not be repeated here.

[0091] Non-transitory computer-readable storage media can be any available media or data storage devices that can be accessed by the processor, including but not limited to magnetic storage (such as floppy disks, hard disks, magnetic tapes, magneto-optical disks (MO)), optical storage (such as CDs, DVDs, BDs, HVDs, etc.), and semiconductor storage (such as ROM, EPROM, EEPROM, non-volatile memory (NANDFLASH), solid-state drives (SSDs)), etc.

[0092] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0093] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0094] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A multivariate prediction method based on a time series large model, characterized in that: include: Mix and sample the multivariate data to obtain univariate data; Slicing the univariate data according to preset fixed time points to obtain multiple non-overlapping continuous segments; Each fragment is vectorized and information of position coding, length coding and frequency coding is added to obtain target data; the target data is used to train the time series large model so that the time series large model can learn the relevant information between different variables through the attention mechanism; The data to be predicted is input into the time series large model, and the time series large model gradually generates the future time series through autoregression. The prediction of each step is used as the input of the next step until the prediction of the entire time period is completed and the prediction result is output.

2. The multivariate prediction method based on a time series large model according to claim 1 is characterized in that: The method of performing mixed sampling on the multivariate data to obtain the univariate data includes: Normalizing the time series data to obtain the multivariate data; The multivariate data is mixed and sampled to obtain the univariate data.

3. The multivariate prediction method based on a time series large model according to claim 1 is characterized in that: The step of performing mixed sampling on the multivariate data to obtain the univariate data includes: Taking one time point of each variable in the multivariate data in turn, and mixing the multivariate data into a single variable; All univariate data were combined to obtain the univariate data.

4. The multivariate prediction method based on a time series large model according to claim 1 is characterized in that: The vectorization process of each fragment includes: Convert each fragment into a fixed-size vector for input into the temporal macromodel; Each segment outputs a vector through a multi-layer perceptron, and the vector dimension matches the input dimension specified by the large temporal model.

5. The multivariate prediction method based on a time series large model according to claim 4 is characterized in that: The time series large model includes a transformer network; The position encoding is used to provide the time series large model with information about the position of each time point in the sequence; The length encoding is used to enable the temporal large model to adapt to different context lengths by randomly masking part or all of the input segment; The frequency code is used to indicate high-frequency data, medium-frequency data and low-frequency data; the high-frequency data represents data with a frequency of minutes, hours and days; the medium-frequency data represents data of weeks and months, and the low-frequency data represents data of quarters and years.

6. The multivariate prediction method based on a time series large model according to claim 1, characterized in that: The slicing and dividing the univariate data according to preset fixed time points to obtain a plurality of non-overlapping continuous segments includes: Segmenting the univariate data into a plurality of non-overlapping initial continuous segments, determining the number of time points in each segment, and obtaining first intermediate data; The first intermediate data is traversed, and consecutive preset fixed time points are taken out each time as a segment, thereby obtaining a plurality of non-overlapping consecutive segments.

7. A multivariate prediction device based on a time series large model, characterized in that: include: A mixed sampling module is used to perform mixed sampling on multivariate data to obtain univariate data; A slicing module, used for slicing the univariate data according to preset fixed time points to obtain a plurality of non-overlapping continuous segments; A vectorization module is used to vectorize each segment and add position coding, length coding and frequency coding information to obtain target data; the target data is used to train the time series large model so that the time series large model can learn the relevant information between different variables through the attention mechanism; The prediction module is used to input the data to be predicted into the time series large model. The time series large model gradually generates the future time series through autoregression. The prediction of each step is used as the input of the next step until the prediction of the entire time period is completed and the prediction result is output.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the multivariate prediction method based on the time series large model as described in any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the multivariate prediction method based on a time series large model as claimed in any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the multivariate prediction method based on a time series large model as claimed in any one of claims 1 to 6 is implemented.

Images