Time sequence prediction method and system based on adaptive period segmentation and parallel decoding

Abstract

The invention discloses a time sequence prediction method and system based on adaptive period segmentation and parallel decoding, and the method comprises the following steps: judging the period length of time sequence data, segmenting the time sequence data according to the period length, and obtaining time sequence segmentation data; adaptively adjusting the hiding dimension of the embedded layer according to the period length, and mapping the time sequence segmentation data into the hiding dimension to obtain a time sequence representation; the obtained time sequence representation is input into a Transform encoder to carry out feature extraction; the decoder input is initialized according to the Transform encoder output, the initialized decoder input and the encoder output are subjected to feature interaction in the decoder based on a cross attention mechanism, and finally prediction output is obtained. According to the method, a more flexible period segmentation scheme and a decoding mechanism are adopted, the reasoning efficiency is improved, accumulative errors are reduced, and the time sequence basic model can achieve a better prediction effect with fewer parameters.

Description

Technical Field

[0001] The present invention relates to the technical field of time series prediction, and in particular to a time series prediction method and system based on adaptive cycle segmentation and parallel decoding. Background Art

[0002] Time series forecasting technology is a core enabler for the Industrial Internet of Things (IIoT) and business decision-making. Existing methods are primarily based on two paradigms: statistical learning and deep learning. Traditional statistical models (such as ARIMA and Prophet) rely on artificial feature engineering and stationarity assumptions, making it difficult to model complex nonlinear trends. Deep learning methods (such as LSTM, TCN, and Transformer) can capture dynamic patterns but require separate training for different downstream scenarios. Without sufficient downstream data, performance cannot be guaranteed.

[0003] Pre-training technology based on large-scale time series data offers a breakthrough in this dilemma. By building a universal time series representation model, this approach employs a "pre-training + transfer learning" paradigm to overcome the limitations of traditional single-scenario modeling. First, a basic time series model is trained on billions of time series data across multiple domains and scenarios to learn the essential laws governing time series dynamics. Subsequently, through transfer mechanisms such as parameter freezing and feature adaptation, the model is rapidly adapted to downstream tasks in zero- or few-shot conditions.

[0004] Although pre-training techniques based on large-scale time series data have shown significant results in zero-shot and few-shot scenarios, current time series basic models still have the following key bottlenecks:

[0005] First, existing models generally use a fixed segmentation strategy, dividing sequences from different scenes into equal-length, non-overlapping segments, which are then input into the encoder after hidden layer mapping. This fixed segmentation mechanism makes it difficult for the model to adaptively adjust the temporal segmentation based on differences in sampling granularity, limiting its ability to capture multi-scale temporal patterns. This in turn forces the model to compensate for representational deficiencies by increasing the number of parameters.

[0006] Second, existing models mainly rely on autoregressive decoding mechanisms to achieve prediction output. Its iterative prediction characteristics not only significantly reduce the inference speed, but also lead to error accumulation, seriously restricting the application value of the model in real-time scenarios.

[0007] Therefore, how to provide a method and system that can adaptively segment time series data according to different periodicities and achieve better time series prediction with a smaller parameter scale is a technical problem that needs to be solved urgently by those skilled in the art. Summary of the Invention

[0008] In response to the above research status, the present invention provides a time series prediction method and system based on adaptive period segmentation and parallel decoding. It proposes adaptive period segmentation to deal with time series data of different sampling granularities, and uses an adaptive embedding layer to map time series segments of different lengths into a unified hidden dimension. Based on the non-autoregressive decoding mechanism, it improves the inference efficiency and reduces the cumulative error, so that the time series basic model can achieve better prediction effects with fewer parameters.

[0009] The present invention provides a time series prediction method based on adaptive period segmentation and parallel decoding. The method is implemented based on a time series basic model. The time series basic model includes an embedding layer, an encoding layer, and a decoding layer. Time series data is input to the embedding layer, and the output layer outputs predicted data. The time series data is a sequence of data points arranged in chronological order. The method includes the following steps:

[0010] S1: Determine the period length of the time series data and segment the time series data according to the period length to obtain time series segmentation data;

[0011] S2: Adaptively adjust the hidden dimension of the embedding layer according to the cycle length, and map the time series segmentation data into the hidden dimension to obtain the time series representation;

[0012] S3: Input the obtained temporal representation into the Transformer encoder for feature extraction;

[0013] S4: Initialize the decoder input according to the Transformer encoder output, and perform feature interaction between the initialized decoder input and the encoder output within the decoder based on the cross-attention mechanism to finally obtain the predicted output.

[0014] Preferably, S1 includes a step of analyzing the input time series data to determine its periodicity, and the analysis method includes any one of the following:

[0015] Determining the cycle length by prior knowledge;

[0016] The time series data is Fourier transformed to obtain frequency domain data, the frequency component with the largest amplitude in the frequency domain data is found, and the period length is determined according to the frequency component.

[0017] Preferably, the step of adaptively adjusting the hidden dimension of the embedding layer according to the period length in S2 includes:

[0018] S21: Obtain non-overlapping equal-length segments X obtained by splitting the time series data according to the new period length P' P ;

[0019] S22: Adjust the original parameter matrix of the embedding layer through parameter interpolation:

[0020]

[0021] Where P* is the original defined period length, θ is the original parameter matrix, A is the linear interpolation matrix, is the scaling factor that adapts θ to the new period length P' through flex-resize.

[0022] Preferably, the step of mapping the time series segmentation data into the hidden dimension to obtain the time series representation in S2 includes:

[0023]

[0024] Where, X e is the temporal representation after mapping, θ e It is the parameter matrix of the input layer, and P' substitutes the value of P to represent the new cycle length.

[0025] Preferably, the step of initializing the decoder input according to the Transformer encoder output in S4 includes:

[0026] Take the last token output by Transformer encoder and copy it K times to get the decoder input H = {h j}, j = 1...K; K is determined according to the preset prediction length of the output layer.

[0027] Preferably, K=F / P, where P is the period length of the time series data in S1.

[0028] Preferably, in S4:

[0029]

[0030] Where E is the output of the Transformer encoder, τ is the index of the decoder input token, is the final predicted output, P* is the original defined cycle length, θ d It is the parameter matrix of the output layer, and P' substitutes the value of P to represent the new cycle length.

[0031] The present invention also provides a time series prediction system according to the time series prediction method based on adaptive period segmentation and parallel decoding, comprising:

[0032] The period segmentation module is used to determine the period length of time series data and segment the time series data according to the period length to obtain time series segmentation data;

[0033] The adaptive period mapping module is used to adaptively adjust the hidden dimension of the embedding layer according to the period length and map the time series segmentation data into the hidden dimension to obtain the time series representation;

[0034] The encoding module is used to input the obtained time series representation into the Transformer encoder for feature extraction;

[0035] The parallel decoding module is used to initialize the decoder input according to the Transformer encoder output, and perform feature interaction between the initialized decoder input and the encoder output within the decoder based on the cross-attention mechanism to finally obtain the predicted output.

[0036] Preferably, the parallel decoding module includes:

[0037] Initialization submodule is used to take the last token output by Transformer encoder and copy it K times to get the decoder input H = {h j}, j = 1...K; K is determined according to the preset prediction length of the output layer.

[0038] Preferably, an output module is further included, for mapping the representation of the prediction value obtained by the decoding module into a time sequence segment of a preset prediction length to complete the prediction.

[0039] Compared with the prior art, the present invention has the following beneficial effects:

[0040] By introducing an adaptive period segmentation mechanism, the model can automatically adjust the time segment division based on the actual sampling granularity and inherent periodicity of the input time series data. This allows the model to effectively capture complex patterns in data with different sampling frequencies without relying on a fixed, pre-set time segmentation strategy.

[0041] Compared to traditional fixed segmentation methods, adaptive period segmentation combined with periodic parallel decoding can achieve better prediction results with fewer parameters. This means the model is more lightweight and can more efficiently utilize computing resources.

[0042] Using a non-autoregressive decoding mechanism, the model generates the entire prediction length in one go, rather than iteratively. This approach not only significantly improves inference speed but also effectively reduces the error accumulation caused by the iterative process, enhancing the model's practicality and accuracy in real-time applications.

[0043] It can maintain consistent performance on data sets of different sampling granularities, solving the problem in traditional methods that it is difficult to uniformly model time series data of multiple scales due to fixed segmentation strategies.

[0044] In summary, the present invention not only overcomes the limitations of the existing technology through the innovative adaptive cycle segmentation and cycle parallel decoding mechanism, but also improves the overall performance of the time series prediction model in multiple dimensions, providing strong support for applications in the fields of industrial Internet of Things, business decision-making, etc. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] To more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only embodiments of the present invention. Those skilled in the art can also derive other drawings based on the provided drawings without inventive effort.

[0046] Figure 1 A flow chart of a timing prediction method based on adaptive cycle segmentation and parallel decoding provided by an embodiment of the present invention;

[0047] Figure 2 An architectural diagram of a basic timing model provided by an embodiment of the present invention;

[0048] Figure 3 A schematic diagram of the prediction effect on data sets of different sampling granularities provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0049] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0050] The first aspect of the embodiment of the present invention discloses a timing prediction method based on adaptive cycle segmentation and parallel decoding, such as Figure 1 As shown, it is implemented based on the time series basic model. The time series basic model includes an embedding layer, an encoding layer, and a decoding layer. Time series data is input to the embedding layer, and the output layer outputs the predicted data. Time series data is a sequence of data points arranged in chronological order. It includes the following steps:

[0051] S1: Determine the period length of the time series data and segment the time series data according to the period length to obtain time series segmentation data;

[0052] S2: Adaptively adjust the hidden dimension of the embedding layer according to the cycle length, and map the time series segmentation data into the hidden dimension to obtain the time series representation;

[0053] S3: Input the obtained temporal representation into the Transformer encoder for feature extraction;

[0054] S4: Initialize the decoder input according to the Transformer encoder output, and perform feature interaction between the initialized decoder input and the encoder output within the decoder based on the cross-attention mechanism to finally obtain the predicted output.

[0055] In one embodiment, S1 analyzes the input time series data to determine its periodicity P:

[0056] P = PeriodsFinding(x).

[0057] The analysis method includes any of the following:

[0058] Determine the cycle length through prior knowledge;

[0059] The time series data is transformed into frequency domain data through Fourier transform, the frequency component with the largest amplitude in the frequency domain data is found, and the cycle length is determined based on the frequency component.

[0060] In this embodiment, the prior knowledge method is used as an example: the cycle of traffic flow data is one day. If the time series sampling granularity is 1 hour, then the cycle length is 24.

[0061] In this embodiment, a Fourier transform method is used as an example: the time series is transformed into the frequency domain through Fourier transform, the frequency component with the largest amplitude is found, and the time series period is obtained according to the frequency component.

[0062] Using the above analysis method, the time series data is segmented to obtain the time series segmentation data X P .

[0063] In one embodiment, in S2, the hidden dimension of the embedding layer is adaptively adjusted according to the period length so as to adapt to the input segment length. The step of adaptively adjusting the hidden dimension of the embedding layer according to the period length includes:

[0064] S21: Obtain non-overlapping equal-length segments X obtained by splitting the time series data according to the new period length P' P ;

[0065] S22: Adjust the original parameter matrix of the embedding layer through parameter interpolation:

[0066]

[0067] Where P* is the original defined period length, θ is the original parameter matrix, A is the linear interpolation matrix, is the scaling factor that adapts θ to the new period length P' through flex-resize.

[0068] In one embodiment, the step of mapping the time series segmented data into the hidden dimension to obtain the time series representation in S2 includes:

[0069]

[0070] Where, X e is the temporal representation after mapping, θ e It is the parameter matrix of the input layer, and P' substitutes the value of P to represent the new cycle length.

[0071] In one embodiment, the step of initializing the decoder input according to the Transformer encoder output in S4 includes:

[0072] Take the last token output by Transformer encoder and copy it K times to get the decoder input H = {h j}, j = 1...K; K is determined according to the preset prediction length of the output layer.

[0073] In one embodiment, K=F / P, where P is the period length of the time series data in S1.

[0074] In one embodiment, H is re-weighted in S4 and input into the decoder together with E to decode the temporal features to obtain the representation of the predicted value:

[0075]

[0076] Where E is the output of the Transformer encoder, τ is the index of the decoder input token, is the representation of the predicted value, P* is the original defined period length, θ d It is the parameter matrix of the output layer, and P' substitutes the value of P to represent the new cycle length.

[0077] In one embodiment, the inference step of the time series basic model is also included: the prediction loss is calculated according to the predicted value and the true value, the prediction loss is used to train the entire time series basic model, and the trained time series basic model is obtained for the prediction of time series data with a set prediction length.

[0078] A second aspect of an embodiment of the present invention further provides a time series prediction system based on a time series prediction method based on adaptive period slicing and parallel decoding according to the first aspect of the embodiment, comprising:

[0079] The period segmentation module is used to determine the period length of time series data and segment the time series data according to the period length to obtain time series segmentation data;

[0080] The adaptive period mapping module is used to adaptively adjust the hidden dimension of the embedding layer according to the period length and map the time series segmentation data into the hidden dimension to obtain the time series representation;

[0081] The encoding module is used to input the obtained time series representation into the Transformer encoder for feature extraction;

[0082] The parallel decoding module is used to initialize the decoder input according to the Transformer encoder output, and perform feature interaction between the initialized decoder input and the encoder output within the decoder based on the cross-attention mechanism to finally obtain the predicted output.

[0083] In one embodiment, the parallel decoding module includes:

[0084] Initialization submodule is used to take the last token output by Transformer encoder and copy it K times to get the decoder input H = {h j}, j = 1...K; K is determined according to the preset prediction length of the output layer.

[0085] In one embodiment, an output module is further included, which is used to map the representation of the prediction value obtained by the decoding module into a time sequence segment of a preset prediction length to complete the prediction.

[0086] The second aspect of the embodiment of the present invention is used to execute all the steps of the first aspect of the embodiment.

[0087] like Figure 1 As shown in the figure, the present invention surpasses the existing time series basic model in the zero-sample scenario, where LightGTS is our model, tiny and mini are its different parameter versions, and the others are other methods. It can be seen that our method has a significant improvement over other methods.

[0088] Table 1 Comparison of prediction results in zero-sample scenario compared with existing time series basic models

[0089]

[0090] As shown in Table 2, the present invention can surpass the existing time series basic model on some data sets in the full sample scenario in the zero sample scenario, and can surpass the existing time series basic model on all data sets in the full sample scenario.

[0091] Table 2 Comparison of prediction results of existing time series basic models in zero-sample and full-sample scenarios

[0092]

[0093] As shown in Table 3, the inference efficiency and cost of the present invention have significant advantages over other time series basic models.

[0094] Table 3 Comparison of inference efficiency compared with existing time series basic models

[0095]

[0096] like Figure 3 As shown, the present invention also has a uniform effect on data sets with different sampling granularities. Figure 3 (a) Mean square error graph not on the dataset ETT2, Figure 3 (b) is the mean square error graph on the dataset ETT1.

[0097] The above is a detailed introduction to the timing prediction method and system based on adaptive periodic segmentation and parallel decoding provided by the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for general technical personnel in this field, according to the ideas of the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

[0098] In this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus comprising the element.

Claims

1. A time series prediction method based on adaptive period segmentation and parallel decoding, characterized in that: This is implemented based on a time series basic model, which includes an embedding layer, an encoding layer, and a decoding layer. Time series data is input into the embedding layer, and the output layer outputs predicted data. The time series data is a sequence of data points arranged in chronological order. The model includes the following steps: S1: Determine the period length of the time series data and segment the time series data according to the period length to obtain time series segmentation data; S2: Adaptively adjust the hidden dimension of the embedding layer according to the cycle length, and map the time series segmentation data into the hidden dimension to obtain the time series representation; S3: Input the obtained temporal representation into the Transformer encoder for feature extraction; S4: Initialize the decoder input according to the Transformer encoder output, and perform feature interaction between the initialized decoder input and the encoder output within the decoder based on the cross-attention mechanism to finally obtain the predicted output.

2. The method for time series prediction based on adaptive period segmentation and parallel decoding according to claim 1, characterized in that: S1 includes the step of analyzing the input time series data to determine its periodicity, and the analysis method includes any one of the following: Determining the cycle length by prior knowledge; The time series data is Fourier transformed to obtain frequency domain data, the frequency component with the largest amplitude in the frequency domain data is found, and the period length is determined according to the frequency component.

3. The method for time series prediction based on adaptive period segmentation and parallel decoding according to claim 1, wherein: The step of adaptively adjusting the hidden dimension of the embedding layer according to the period length in S2 includes: S21: Obtain non-overlapping equal-length segments X obtained by splitting the time series data according to the new period length P' P ; S22: Adjust the original parameter matrix of the embedding layer through parameter interpolation: Where P* is the original defined period length, θ is the original parameter matrix, A is the linear interpolation matrix, is the scaling factor that adapts θ to the period length P' through flex-resize.

4. The method for time series prediction based on adaptive period segmentation and parallel decoding according to claim 3, characterized in that: The step of mapping the time series segmentation data into the hidden dimension to obtain the time series representation in S2 includes: Where, X e is the temporal representation after mapping, θ e It is the parameter matrix of the input layer, and P' substitutes the value of P to represent the new cycle length.

5. The method for time series prediction based on adaptive period segmentation and parallel decoding according to claim 1, wherein: The step of initializing the decoder input according to the Transformer encoder output in S4 includes: Take the last token output by Transformer encoder and copy it K times to get the decoder input H = {h j }, j = 1...K; K is determined according to the preset prediction length of the output layer.

6. The method for time series prediction based on adaptive period segmentation and parallel decoding according to claim 5, characterized in that: K=F / P, where P is the period length of the time series data described in S1.

7. The method for time series prediction based on adaptive period segmentation and parallel decoding according to claim 3, characterized in that: In said S4: Where E is the output of the Transformer encoder, τ is the index of the decoder input token, is the final predicted output, P* is the original defined cycle length, θ d It is the parameter matrix of the output layer, and P' substitutes the value of P to represent the new cycle length.

8. A time series prediction system according to any one of claims 1 to 7, characterized in that: include: The period segmentation module is used to determine the period length of time series data and segment the time series data according to the period length to obtain time series segmentation data; The adaptive period mapping module is used to adaptively adjust the hidden dimension of the embedding layer according to the period length and map the time series segmentation data into the hidden dimension to obtain the time series representation; The encoding module is used to input the obtained time series representation into the Transformer encoder for feature extraction; The parallel decoding module is used to initialize the decoder input according to the Transformer encoder output, and perform feature interaction between the initialized decoder input and the encoder output within the decoder based on the cross-attention mechanism to finally obtain the predicted output.

9. The time series prediction system according to claim 8, characterized in that The parallel decoding module includes: Initialization submodule is used to take the last token output by Transformer encoder and copy it K times to get the decoder input H = {h j }, j = 1...K; K is determined according to the preset prediction length of the output layer.

10. The time series prediction system according to claim 8, characterized in that It also includes an output module for mapping the representation of the prediction value obtained by the decoding module into a time sequence segment of a preset prediction length to complete the prediction.

Images