Time series data processing method

Abstract

The invention relates to the technical field of artificial intelligence, in particular to a time series data processing method, an autocorrelation coefficient matrix is a matrix with a larger scale, the scales of the matrix are different along with different values of tau, for example, when tau is equal to 1, a sub-sequence Y1 is equal to x1, x2,..., xL, a sub-sequence Y2 is equal to x2, x3,..., xL + 1, and from the perspective of an original sequence X, the sub-sequence Y1 is equal to x1, x2,..., xL + 1; the difference between the positions of only initial elements of the two subsequences Y1 and Y2 is 1, and so on, when tau is taken as other values, the difference between the initial positions of the subsequences relative to the original sequence X is tau, autocorrelation coefficients among all the subsequences form an autocorrelation coefficient matrix, the autocorrelation coefficients are calculated by taking the autocorrelation coefficients as the core, and the difference between the initial positions of all the subsequences is 1; according to the method, future time sequence data can be predicted according to historical time sequence data, prediction can be performed by integrating multiple time spans, meanwhile, the relationship between the time sequence and historical small sequences of multiple time spans is considered, the minimum unit of a problem is considered to be a small sequence, and the historical small sequences of large time spans can be processed.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, specifically to a method for processing time series data. Background Technology

[0002] Existing methods for time series forecasting mainly include ARMA models, RNN neural networks, and LSTM neural networks. However, they lack small sequences, making it impossible to extract relationships between them (i.e., small sequence attention). Furthermore, the time span for historical data cannot be too large; data with medium to long time spans has little impact on future time series forecasts, while data with short time spans has a significant impact. Therefore, this paper proposes a time series data processing method to address these issues. Summary of the Invention

[0003] The purpose of this invention is to provide a time series data processing method to solve the problem of not being able to extract the relationships between small series and the relationships that have a significant impact on the prediction of future time series.

[0004] To achieve the above objectives, the present invention provides the following technical solution:

[0005] A time series data processing method includes the following steps:

[0006] Step 1:

[0007] The original sequence X is divided into two subsequences, resulting in the subsequence Y, where the division is overlapped. The sequence X = x1, x2, ..., x... LX Given a historical time series, the sequence length is LX. The sequence X is then divided into subsequences Y of equal length, each of length L. i When the number of elements in sequence X is not divisible by the number of elements in sequence X, the last element x of sequence X is used as the dividing point. LX The padding is performed, where L is a variable. When the value of L is determined, the subsequence Y... i The length of each subsequence is L, and the subsequence Y is... i The sequences are concatenated in chronological order to form the sequence Y = Y1, Y2, ..., which is similar to the original sequence X = x1, x2, ..., x... LX Different, where Y1 = x1, x2, ..., x L Y2 = x2, x3, ..., x L+1 By analogy, we can conclude that Y τ =x τ x τ+1 , ..., x τ+L τ represents the step size, which takes values ​​in the range {1, LX-L+1};

[0008] Step Two:

[0009] By calculating the autocorrelation coefficients among all subsequences in the Y sequence, an autocorrelation coefficient matrix is ​​obtained. This invention provides the following estimation method for this autocorrelation coefficient matrix, which is a large matrix whose size varies with the value of τ. For example, when τ = 1, the subsequences Y1 = x1, x2, ..., x... are obtained. L Subsequence Y2 = x2, x3, ..., x L+1 From the perspective of the original sequence X, the subsequences Y1 and Y2 differ by only 1 in the position of their starting elements. Similarly, when τ takes other values, the starting positions of all subsequences relative to the original sequence X differ by τ. The autocorrelation coefficients between all subsequences form an autocorrelation coefficient matrix, with the following dimensions: number of columns... The number of rows is in This indicates rounding down, where the subsequences Y1 and Y2 in the autocorrelation coefficient matrix are used as the basis for rounding. τ Taking the autocorrelation coefficient as an example, the autocorrelation coefficient is calculated as R(τ).

[0010] Therefore,

[0011] At this point, the first row and τth column of the autocorrelation coefficient matrix is ​​the result R(τ) calculated here;

[0012] τ represents the interval at which the subsequence takes values ​​from the original sequence. Each value of τ corresponds to an autocorrelation coefficient matrix. After traversing all τ, a set of accompanying autocorrelation coefficient matrices will be generated. By concatenating the above matrices column by column according to the order of the values ​​of τ, a complete autocorrelation coefficient matrix is ​​formed.

[0013] When LX is large, the autocorrelation matrix will be extremely large. Not only does LX have an impact, but traversal and τ also contribute to the large matrix size. Based on the above, this invention provides the following estimation method for estimating the autocorrelation coefficient matrix. After the autocorrelation coefficient matrix is ​​estimated, the result is denoted as matrix M. Performing SVD decomposition on M yields M = U∑V T Where U matrix is ​​the matrix composed of left singular vectors, V matrix is ​​the matrix composed of right singular vectors, and ∑ is the diagonal matrix composed of singular values, which can also be written as diag(σ1, σ2, ..., σ). n );

[0014] At this point, we take the elements σ1, σ2, ..., σ on the diagonal. n Summing yields a scalar σ. We take σ as a multiple of λ ∈ (0,1), meaning the summation of the diagonal elements stops when it reaches λσ. At this point, the corresponding σ... kThis yields the index set {1,...,k};

[0015] Step 3:

[0016] Calculate the Q sequence, K sequence, and V sequence, and then use the subsequence Y. i The projection of Y, where i takes values ​​in the range {1,...,k}, depends on different Y values. i Calculate the corresponding Q, K, and V sequences. The Q, K, and V sequences are the basic sequences used to construct the encoder-decoder structure. Perform cross-correlation calculations on the Q and K sequences to obtain the result R. Q,K (1),R Q,K (2),...,R Q,K (k), denoted as vector R Q,K Using this vector as a weight, a weighted sum is performed on the set of sequences formed by the V sequence to obtain R. Q,K (1)*V1+R Q,K (2)*V2+...,R Q,K (k)*V k , denoted as vector RV Q,K ;

[0017] Step Four:

[0018] The resulting matrix is ​​directly used as the input to the feedforward network to calculate its output, which is a vector FRV. Q,K At this point, the original input sequence X = x1, x2, ..., x from the first step is... LX Transform into a vector FRV Q,K ;

[0019] Step 5:

[0020] The decoder calculates the output of the decoder, which converts the vector FRV based on the Q sequence, K sequence, and V sequence. Q,K Transform into vector The vector at this point is the input sequence X = x1, x2, ..., x from the first step. LX The generated new sequence.

[0021] Preferably, assuming that the autocorrelation coefficients of each pair of subsequences are calculated based on a finite number of subsequences and written in matrix form, denoted as D, for example, the elements in the first row and first column of matrix D are: the autocorrelation coefficients of subsequence Y1 and subsequence Y2; the elements in the first row and second column of matrix D are: the autocorrelation coefficients of subsequence Y1 and subsequence Y2 when τ = 1.

[0022] Next, we will calculate matrix M using matrix D. The specific calculation steps are as follows:

[0023] According to data D m×nThe matrices are calculated to obtain the W and Z matrices. The calculation process is as follows:

[0024] (1) Perform orthogonal decomposition on matrix D: Where U∈R m×m , Q∈R n×n All are orthogonal matrices, R∈R r×r W is an invertible upper triangular matrix, O is the zero matrix, and the initial value of W is W0 = R;

[0025] (2) Take the maximum number of loops as k = k max ;

[0026] (3) Calculation

[0027] (4) Let

[0028] (5) Order

[0029] (6) If ||W k+1 -W k || F ≤ε||D|| F Stop the iteration, and at this point let W = W k+1 , If ||W k+1 -W k || F ≤ε||D|| F Then continue iterating from step (3) to step (6) until the condition is met or k = k max ;

[0030] Calculate the projection matrix P of Z, calculate the matrix M. Each matrix has been defined or calculated in steps 1 and 2. Compared with the prior art, the beneficial effects of this invention are:

[0031] In this invention, it is possible to predict future time series data based on historical time series data, and to make predictions across multiple time spans, including short, medium, and long time spans. By dividing the time series into smaller series, a relationship is established between the latest smaller series and multiple historical smaller series with different time spans. This relationship is called the attention of the smaller series, which is then used to predict the time series at future moments. In other words, when generating the time series at future moments, the relationship between the time series and multiple historical smaller series with different time spans is considered simultaneously. The smallest unit of consideration is the smaller series, which can handle historical smaller series with large time spans. Attached Figure Description

[0032] Figure 1 This is a schematic diagram of the relationship between time and amplitude in this invention. Detailed Implementation

[0033] Please see Figure 1 The present invention provides a technical solution:

[0034] A time series data processing method,

[0035] like Figure 1 As shown, the horizontal axis represents time, and the vertical axis represents amplitude. The small sequence within the red box on the right is called the current latest small sequence. This invention establishes the relationship between the current latest small sequence and historical small sequences. For example, the small sequence within the first pink box before the time period τ1, τ... k The small sequence within the k-th pink box before the time period, τ K-1 The small sequence within the (K-1)th pink box before the time period, τ K The small sequences within the Kth pink box preceding a time period are considered by establishing relationships with these historical small sequences—that is, attention to these small sequences. This allows us to simultaneously consider historical small sequences across multiple time spans when predicting time series at future moments. For example, in this case, there are K time spans τ1, τ2, τ3, τ4, τ5, τ6, τ7, τ8, τ9, τ1, τ1, τ1, τ2, τ3, τ4, τ5, τ1, τ1, τ2, τ3, τ4, τ5, τ6, τ7, τ8, τ9, τ1, τ1, τ1, τ2 ...2, τ1, τ2, τ2, τ k , τ K-1 , τ K This invention predicts future time series data by incorporating the relationship between the latest small sequence and multiple historical small sequences with different time spans, namely: short time span, medium time span, and long time span. When the sampling period is determined as st, there is a one-to-one correspondence between the data length and the time span. For example, if the data length is L, it means that there are L sampling points, L sampling periods are required, and a total duration of L×st is needed to construct L data points. There is no unified standard for the classification of short, medium, and long time spans. Usually, it is based on the data length contained in the small sequence. For example, if the data length contained in the small sequence is L, then a time span less than one-third of L×st is considered a short time span, a time span greater than two-thirds of L*st is considered a long time span, and a time span in between is considered a medium time span.

[0036] Step 1:

[0037] The original sequence X is divided into two subsequences, resulting in the subsequence Y, where the division is overlapped. The sequence X = x1, x2, ..., x... LX Given a historical time series, the sequence length is LX. The sequence X is then divided into subsequences Y of equal length, each of length L. i When the number of elements in sequence X is not divisible by the number of elements in sequence X, the last element x of sequence X is used as the dividing point. LX The padding is performed, where L is a variable. When the value of L is determined, the subsequence Y... i The length of each subsequence is L, and the subsequence Y is... iThe sequences are concatenated in chronological order to form the sequence Y = Y1, Y2, ..., which is similar to the original sequence X = x1, x2, ..., x... LX Different, where Y1 = x1, x2, ..., x L Y2 = x2, x3, ..., x L+1 By analogy, we can conclude that Y τ =x τ x τ+1 , ..., x τ+L τ represents the step size, with a value range of {1, LX-L+1}. This sequence is similar to the original sequence X = x1, x2, ..., x... LX The difference lies in the fact that sequence Y is an overlapping partition of sequence X. In this case, L is a variable, and the maximum value of L is LX, which is the subsequence Y. i The maximum length of τ does not exceed the length of the original sequence X. When LX is large, the range of values ​​for τ is large, resulting in a large number of sequences Y.

[0038] Step Two:

[0039] By calculating the autocorrelation coefficients among all subsequences in the Y sequence, an autocorrelation coefficient matrix is ​​obtained. This invention provides the following estimation method for this autocorrelation coefficient matrix, which is a large matrix whose size varies with the value of τ. For example, when τ = 1, the subsequences Y1 = x1, x2, ..., x... are obtained. L Subsequence Y2 = x2, x3, ..., x L+1 From the perspective of the original sequence X, the subsequences Y1 and Y2 differ by only 1 in the position of their starting elements. Similarly, when τ takes other values, the starting positions of all subsequences relative to the original sequence X differ by τ. The autocorrelation coefficients between all subsequences form an autocorrelation coefficient matrix, with the following dimensions: number of columns... The number of rows is in This indicates rounding down, where the subsequences Y1 and Y2 in the autocorrelation coefficient matrix are used as the basis for rounding. τ Taking the autocorrelation coefficient as an example, the autocorrelation coefficient is calculated as R(τ).

[0040] Therefore,

[0041] At this point, the first row and τth column of the autocorrelation coefficient matrix is ​​the result R(τ) calculated here;

[0042] τ represents the interval at which the subsequence takes values ​​from the original sequence. Each value of τ corresponds to an autocorrelation coefficient matrix. After traversing all τ, a set of accompanying autocorrelation coefficient matrices will be generated. By concatenating the above matrices column by column according to the order of the values ​​of τ, a complete autocorrelation coefficient matrix is ​​formed.

[0043] When LX is large, the autocorrelation matrix will be extremely large. Not only does LX have an impact, but traversing τ also contributes to the large matrix size. Based on the above, this invention provides the following estimation method for estimating the autocorrelation coefficient matrix. After the autocorrelation coefficient matrix is ​​estimated, the result is denoted as matrix M. Performing SVD decomposition on M yields M = UΣV T Where U is the matrix composed of left singular vectors, V is the matrix composed of right singular vectors, and Σ is the diagonal matrix composed of singular values, which can also be denoted as diag(σ1,σ2,...,σ). n );

[0044] At this point, we take the elements σ1, σ2, ..., σ on the diagonal. n Summing yields a scalar σ. We take σ as a multiple of λ ∈ (0,1), meaning the summation of the diagonal elements stops when it reaches λσ. At this point, the corresponding σ... k Thus, we obtain the index set {1,...,k}.

[0045] Step 3:

[0046] Calculate the Q sequence, K sequence, and V sequence, and then use the subsequence Y. i The projection of Y, where i takes values ​​in the range {1,...,k}, depends on different Y values. i Calculate the corresponding Q, K, and V sequences. The Q, K, and V sequences are the basic sequences used to construct the encoder-decoder structure. Perform cross-correlation calculations on the Q and K sequences to obtain the result R. Q,K (1),R Q,K (2),...,R Q,K (k), denoted as vector R Q,K Using this vector as a weight, a weighted sum is performed on the set of sequences formed by the V sequence to obtain R. Q,K (1)*V1+R Q,K (2)*V2+...,R Q,K (k)*V k , denoted as vector RV Q,K .

[0047] Step Four:

[0048] The resulting matrix is ​​directly used as the input to the feedforward network to calculate its output, which is a vector FRV. Q,KAt this point, the original input sequence X = x1, x2, ..., x from the first step is... LX Transform into a vector FRV Q,K .

[0049] Step 5:

[0050] The resulting matrix is ​​directly used as the input to the feedforward network to calculate its output, which is a vector FRV. Q，K At this point, the original input sequence X = x1, x2, ..., x from the first step is... LX Transform into a vector FRV Q，K .

[0051] In step two, the autocorrelation coefficient matrix is ​​very large and computationally intensive. Performing a complete calculation is not realistic. Step two allows for an accurate estimation of the matrix. Based on the estimated matrix, the subscript set {1, ..., k} is further calculated. This subscript set is convenient for use in subsequent processing steps.

[0052] In the above steps, the estimation of the autocorrelation coefficient matrix in step two is as follows:

[0053] By calculating the autocorrelation coefficients of subsequences, we can obtain the autocorrelation coefficients of some subsequences. In practical engineering applications, the scale of LX is usually tens of thousands or even hundreds of thousands or millions. Therefore, relying on traversing all subsequences of X sequence to calculate the autocorrelation coefficients and then constructing the autocorrelation coefficient matrix is ​​very time-consuming and laborious, and may even be impossible. At this time, we can estimate the autocorrelation coefficient matrix. Before estimation, we can calculate the autocorrelation coefficients between some subsequences to obtain some elements in the autocorrelation coefficient matrix, and estimate the autocorrelation coefficient matrix using these elements.

[0054] Suppose that we calculate the pairwise autocorrelation coefficients of a finite number of subsequences and write them in matrix form, denoted as D. For example, the first row and first column of matrix D contains the autocorrelation coefficients of subsequence Y1 and subsequence Y2; the first row and second column of matrix D contains the autocorrelation coefficients of subsequence Y1 and subsequence Y2 when τ = 1.

[0055] Next, we will calculate matrix M using matrix D. The specific calculation steps are as follows:

[0056] According to data D m×n The matrices are calculated to obtain the W and Z matrices. The calculation process is as follows:

[0057] (1) Perform orthogonal decomposition on matrix D: Where U∈R m×m , Q∈R n×n All are orthogonal matrices, R∈R r×rW is an invertible upper triangular matrix, O is the zero matrix, and the initial value of W is W0 = R;

[0058] (2) Take the maximum number of loops as k = k max ;

[0059] (3) Calculation

[0060] (4) Let

[0061] (5) Order

[0062] (6) If ||W k+1 -W k || F ≤ε||D|| F Stop the iteration, and at this point let W = W k+1 , If ||W k+1 -W k || F ≤ε||D|| F Then continue iterating from step (3) to step (6) until the condition is met or k = k max .

[0063] This article uses specific examples to illustrate the principles and implementation methods of the present invention. The above examples are only for the purpose of helping to understand the method and core ideas of the present invention. The above descriptions are only preferred embodiments of the present invention. It should be noted that due to the limitations of textual expression, while there are objectively infinite specific structures, those skilled in the art can make several improvements, modifications, or changes without departing from the principles of the present invention, and can also combine the above technical features in an appropriate manner. These improvements, modifications, changes, or combinations, or the direct application of the inventive concept and technical solution to other situations without modification, should all be considered within the scope of protection of the present invention.

Claims

1. A time series data processing method, characterized in that, Includes the following steps: Step 1: The original sequence X is divided into two subsequences, resulting in the subsequence Y, where the division is overlapped. The sequence X = x1, x2, ..., x... LX Given a historical time series, the sequence length is LX. The sequence X is then divided into subsequences Y of equal length, each of length L. i When the number of elements in sequence X is not divisible by the number of elements in sequence X, the last element x of sequence X is used as the dividing point. LX The padding is performed, where L is a variable. When the value of L is determined, the subsequence Y... i The length of each subsequence is L, and the subsequence Y is... i The sequences are concatenated in chronological order to form the sequence Y = y1, y2, ..., which is similar to the original sequence X = x1, x2, ..., x... LX Different, where Y1 = x1, x2, ..., x L Y2 = x2, x3, ..., x L+1 By analogy, we can conclude that Y τ =x τ x τ+1 , ..., x τ+L τ represents the step size, which takes values ​​in the range {1, LX-L+1}; Step Two: By calculating the autocorrelation coefficients among all subsequences in the Y sequence, an autocorrelation coefficient matrix is ​​obtained. This invention provides the following estimation method for this autocorrelation coefficient matrix, which is a large matrix whose size varies with the value of τ. For example, when τ = 1, the subsequences Y1 = x1, x2, ..., x... are obtained. L Subsequence Y2 = x2, x3, ..., x L+1 From the perspective of the original sequence X, the subsequences Y1 and Y2 differ by only 1 in the position of their starting elements. Similarly, when τ takes other values, the starting positions of all subsequences relative to the original sequence X differ by τ. The autocorrelation coefficients between all subsequences form an autocorrelation coefficient matrix, with the following dimensions: number of columns... The number of rows is in This indicates rounding down, where the subsequences Y1 and Y2 in the autocorrelation coefficient matrix are used as the basis for rounding. τ Taking the autocorrelation coefficient as an example, the autocorrelation coefficient is calculated as R(τ). Therefore, At this point, the first row and τth column of the autocorrelation coefficient matrix is ​​the result R(τ) calculated here; τ represents the interval at which the subsequence takes values ​​from the original sequence. Each value of τ corresponds to an autocorrelation coefficient matrix. After traversing all τ, a set of accompanying autocorrelation coefficient matrices will be generated. By concatenating the above matrices column by column according to the order of the values ​​of τ, a complete autocorrelation coefficient matrix is ​​formed. When LX is large, the autocorrelation matrix will be extremely large. Not only does LX have an impact, but traversing τ also contributes to the large matrix size. Based on the above, this invention provides the following estimation method for estimating the autocorrelation coefficient matrix. After the autocorrelation coefficient matrix is ​​estimated, the result is denoted as matrix M. Performing SVD decomposition on M yields M = UΣV T Where U is the matrix composed of left singular vectors, V is the matrix composed of right singular vectors, and Σ is the diagonal matrix composed of singular values, which can also be denoted as diag(σ1,σ2,...,σ). n ); At this point, we take the elements σ1, σ2, ..., σ on the diagonal. n Summing yields a scalar σ. We take σ as a multiple of λ ∈ (0,1), meaning the summation of the diagonal elements stops when it reaches λσ. At this point, the corresponding σ... k This yields the index set {1,...,k}; Step 3: Calculate the Q sequence, K sequence, and V sequence, and then use the subsequence Y. i The projection of Y, where i takes values ​​in the range {1,...,k}, depends on different Y values. i Calculate the corresponding Q, K, and V sequences. The Q, K, and V sequences are the basic sequences used to construct the encoder-decoder structure. Perform cross-correlation calculations on the Q and K sequences to obtain the result R. Q,K (1),R Q,K (2),...,R Q,K (k), denoted as vector R Q,K Using this vector as a weight, a weighted sum is performed on the set of sequences formed by the V sequence to obtain R. Q,K (1)*V1+R Q,K (2)*V2+...,R Q,K (k)*V k , denoted as vector RV Q,K ; Step Four: The resulting matrix is ​​directly used as the input to the feedforward network to calculate its output, which is a vector FRV. Q,K At this point, the original input sequence X = x1, x2, ..., x from the first step is... LX Transform into a vector FRV Q,K ; Step 5: The resulting matrix is ​​directly used as the input to the feedforward network to calculate its output, which is a vector FRV. Q,K At this point, the original input sequence X = x1, x2, ..., x from the first step is... LX Transform into a vector FRV Q,K .

2. The time series data processing method according to claim 1, wherein the estimation of the autocorrelation coefficient matrix in step two is characterized in that: Suppose that we calculate the pairwise autocorrelation coefficients of a finite number of subsequences and write them in matrix form, denoted as D. For example, the first row and first column of matrix D contains the autocorrelation coefficients of subsequence Y1 and subsequence Y2; the first row and second column of matrix D contains the autocorrelation coefficients of subsequence Y1 and subsequence Y2 when τ = 1. Next, we will calculate matrix M using matrix D. The specific calculation steps are as follows: According to data D m×n The matrix is ​​calculated to obtain the W and Z matrices. The calculation process is as follows: (1) Perform orthogonal decomposition on matrix D: Where U∈R m×m , Q∈R n×n All are orthogonal matrices, R∈R r×r W is an invertible upper triangular matrix, O is the zero matrix, and the initial value of W is W0 = R; (2) Take the maximum number of loops as k = k max ; (3) Calculation (4) Let (5) Order (6) If ||W k+1 -W k || F ≤ε||D|| F Stop the iteration, and at this point let W = W k+1 , If ||W k+1 -W k || F ≤ε||D|| F Then continue iterating from step (3) to step (6) until the condition is met or k = k max .

Images