Time series prediction model training method, time series prediction method and equipment

By discretizing time series samples and generating attention weight matrices, the interpretability and accuracy issues of time series prediction models are resolved, achieving interpretable and efficient long-term prediction.

CN119025924AActive Publication Date: 2024-11-26INSPUR SUZHOU INTELLIGENT TECH CO LTD
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
CN202411498056.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-10-25
Publication Date
2024-11-26
Estimated Expiration
2044-10-25

AI Technical Summary

Technical Problem

Existing time series forecasting models are difficult to interpret and have low accuracy in long-term forecasting.

Method used

By discretizing time series samples, the segment features and information of sequence fragment samples are obtained. The attention operation mechanism of the time series prediction model is used to generate an attention weight matrix, which quantifies the importance of each sequence fragment sample to the prediction result, thereby achieving the interpretability and accuracy of the model.

Benefits of technology

It improves the accuracy of long-period time series forecasting, enhances model interpretability, and reduces computational costs and data processing volume.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119025924A_ABST
    Figure CN119025924A_ABST
Patent Text Reader

Abstract

The invention provides a time sequence prediction model training method, a time sequence prediction method and equipment, and is applied to the technical field of data processing, and the method comprises the steps: obtaining a time sequence sample, and carrying out the discretization of the time sequence sample, and obtaining a sequence fragment sample; obtaining fragment sample features and fragment sample information of the sequence fragment samples; obtaining an attention weight matrix of the time sequence sample according to an attention operation mechanism of the time sequence prediction model, the fragment sample features and the fragment sample information; obtaining a time sequence sample prediction result according to the attention weight matrix; and training the time sequence prediction model through the time sequence sample prediction result to obtain a trained time sequence prediction model. The time sequence prediction model obtained by the method has interpretability, and the accuracy of the prediction result is high.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to the field of data processing technology, and in particular to a time series prediction model training method, a time series prediction method and a device. Background Art

[0002] Time series data is a data series that includes multiple data sequences arranged in chronological order. Time series prediction can be applied to a variety of application scenarios. For example, weather information in the future can be predicted based on weather information in the historical time period. For another example, traffic flow in the future can be predicted based on traffic flow information in the historical time period.

[0003] In the related technology, a time series prediction model (Sample Convolution and Interaction Network, SCINet) can be used for time series prediction, or a deep learning model for long-term time series prediction (Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting, AuToFormer) can be used for time series prediction.

[0004] However, the time series prediction models in the related art have the disadvantage of being difficult to explain, and these time series prediction models have the problem of low accuracy of prediction results when predicting long-period time series data. Summary of the invention

[0005] The embodiments of the present disclosure provide a time series prediction model training method, a time series prediction method and a device to solve the problem that the time series prediction model in the related art is difficult to explain and the prediction result has low accuracy.

[0006] In a first aspect, an embodiment of the present disclosure provides a time series prediction model training method, comprising: Acquire time series samples, and discretize the time series samples to obtain sequence fragment samples; Acquire fragment sample features and fragment sample information of the sequence fragment samples; Obtaining an attention weight matrix of the time series samples according to the attention operation mechanism of the time series prediction model, the characteristics of the fragment samples, and the fragment sample information; Obtaining a time series sample prediction result of the time series sample according to the attention weight matrix; The time series prediction model is trained by using the time series prediction results of the time series samples to obtain a trained time series sample prediction model.

[0007] In a second aspect, an embodiment of the present disclosure provides a time series prediction method, comprising: Get the target time series to be predicted; Discretizing the target time series to obtain multiple target sequence segments; The target sequence segment is input into the trained time prediction model to obtain the time series prediction result of the target time series; wherein the trained time prediction model is obtained by the method for obtaining the time series prediction model described in the first aspect.

[0008] In a third aspect, the present disclosure provides a time series prediction model training device, including: A first acquisition module is used to acquire time series samples and discretize the time series samples to obtain sequence fragment samples; A second acquisition module is used to acquire a fragment sample feature of the sequence fragment sample; the fragment sample feature includes sample information of the sequence fragment sample corresponding to the fragment sample feature; A third acquisition module is used to obtain an attention weight matrix according to the attention operation mechanism of the time series prediction model and the characteristics of the fragment samples; A fourth acquisition module is used to obtain a time series sample prediction result according to the attention weight matrix; The fifth acquisition module is used to train the time series prediction model through the time series sample prediction results to obtain the time series prediction model.

[0009] In a fourth aspect, an embodiment of the present disclosure provides a time series prediction device, comprising: A sixth acquisition module, used to acquire a target time series to be predicted; A seventh acquisition module, used for discretizing the target time series to obtain a plurality of target sequence segments; An eighth acquisition module is used to input the target sequence segment into the trained time prediction model to obtain the time series prediction result of the target time series; Wherein, the time prediction model is obtained by the method for obtaining the time series prediction model described in the first aspect.

[0010] In a fifth aspect, an embodiment of the present disclosure provides an electronic device, including: Memory, used to store computer programs; A processor is used to execute the computer program to implement the method as described in the first aspect or the second aspect.

[0011] In a sixth aspect, an embodiment of the present disclosure provides a computer-readable storage medium for storing a computer program; wherein when the computer program is executed by a processor, the method described in the first aspect or the second aspect is implemented.

[0012] In a seventh aspect, an embodiment of the present disclosure provides a computer program product, comprising a computer program / instruction, which, when executed by a processor, implements the steps of the method as described in any one of the first aspect or the second aspect.

[0013] Compared with the prior art, the present invention has the following advantages: In the disclosed embodiment, in this embodiment, the time series samples are discretized to obtain sequence fragment samples, and the fragment sample features and sample information of the sequence fragment samples are obtained. According to the attention operation mechanism of the time series prediction model and the fragment sample features, an attention weight matrix is ​​obtained, and the attention weight matrix thus obtained includes the sample information of the fragment sample features used to calculate the attention weights. Based on the attention weights in the attention weight matrix and the sample information of the fragment sample features used to calculate the attention weights, the importance of each sequence fragment sample input into the time series prediction model to the prediction result can be quantified, and the importance of the sequence fragment samples corresponding to the fragment sample information to the time series prediction result can be determined according to the sample information of the attention weight matrix. In other words, based on the attention weight matrix containing the sample information of the fragment sample features used to calculate the attention weights, the model interpretation of the time prediction model is realized, and the time series prediction model obtained based on the attention weight matrix can predict the long-term time series in an interpretable manner.

[0014] The above description is only an overview of the technical solution of the present disclosure. In order to more clearly understand the technical means of the present disclosure, it can be implemented according to the contents of the specification. In order to make the above and other purposes, features and advantages of the present disclosure more obvious and easy to understand, the specific implementation methods of the present disclosure are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure or related technologies, the drawings required for describing the embodiments are briefly introduced below.

[0016] Figure 1 A step diagram of a time series prediction model training method provided by an embodiment of the present disclosure; Figure 2 A schematic diagram of a time series sample discretization processing result provided by an embodiment of the present disclosure; Figure 3 A schematic diagram of the structure of a time series prediction model provided by an embodiment of the present disclosure; Figure 4 A step diagram of another time series prediction model training method provided by an embodiment of the present disclosure; Figure 5 A step diagram of another time series prediction model training method provided by an embodiment of the present disclosure; Figure 6 A schematic diagram of another time series sample discretization processing result provided by an embodiment of the present disclosure; Figure 7 A step diagram of a time series prediction method provided by an embodiment of the present disclosure; FIG8 (a) is a schematic diagram of an attention weight matrix corresponding to parameter 1 provided in an embodiment of the present disclosure; FIG8 (b) is a schematic diagram of an attention weight matrix corresponding to parameter 2 provided in an embodiment of the present disclosure; FIG8 (c) is a schematic diagram of an attention weight matrix corresponding to parameter 3 provided in an embodiment of the present disclosure; FIG8 (d) is a schematic diagram of an attention weight matrix corresponding to parameter 4 provided in an embodiment of the present disclosure; FIG8 (e) is a schematic diagram of an attention weight matrix corresponding to parameter 5 provided in an embodiment of the present disclosure; FIG8( f ) is a schematic diagram of an attention weight matrix corresponding to parameter 6 provided in an embodiment of the present disclosure; FIG8( g) is a schematic diagram of an attention weight matrix corresponding to parameter 7 provided in an embodiment of the present disclosure; FIG9 (a) is a schematic diagram of an attention weight matrix corresponding to a single prediction parameter 1 provided in an embodiment of the present disclosure; FIG9( b ) is a schematic diagram of an attention weight matrix corresponding to a single prediction parameter 2 provided in an embodiment of the present disclosure; FIG9 (c) is a schematic diagram of an attention weight matrix corresponding to a single prediction parameter 3 provided in an embodiment of the present disclosure; FIG9( d ) is a schematic diagram of an attention weight matrix corresponding to a single prediction parameter 4 provided in an embodiment of the present disclosure; FIG9 (e) is a schematic diagram of an attention weight matrix corresponding to a single prediction parameter 5 provided in an embodiment of the present disclosure; FIG9( f ) is a schematic diagram of an attention weight matrix corresponding to a single prediction parameter 6 provided in an embodiment of the present disclosure; FIG9( g ) is a schematic diagram of an attention weight matrix corresponding to a single prediction parameter 7 provided in an embodiment of the present disclosure; Fig.10 A schematic diagram of the true value and predicted value of OT provided in an embodiment of the present disclosure; FIG11( a ) is a schematic diagram of a global statistical attention weight matrix corresponding to parameter 1 provided in an embodiment of the present disclosure; FIG11( b ) is a schematic diagram of a global statistical attention weight matrix corresponding to parameter 2 provided in an embodiment of the present disclosure; FIG11( c ) is a schematic diagram of a global statistical attention weight matrix corresponding to parameter 3 provided in an embodiment of the present disclosure; FIG11( d ) is a schematic diagram of a global statistical attention weight matrix corresponding to parameter 4 provided in an embodiment of the present disclosure; FIG11( e ) is a schematic diagram of a global statistical attention weight matrix corresponding to parameter 5 provided in an embodiment of the present disclosure; FIG11( f ) is a schematic diagram of a global statistical attention weight matrix corresponding to parameter 6 provided in an embodiment of the present disclosure; FIG11( g ) is a schematic diagram of a global statistical attention weight matrix corresponding to parameter 7 provided in an embodiment of the present disclosure; FIG. 12 ( a ) is a schematic diagram showing the degree of influence of various parameters on the prediction result of parameter 1 provided in an embodiment of the present disclosure; FIG12( b ) is a schematic diagram showing the influence of various parameters on the prediction result of parameter 2 provided in an embodiment of the present disclosure; FIG12( c ) is a schematic diagram showing the influence of various parameters on the prediction result of parameter 3 provided in an embodiment of the present disclosure; FIG12( d ) is a schematic diagram showing the influence of various parameters on the prediction result of parameter 4 provided in an embodiment of the present disclosure; FIG12( e ) is a schematic diagram showing the influence of various parameters on the prediction result of parameter 5 provided in an embodiment of the present disclosure; FIG12( f ) is a schematic diagram showing the influence of various parameters on the prediction result of parameter 6 provided in an embodiment of the present disclosure; FIG12( g ) is a schematic diagram showing the influence of various parameters on the prediction result of parameter 7 provided in an embodiment of the present disclosure; Fig.13 A screening matrix for an ETTH2 data set provided in an embodiment of the present disclosure; Fig.14 A schematic diagram of a time series prediction model training device provided by an embodiment of the present disclosure; Fig.15 A schematic diagram of a time series prediction device provided by an embodiment of the present disclosure; Fig.16 A block diagram of an electronic device provided for an embodiment of the present disclosure. DETAILED DESCRIPTION

[0017] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can also be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0018] Some nouns or terms appearing in this application are subject to the following interpretations: The Electricity Transformer Temperature (ETT) dataset contains the transformer power load and oil temperature in different time periods, as well as the time series of the power load and oil temperature.

[0019] It should be noted that the data samples involved in this application (including but not limited to data used for analysis, stored data, displayed data, etc.) are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0020] Figure 1 is a step diagram of a time series prediction model training method provided by an embodiment of the present disclosure, referring to Figure 1 , the time series prediction model training method includes: Step 101: obtain time series samples, and discretize the time series samples to obtain multiple sequence segment samples.

[0021] The time series sample is obtained by sorting multiple data samples according to the collection time of the data samples, and the sequence fragment sample includes at least one data sample.

[0022] Furthermore, data samples at multiple preset moments are obtained, and the multiple data samples are arranged in chronological order of the preset moments to obtain time series samples.

[0023] The data in the time series sample may be weather data, traffic flow data, power load data of equipment in the power system, or other data. This embodiment does not limit the type of data.

[0024] For example, the time series samples can be discretized according to a preset sequence length to obtain multiple sequence fragment samples, wherein the sequence length reflects the number of data in the sequence.

[0025] For example, refer to Figure 2, the time series samples include six data samples from data a1 to data a6. Each data sample has a corresponding collection time, and data a1 to data a6 are arranged in the order of collection time. For example, the collection time of data a1 is earlier than the collection time of other data, and the collection time of data a2 is earlier than the collection time of data a3 to data a6.

[0026] For example, the preset sequence length is 3. According to the preset sequence length of 3, Figure 2 The time series samples in are discretized to obtain 3 sequence fragment samples. Each sequence fragment sample includes 3 data samples.

[0027] In order to preserve the context information of the sequence fragment samples, the data of two adjacent sequence fragment samples need to be connected end to end. Specifically, in two adjacent sequence fragment samples, the data at the end of the first sequence fragment sample is the same as the data at the beginning of the second sequence fragment sample. Figure 2 , the first sequence segment sample includes data a1, data a2 and data a3, and the second sequence segment sample includes data a3, data a4 and data a5. The last data in the first sequence segment sample is a3, and the first data in the second sequence segment sample is also data a3.

[0028] Step 102: Obtain fragment sample features and fragment sample information of the sequence fragment samples.

[0029] The sequence segment sample features and the sample information of the sequence segment samples are obtained, thereby ensuring that the corresponding relationship between the sequence segment samples and the segment sample features input into the time series prediction model is retained in the subsequent processing process.

[0030] For example, the fragment sample information may include the identifier of the sequence fragment sample and the time corresponding to the sequence fragment sample. The identifier of the sequence fragment sample may be a data identifier of the sequence fragment sample, and the data identifier may be a data type name, a data type code, or other data identifier. For example, if the sequence fragment sample includes the power load data of device A in the power system, the fragment sample information may include a code for identifying the power load of device A and the acquisition time of each power load.

[0031] For example, a one-dimensional convolution layer is used to perform convolution processing on the sequence fragment samples to obtain the fragment sample features of the sequence fragment samples.

[0032] Step 103, obtaining the attention weight matrix of the time series samples according to the attention operation mechanism of the time series prediction model, the fragment sample characteristics, and the fragment sample information.

[0033] Among them, the attention weight matrix includes fragment sample information.

[0034] Furthermore, the attention weight matrix includes: attention weights, and fragment sample information corresponding to sequence fragment sample features used to calculate the attention weights.

[0035] For example, refer to Figure 3 , perform convolution processing on the fragment sample features to obtain the query vector, key vector, and value vector of the sample features. According to the query connection and key vector, the attention weight matrix of the fragment sample features is obtained.

[0036] Step 104, obtaining the time series sample prediction result according to the attention weight matrix.

[0037] For example, dot product attention operation is performed on the attention weight matrix and the value vector to obtain the attention feature, and the attention feature is input into the forward channel of the time series prediction model to obtain the time series sample prediction result. The forward channel may include multiple fully connected layers. The forward channel composed of multiple fully connected layers plays the role of a predictor in the time series prediction model.

[0038] Step 105: training the time series prediction model through the time series sample prediction results to obtain a trained time series prediction model.

[0039] For example, the training of the time series prediction model includes multiple training rounds. In each training round, the attention weight matrix is ​​obtained according to the fragment sample characteristics of the sequence fragment samples, the fragment sample information and the attention operation mechanism of the time series prediction model, and the time series sample prediction result is obtained according to the time series prediction model and the attention weight matrix.

[0040] When the current round does not meet the training stop conditions, the model parameters of the time series prediction model are optimized according to the time series sample prediction results and the time series results as labels, and the training of the time series prediction model is continued until the model training meets the training stop conditions. Then the time series prediction model obtained in the last round is determined as the time series prediction model.

[0041] For example, when the number of rounds reaches a preset round number threshold, it is determined that the model training meets the training stop condition, and the time series prediction model obtained from the last training is determined as the time series model.

[0042] For another example, in each round of training, the loss function value is obtained, and the model parameters in the time series prediction model are optimized according to the loss function value, completing a time series prediction model training. Furthermore, in each training round, the time series sample prediction results output by the time series prediction model are obtained, and the loss function value is obtained according to the time series prediction sample results and the time series data as labels. When the loss function value meets the preset loss function value requirements, it is determined that the model training meets the training stop condition, and the time series prediction model obtained from the last training is determined as the time series model.

[0043] Furthermore, the loss function value of each training round in the training process of the most recent preset number of rounds can be obtained. When the loss function value of the preset round does not decrease as the number of rounds increases, it is determined that the loss function value meets the preset loss function requirement.

[0044] In the related technology, commonly used time series prediction models include: Long Short-Term Memory (LSTM) model, Recurrent Neural Network (RNN) model, Transformer structure model, etc. There are two problems that need to be solved in the time series prediction model. One is how to improve the accuracy of long sequence time series prediction (LSTF), and the other is how to explain the reasons why the deep neural network model obtains the prediction results. Before the Transformer structure was proposed, the Recurrent Neural Network (RNN) model and the Long Short-Term Memory (LSTM) model dominated the field of time series prediction. Although the RRN model and the LSTM model can realize time series prediction, their cyclic structure will lead to low computational efficiency and difficulties in feature extraction when processing long sequences, which limits their application in time series prediction. Among them, the Transformer structure model and its branches have better time series prediction effects than other models.

[0045] However, for the Transformer model of the related technology, its computational cost is proportional to the square of the input and output data length. The input and output data length of long-period time series is relatively long. The long-period time series prediction based on the Transform model of the related technology has high computational cost, and its attention value cannot directly represent the importance of the variable, resulting in unexplainable problems in the Transform model of the related technology.

[0046] In the related art, the long-term time series prediction (AuToFormer) model can also be used for long-term time series prediction, but the AuToFormer model relies on the assumption of the periodicity of the time series, and its computational complexity is very high, and there are problems that cannot be explained. Models based on the Transformer structure include the LogSparseTransformer model, the model that can process long texts (Longformer) model, and the Beyond Efficient Transformer for Long equence Time-Series Forecasting (Informer) model, etc. These models also have problems that are difficult to explain. Based on these models, it is impossible to clarify the importance of parameters to the prediction results. The deep learning multi-level time series prediction (Temporal FusionTransformer, TFT) model in the related art can explain the data basis of the model, but it uses the importance of attention to explain the variables without considering the impact of the model structure on the attention value, so its interpretability is inaccurate. That is, the time series prediction model in the related art at least has the problem of lack of interpretability.

[0047] In this embodiment, the time series samples are discretized to obtain sequence fragment samples, and the fragment sample features and sample information of the sequence fragment samples are obtained. According to the attention operation mechanism and fragment sample features of the time series prediction model, the attention weight matrix is ​​obtained, and the attention weight matrix thus obtained includes the sample information of the fragment sample features used to calculate the attention weights. Based on the attention weights in the attention weight matrix and the sample information of the fragment sample features used to calculate the attention weights, the importance of each sequence fragment sample input into the time series prediction model to the prediction result can be quantified, and the importance of the sequence fragment samples corresponding to the sample information to the time series sample prediction result can be clearly obtained, thereby realizing the interpretability of the time series prediction model. That is, in this embodiment, based on the attention weight matrix containing the sample information of the fragment sample features used to calculate the attention weights, the model interpretation of the time prediction model is realized. In other words, the time series prediction model obtained based on the attention weight matrix can predict the long-term time series in an interpretable manner.

[0048] In addition, the time series samples are discretized, and the sequence fragment samples obtained based on the discretization processing are also discrete. The fragment sample features obtained based on the sequence fragment samples are also discrete. Based on the discrete fragment sample features, the degree of influence of the sequence fragment samples on the attention weights in the attention weight matrix can be obtained. Based on the discrete fragment sample features, the attention weights obtained from the subsequent fragment sample features are used to obtain a screening matrix. When a part of the sequence fragment samples is screened out according to the screening matrix, the impact on the prediction results of the time series samples when shielding unnecessary sequence fragment samples can be minimized. In other words, the sequence fragment samples obtained based on the discretization processing can reduce the computational cost of the time series prediction model for predicting long-period time series, improve the prediction accuracy, and achieve the interpretability of the time series prediction model.

[0049] Figure 4 Another time series model training method provided in the present application embodiment is referred to Figure 4 , the method may include the following steps: Step 201 : discretize the time series samples according to the preset sequence length to obtain sequence segment samples.

[0050] Among two adjacent sequence segment samples, a first number of data samples located at the end of a first sequence segment sample is the same as a first number of data samples located at the beginning of a second sequence segment sample.

[0051] The first number may be a preset value and may be set according to user requirements.

[0052] The sequence fragment sample includes a plurality of data arranged in chronological order, and the preset sequence length is equal to the number of data samples in the sequence fragment sample.

[0053] For example, refer to Figure 2 , the preset sequence length is 3, then each sequence fragment sample obtained includes 3 data samples, and the data sample at the end of the first sequence fragment sample is the same as the data sample at the beginning of the second sequence fragment sample, both of which are a3.

[0054] Step 202: Obtain the fragment sample features and fragment sample information of the sequence fragment samples.

[0055] The fragment sample information includes at least one of the following: a sample identifier of a sequence fragment sample, and time information of the sequence fragment sample.

[0056] The sample identifier may be the name or number of the time series sample to which the sequence fragment sample belongs, or other identification information of the time series sample.

[0057] The time information of the sequence segment samples may be the collection time information of the data samples in the sequence segment samples, or the time period information of the sequence segment samples.

[0058] When there are multiple sequence fragment samples, the sequence fragment samples and fragment sample features correspond one-to-one, and the fragment sample features correspond one-to-one to the fragment sample information. For example, if the fragment sample information includes a sample identifier and time information, the fragment sample feature has the corresponding sample identifier and time information.

[0059] For example, refer to Figure 3 The schematic diagram of the structure of the time series prediction model shown in FIG. divides the time series samples into discrete sequence fragment samples through the interpretable discrete feature mapping layer in the model. Through the interpretable discrete feature mapping process, the sequence fragment samples are mapped into fragment sample features.

[0060] For example, through the one-dimensional convolution layer in the discrete feature mapping layer, the fragment sample features of the sequence fragment samples are extracted from the small-scale sequence fragment samples obtained after the discrete processing. Among them, the feature extraction is performed through the one-dimensional convolution layer to effectively retain the fragment sample information corresponding to the extracted fragment sample features.

[0061] For example, the width and step size of the kernel function in the one-dimensional convolution layer can be set according to user needs. Preferably, the width and step size of the kernel function of the one-dimensional convolution layer are 1, and based on the kernel function with a width of 1, it can be ensured that the fragment sample features of one sequence fragment sample are extracted at a time, and the fragment sample features and the sequence fragment samples can correspond one to one. Setting the kernel function step size to 1 can ensure that the features of each sequence fragment sample are extracted to avoid data omission.

[0062] Specifically, the time series prediction module may include a feature mapping module (Feature Encoding, FE) and a positional encoding mapping module (Positional Encoding, PE). The feature mapping module FE and the positional encoding mapping module PE can be used to obtain the fragment sample features of the sequence fragment samples, and the fragment sample features can be labeled with the fragment sample information. Among them, in the time series prediction model, the positional encoding mapping module PE is used to mark the sequence order of the sequence variables. Specifically, the positional encoding mapping module PE is used to mark the position of the sequence fragment samples in the time series samples. By retaining the positional encoding for the sequence fragment samples, the fragment sample features of each sequence fragment sample can be labeled.

[0063] Step 203: Perform convolution processing on the features of the fragment samples to obtain a query vector and a key vector.

[0064] For example, refer to Figure 3, the fragment sample features are convolved through a one-dimensional convolutional layer to obtain a query vector and a key vector. Among them, the fragment sample features correspond to the sequence fragment samples one by one. There are multiple sequence fragment samples and multiple fragment sample features. Figure 3 As shown, the sequence fragment sample h is convolved to obtain the query vector Qh, key vector Kh, and value vector Vh of the sequence fragment sample h.

[0065] Step 204, obtaining the attention weight matrix of the time series sample according to the query vector and the key vector, as well as the fragment sample information.

[0066] The attention weight matrix includes multiple attention weights, each of which corresponds to sample information of a fragment sample feature used to calculate the attention weight.

[0067] Specifically, the query vector and the key vector are multiplied to obtain an initial attention weight matrix, and each attention weight in the attention weight matrix is ​​labeled with the fragment sample information to obtain an attention weight matrix containing the fragment sample information.

[0068] Among them, the attention weight matrix includes the fragment sample information of the sequence fragment samples. According to the attention weights in the attention weight matrix and the corresponding fragment sample information, it can be known how important the sequence fragment samples corresponding to the fragment sample information are to the subsequently acquired time series sample prediction results, thereby achieving the interpretability of the time series prediction model.

[0069] Step 205: Obtain a target attention weight greater than or equal to a preset weight threshold from the attention weight matrix.

[0070] The preset weight threshold can be set according to user needs. For example, when the importance of sequence fragment samples to the prediction results of time series samples is relatively high, the preset weight threshold can be set larger. Conversely, the preset weight threshold can be set smaller.

[0071] Step 206, obtaining a target segment sample feature corresponding to the target attention weight from the multiple segment sample features.

[0072] Among them, a fragment sample feature corresponds to a sequence fragment sample.

[0073] For example, a screening matrix is ​​constructed according to the target attention weight, and according to the screening matrix, target fragment sample features corresponding to the target attention weight are screened out from multiple fragment sample features.

[0074] For example, step 206 may include the following sub-steps (sub-steps 2061 to 2063): Sub-step 2061, configure a first weight for the target attention weight, and configure a second weight for other attention weights except the target attention weight.

[0075] Among them, the first weight corresponds to the fragment sample feature used to calculate the target attention weight, and the second weight corresponds to the fragment sample feature used to calculate other attention weights.

[0076] For example, the first weight is 1, and the second weight is 0. Specifically, the first weight corresponding to the sequence fragment sample used to calculate the target attention weight is 1, and the second weight corresponding to the sequence fragment sample used to calculate other attention weights is 0.

[0077] Sub-step 2062, constructing a screening matrix according to the first weight, the fragment sample information of the fragment sample feature corresponding to the first weight, the second weight, and the fragment sample information of the fragment sample feature corresponding to the second weight.

[0078] For example, the first weight is 1 and the second weight is 0, then the constructed screening matrix is ​​a matrix whose matrix elements are 1 or 0.

[0079] Sub-step 2063, screening out target segment sample features from a plurality of segment sample features according to the screening matrix.

[0080] For example, the screening matrix and the matrix composed of multiple fragment sample features are multiplied to obtain an evaluation value matrix, wherein the elements in the evaluation value matrix are evaluation values, and the evaluation values ​​correspond to the fragment sample features one by one, indicating that the probability of the fragment sample features being selected is positively correlated.

[0081] From the evaluation value matrix, a target evaluation value whose evaluation value is greater than or equal to a preset threshold is obtained, and the segment sample feature corresponding to the target evaluation value is determined as the target segment sample feature.

[0082] For example, the first weight is 1, the second weight is 0, and the screening matrix is ​​a matrix whose matrix elements are 1 or 0. The screening matrix and the matrix composed of multiple fragment sample features are multiplied to obtain an evaluation value matrix, and the evaluation value in the evaluation value matrix is ​​equal to 1 or 0. Specifically, the evaluation value corresponding to the fragment sample feature of the first weight is 1, and the evaluation value corresponding to the fragment sample feature of the second weight is 0. The fragment sample feature with the evaluation value equal to 1 is obtained and determined as the target fragment sample feature.

[0083] Based on the screening matrix whose matrix elements are 1 or 0, the target fragment sample features can be quickly obtained from the fragment sample features.

[0084] Step 207: Input the target segment sample features into the time series prediction model to obtain the time series sample prediction results.

[0085] Specifically, the target segment sample features are input into the time series prediction model, and the query vector, key vector and value vector corresponding to the target sample features are obtained through the one-dimensional convolutional layer in the time series prediction model. The attention weight matrix is ​​obtained through the query vector and the key vector, and the dot product attention operation is performed on the attention weight matrix and the value vector to obtain the attention feature. The attention feature is input into the forward channel to obtain the time series sample prediction result.

[0086] Step 208: training the time series prediction model using the time series sample prediction results to obtain a trained time series prediction model.

[0087] The method of this step has been described in the aforementioned step 105 and will not be repeated here.

[0088] For example, the attention weight matrix can be obtained through the attention operation mechanism in the Transform structure, and an interpretable time series prediction model can be obtained based on the attention weight matrix. The time series prediction model has the advantages of high prediction accuracy and interpretability. The interpretable time series prediction model based on the Transformer structure can accurately predict long-term time series while explaining the time series prediction model.

[0089] In this embodiment, the attention weight matrix contains sequence fragment information of the fragment sample features. According to the attention weight matrix, the interpretability of the time series prediction model is achieved. Based on the interpretable attention weight matrix, a screening matrix is ​​obtained. According to the screening matrix, key target fragment sample features can be obtained from multiple fragment sample features. The time series prediction model is obtained by training the target fragment sample features. When the time series prediction model is used to predict the time series, the prediction result can be obtained through the key time series fragments, thereby reducing the amount of data used for prediction, reducing the prediction time, reducing the prediction cost, and improving the prediction efficiency. The method of this embodiment is suitable for time series prediction, and is particularly suitable for long-period time series prediction. It solves the problem in the related art that when performing long-period time series prediction, a large amount of data needs to be input into the time series prediction model, resulting in high prediction cost.

[0090] In one embodiment, the time series sample has at least one, reference Figure 5 , the time series prediction model training method may include the following steps: Step 301 : Discretize each time series sample to obtain at least one group of sequence segment samples.

[0091] Among them, one time series sample corresponds to a group of sequence fragment samples.

[0092] For example, refer to Figure 6The time series samples include a first time series sample and a second time series sample. The first time series sample includes 6 data samples from data sample a1 to data sample a6, and the second time series sample includes 6 data samples from data sample b1 to data sample b6.

[0093] Discretization processing is performed on the first time series samples to obtain three groups of sequence fragment samples. The three groups of sequence fragment samples are: sequence fragment samples including data samples a1 to data samples a3, sequence fragment samples including data samples a3 to data samples a5, and sequence fragment samples including data samples a5 to data samples a6.

[0094] The second time series samples are processed discretely to obtain three groups of sequence fragment samples, which are: sequence fragment samples including data samples b1 to data samples b3, sequence fragment samples including data samples b3 to data samples b5, and sequence fragment samples including data samples b5 to data samples b6.

[0095] For example, a time series sample corresponds to at least one parameter, and the time series sample is used to represent data of the corresponding parameter at different times.

[0096] For example, a time series prediction model is used to predict the weather, and the time series samples used to train the time series prediction model include multiple, specifically including: a first time series sample including humidity data, and a second time series sample including temperature data. The first time series sample is discretized to obtain multiple sequence segment samples including humidity data, wherein each sequence segment sample includes multiple humidity sample data arranged according to the acquisition time.

[0097] Step 302: extract features from at least one group of sequence segment samples to obtain segment sample features of the sequence segment samples.

[0098] Among them, one time series sample corresponds to a segment sample feature of one dimension, and the segment sample feature of one dimension includes all sequence segment sample features of the time series sample. Figure 3 The interpretable discrete feature mapping layer extracts features from at least one set of sequence fragment samples to obtain fragment sample features of at least one dimension. The interpretable discrete feature mapping layer includes a feature mapping module FE and a position encoding mapping module PE.

[0099] For example, label the time series sample X of at least one dimension as:

[0100] in, Indicates the number of dimensions, represents the length of time, R represents the real number space, X1, X i , X T Respectively represent the data of the sequence fragment sample of the 1st time length, the data of the sequence fragment sample of the i-th time length and the data of the sequence fragment sample of the T-th time length, Indicates d x dimensional real number space.

[0101] The calculation method of the position encoding mapping module PE is shown in the following formula.

[0102]

[0103]

[0104] in, It represents the position encoding result obtained by encoding the 2jth data of the tth time length using the position encoder function. It represents the position encoding result obtained by encoding the 2j+1th data of the tth time length using the position encoder function. .

[0105] Furthermore, the feature mapping module FE and the position encoding mapping module PE constitute an interpretable discrete feature mapping layer EF. The mapping layer EF discretizes the input time series samples to obtain discrete sequence fragment samples, and maps the sequence fragment samples to the fragment sample features corresponding to the sequence fragment samples. The mapping formula is as follows:

[0106] in, represents the fragment sample features obtained by encoding the time series sample X obtained by the feature encoder; Represents the position encoding of the time series sample X. It represents the mapping result of mapping the fragment sample features obtained by encoding the time series sample X and the time series sample X.

[0107] The fragment sample features extracted based on the interpretable discrete feature mapping layer correspond one-to-one to the input sequence fragment samples, thereby ensuring the intrinsic interpretability of the time series model.

[0108] Further, see Figure 3 ,After the discretized sequence fragment samples pass through the interpretable discrete feature mapping layer, they are labeled with position labels and then input into the dot product attention structure.

[0109] By extracting features from sequence fragment samples of the same time series sample, a fragment sample feature of one dimension is obtained, wherein, among the fragment sample features belonging to the same dimension, the fragment sample features and the sequence fragment samples correspond one-to-one. Based on the fragment sample features that correspond one-to-one to the sequence fragment samples, when the attention weight matrix is ​​subsequently obtained according to the fragment sample features, the attention weight matrix can present the importance of the sequence fragment samples to the prediction results, thereby realizing the interpretability of the time series model.

[0110] Furthermore, one time series sample corresponds to one dimension, and each time series sample is discretized separately, which can achieve isolation according to different dimensions and different time periods. By performing isolated discretization on multi-dimensional time series samples, the discretized sequence fragment samples corresponding to the time series samples can be obtained, and features can be extracted from the discretized sequence fragment sample set. Therefore, in the process of feature extraction, interaction between different data can be strictly prevented to ensure a one-to-one correspondence between sequence fragment samples and fragment sample features, making the data flow clear and simple, so as to clearly realize the interpretability of the time series prediction model.

[0111] For example, there is at least one time series sample. For example, in an application scenario of weather prediction, the time series samples may include humidity time series samples, temperature time series samples, and other time series samples related to weather prediction.

[0112] Step 303, obtaining the attention weight matrix of the time series samples according to the attention operation mechanism of the time series prediction model, the fragment sample characteristics, and the fragment sample information.

[0113] The method of this step has been described in the aforementioned step 103 and will not be repeated here.

[0114] Step 304, obtaining the head attention features of the time series samples according to the attention weight matrix of the time series samples.

[0115] For example, dot product attention calculation is performed on the attention weight matrix of the time series sample and the value vector of the fragment sample feature to obtain the head attention feature of the time series sample.

[0116] For example, refer to Figure 3 , based on the interpretable independent multi-head dot product attention layer, the weighted attention matrix corresponding to the fragment sample features is obtained. The interpretable independent multi-head dot product attention layer can effectively fuse the input fragment sample features and generate the attention weight matrix corresponding to the fragment sample features, and then output each head attention independently according to the attention weight matrix.

[0117] Multi-head attention is interpretable, and the interpretable head attention features of independent multi-head attention are positively correlated with the importance of the corresponding fragment sample features, which can be used to characterize the relative importance of fragment sample features.

[0118] For example, the head attention is obtained through the attention operation mechanism based on the Transform structure. Compared with the Transform structure in the related art, this embodiment makes the following improvements to the Transform structure: the residual link structure is abandoned.

[0119] In order to accurately calculate the importance of the variable, it is necessary to adjust the weight of the attention value according to the gradient of the attention matrix and the model structure. The Transformer model in the related art has a residual connection layer. Because of the stacking of the residual link and the attention layer, the weight calculation is very difficult, which makes it difficult to explain the neural network based on attention. In this embodiment, the interpretable independent multi-head dot product attention mechanism abandons the residual link, ensuring that the input data with low attention value cannot be forward propagated through the residual connection, making the data screening based on the attention mechanism more reliable. That is, after abandoning the residual link structure, the output of the attention mechanism is no longer added to the original input, thereby ensuring that only the fused features adjusted by the attention weight can participate in the subsequent calculation process of the network model.

[0120] By eliminating residual links through an interpretable independent multi-head dot product attention mechanism, the time series prediction model of this embodiment can easily filter out unnecessary sequence fragments, fundamentally reduce the length of the input sequence, and significantly reduce the computational cost of long-term prediction. In addition, the time series prediction model of this embodiment can avoid the errors caused by cyclic calculations and improve the calculation speed.

[0121] For example, refer to Figure 3 , the discretized sequence fragment samples are feature extracted through a one-dimensional convolutional layer to obtain the query vector, value vector and key vector.

[0122] Preferably, the kernel function width of the one-dimensional convolution layer is 1 and the step length is 1. Based on the kernel function with a width of 1 and a step length of 1, the query vector, value vector and key vector obtained by the one-dimensional convolution layer can correspond one-to-one with the mathematical meaning of the segment sample features obtained by the interpretable discrete feature mapping layer, and the query vector, value vector and key vector obtained based on the one-dimensional convolution layer correspond one-to-one with the sequence segment samples input into the time series prediction model.

[0123] Step 305: Obtain a first time series sample prediction result of the time series sample according to the head attention feature.

[0124] For example, the head attention feature is input into the predictor corresponding to the time series sample to obtain the first time series sample prediction result corresponding to the fragment sample feature.

[0125] For example, the predictor consists of two fully connected layers. Figure 3 shows the independent forward pass layer, Figure 3 The forward channel in constitutes the predictor in this step. Among them, each forward channel predicts the single-dimensional fragment sample features according to the input head attention features, and obtains the first time series sample prediction results corresponding to the single-dimensional fragment sample features.

[0126] The fragment sample features of all sequence fragment samples of a time series sample correspond to the same dimension. In other words, a dimensional fragment sample feature includes the fragment sample features of all sequence fragment samples belonging to the same time series sample.

[0127] For example, if the time series samples are humidity data samples within 24 hours, and the collection interval of humidity data samples is 4 hours, there are 6 humidity data samples, and the sequence length of the time series samples is 6. Figure 2 , according to the sequence length equal to 3, the time series samples with a sequence length of 6 are discretized, and 3 sequence fragment samples can be obtained. The fragment sample features of these 3 sequence fragment samples correspond to a dimension, which is humidity.

[0128] Further, see Figure 3 , Figure 3 The forward pass in is a predictor for predicting the time series prediction results. An independent forward pass layer consists of multiple independent forward passes, the number of forward passes is the same as the number of head attention features, and the forward passes and head attention features correspond one to one. The number of forward passes is the same as the dimension of the segment sample features, and the dimensions of the forward passes and segment sample features correspond one to one. For example, each independent forward pass consists of two fully connected layers.

[0129] The independent prediction channels correspond one-to-one to the head attention features and are used to predict the first time series sample prediction result of the time series samples of a single dimension according to the head attention features.

[0130] Step 306: splice the first time series sample prediction result of at least one time series sample to obtain a time series sample prediction result.

[0131] According to the fragment sample features of one dimension, the attention weight matrix of the time series samples corresponding to the fragment sample features of the dimension is obtained, and according to the attention weight matrix, the head attention features of the time series samples are obtained. The head attention features independently output by the fragment sample features of one dimension are input into the predictor (forward channel) to obtain the first time series sample prediction result of the time series sample. This embodiment does not couple the output results of all the head attention features into a single output result and then make a prediction. Instead, for the fragment sample features of one dimension, its attention weight matrix is ​​obtained separately, and then a head attention feature of the fragment sample features is obtained according to the attention weight matrix and the value vector, thereby clearly indicating the importance of the fragment sample features in the value vector to the prediction result.

[0132] Furthermore, according to the method of this embodiment, when predicting multi-dimensional time series samples, an attention weight matrix can be obtained for each dimension. The attention weight matrix can directly represent the importance of the corresponding fragment sample features and the sequence fragment samples corresponding to the fragment sample features to predicting the fragment sample features of the dimension. Based on the method of this embodiment, the attention weight matrix of all head attention can be obtained, and the importance of all sequence fragment samples to the sequence fragment samples of any dimension can be obtained through one prediction.

[0133] Reference Figure 3 After obtaining the first time series sample prediction results of the fragment sample features of at least one dimension, the first time series sample prediction results of all dimensions are spliced ​​and fused to finally obtain the multi-dimensional time series sample prediction results.

[0134] For example, the time series samples include multiple dimensions. The head attention feature obtained according to the time series samples of the first dimension is A1, the head attention feature obtained according to the time series samples of the hth dimension is Ah, and the head attention feature obtained according to the time series samples of the Hth dimension is AH. The head attention features A1 to AH are respectively input into their corresponding forward channels to obtain their respective first time series sample prediction results, and all the first time series sample prediction results are concatenated to obtain the predicted multi-dimensional time series.

[0135] Step 307: training the time series prediction model using the time series sample prediction results to obtain a trained time series prediction model.

[0136] The method of this step has been described in the aforementioned step 105 and will not be repeated here.

[0137] According to this embodiment, and Figure 3The model structure diagram shown in the figure can extract the fragment sample features of the multi-dimensional time series samples through the interpretable discrete feature mapping layer, and the intrinsic interpretability of the time series prediction model can be achieved through the fragment sample features of the multi-dimensional time series samples.

[0138] Reference Figure 3 ,The interpretable discrete feature mapping layer first divides the single-dimensional or multi-dimensional long time series samples into discrete sequence fragment samples.,The basic unit of the discrete sequence fragment samples is a data sample within a specified,length of a single dimension.

[0139] In order to preserve the context information, the data in the discretized sequence fragment samples are connected end to end. The discretization process is as follows: Figure 2 As shown, the discretization process has been described in the aforementioned embodiment and will not be repeated here.

[0140] According to the discrete sequence fragment samples, the fragment sample features and the fragment sample information are obtained, and based on the fragment sample features and the fragment sample information, the weighted attention matrix containing the fragment sample information is obtained, and the interpretability of the model is achieved according to the weighted attention matrix. The interpretability of the time series prediction model in this embodiment is achieved by generating an attention weight matrix that can measure the importance of all input time series fragment samples when predicting the column. In the attention weight matrix, the attention weight is positively correlated with the importance of the input sequence fragment sample.

[0141] Based on the intrinsic interpretability of the interpretable time series prediction model, the data flow of the time series prediction model is clear and explicit during the process of using the model for time series prediction, and the importance of the input sequence fragments can be directly measured based on the dot product attention.

[0142] The time series prediction model of this embodiment has an independent multi-head attention structure. Based on the independent multi-head attention structure, each head attention can be decoupled. Therefore, in the process of model training and prediction, it is not necessary to perform gradient correction on the time series model, and the importance of all the sequence fragment samples input when the time series sample of any dimension is predicted by the time series prediction model can be measured. The time series prediction model proposed in this embodiment can obtain the importance of all the sequence fragment samples input when predicting the time series sample of any dimension in a single prediction, and then explain the prediction of any dimension.

[0143] Furthermore, in this embodiment, the fragment sample features with clear mathematical meanings are processed by an interpretable independent multi-head dot product attention mechanism. Based on the independent multi-head attention mechanism, while effectively fusing the input fragment sample features, an attention weight matrix that characterizes the importance of each fragment sample feature in the network model can be generated. The interpretable independent multi-head dot product attention mechanism independently inputs the fused features represented by each head attention and adjusted by the attention weight into the forward channel corresponding to the time series sample one by one, so as to predict the single-dimensional time series samples through each forward channel, and finally splices all the single-dimensional prediction results (first time series sample prediction results) into the final multi-dimensional prediction results (time series sample prediction results).

[0144] The time series prediction model based on the independent multi-head attention mechanism in this embodiment can obtain data clearly and simply. Based on the time series prediction model of this structure, the obtained attention value can directly reflect the importance of the variable without complex correction.

[0145] The present application also provides a time series prediction method, referring to Figure 7 , the method may include the following steps: Step 401, obtaining the target time series to be predicted.

[0146] The target time series is obtained by sorting multiple data according to the time when the data were collected.

[0147] It is possible to obtain data collected at different times, arrange multiple data in chronological order, and form a target time series.

[0148] Step 402, discretizing the target time series to obtain a plurality of target sequence segments; The target sequence segment includes at least one data.

[0149] The method of discretizing the target time series to obtain multiple target sequence segments can refer to the description of the method of discretizing the time series samples in the aforementioned step 101, which will not be repeated here.

[0150] Step 403: Input the target sequence segment into the trained time prediction model to obtain the time series sample prediction result.

[0151] The trained time prediction model is obtained by the method for obtaining the time series prediction model of any of the aforementioned embodiments.

[0152] For example, step 403 may include the following steps: Step 4031, obtaining the fragment features of the target sequence fragment.

[0153] For example, through a one-dimensional convolutional layer, the segment features of the target sequence segment are obtained.

[0154] Sub-step 4032, screening out target segment features from multiple segment features according to the screening matrix.

[0155] For example, the screening matrix is ​​a matrix whose elements are 0 or 1, and the screening matrix and the matrix of fragment features are multiplied to obtain the target fragment features.

[0156] Sub-step 4033, obtaining a time series sample prediction result for the target time series according to the target segment features and the trained time series prediction model.

[0157] The screening matrix is ​​obtained by the screening matrix acquisition method of the above-mentioned embodiment. According to the attention mechanism of the time series prediction model, the time series sample prediction result of the target time series is obtained.

[0158] The target segment features are screened out from multiple segment features through the screening matrix, and the time series sample prediction results of the target time series are obtained based on the target segment features and the time series prediction model, which reduces the amount of data processing and improves the prediction efficiency of the model for long time series.

[0159] In this embodiment, the target time series is discretized to obtain target sequence fragments, and the target sequence fragments are input into the trained time prediction model to obtain the time series sample prediction results of the target time series. The time series prediction model obtained according to the method of the present application has the characteristics of interpretability and high accuracy of prediction results. Therefore, the time prediction results obtained based on this embodiment have the characteristics of high accuracy.

[0160] Below, taking the time series sample composed of ETT data as an example, the time series prediction model of the embodiment of the present application is further exemplified. In this embodiment, the public ETT data set is used to verify the accuracy and interpretability of the interpretable time series prediction model proposed in this application when predicting long-term time series. Among them, the ETT data set records the power system data collected from two counties within two years. The data set contains 1-hour sub-datasets: sub-dataset ETTH1 reflecting the power system data of the first county, and sub-dataset ETTH2 reflecting the power system data of the second county. The data set also includes a 15-minute sub-dataset ETTM1, which is used to reflect the power system data of one of the counties.

[0161] The ETT dataset is a seven-dimensional time series. Each dimension of the time series corresponds to a type of data, and each type of data has a corresponding collection time. For each preset time, seven types of data are collected. For each type of data, the data is arranged in chronological order to obtain a one-dimensional time series.

[0162] Specifically, each data point has six power load characteristics and a target value. The six power loads are high-voltage side effective load (High Usefull Load, HUFL), high-voltage side invalid load (High Useless Load, HULL), medium-voltage side effective load (Medium Usefull Load, MUFL), medium-voltage side invalid load (Medium Useless Load, MULL), low-voltage side effective load (Low Usefull Load, LUFL), low-voltage side invalid load (Low Useless Load, LULL), and the target value is oil temperature (Oil Temperature, OT). The time series samples are divided into training set, validation set and test set, wherein the training set, validation set and test set contain ETT data within 12 months, ETT data within 4 months and ETT data within 4 months, respectively. When training the time series model, the Adam optimizer is used to optimize the interpretable time series prediction model. Preferably, the learning rate of the Adam optimizer is set to 0.001, and the number of samples processed at a single time is 64.

[0163] When predicting time series through the forward channel, two evaluation indicators are used on each prediction window, namely Mean Squared Error (MSE) and Mean Absolute Error (MAE). Furthermore, when predicting multi-dimensional time series, the time series prediction results of each dimension are obtained respectively, and the MSE and MAE of the time series prediction results are obtained. The MSE of the time series prediction results of multiple dimensions is averaged to obtain the MSE average value of all dimensions, and the MAE of the time series prediction results of multiple dimensions is averaged to obtain the MAE average value of all dimensions, and all predictions are rolled and tested with a step size of 1. When analyzing the accuracy of the prediction results, the MSE average value of all dimensions and the MAE average value of all dimensions are used for analysis.

[0164] After obtaining an interpretable time series prediction model based on the time series prediction model training method in the aforementioned embodiment, the interpretable time series prediction model is used to perform single-dimensional and multi-dimensional long-period time series predictions on the ETT data set, respectively, and the model is compared with the following time series prediction algorithms: Autoregressive Integrated Moving Average Model (ARIMA), Prophet prediction model, Probabilistic Forecasting with Autoregressive Recurrent (DeepAR) structure, Neural Basis Expansion Analysis for Interpretable Time Series Forecasting (N-Beats) model, Informer, LogSparse Transformer (LogTrans), Reversible Transformer (Reversible Transformer, Reformer) model, Long Short Term Memory-a (LSTMa), Long and Short-term Memory Networks (LSTNet) model, Temporal Convolutional Network (Temporal Convolutional The prediction errors of the single-dimensional time series under different prediction periods in the ETTH1 and ETTH2 datasets based on each prediction model are shown in Table 1:

[0165] The prediction errors of the single-dimensional time series under different prediction periods in some ETTH2 and ETTM1 datasets are shown in Table 2:

[0166] Among them, based on each prediction model, the prediction error of the multi-dimensional time series under different prediction periods in the ETTH1 and ETTH2 data sets is shown in 3:

[0167] The prediction errors of multi-dimensional sequences under different prediction periods in some ETTH2 datasets and ETTM1 datasets are shown in Table 4:

[0168] According to the error results shown in Tables 1 to 4, among all the models, the time series prediction model obtained by this embodiment has the smallest total error and obtains the smallest error the most times. In other words, compared with other time series prediction models in the related art, the time series prediction model obtained by the method of this embodiment has a higher prediction accuracy on the basis of achieving interpretability.

[0169] In this embodiment, a prediction result is selected from the ETT data set for specific analysis. The ETT data set is a seven-dimensional time series, and the length of the time series of each dimension is 256, that is, the time series of each dimension includes 256 data, and each data corresponds to a collection time.

[0170] The time series data with a sequence length of 256 is input into the interpretable time series prediction model to predict the time series with a sequence length of 720. That is, in this embodiment, the data of 720 moments in the future time are predicted based on the data of 256 moments in the historical time period.

[0171] For example, the multidimensional time series data is discretized into a set of multiple single-dimensional sequence fragments with a sequence length of 8. The time series prediction results of the time series data of each dimension and the attention weight matrix of the time series data of each dimension are obtained respectively.

[0172] According to the time series data of the seven prediction dimensions, seven attention weight matrices can be obtained. The size of each attention weight matrix is ​​7 rows and 32 columns. The attention weight matrix is ​​used to measure the importance of the 32 sequence fragments of the seven dimensions of the input. For example, the attention weight matrix for predicting the seven-dimensional time series data is shown in Figure 8. Among them, Figure 8 (a) is the attention weight matrix of parameter 1, Figure 8 (b) is the attention weight matrix of parameter 2, Figure 8 (c) is the attention weight matrix of parameter 3, Figure 8 (d) is the attention weight matrix of parameter 4, Figure 8 (e) is the attention weight matrix of parameter 5, Figure 8 (f) is the attention weight matrix of parameter 6, and Figure 8 (g) is the attention weight matrix of parameter 7. Among them, in Figures 8 (a) to 8 (g), the horizontal axis is the preset time, and the vertical axis is the heat value of the attention weight.

[0173] Among them, parameters 1 to 7 are HUFL, HULL, MUFL, MULL, LUFL, LULL, and OT, respectively.

[0174] In Figures 8 (a) to 8 (g), the higher the brightness of the color block, the greater the attention weight corresponding to the color block, and the more important the parameter at the preset moment corresponding to the color block is to the prediction result. For example, referring to Figure 8 (a), the color block brightness of parameter 6 at the 6th preset moment is the highest, indicating that the data sample of parameter 6 at the 6th preset moment is the most important to the prediction result of parameter 1.

[0175] Furthermore, the attention weight matrix when predicting the variable OT sequence is matched with the seven-dimensional time series data to obtain the interpretability analysis result diagram of the input time series data shown in FIG9 , and Fig.10 The comparison diagram of the prediction curve and the target curve of OT is shown. Among them, Figure 9 (a) is a schematic diagram of the interpretability analysis results of parameter 1, Figure 9 (b) is a schematic diagram of the interpretability analysis results of parameter 2, Figure 9 (c) is a schematic diagram of the interpretability analysis results of parameter 3, Figure 9 (d) is a schematic diagram of the interpretability analysis results of parameter 4, Figure 9 (e) is a schematic diagram of the interpretability analysis results of parameter 5, Figure 9 (f) is a schematic diagram of the interpretability analysis results of parameter 6, and Figure 9 (g) is a schematic diagram of the interpretability analysis results of parameter 7.

[0176] Among them, Figure 9 can reflect the time series segments that play a key role in the input time series segments when the model predicts the specified time series data, as well as the time series segments that are irrelevant to or have little impact on the prediction results. Based on Figure 9, the explanation results of a prediction using the time series model can be obtained. Based on the explanation results presented in Figure 9, the operating logic and data basis of the time series prediction model can be analyzed. Among them, the horizontal axis in Figure 9 represents the time, and the vertical axis represents the thermal value corresponding to the attention weight. The brighter the color block, the greater the attention weight, and the more important the parameter is to the prediction result.

[0177] According to Figure 9, the time series prediction model assigns higher attention weights to several obvious segment features. The analysis of the time series segments corresponding to these segment features is as follows: refer to Figure 9 (a) for the trough interval of the input parameter 1 (HUFL) sequence; refer to Figure 9 (b) for the two segments where the input parameter 2 (HULL) sequence first fluctuates upward and downward; refer to Figure 9 (d) for the segment where the input parameter 4 (MULL) sequence suddenly drops; refer to Figure 9 (g) for the fluctuation period of the input parameter 7 (OT) sequence. Based on the intrinsic interpretability of the interpretable time series prediction model, the data analysis can obtain the potential data rules in the process of processing data through the time series prediction model. According to the heat map shown in Figure 9, the intrinsic interpretability of the interpretable time series prediction model is stable and reliable.

[0178] based on Fig.10It can be seen that the error between the predicted curve and the true curve obtained by the time series prediction model of this embodiment is small, indicating that the prediction result of the time series prediction model of this embodiment is highly accurate.

[0179] In this embodiment, the inherent interpretability of the time series prediction model is analyzed when a single prediction is made. The overall data can also be analyzed by statistically analyzing the attention weight matrix in all prediction experiments.

[0180] Specifically, the attention weight matrices obtained when predicting the entire data set are accumulated to obtain the global statistical attention matrix shown in Figure 11.

[0181] Figure 11 shows 7 attention weight matrices. Among them, Figure 11 (a) is the attention weight matrix of parameter 1, Figure 11 (b) is the attention weight matrix of parameter 2, Figure 11 (c) is the attention weight matrix of parameter 3, Figure 11 (d) is the attention weight matrix of parameter 4, Figure 11 (e) is the attention weight matrix of parameter 5, Figure 11 (f) is the attention weight matrix of parameter 6, and Figure 11 (g) is the attention weight matrix of parameter 7. In Figures 11 (a) to 11 (g), the horizontal axis is the time, and the vertical axis is the thermal value of the attention weight, wherein the higher the brightness of the color block, the greater the attention weight corresponding to the color block, and the more important the parameter at the preset time corresponding to the color block is to the prediction result.

[0182] The size of each attention weight matrix is ​​7 rows and 32 columns. The attention weight matrix is ​​used to measure the importance of 32 segments of 7 dimensions of the input. According to the global statistical attention weight matrix shown in Figure 11, it can be seen that the time series prediction model extracts features from equally spaced time series, which is consistent with the characteristics of time series. For example, referring to Figure 11 (a), the segments with a relatively high degree of importance of parameter 3 to the prediction result of parameter 1 are the segments with higher brightness of the color blocks in the figure. These segments are parameter 3 at the 4th moment, the 15th moment, and the 23rd moment, respectively. The importance changes periodically, and the change period is 9, indicating that the change of attention weight is related to the extraction time of the segment sample features, and it shows periodicity with the change of the extraction time.

[0183] In order to facilitate the evaluation of the relationship between each data type, the total attention weight of each data type when predicting the specified dimension is counted one by one, and the statistical results are shown in Figure 12. Among them, Figure 12 (a) is the importance of each parameter to parameter 1, Figure 12 (b) is the importance of each parameter to parameter 2, Figure 12 (c) is the importance of each parameter to parameter 3, Figure 12 (d) is the importance of each parameter to parameter 4, Figure 12 (e) is the importance of each parameter to parameter 5, Figure 12 (f) is the importance of each parameter to parameter 6, and Figure 12 (g) is the importance of each parameter to parameter 7. In Figures 12 (a) to 12 (g), the horizontal axis is the time and the vertical axis is the thermal value of the attention weight.

[0184] According to the statistical results shown in Figure 12, we can specifically analyze the role of various data in predicting a certain data sequence. For example, referring to Figure 12 (g), the attention weight heat values ​​corresponding to parameters 3 (MUFL), 4 (MULL), and 7 (OT) are the largest. According to Figure 12 (g), the variables MUFL, MULL, and OT are most important for predicting OT. In other words, the prediction of variable OT mainly depends on variables MUFL, MULL, and OT. The statistical results shown in Figure 12 provide an important reference for further exploring the internal relationship of the data set.

[0185] Based on the intrinsic interpretability of the time series forecasting model, the relationship between data types can be analyzed qualitatively and quantitatively. In addition to providing a reference for data analysis, intrinsic interpretability can also effectively screen data.

[0186] Specifically, from the discretized set of time series fragments, a subset of input sequence fragments with high attention weights is selected, and time series prediction is performed based on the subset of input sequence fragments with high attention weights. This can fundamentally reduce the length of the input time series, thereby significantly reducing the computational cost of long-term predictions. Based on the inherent data screening function of the time series prediction model, the time series prediction model can reduce the computational cost when predicting long-period series. In addition, based on the discretized data structure within the time series prediction model, the negative impact of shielding data can be minimized.

[0187] For example, in the global statistical attention weight matrix of each dimension of the prediction, the most critical specified number of sequence fragment positions are obtained, the weights of these positions are set to 1, and the weights of the remaining positions are set to 0 to obtain the screening matrix. When filtering data according to the screening matrix, the sequence fragments at the positions with a weight of 1 are retained, and the fragments at the positions with a weight of 0 are no longer input into the prediction model.

[0188] By filtering the data, retraining and testing the prediction model, the final time series prediction model is obtained. Compared with the original time series prediction model, the computational cost of the prediction model based on the filtered data is significantly reduced. Fig.13 The figure shows the screening matrix when screening the original 7×32 sequence fragments of the dataset ETTH2. The horizontal axis is the time, and the vertical axis is the thermal value of the attention weight. Among them, the color block with high brightness has a large weight, and the color block with low brightness has a small weight. For example, the weight corresponding to the color block with high brightness is 1, and the weight corresponding to the color block with low brightness is 0.

[0189] according to Fig.13 It can be seen that when predicting the 7 dimensions corresponding to the 7 parameters 1 to 7, parameter 1 is most important at the 7th moment, the 9th moment, the 12th moment, the 18th moment, the 20th moment, the 23rd moment, and the 29th to the 31st moment. When performing time series prediction, the fragment sample features of these moments of parameter 1 are screened out, the time series prediction samples are trained, and in the subsequent model inference process, the time series data at these moments are used to perform time series prediction.

[0190] In this embodiment, the interpretable time series prediction model relies only on the screened data for long-term series prediction, which is marked as the interpretable time series prediction model (screening). The interpretable time series prediction model relies on 24-length multidimensional data for long-term series prediction, which is marked as the interpretable time series prediction model (short input). The prediction errors of different time series prediction models are obtained, and the prediction errors are shown in Table 3:

[0191] According to Table 5, the prediction error of the time series prediction model based on the filtered data is very close to the prediction error of the original time series prediction model, and is much smaller than the prediction error when only 24-length data is input. However, the computational cost of the prediction model based on the filtered data is smaller than that of the model with only 24-length data input.

[0192] According to the prediction error comparison results shown in Table 5, by screening the prediction model data, the accuracy of the prediction results can be improved while ensuring the interpretability of the time series model. In other words, the interpretable time series prediction model obtained in this embodiment can use the data screening function to predict long-term time series at a low cost, and the prediction results are highly accurate.

[0193] In the related art, the SCINet model uses a fully convolutional network without a dot product attention structure and has low computational cost. However, the SCINet model is not interpretable and has large prediction errors when predicting long-term sequences. The AuToFormer model uses a dot product attention structure, but the computational cost of the dot product attention structure is proportional to the square of the input data length.

[0194] The maximum data length of the time series prediction model obtained in this application is , the maximum data length of the AuToFormer model input dot product attention structure .

[0195] in, Indicates the time length of the input data, represents the length of the forecast time, represents the dimension of the original data, Represents the unit time length of the proposed interpretable time series forecasting model.

[0196] The maximum data length of the time series prediction model obtained in the present application is smaller than the maximum data length of the input dot product attention structure of the AuToFormer model. Therefore, compared with the AuToFormer model, the time series prediction model of this embodiment has low computational consumption.

[0197] The time series prediction model of this application has a data screening function, which can shorten the length of input data to a very short length, greatly reducing the computational cost. Therefore, when predicting long-term series, the computational complexity of the time series prediction model of this application is significantly lower than that of the AuToFormer model, and data screening can even be implemented according to demand, compressing the computational cost to the extreme.

[0198] In an embodiment of the present application, a new interpretable long-period time series prediction model is built through discrete feature mapping, interpretable independent multi-head dot product attention and independent forward channel; the proposed interpretable long-period time series prediction model surpasses many current excellent sequence prediction algorithms in terms of time series prediction accuracy; the proposed interpretable long-period time series prediction model gives full play to its interpretable advantages, performs importance analysis on the input data, and reveals the reasons for the model output prediction results; according to the results of the importance analysis of the input data by the proposed interpretable long-period time series prediction model, the screening of the original data is realized, and a method is proposed to fundamentally reduce the computational complexity of the time series prediction algorithm.

[0199] This invention is an exploration and improvement of the Transformer structure for time series prediction, which can provide an important basic technology for the improvement of large language models in the future. For example, the improvements of the Transformer structure in the present invention for time series prediction tasks can be applied to a large language model to achieve an interpretable large language model with improved performance.

[0200] Fig.14 is a schematic diagram of a time series prediction model training device provided by an embodiment of the present disclosure, referring to Fig.14 , the device 50 comprises: The first acquisition module 501 is used to discretize the time series samples to obtain a plurality of sequence segment samples; The second acquisition module 502 is used to acquire the fragment sample characteristics and fragment sample information of the sequence fragment samples; The third acquisition module 503 is used to obtain the attention weight matrix of the time series sample according to the attention operation mechanism of the time series prediction model, the fragment sample characteristics, and the fragment sample information; The fourth acquisition module 504 is used to obtain the time series sample prediction result according to the attention weight matrix; The fifth acquisition module 505 is used to train the time series prediction model through the time series sample prediction results to obtain a trained time series prediction model.

[0201] Optionally, the fourth acquisition module 504 may include: A first acquisition submodule is used to obtain a target attention weight greater than or equal to a preset weight threshold from the attention weight matrix; The second acquisition submodule is used to acquire the target sequence segment sample feature corresponding to the target attention weight from the multiple segment sample features; The third acquisition submodule is used to input the sample features of the target sequence fragment into the time series prediction model to obtain the time series sample prediction result.

[0202] Optionally, the second acquisition submodule includes: A first configuration unit, configured to configure a first weight for a target attention weight, and to configure a second weight for other attention weights except the target attention weight; A first construction unit, configured to construct a screening matrix according to the first weight, the fragment sample information of the fragment sample feature corresponding to the first weight, the second weight, and the fragment sample information of the fragment sample feature used to calculate other attention weights; The first screening unit is used to screen out target sequence fragment sample features from a plurality of fragment sample features according to a screening matrix.

[0203] Optionally, the time series sample has at least one; the first acquisition module 501 may include: The fourth acquisition submodule is used to discretize each time series sample to obtain at least one group of sequence segment samples; The second acquisition module 502 includes: The fifth acquisition submodule is used to extract features from at least one group of sequence fragment samples to obtain fragment sample features of the sequence fragment samples.

[0204] Optionally, the fourth acquisition module 504 includes: The sixth acquisition submodule is used to obtain the head attention feature of the time series sample according to the attention weight matrix of the time series sample; a seventh acquisition submodule, for obtaining a first time series sample prediction result of the time series sample according to the head attention feature; The eighth acquisition submodule is used to splice the first time series sample prediction result of at least one time series sample to obtain the time series sample prediction result.

[0205] Optionally, the sixth acquisition submodule includes: The first acquisition unit is used to perform dot product attention calculation on the attention weight matrix of the time series sample and the value vector of the fragment sample feature to obtain the head attention feature of the time series sample; wherein the value vector is obtained by convolution processing of the fragment sample feature.

[0206] Optionally, the seventh acquisition submodule includes: The second acquisition unit is used to input the head attention feature into a predictor corresponding to the time series sample to obtain a first time series sample prediction result corresponding to the fragment sample feature.

[0207] Optionally, the first acquisition module 501 includes: The ninth acquisition submodule is used to discretize the time series samples according to a preset sequence length to obtain sequence segment samples, wherein, in two adjacent sequence segment samples, the first number of data samples located at the end of the first sequence segment sample is the same as the first number of data samples located at the beginning of the second sequence segment sample.

[0208] Optionally, the sample information includes at least one of the following: a sample identifier of a sequence fragment sample, and time information of the sequence fragment sample.

[0209] Optionally, the third acquisition module 503 includes: The tenth acquisition submodule is used to perform convolution processing on the features of the fragment samples to obtain a query vector and a key vector; The tenth acquisition submodule is used to obtain an attention weight matrix according to the query vector and the key vector, as well as the fragment sample information.

[0210] In summary, in this embodiment, based on the attention weights in the attention weight matrix and the sample information of the fragment sample features used to calculate the attention weights, the importance of each sequence fragment sample in the input time series prediction model to the prediction result can be quantified, and the importance of the sequence fragment samples corresponding to the sample information to the time series prediction result can be clearly obtained. In other words, based on the attention weight matrix containing the sample information of the fragment sample features used to calculate the attention weights, the model interpretation of the time prediction model is realized, and the trained time series prediction model obtained based on the attention weight matrix has the interpretability to predict long-term time series.

[0211] Fig.15 is a schematic diagram of a time series prediction device provided by an embodiment of the present disclosure, referring to Fig.15 , the device 60 comprises: The sixth acquisition module 601 is used to acquire a target time series to be predicted; The seventh acquisition module 602 is used to discretize the target time series to obtain multiple target sequence segments; The eighth acquisition module 603 is used to input the target sequence segment into the trained time prediction model to obtain the time series prediction result; wherein the trained time prediction model is obtained by the time series prediction model acquisition method in the above-mentioned embodiment.

[0212] For example, the eighth acquisition module 603 includes: A twelfth acquisition submodule is used to acquire the fragment features of the target sequence fragment; A thirteenth acquisition submodule is used to screen out a target fragment feature from a plurality of fragment features according to a screening matrix; A fourteenth acquisition submodule is used to obtain a time series prediction result of a target time series according to the target segment characteristics and the trained time series prediction model; The screening matrix is ​​obtained by the screening matrix acquisition method of the aforementioned embodiment.

[0213] In this embodiment, in this embodiment, the target time series is discretized to obtain target sequence fragments, and the target sequence fragments are input into the trained time prediction model to obtain a time series prediction result for the target time series. The time series prediction model obtained according to the method of the present application has the characteristics of interpretability and high accuracy of prediction results. Therefore, the time prediction result obtained based on this embodiment has the characteristic of high accuracy.

[0214] The present disclosure also provides an electronic device, such as Fig.16As shown, it includes a processor 1101 , a communication interface 1102 , a memory 1103 and a communication bus 1104 , wherein the processor 1101 , the communication interface 1102 , and the memory 1103 communicate with each other via the communication bus 1104 .

[0215] The memory 1103 is used to store computer programs.

[0216] When the processor 1101 is used to execute the program stored in the memory 1103, the following steps are implemented: obtaining time series samples, and discretizing the time series samples to obtain sequence fragment samples; obtaining fragment sample features and fragment sample information of the sequence fragment samples; obtaining the attention weight matrix of the time series samples according to the attention operation mechanism, fragment sample features, and fragment sample information of the time series prediction model; the attention weight matrix includes the fragment sample information; obtaining the time series sample prediction results according to the attention weight matrix; training the time series prediction model through the time series sample prediction results to obtain the trained time series prediction model.

[0217] Among them, the processor 1101 can also implement other steps in the above-mentioned time series prediction model training method or time series prediction method, which will not be repeated here.

[0218] The communication bus mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The communication bus can be divided into an address bus, a data bus, and a control bus. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0219] The communication interface is used for communication between the above electronic device and other devices.

[0220] The memory may include a random access memory (RAM) or a non-volatile memory, such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0221] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP); it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0222] In another embodiment provided by the present disclosure, a computer-readable storage medium is also provided, which stores instructions. When the computer-readable storage medium is executed on a computer, the computer executes the time series prediction model training method or the time series prediction method in the above-mentioned embodiment.

[0223] In another embodiment provided by the present disclosure, a computer program product including instructions is also provided, which, when executed on a computer, enables the computer to execute the time series prediction model training method or the time series prediction method in the above-mentioned embodiment.

[0224] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on the computer, the process or function according to the embodiment of the present disclosure is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave) mode. The computer-readable storage medium can be any available medium that can be accessed by the computer or a server, data center or other data storage device that contains one or more available media integration. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive Solid State Disk (SSD)).

[0225] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0226] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. For the embodiments of the device, electronic device, computer-readable storage medium and computer program products containing instructions, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0227] The above are only preferred embodiments of the present disclosure and are not intended to limit the protection scope of the present disclosure. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present disclosure are included in the protection scope of the present disclosure.

Claims

1. A time series prediction model training method, characterized in that: include: Acquire time series samples, and discretize the time series samples to obtain sequence fragment samples; Acquire fragment sample features and fragment sample information of the sequence fragment samples; Obtaining an attention weight matrix of the time series sample according to the attention operation mechanism of the time series prediction model, the segment sample features, and the segment sample information; Obtaining a time series sample prediction result according to the attention weight matrix; The time series prediction model is trained by using the time series sample prediction results to obtain a trained time series prediction model.

2. The method according to claim 1, characterized in that According to the attention weight matrix, the time series sample prediction result is obtained, including: Obtaining a target attention weight greater than or equal to a preset weight threshold from the attention weight matrix; Obtaining a target segment sample feature corresponding to the target attention weight from a plurality of segment sample features; The target segment sample features are input into the time series prediction model to obtain a time series sample prediction result.

3. The method according to claim 2, characterized in that Obtaining a target segment sample feature corresponding to the target attention weight from the plurality of segment sample features, including: Configuring a first weight for the target attention weight, and configuring a second weight for other attention weights except the target attention weight; constructing a screening matrix according to the first weight, the fragment sample information of the fragment sample feature corresponding to the first weight, the second weight, and the fragment sample information of the fragment sample feature used to calculate other attention weights; The target segment sample feature is screened out from a plurality of segment sample features according to the screening matrix.

4. The method according to claim 1, characterized in that: The time series sample has at least one; the time series sample is discretized to obtain a sequence fragment sample, including: Discretizing each of the time series samples to obtain at least one group of sequence fragment samples; The obtaining of the fragment sample features of the sequence fragment samples comprises: Feature extraction is performed on at least one group of the sequence fragment samples to obtain fragment sample features of the sequence fragment samples.

5. The method according to claim 1, characterized in that: The time series sample has at least one, and obtaining a prediction result of the time series sample according to the attention weight matrix includes: According to the attention weight matrix of the time series sample, a head attention feature of the time series sample is obtained; Obtaining a first time series sample prediction result of the time series sample according to the head attention feature; The first time series sample prediction result of at least one of the time series samples is concatenated to obtain the time series sample prediction result.

6. The method according to claim 5, characterized in that According to the attention weight matrix of the time series sample, the head attention feature of the time series sample is obtained, including: Performing dot product attention calculation on the attention weight matrix of the time series sample and the value vector of the segment sample feature to obtain the head attention feature of the time series sample; The value vector is obtained by performing convolution processing on the features of the fragment samples.

7. The method according to claim 5, characterized in that Obtaining a first time series sample prediction result of the time series sample according to the head attention feature, including: The head attention feature is input into a predictor corresponding to the time series sample to obtain a first time series sample prediction result corresponding to the segment sample feature.

8. The method according to claim 1, characterized in that Discretization processing is performed on the time series samples to obtain sequence fragment samples, including: According to a preset sequence length, the time series samples are discretized to obtain sequence fragment samples; Among two adjacent sequence segment samples, a first number of data samples located at the end of a first sequence segment sample is the same as a first number of data samples located at the beginning of a second sequence segment sample.

9. The method according to claim 1, characterized in that: The sample information includes at least one of the following: a sample identifier of the sequence fragment sample, and time information of the sequence fragment sample.

10. The method according to claim 1, characterized in that According to the attention operation mechanism of the time series prediction model, the characteristics of the fragment samples, and the fragment sample information, an attention weight matrix of the time series samples is obtained, including: Performing convolution processing on the features of the fragment samples to obtain a query vector and a key vector; An attention weight matrix of the time series sample is obtained according to the query vector, the key vector, and the fragment sample information.

11. A time series prediction method, characterized in that: include: Get the target time series to be predicted; Discretize the target time series to obtain target sequence fragments; The target sequence segment is input into a trained time prediction model to obtain a time series prediction result; wherein the trained time prediction model is obtained by the time series prediction model training method according to any one of claims 1 to 10.

12. The method according to claim 11, characterized in that Inputting the target sequence fragment into the trained time prediction model to obtain the time series prediction result includes: Acquiring fragment features of the target sequence fragment; Screening out a target fragment feature from the plurality of fragment features according to a screening matrix; Obtaining a time series prediction result of a target time series according to the target segment features and the trained time series prediction model; Wherein, the screening matrix is ​​obtained by the method of claim 3.

13. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor, configured to execute the computer program to implement the method according to any one of claims 1 to 12.

14. A computer-readable storage medium, characterized in that: Used to store a computer program; wherein when the computer program is executed by a processor, the method according to any one of claims 1 to 12 is implemented.

15. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the method according to any one of claims 1 to 12 is implemented.

Citation Information

Patent Citations

  • Power load prediction method based on TPA-Seq2Seq, and related components

    CN111553543A

  • Time sequence prediction method based on A2form

    CN117574958A

  • Multivariable time sequence prediction method based on spatio-temporal embedding independence

    CN118446356A

  • Photovoltaic power prediction method and system based on multi-characteristic variable and time sequence correlation

    CN118734079A