Time sequence prediction method and device, equipment, storage medium and program product

Abstract

The invention discloses a time sequence prediction method and device, equipment, a storage medium and a program product, and the method comprises the steps: obtaining an input time sequence of an input data set, carrying out the preprocessing of the input time sequence, obtaining an initial time sequence, carrying out the feature analysis of each data point in the input data set, and obtaining a time sequence; the method comprises the steps that input sequence features and prediction sequence features are obtained, attention analysis is conducted on the input sequence features and an initial time sequence through a pre-constructed attention analysis module, a target attention score is obtained, and the attention analysis module is a neural network module packaged with a plurality of attention layers; according to the method, time sequence prediction is performed on the input data set based on the prediction sequence features and the target attention score to obtain a time sequence prediction result, so that the feature capture capability of complex data is effectively enhanced, the accuracy of time sequence prediction is greatly improved, and the prediction complexity is effectively reduced.

Description

Technical Field

[0001] The present invention relates to the field of data processing, and in particular to a time series prediction method, device, equipment, storage medium and program product. Background Art

[0002] In recent years, research on time series forecasting technology has mainly focused on using deep learning models to capture complex patterns and trends in data. Existing time series forecasting methods have multiple technical shortcomings, which limit their scope of application and prediction accuracy. First, many traditional methods rely on the assumption of stationarity of historical data, but actual data are often non-stationary, such as seasonal changes and trend changes, resulting in poor prediction results. Secondly, when processing high-dimensional data, traditional methods such as ARIMA and exponential smoothing face the "curse of dimensionality". The computational complexity and number of parameters increase exponentially with the increase in dimensions, resulting in long training time or even failure to converge. In addition, multivariate time series forecasting needs to consider internal relationships and mutual influences, but traditional methods often ignore complex interactions, reducing prediction accuracy. Summary of the invention

[0003] The main purpose of the present invention is to provide a time series prediction method, device, equipment, storage medium and program product, aiming to solve the technical problems of low accuracy and high computational complexity of time series prediction in the prior art.

[0004] To achieve the above object, the present invention provides a time series prediction method, which comprises the following steps: Obtaining an input time series of an input data set, and preprocessing the input time series to obtain an initial time series; Performing feature analysis on each data point in the input data set to obtain input sequence features and predicted sequence features; Performing attention analysis on the input sequence features and the initial time series through a pre-built attention analysis module to obtain a target attention score, wherein the attention analysis module is a neural network module encapsulating multiple attention layers; Based on the prediction sequence features and the target attention score, time series prediction is performed on the input data set to obtain a time series prediction result.

[0005] Optionally, performing feature analysis on each data point in the input data set to obtain input sequence features and predicted sequence features includes: Perform feature analysis according to the timestamp of each data point in the input data set to obtain time feature information of the input data set; Performing feature enhancement on the time feature information through a pre-built feature analysis model to obtain candidate feature information; Performing dimension mapping on the candidate feature information by using the feature analysis model to obtain dimension mapping feature information; Normalizing the dimension mapping feature information to obtain target feature information; The target feature information is divided to obtain input sequence features and predicted sequence features.

[0006] Optionally, the acquiring an input time series of an input data set and preprocessing the input time series to obtain an initial time series includes: Get the input time series of the input dataset; Compute the mean and standard deviation of the input time series: in, represents the mean value of the input time series, L is the length of the input time series, is the first elements, Represents the standard deviation of the input time series; The input time series is standardized based on the mean value and standard deviation to obtain an initial time series: in, represents the initial time series obtained after standardization, Indicates a preset constant.

[0007] Optionally, the attention analysis module includes a convolutional attention analysis module, a multi-head attention analysis module and a feedforward network module; the pre-built attention analysis module is used to perform attention analysis on the input sequence features and the initial time series to obtain a target attention score, including: Performing linear projection on the input sequence features and the initial time series through a convolutional attention analysis module to obtain a query projection matrix, a key projection matrix, and a value projection matrix; Calculate the initial attention score based on the query projection matrix and the key projection matrix: in, represents the initial attention score, represents the query projection matrix, represents the key projection matrix, represents the matrix transpose, Represents feature dimension; Processing the initial attention score by the convolutional attention analysis module and the multi-head attention analysis module to obtain a candidate attention score; The candidate attention scores are nonlinearly changed through the feedforward network module to obtain a target attention score.

[0008] Optionally, the processing the initial attention score by the convolutional attention analysis module and the multi-head attention analysis module to obtain a candidate attention score includes: The initial attention score is normalized by the convolutional attention analysis module to obtain the attention weight: in, represents the attention weight, express function, represents the initial attention score; The value projection matrix is ​​weighted summed based on the attention weight to obtain a context vector: in, represents the context vector, Represents the value projection matrix; The context vector is subjected to residual connection and layer normalization processing to obtain a first output result of the input sequence feature and a second output result of the initial time series: in, The first output result representing the input sequence features, represents the second output result of the initial time series, represents the input sequence features, represents the initial time series, Representation layer normalization function, Represents the residual connection between the input sequence features and the context vector, Represents the residual connection between the initial time series and the context vector; The first output result and the second output result are linearly transformed by the multi-head attention analysis module to obtain a target query vector and a target key vector: in, represents the target query vector, represents the target key vector, represents the linear transformation function of the query vector, represents the linear transformation function of the key vector; Determine candidate attention scores based on the target query vector and the target key vector: in, represents the candidate attention score, Represents the dimension scaling factor.

[0009] Optionally, performing time series prediction on the input data set based on the prediction sequence feature and the target attention score to obtain a time series prediction result includes: Performing feature enhancement processing on the predicted sequence features, and generating an initial predicted sequence based on the predicted sequence features after the feature enhancement processing; Mapping the initial prediction sequence to a dimension corresponding to the target attention score to obtain a candidate prediction sequence; Performing matrix multiplication of the target attention score and the candidate prediction sequence to obtain an initial prediction result; The initial prediction result is dimensionally transformed, and the initial prediction result after dimension transformation is de-standardized to obtain a time series prediction result.

[0010] In addition, to achieve the above-mentioned purpose, the present invention also proposes a time series prediction device, which includes: A data processing module is used to obtain an input time series of an input data set, and preprocess the input time series to obtain an initial time series; A feature analysis module is used to perform feature analysis on each data point in the input data set to obtain input sequence features and predicted sequence features; An attention analysis module, used to perform attention analysis on the input sequence features and the initial time series through a pre-built attention analysis module to obtain a target attention score, wherein the attention analysis module is a neural network module encapsulating multiple attention layers; A time series prediction module is used to perform time series prediction on the input data set based on the prediction sequence features and the target attention score to obtain a time series prediction result.

[0011] In addition, to achieve the above-mentioned purpose, the present application also proposes a time series prediction device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the time series prediction method described above.

[0012] In addition, to achieve the above-mentioned purpose, the present application also proposes a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the time series prediction method described above are implemented.

[0013] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the steps of the time series prediction method described above are implemented.

[0014] The present invention obtains an input time series of an input data set, preprocesses the input time series, obtains an initial time series, performs feature analysis on each data point in the input data set, obtains input sequence features and predicted sequence features, performs attention analysis on the input sequence features and the initial time series through a pre-built attention analysis module, and obtains a target attention score. The attention analysis module is a neural network module encapsulated with multiple attention layers. Time series prediction is performed on the input data set based on the predicted sequence features and the target attention score to obtain a time series prediction result. Since the present invention adaptively focuses on key information in the sequence based on the attention mechanism, while reducing computational complexity and enhancing the feature capture capability of complex data, it effectively integrates the attention score and predicted sequence features, generates a final prediction result, and effectively improves the accuracy of time series prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The accompanying drawings herein are incorporated in and constitute a part of the specification, illustrate embodiments consistent with the present application, and together with the description, are used to explain the principles of the present application.

[0016] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0017] Figure 1 It is a schematic diagram of the structure of a time series prediction device in a hardware operating environment involved in an embodiment of the present invention; Figure 2 It is a schematic diagram of the flow chart of the first embodiment of the time series prediction method of the present invention; Figure 3 It is a schematic diagram of the flow chart of the second embodiment of the time series prediction method of the present invention; Figure 4 It is a structural block diagram of the first embodiment of the time series prediction device of the present invention.

[0018] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0019] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0020] Reference Figure 1 , Figure 1 It is a schematic diagram of the structure of a time series prediction device in a hardware operating environment involved in an embodiment of the present invention.

[0021] like Figure 1 As shown, the time series prediction device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a wireless fidelity (Wireless-Fidelity, WI-FI) interface). The memory 1005 may be a high-speed random access memory (RandomAccess Memory, RAM), or a stable non-volatile memory (Non-Volatile Memory, NVM), such as a disk memory. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0022] Those skilled in the art will understand that Figure 1 The structure shown in does not constitute a limitation on the time series prediction device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0023] like Figure 1 As shown, the memory 1005 as a computer-readable storage medium may include an operating system, a network communication module, a user interface module, and a time series prediction program.

[0024] exist Figure 1In the time series prediction device shown, the network interface 1004 is mainly used for data communication with the network server; the user interface 1003 is mainly used for data interaction with the user; the processor 1001 and the memory 1005 in the time series prediction device of the present invention can be set in the time series prediction device, and the time series prediction device calls the time series prediction program stored in the memory 1005 through the processor 1001, and executes the time series prediction method provided by the embodiment of the present invention.

[0025] The embodiment of the present invention provides a time series prediction method, referring to Figure 2 , Figure 2 Schematic diagram of the flow chart of the first embodiment of the time series prediction method of the present invention.

[0026] In this embodiment, the time series prediction method includes the following steps: Step S10: obtaining an input time series of an input data set, and preprocessing the input time series to obtain an initial time series.

[0027] It should be noted that the execution subject of this embodiment can be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or a terminal electronic device capable of realizing the above functions, etc. The following takes a time series prediction device (referred to as the prediction device) as an example to illustrate this embodiment and the following embodiments.

[0028] It should be noted that the input data set may be a data set with time characteristics that needs to be predicted for time series, for example, the input data set may be meteorological data, traffic data, financial data, etc. The above-mentioned input time series may be a time series generated based on the time characteristic information of each data point in the input data set. The above-mentioned preprocessing may include data cleaning, dimension mapping, feature enhancement, normalization processing, etc.

[0029] It can be understood that the prediction device can obtain the timestamp information of each data point in the input data set, perform time feature analysis on the data point based on the timestamp information, obtain the event feature information of the input data set, generate an input time series based on the time feature information, preprocess the input time series, and obtain an initial time series.

[0030] Furthermore, in order to improve the analysis efficiency, the above step S10 may include: Get the input time series of the input dataset; Calculate the mean and standard deviation of the input time series; The input time series is standardized based on the mean value and the standard deviation to obtain an initial time series.

[0031] It should be noted that the prediction device can calculate the average of all data in the input data set: The prediction device calculates the standard deviation of each data point based on the average value. The standard deviation is the average distance of each data point from the average value. The larger the standard deviation, the greater the fluctuation of the data: in, represents the mean value of the input time series, is the length of the input time series, is the first elements, Represents the standard deviation of the input time series.

[0032] In some embodiments, the prediction device inputs the time series Normalize to have a mean of 0 and a standard deviation of 1: in, represents the initial time series obtained after standardization, Represents a preset constant, Can be a very small constant to avoid division by zero.

[0033] Step S20: performing feature analysis on each data point in the input data set to obtain input sequence features and predicted sequence features.

[0034] In some embodiments, for each data point in the input data set, the prediction device may extract time features based on the timestamp of each data point as feature information (mark information) to assist in input, and to provide additional information or context to improve time series prediction performance.

[0035] In some embodiments, the prediction device may perform feature analysis on each data point in the input data set, obtain time feature information of the input data set, and divide the time feature information into input sequence features and prediction sequence features according to the prediction ratio.

[0036] Furthermore, in order to improve the accuracy of timing prediction, the above step S20 may include: Step S201: performing feature analysis according to the timestamp of each data point in the input data set to obtain time feature information of the input data set; Step S202: performing feature enhancement on the time feature information through a pre-built feature analysis model to obtain candidate feature information; Step S203: performing dimension mapping on the candidate feature information through the feature analysis model to obtain dimension mapping feature information; Step S204: normalizing the dimension mapping feature information to obtain target feature information; Step S205: Divide the target feature information to obtain input sequence features and predicted sequence features.

[0037] It should be noted that the above feature analysis model can be KAN Model, prediction equipment can pass time feature information through KAN The model extracts additional latent features and maps them to higher-dimensional feature representations, and normalizes them to obtain the final feature information.

[0038] In some embodiments, the prediction device may KAN The model is initialized with Dimensional input and Dimensional output KAN A layer can be defined as a matrix of one-dimensional functions, referring to the following formula: in, express KAN The matrix of the layer, Indicates the number of dimensions of the input features, Indicates the number of dimensions of the output features, represents the index of the input dimension, Indicates the index of the output dimension.

[0039] can be viewed as a set of mappings from input space to output space, each The function is responsible for converting specific input features Mapping to specific output features ,in The function has trainable parameters. In the Kolmogorov-Arnold representation theorem, the inner function forms a KAN Layer, where and , the external function forms a KAN Layer, where and .

[0040] In some embodiments, the following integer array may be used to represent KAN The shape of the model: in, It is the computational graph The number of nodes in the layer, Used to indicate the The first neurons, and use express The activation value of the neuron. Layer and There are activation function, connection and The activation function is expressed as: In some embodiments, the prediction device may KAN Introducing a basis function , so that the activation function is the basis function and spline functions The sum of , refer to the following formula: The basis functions include function, Functions are used to improve the performance of neural networks. The function combines linear and nonlinear properties and is used as an activation function in neural networks. The function is close to 0 when the input value is small, and close to the input value itself when the input value is large.

[0041] In some embodiments, the spline function is parameterized as a linear combination of B-splines, namely: in, It is trainable. and Can be absorbed and spline functions, but using these factors gives you more control over the overall magnitude of the activation function.

[0042] The pre-activation value can be , The post-activation value is Indicates that . No. The activation value of a neuron is the sum of all incoming activation values: In matrix form this can be expressed as: in, It is the first KAN The function matrix corresponding to the layer.

[0043] In some embodiments, KAN The model can be Combination of layers, given an input vector , KAN The output is: Restating the above equation, assuming the output dimension is ， And define : Step S30: Performing attention analysis on the input sequence features and the initial time series through a pre-built attention analysis module to obtain a target attention score.

[0044] It should be noted that the attention analysis module is a neural network module encapsulated with multiple attention layers.

[0045] In some embodiments, the attention analysis module can be a SICM module, and the prediction device can build an efficient SICM module by fusing convolutional attention, feedforward / Neural / Network (FNN) and multi-head attention mechanism, aiming to improve the accuracy and robustness of time series prediction.

[0046] In some embodiments, the prediction device may construct a SICM module that integrates convolutional attention, FNN, and multi-head attention mechanisms. The multi-head attention mechanism in the SICM module captures local features and long-distance dependencies through local convolution operations and global attention mechanisms, respectively, thereby more comprehensively extracting features from time series data.

[0047] Step S40: Perform time series prediction on the input data set based on the prediction sequence features and the target attention score to obtain a time series prediction result.

[0048] In some embodiments, the prediction device may KAN The module (Kolmogorov-Arnold Networks) extracts additional implicit features from the prediction sequence features and maps them to the dimensions corresponding to the attention scores to obtain the basic prediction sequence. The target attention score is matrix multiplied with the basic prediction sequence to generate the time series prediction results.

[0049] In some embodiments, the prediction device may be combined with KANThe algorithm's adaptive grid and spline interpolation techniques, as well as the multi-head attention mechanism and feedforward neural network in the SICM model, significantly enhance the model's feature extraction and nonlinear transformation capabilities. Specifically, KAN The algorithm enhances the expressiveness of the model by dynamically adjusting the internal grid to better match the data distribution; spline interpolation generates richer feature representations and improves the ability to capture complex patterns.

[0050] Furthermore, in order to effectively capture the implicit features of the data and improve the prediction performance, the above step S40 may include: Step S401: performing feature enhancement processing on the prediction sequence features, and generating an initial prediction sequence based on the prediction sequence features after feature enhancement processing; Step S402: Mapping the initial prediction sequence to a dimension corresponding to the target attention score to obtain a candidate prediction sequence; Step S403: performing matrix multiplication on the target attention score and the candidate prediction sequence to obtain an initial prediction result; Step S404: performing dimension transformation on the initial prediction result, and performing de-standardization processing on the initial prediction result after dimension transformation to obtain a time series prediction result.

[0051] In some embodiments, the prediction device may KAN The module integrates existing features and extracts additional implicit features from the input dataset, maps them to dimensions corresponding to the attention scores, obtains candidate prediction sequences, performs matrix multiplication on the target attention scores and candidate prediction sequences, and generates initial prediction results: in, represents the candidate prediction sequence, , represents the length of the input time series, Represents the number of independent attention calculation units running in parallel in the multi-head attention mechanism. Each independent attention calculation unit corresponds to an independent attention function. represents the initial prediction result, .

[0052] Predicted equipment passing KAN The module maps the initial prediction results to Dimension. After the initial prediction results are processed, they need to be restored to the scale corresponding to the input dataset, which can be done by denormalization: In the above formula, Is the input data The mean of yes The standard deviation of is a very small value, and finally predict The dimension is .

[0053] This embodiment obtains an input time series of an input data set, preprocesses the input time series, obtains an initial time series, performs feature analysis on each data point in the input data set, obtains input sequence features and predicted sequence features, performs attention analysis on the input sequence features and the initial time series through a pre-built attention analysis module, and obtains a target attention score. The attention analysis module is a neural network module encapsulated with multiple attention layers. Time series prediction is performed on the input data set based on the predicted sequence features and the target attention score to obtain a time series prediction result. Since this embodiment adaptively focuses on key information in the sequence based on the attention mechanism, while reducing computational complexity and enhancing the ability to capture features of complex data, it effectively integrates attention scores and predicted sequence features, generates a final prediction result, and effectively improves the accuracy of time series prediction.

[0054] refer to Figure 3 , Figure 3 Schematic diagram of the flow chart of the second embodiment of the time series prediction method of the present invention.

[0055] Based on the above first embodiment, in this embodiment, the attention analysis module includes a convolutional attention analysis module, a multi-head attention analysis module and a feedforward network module; In this embodiment, the step S30 further includes: Step S301: linearly project the input sequence features and the initial time series through a convolutional attention analysis module to obtain a query projection matrix, a key projection matrix, and a value projection matrix.

[0056] In some embodiments, the prediction device may map the input sequence features and the initial time series to 、 、 , refer to the following formula: Among them, the initial time series , represents the batch size, Indicates the base quantity, Represents feature dimension, input sequence feature , Indicates the number of channels.

[0057] Step S302: Calculate an initial attention score based on the query projection matrix and the key projection matrix.

[0058] It should be noted that the prediction device can use the convolutional attention mechanism to calculate the attention output of the basis and time series, referring to the following formula: in, represents the initial attention score, represents the query projection matrix, represents the key projection matrix, represents the matrix transpose, Represents the feature dimension.

[0059] Step S303: Process the initial attention score through the convolutional attention analysis module and the multi-head attention analysis module to obtain a candidate attention score.

[0060] In some embodiments, the prediction device may perform normalization, context analysis, and linear change processing on the initial attention score to obtain a candidate attention score.

[0061] Further, in order to improve the accuracy of the attention score, the above step S303 may include: Step S3031: normalizing the initial attention score through the convolutional attention analysis module to obtain an attention weight; Step S3032: performing weighted summation on the value projection matrix based on the attention weight to obtain a context vector; Step S3033: performing residual connection and layer normalization processing on the context vector to obtain a first output result of the input sequence feature and a second output result of the initial time series; Step S3034: linearly transform the first output result and the second output result through the multi-head attention analysis module to obtain a target query vector and a target key vector; Step S3035: Determine candidate attention scores based on the target query vector and the target key vector.

[0062] It should be noted that the prediction device can be applied The function normalizes the attention score to obtain the attention weight: in, represents the attention weight, express function, represents the initial attention score.

[0063] The values ​​are weighted summed using the attention weights to get the context vector: in, represents the context vector, Represents the value projection matrix.

[0064] The context vector is passed through a linear layer for residual connection and layer normalization to obtain the first output result and the second output result: in, The first output result representing the input sequence features, represents the second output result of the initial time series, represents the input sequence features, represents the initial time series, Representation layer normalization function, Represents the residual connection between the input sequence features and the context vector, It means that the initial time series is connected to the residual of the context vector, and the dimension of the final output is consistent with the dimension of the input. ， .

[0065] In some embodiments, the prediction device may use a multi-head attention mechanism to further process the data. Assume that the dimension of each head is ,Will , Mapped to , ,in , Indicates the number of heads. The query and key are divided into multiple heads so that each head can calculate the attention weight independently. After that, the similarity between the query and the key is calculated by dot product and divided by The square root of is scaled to stabilize the gradient, referring to the following formula: in, represents the target query vector, represents the target key vector, represents the linear transformation function of the query vector, represents the linear transformation function of the key vector; in, represents the candidate attention score, represents the dimension scaling factor, Can be , The dimension is .

[0066] Step S304: Perform nonlinear changes on the candidate attention scores through the feedforward network module to obtain a target attention score.

[0067] In some embodiments, the prediction device may perform a nonlinear transformation on the result of the convolutional attention mechanism through a feedforward network module: For each feature, , , firstly, a linear transformation is used to map it to a higher-dimensional space, then an activation function is applied to introduce nonlinearity, and then regularization is performed to prevent overfitting. After that, another linear transformation is used to map the features back to the original dimensional space. Finally, the features processed by the feedforward network are added to the original features to complete the residual connection to retain the information of the input features, and its output dimension is consistent with the input dimension.

[0068] In this embodiment, a convolutional attention analysis module is used to linearly project the input sequence features and the initial time series to obtain a query projection matrix, a key projection matrix and a value projection matrix, an initial attention score is calculated based on the query projection matrix and the key projection matrix, the initial attention score is processed by the convolutional attention analysis module and the multi-head attention analysis module to obtain a candidate attention score, and the candidate attention score is nonlinearly changed by the feedforward network module to obtain a target attention score; this embodiment uses a convolutional attention mechanism to adaptively focus on key information in a sequence while reducing computational complexity, and uses a multi-head attention mechanism to perform parallel learning in different subspaces to enhance the ability to capture features of complex data.

[0069] In addition, an embodiment of the present invention further proposes a computer-readable storage medium, on which a time series prediction program is stored. When the time series prediction program is executed by a processor, the steps of the time series prediction method described above are implemented.

[0070] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM: Random Access Memory), a read-only memory (ROM: Read Only Memory), an erasable programmable read-only memory (EPROM: Erasable Programmable Read Only Memory or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM: CD-Read Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency: Radio Frequency), etc., or any suitable combination of the above.

[0071] The above-mentioned computer-readable storage medium may be included in the time series prediction device; or it may exist independently without being assembled into the time series prediction device.

[0072] In addition, an embodiment of the present invention further provides a computer program product, including a time series prediction program, which implements the steps of the time series prediction method described above when executed by a processor.

[0073] The specific implementation of the computer program product of the present invention is basically the same as the various embodiments of the above-mentioned time series prediction method, and will not be repeated here.

[0074] Reference Figure 4 , Figure 4 It is a structural block diagram of the first embodiment of the time series prediction device of the present invention.

[0075] like Figure 4 As shown, the time series prediction device proposed in the embodiment of the present invention includes: The data processing module 10 is used to obtain an input time series of an input data set and preprocess the input time series to obtain an initial time series; A feature analysis module 20 is used to perform feature analysis on each data point in the input data set to obtain input sequence features and predicted sequence features; An attention analysis module 30, configured to perform attention analysis on the input sequence features and the initial time series through a pre-built attention analysis module to obtain a target attention score, wherein the attention analysis module is a neural network module encapsulating multiple attention layers; The time series prediction module 40 is used to perform time series prediction on the input data set based on the prediction sequence features and the target attention score to obtain a time series prediction result.

[0076] Furthermore, the feature analysis module 20 is also used to perform feature analysis based on the timestamp of each data point in the input data set to obtain time feature information of the input data set; perform feature enhancement on the time feature information through a pre-built feature analysis model to obtain candidate feature information; perform dimension mapping on the candidate feature information through the feature analysis model to obtain dimension mapping feature information; normalize the dimension mapping feature information to obtain target feature information; and divide the target feature information to obtain input sequence features and predicted sequence features.

[0077] Furthermore, the data processing module 10 is also used to obtain an input time series of an input data set; and calculate the average value and standard deviation of the input time series: in, represents the mean value of the input time series, L is the length of the input time series, is the first i elements, Represents the standard deviation of the input time series; The input time series is standardized based on the mean value and standard deviation to obtain an initial time series: in, represents the initial time series obtained after standardization, Indicates a preset constant.

[0078] Furthermore, the attention analysis module includes a convolutional attention analysis module, a multi-head attention analysis module and a feedforward network module; the attention analysis module 30 is further used to perform linear projection on the input sequence features and the initial time series through the convolutional attention analysis module to obtain a query projection matrix, a key projection matrix and a value projection matrix; and calculate an initial attention score based on the query projection matrix and the key projection matrix: in, represents the initial attention score, represents the query projection matrix, represents the key projection matrix, represents the matrix transpose, Represents feature dimension; The initial attention score is processed by the convolutional attention analysis module and the multi-head attention analysis module to obtain a candidate attention score; the candidate attention score is nonlinearly changed by the feedforward network module to obtain a target attention score.

[0079] Furthermore, the attention analysis module 30 is further configured to normalize the initial attention score through the convolutional attention analysis module to obtain an attention weight: in, represents the attention weight, express function, represents the initial attention score; The value projection matrix is ​​weighted summed based on the attention weight to obtain a context vector: in, represents the context vector, Represents the value projection matrix; The context vector is subjected to residual connection and layer normalization processing to obtain a first output result of the input sequence feature and a second output result of the initial time series: in, The first output result representing the input sequence features, represents the second output result of the initial time series, represents the input sequence features, represents the initial time series, Representation layer normalization function, Represents the residual connection between the input sequence features and the context vector, Represents the residual connection between the initial time series and the context vector; The first output result and the second output result are linearly transformed by the multi-head attention analysis module to obtain a target query vector and a target key vector: in, represents the target query vector, represents the target key vector, represents the linear transformation function of the query vector, represents the linear transformation function of the key vector; Determine candidate attention scores based on the target query vector and the target key vector: in, represents the candidate attention score, Represents the dimension scaling factor.

[0080] Furthermore, the time series prediction module 40 is also used to perform feature enhancement processing on the prediction sequence features, and generate an initial prediction sequence based on the prediction sequence features after feature enhancement processing; map the initial prediction sequence to a dimension corresponding to the target attention score to obtain a candidate prediction sequence; perform matrix multiplication of the target attention score and the candidate prediction sequence to obtain an initial prediction result; perform dimensionality transformation on the initial prediction result, and perform de-normalization processing on the initial prediction result after dimensionality transformation to obtain a time series prediction result.

[0081] This embodiment obtains an input time series of an input data set, preprocesses the input time series, obtains an initial time series, performs feature analysis on each data point in the input data set, obtains input sequence features and predicted sequence features, performs attention analysis on the input sequence features and the initial time series through a pre-built attention analysis module, and obtains a target attention score. The attention analysis module is a neural network module encapsulated with multiple attention layers. Time series prediction is performed on the input data set based on the predicted sequence features and the target attention score to obtain a time series prediction result. Since this embodiment adaptively focuses on key information in the sequence based on the attention mechanism, while reducing computational complexity and enhancing the ability to capture features of complex data, it effectively integrates attention scores and predicted sequence features, generates a final prediction result, and effectively improves the accuracy of time series prediction.

[0082] The time series prediction device provided by the present application adopts the time series prediction method in the above embodiment, which can solve the technical problem of time series prediction. Compared with the prior art, the beneficial effects of the time series prediction device provided by the present application are the same as the beneficial effects of the time series prediction method provided by the above embodiment, and other technical features in the time series prediction device are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0083] It should be understood that the above is only an example and does not constitute any limitation on the technical solution of the present invention. In specific applications, technicians in this field can make settings as needed, and the present invention does not limit this.

[0084] It should be noted that the workflow described above is merely illustrative and does not limit the scope of protection of the present invention. In practical applications, technicians in this field can select part or all of them according to actual needs to achieve the purpose of the present embodiment, and no limitation is made here.

[0085] In addition, for technical details not fully described in this embodiment, reference can be made to the time series prediction method provided in any embodiment of the present invention, and will not be repeated here.

[0086] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0087] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0088] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as a read-only memory / random access memory, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0089] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A time series prediction method, characterized in that: The time series prediction method comprises: Obtaining an input time series of an input data set, and preprocessing the input time series to obtain an initial time series; Performing feature analysis on each data point in the input data set to obtain input sequence features and predicted sequence features; Performing attention analysis on the input sequence features and the initial time series through a pre-built attention analysis module to obtain a target attention score, wherein the attention analysis module is a neural network module encapsulating multiple attention layers; Based on the prediction sequence features and the target attention score, time series prediction is performed on the input data set to obtain a time series prediction result.

2. The time series prediction method according to claim 1, characterized in that: The performing feature analysis on each data point in the input data set to obtain input sequence features and predicted sequence features includes: Perform feature analysis according to the timestamp of each data point in the input data set to obtain time feature information of the input data set; Performing feature enhancement on the time feature information through a pre-built feature analysis model to obtain candidate feature information; Performing dimension mapping on the candidate feature information by using the feature analysis model to obtain dimension mapping feature information; Normalizing the dimension mapping feature information to obtain target feature information; The target feature information is divided to obtain input sequence features and predicted sequence features.

3. The time series prediction method according to claim 1, characterized in that: The step of obtaining an input time series of an input data set and preprocessing the input time series to obtain an initial time series includes: Get the input time series of the input dataset; Compute the mean and standard deviation of the input time series: in, represents the mean value of the input time series, L is the length of the input time series, is the first elements, Represents the standard deviation of the input time series; The input time series is standardized based on the mean value and standard deviation to obtain an initial time series: in, represents the initial time series obtained after standardization, Indicates a preset constant.

4. The time series prediction method according to claim 1, characterized in that: The attention analysis module includes a convolutional attention analysis module, a multi-head attention analysis module and a feedforward network module; the attention analysis is performed on the input sequence features and the initial time series by the pre-built attention analysis module to obtain a target attention score, including: Performing linear projection on the input sequence features and the initial time series through a convolutional attention analysis module to obtain a query projection matrix, a key projection matrix, and a value projection matrix; Calculate the initial attention score based on the query projection matrix and the key projection matrix: in, represents the initial attention score, represents the query projection matrix, represents the key projection matrix, represents the matrix transpose, Represents feature dimension; Processing the initial attention score by the convolutional attention analysis module and the multi-head attention analysis module to obtain a candidate attention score; The candidate attention scores are nonlinearly changed through the feedforward network module to obtain a target attention score.

5. The time series prediction method according to claim 4, characterized in that: The processing of the initial attention score by the convolutional attention analysis module and the multi-head attention analysis module to obtain a candidate attention score includes: The initial attention score is normalized by the convolutional attention analysis module to obtain the attention weight: in, represents the attention weight, express function, represents the initial attention score; The value projection matrix is ​​weighted summed based on the attention weight to obtain a context vector: in, represents the context vector, Represents the value projection matrix; The context vector is subjected to residual connection and layer normalization processing to obtain a first output result of the input sequence feature and a second output result of the initial time series: in, The first output result representing the input sequence features, represents the second output result of the initial time series, represents the input sequence features, represents the initial time series, Representation layer normalization processing function, Represents the residual connection between the input sequence features and the context vector, Represents the residual connection between the initial time series and the context vector; The first output result and the second output result are linearly transformed by the multi-head attention analysis module to obtain a target query vector and a target key vector: in, represents the target query vector, represents the target key vector, represents the linear transformation function of the query vector, represents the linear transformation function of the key vector; Determine candidate attention scores based on the target query vector and the target key vector: in, represents the candidate attention score, Represents the dimension scaling factor.

6. The time series prediction method according to any one of claims 1 to 5, characterized in that: The performing time series prediction on the input data set based on the prediction sequence feature and the target attention score to obtain a time series prediction result includes: Performing feature enhancement processing on the predicted sequence features, and generating an initial predicted sequence based on the predicted sequence features after the feature enhancement processing; Mapping the initial prediction sequence to a dimension corresponding to the target attention score to obtain a candidate prediction sequence; Performing matrix multiplication of the target attention score and the candidate prediction sequence to obtain an initial prediction result; The initial prediction result is dimensionally transformed, and the initial prediction result after dimension transformation is de-standardized to obtain a time series prediction result.

7. A time series prediction device, characterized in that: The time series prediction device comprises: A data processing module is used to obtain an input time series of an input data set, and preprocess the input time series to obtain an initial time series; A feature analysis module is used to perform feature analysis on each data point in the input data set to obtain input sequence features and predicted sequence features; An attention analysis module, used to perform attention analysis on the input sequence features and the initial time series through a pre-built attention analysis module to obtain a target attention score, wherein the attention analysis module is a neural network module encapsulating multiple attention layers; A time series prediction module is used to perform time series prediction on the input data set based on the prediction sequence features and the target attention score to obtain a time series prediction result.

8. A time series prediction device, characterized in that: The time series prediction device comprises: a memory, a processor, and a time series prediction program stored in the memory and executable on the processor, wherein the time series prediction program is configured to implement the time series prediction method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a time series prediction program, and when the time series prediction program is executed by a processor, the time series prediction method according to any one of claims 1 to 6 is implemented.

10. A computer program product, characterized in that The computer program product comprises a time series prediction program, and when the time series prediction program is executed by a processor, the steps of the time series prediction method according to any one of claims 1 to 6 are implemented.

Images