Multivariable time sequence prediction method based on iTransform and computer program product

Abstract

The invention discloses a multivariable time sequence prediction method based on iTransform and a computer program product. The method comprises a prototype learning module for defining all variables into a group of learnable prototypes and calculating the similarity between each variable and all prototypes to realize prototype distribution; the prototype features and the variable features are fused to generate enhanced variable representation; performing multi-scale feature fusion, and respectively extracting short, medium and long-term features by using convolution kernels of different sizes; each volume integral branch is output and spliced, the importance weight of each scale in the current context is calculated, and a fusion result is dynamically adjusted; carrying out multi-scale frequency modeling, dividing time slices with different lengths, respectively executing inverse Fourier attention on each scale, and capturing local and global frequency characteristics; a lightweight feed-forward network is introduced on the basis of frequency domain output, and fine disturbance of a frequency spectrum is learned and corrected in a residual form. Experimental results show that the prediction precision can be improved on a data set.

Description

Technical Field

[0001] This invention pertains to a prediction method, specifically a multivariate time series prediction method and computer program product based on iTransformer. Background Technology

[0002] Time series data is a type of observational data recorded in chronological order and exhibiting time dependence. It is widely used in various fields. Time series forecasting is an important research direction in machine learning, with broad applications in energy consumption forecasting, weather forecasting, traffic flow prediction, financial market analysis, and industrial control, among other practical scenarios.

[0003] Multivariate time series forecasting holds significant value in numerous practical applications, such as renewable energy scheduling, financial trend prediction, traffic flow control, and industrial condition monitoring. Its core task is to accurately predict the future dynamic evolution of multiple interrelated variables based on historical observations. However, due to the non-stationarity, long-term dependencies, and high-dimensional interactions typically exhibited by multivariate time series, even with recent breakthroughs based on deep learning, particularly the Transformer architecture, achieving high-precision and highly generalized forecasts remains a significant challenge.

[0004] Patent (CN120893016A) discloses a time series prediction method based on multi-scale decomposition and gated fusion. The method involves multi-level downsampling of the original meteorological time series data to output downsampled time series at different scales. Based on the downsampled meteorological time series, a neural Fourier trend decomposition mechanism is applied to extract trend and seasonal components at different scales from the original meteorological time series data. These trend and seasonal components at different scales are then fused using a gated multilayer sensing network to form a multi-periodic meteorological time feature. After normalization, the multi-periodic meteorological time feature is used by prediction branches at each scale to generate prediction results. Finally, a weighted fusion is performed to obtain the multi-scale prediction output. The method is then trained and optimized by combining multiple loss functions to form a total loss function.

[0005] Patent (CN120873572A) discloses an industrial time series forecasting method and system, relating to the field of data processing technology. It performs multi-scale segmentation on multi-factor time series data to generate feature representations at different time granularities; extracts temporal features from the segmented feature representations using a dual attention mechanism; constructs an adaptive graph structure based on dynamic embedding technology and uses the extracted temporal features to characterize the dynamic relationships between multiple factors at different time steps and scales; employs a multi-scale graph convolutional network to process the graph structure data at each scale, capturing spatial dependencies between variables; models the temporal dimension of the output from spatial modeling at each scale; and fuses the prediction results from multiple scales to generate the final target sequence prediction value. This achieves the technical effect of improving the accuracy and practicality of industrial time series forecasting.

[0006] Patent (CN120763597A) discloses a method and system for predicting non-stationary time series based on hierarchical wavelet attention. This invention proposes a refined prediction paradigm of decomposition-independent modeling-adaptive fusion. First, through wavelet derivative transformation, the dynamic characteristics of endogenous time series data are effectively separated into multiple frequency scales, and non-stationary components such as abrupt changes are amplified. For each frequency scale, an independent time encoder is used for hierarchical independent modeling. Subsequently, through an adaptive scale fusion mechanism, the encoded features of all frequency scales are dynamically weighted and fused at each time point, enabling the model to focus on the most critical dynamic information at present. This method fundamentally solves the modeling problem of complex dynamics in non-stationary time series, significantly improving the accuracy and robustness of prediction.

[0007] In summary, while existing deep learning-based time series forecasting methods have achieved excellent performance in many applications, they still face several challenges. For example, when handling complex forecasting tasks, they still suffer from insufficient multi-scale temporal pattern modeling capabilities, inadequate modeling of inter-variable dependencies, and prominent issues with interpretability and data dependence. Most methods fail to explicitly model the underlying structural relationships between variables, typically treating variables as independent or equivalent, lacking the ability to model semantic clustering or functional groups. Although existing methods attempt to model multi-scale temporal dynamics through convolution or hierarchical attention, the fusion mechanisms are mostly static splicing or averaging, lacking learnable selectivity. Current frequency domain modeling largely relies on global Fourier transform or static dominant frequency analysis, making it difficult to effectively capture local perturbations or fine-grained frequency changes. Therefore, there is an urgent need for a method that provides sufficient inter-variable modeling, strong multi-scale temporal modeling capabilities, and the ability to effectively capture local perturbations or fine-grained frequency changes to achieve better forecasting results. Summary of the Invention

[0008] The purpose of this invention is to provide a multivariate time series forecasting method based on iTransformer, which can effectively improve the accuracy of forecasting results.

[0009] To achieve the above objectives, the present invention provides the following solution:

[0010] A multivariate time series forecasting method, comprising:

[0011] Prototype initialization and learnable modeling: By defining a set of learnable prototype vectors, cluster centers of latent variable clusters are formed.

[0012] Prototype assignment utilizes a feedforward network to calculate the similarity between each variable and all prototypes, and achieves soft clustering through Softmax weights;

[0013] Prototype feature fusion combines aggregated prototype features with original variable features to generate enhanced variable representations;

[0014] Multi-scale convolution extraction uses different kernel sizes (e.g., 3, 5, 7) to extract short-term, medium-term, and long-term features respectively.

[0015] Feature concatenation and fusion: After concatenating the outputs of each convolutional branch, a unified mapping and information integration are achieved through a feedforward layer;

[0016] Adaptive weighted fusion calculates the importance weight of each scale in the current context and dynamically adjusts the fusion result;

[0017] Multi-scale inverse Fourier attention branch: By dividing time segments of different lengths, inverse Fourier attention modeling is performed at each scale to achieve hierarchical capture of local and global frequency features;

[0018] Frequency perturbation residual module: Based on the frequency domain output, a lightweight feedforward network is introduced to learn the subtle perturbation components in the spectrum space and perform frequency correction in the form of residuals to enhance spectrum reconstruction capability and robustness.

[0019] Optionally, the prototype initialization and learnable modeling, by defining a set of learnable prototype vectors to form cluster centers of latent variable clusters, specifically includes:

[0020] Before initialization, all variables are defined as K learnable prototype vectors, forming a prototype matrix: Each p k This represents a potential cluster center in the feature space. At the start of model training, the prototype matrix P is randomly initialized using a normal distribution: σ controls the initial distribution range, ensuring that each prototype uniformly covers the potential feature space in the initial stage. During backpropagation, the prototype matrix P is updated as a trainable parameter. The gradient propagates to the center of each prototype through the prototype allocation stage, causing p to... k Adaptively approaching variable clusters with high similarity, thereby automatically learning the optimal cluster centers during training: Where η is the learning rate. This is the prediction loss function.

[0021] Optionally, the prototype assignment utilizes a feedforward network to calculate the similarity between each variable and all prototypes, and achieves soft clustering through Softmax weights, specifically including:

[0022] Given variable representation Through a feedforward mapping function f assign (·) Calculate its similarity score with the prototype matrix P: s i，k =f assign (x i The formula is: ) = W2σ(W1x1+b1)+b2, where W1 and W2 are the linear layer parameters, σ(·) is the nonlinear activation function (ReLU), and b1 and b2 are the bias terms of the linear layer, corresponding to the bias parameters of the first and second layers, respectively. Then, a temperature-scaled Softmax function is used to generate the prototype weight distribution: Where τ > 0 controls the smoothness of the assignment. When τ → 0, variables tend to be assigned to a single prototype, forming "hard clusters"; when τ is large, the assignment is smoother, and variables tend to be assigned to multiple prototypes, which helps to preserve multi-prototype feature information.

[0023] Optionally, the prototype feature fusion, which fuses the aggregated prototype features with the original variable features to generate an enhanced variable representation, specifically includes:

[0024] Obtain the weight allocation matrix Then, the prototype aggregation feature can be obtained: Z = WP, where This represents the feature of each variable after weighted fusion based on semantic prototypes. Subsequently, it is concatenated with the original variable features and processed through a fusion layer: X′=FFN([X;Z]), where [X;Z] is the concatenation operation, and FFN is the fusion function combining a linear layer and a normalization layer. This process achieves dual-domain fusion of the original variable features and the clustering semantic representation, thereby enhancing the structural sharing and high-level abstract expression among variables.

[0025] Optionally, the multi-scale convolution extraction uses different convolution kernel sizes (e.g., 3, 5, 7) to extract short-term, medium-term, and long-term features respectively, specifically including:

[0026] The model captures multi-level features of time series data from different scales through multiple sets of parallel convolutional layers. Given input features... Where B is the batch size, T is the sequence length, and D is the feature dimension. First, features are extracted using convolutional kernels with multiple different receptive fields: F (k) =Conv k (X), k∈{3, 5, 7}, each This represents the local representation extracted at the corresponding scale. Smaller convolutional kernels can capture short-term patterns such as rapid fluctuations, while larger kernels can identify periodic or trend features, thus complementing each other in local and global modeling.

[0027] Optionally, the feature concatenation and fusion involves concatenating the outputs of each convolutional branch and then using a feedforward layer to achieve unified mapping and information integration, specifically including:

[0028] The features output by multi-scale convolutional layers are concatenated along the feature dimension to obtain combined features: Then, nonlinear transformation and information fusion are performed through a two-layer feedforward network (Linear, LayerNorm, ReLU, Linear): The core function of this step is to compress features at different scales into a unified dimensional space, eliminate the differences in distribution between scales, and enhance the global expressive power of the features.

[0029] Optionally, the adaptive weighted fusion calculates the importance weight of each scale in the current context and dynamically adjusts the fusion result, specifically including:

[0030] After obtaining the fused features, to enhance the dynamic modeling capability across scales, the module further introduces an attention mechanism to assign different importance weights to features at different scales. Specifically, scale attention weights are obtained through a linear mapping plus Softmax: Then, these weights are used to sum the features at each scale to obtain dynamically weighted features. Finally, the two information sources (weighted features and nonlinear fusion features) are fused together and residual connections are added: This approach ensures feature trade-offs across different time scales, allowing the model to dynamically adjust its focus based on the input.

[0031] Optionally, the multi-scale inverse Fourier attention branch: by dividing time segments of different lengths, inverse Fourier attention modeling is performed at each scale to achieve hierarchical capture of local and global frequency features, specifically including:

[0032] First, the time dimension is divided into multiple segments with different proportions. For each segment length s iThe corresponding subsequences are extracted and processed through a dedicated inverse Fourier attention branch. Each inverse Fourier attention branch integrates inverse Fourier transform filtering and a full attention mechanism. Specifically, for each segmented tensor... Reshape it into a three-dimensional form to fit the standard attention structure:

[0033] It is then input into the attention encoding layer: The output is reshaped and multiple scales of output are stitched together to obtain a fused representation: The fusion results were then stabilized using a layer normalization operation: This structure can simultaneously capture local high-frequency disturbances and global low-frequency trends, enabling frequency-dependent modeling across time scales.

[0034] Optionally, the frequency perturbation residual module introduces a lightweight feedforward network based on the frequency domain output to learn subtle perturbation components in the spectral space and perform frequency correction in the form of residuals to enhance spectral reconstruction capability and robustness, specifically including:

[0035] This invention introduces a frequency residual learning module for explicitly learning the fine-grained variation Δf of each variable in the frequency direction. For the normalized output... Perturbation estimation is performed using a lightweight feedforward neural network. in h represents the hidden layer dimension, σ(·) denotes the ReLU activation function, and b1 and b2 are the bias terms of the linear layers, corresponding to the bias parameters of the first and second layers, respectively. This structure can effectively capture local frequency perturbation information with low computational cost. The final output is: This residual perturbation modeling method preserves the core feature information while introducing fine-grained corrections in the frequency direction, further enhancing the model's representational capabilities.

[0036] A computer device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement a multivariate time series forecasting method according to any one of claims 1-9.

[0037] According to specific embodiments provided by the present invention, the following technical effects are disclosed:

[0038] This invention provides a multivariate time series forecasting method and computer program product based on iTransformer. The method includes prototype initialization and learnable modeling, which defines a set of learnable prototype vectors to form cluster centers for latent variable clusters; prototype assignment, which uses a feedforward network to calculate the similarity between each variable and all prototypes, and achieves soft clustering through Softmax weights; prototype feature fusion, which fuses the aggregated prototype features with the original variable features to generate an enhanced variable representation; multi-scale convolutional extraction, which uses different convolutional kernel sizes to extract short-term, medium-term, and long-term features respectively; and feature concatenation and fusion. The model employs a concatenation mechanism, where the outputs of each convolutional branch are stitched together and then fed forward to achieve unified mapping and information integration. Adaptive weighted fusion calculates the importance weights of each scale within the current context and dynamically adjusts the fusion result. A multi-scale inverse Fourier attention branch performs inverse Fourier attention modeling at each scale by dividing the time segment into different lengths, achieving hierarchical capture of local and global frequency features. A frequency perturbation residual module introduces a lightweight feedforward network based on the frequency domain output to learn subtle perturbation components in the spectral space and performs frequency correction in the form of residuals, enhancing spectral reconstruction capability and robustness. This application proposes a prototype learning module, introducing a set of learnable prototype vectors and dynamically establishing the association between variables and prototypes through a soft assignment mechanism, thereby modeling the semantic structural relationships between variables. This mechanism effectively introduces structural priors and improves the model's ability to model interactions between variables. Multiple convolutional branches are used to extract temporal features under different receptive fields, and multi-scale features are adaptively weighted and fused through an attention mechanism. This enables the model to dynamically select the most suitable scale, effectively addressing multi-granular temporal variations. By introducing a multi-scale frequency modeling mechanism and combining inverse Fourier transform with frequency domain residual compensation, this model effectively models local frequency dynamics and weak perturbations, significantly improving the frequency response capability of long-term series. This model can effectively capture complex dynamic patterns in time series, significantly improving prediction accuracy and robustness, and providing more accurate and stable prediction results in variable time series data. Attached Figure Description

[0039] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0040] Figure 1 A flowchart of the iTransformer-based multivariate time series prediction method provided in this application;

[0041] Figure 2A flowchart illustrating the iTransformer-based multivariate time series forecasting method provided in this application. Detailed Implementation

[0042] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0043] This invention proposes a multivariate time series prediction method and computer program product based on iTransformer. First, prototype initialization and learnable modeling are performed by defining a set of learnable prototype vectors to form cluster centers for latent variable clusters. Then, prototype assignment is performed, using a feedforward network to calculate the similarity between each variable and all prototypes, and soft clustering is achieved through Softmax weights. Next, prototype feature fusion is performed, fusing the aggregated prototype features with the original variable features to generate an enhanced variable representation. Following this, multi-scale convolution extraction is conducted, using different convolution kernel sizes to extract short-term, medium-term, and long-term features respectively. Finally, feature concatenation is performed. This invention employs a concatenation and fusion process. The outputs of each convolutional branch are concatenated and then fed forward to achieve unified mapping and information integration. Adaptive weighted fusion is then performed, calculating the importance weights of each scale within the current context and dynamically adjusting the fusion result. Next, a multi-scale inverse Fourier attention branch is used, dividing time segments into different lengths and performing inverse Fourier attention modeling at each scale to achieve hierarchical capture of local and global frequency features. Finally, a frequency perturbation residual module introduces a lightweight feedforward network based on the frequency domain output to learn subtle perturbation components in the spectral space and performs frequency correction in the form of residuals, thereby enhancing spectral reconstruction capability and robustness. This invention achieves multivariate time series prediction, solving the problems of low prediction efficiency and insufficient prediction accuracy in existing models.

[0044] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0045] In one exemplary embodiment, this application provides a multivariate time series forecasting method based on iTransformer. For example... Figure 1 and Figure 2 As shown, the multivariate time series prediction method based on iTransformer provided in this embodiment of the invention includes the following S1 to S8.

[0046] S1: Prototype initialization and learnable modeling, which forms cluster centers of latent variable clusters by defining a set of learnable prototype vectors.

[0047] First, before initialization, define all variables as K learnable prototype vectors, forming a prototype matrix:

[0048]

[0049] Where each p k It represents a potential cluster center in the feature space.

[0050] First, at the start of model training, the prototype matrix P is randomly initialized using a normal distribution:

[0051]

[0052] σ controls the initial distribution range, ensuring that each prototype uniformly covers the potential feature space in the initial stage. This approach avoids initial clustering bias and is beneficial for stable training.

[0053] Then, during backpropagation, the prototype matrix P is updated as a trainable parameter. The gradient propagates to each prototype center through the prototype allocation phase, causing p... k Adaptively approaching variable clusters with high similarity, thereby automatically learning the optimal cluster centers during training:

[0054]

[0055] Where η is the learning rate. The loss function is used for prediction. This end-to-end optimization mechanism enables the model to continuously adjust the prototype distribution during dynamic training, achieving self-organizing modeling of the semantic space.

[0056] S2: Prototype assignment, which uses a feedforward network to calculate the similarity between each variable and all prototypes, and achieves soft clustering through Softmax weights.

[0057] The goal of the prototype assignment phase is to determine the strength of the association between each variable and each prototype. Given a variable, represent... Through a feedforward mapping function f assign (·) Calculate its similarity score with the prototype matrix P:

[0058] s i,k =f assign (x i )=W2σ(W1x i +b1)+b2 (4)

[0059] Where W1 and W2 are linear layer parameters, σ(·) is a nonlinear activation function (ReLU), and b1 and b2 are bias terms of the linear layer, corresponding to the bias parameters of the first and second layers, respectively.

[0060] The prototype weight distribution is then generated using a temperature-scaled Softmax function:

[0061]

[0062] Among them, τ>0 controls the smoothness of the allocation.

[0063] As τ→0, variables tend to be assigned to a single prototype, forming "hard clusters";

[0064] When τ is large, the assignment is smoother, and variables tend to be assigned to multiple prototypes, which helps to preserve multi-prototype feature information.

[0065] This mechanism essentially achieves semantic-level soft clustering assignment, enabling the model to retain information from different cluster centers simultaneously, effectively avoiding over-clustering and feature loss.

[0066] S3: Prototype feature fusion, which fuses the aggregated prototype features with the original variable features to generate an enhanced variable representation.

[0067] Obtain the weight allocation matrix Then, the prototype aggregation features can be obtained:

[0068]

[0069] in, This represents the feature of each variable after semantic prototype-based weighted fusion. Subsequently, it is concatenated with the original variable features and processed through a fusion layer.

[0070] X′=FFN([X;Z]) (7)

[0071] Where [X; Z] represents the concatenation operation, and FFN is the fusion function combining the linear layer and the normalization layer. This process achieves dual-domain fusion of original variable features and cluster semantic representation, thereby enhancing the structural sharing and high-level abstract expression among variables.

[0072] S4: Multi-scale convolution extraction, using different convolution kernel sizes (such as 3, 5, 7) to extract short-term, medium-term, and long-term features respectively.

[0073] In this stage, the model captures multi-level features of the time series from different scales through multiple sets of parallel convolutional layers. Given input features... Where B is the batch size, T is the sequence length, and D is the feature dimension. First, features are extracted using convolutional kernels with multiple different receptive fields:

[0074]

[0075] Each This represents the local representation extracted at the corresponding scale. Smaller convolutional kernels can capture short-term patterns such as rapid fluctuations, while larger kernels can identify periodic or trend features, thus complementing each other in local and global modeling.

[0076] S5: Feature concatenation and fusion. The outputs of each convolutional branch are concatenated and then unified mapping and information integration are achieved through a feedforward layer.

[0077] The features output by multi-scale convolutional layers are concatenated along the feature dimension to obtain combined features:

[0078]

[0079] Then, nonlinear transformation and information fusion are performed through a two-layer feedforward network (Linear, LayerNorm, ReLU, Linear):

[0080]

[0081] The core function of this step is to compress features at different scales into a unified dimensional space, eliminate the differences in distribution between scales, and enhance the global expressive power of the features.

[0082] S6: Adaptive weighted fusion, calculates the importance weight of each scale in the current context, and dynamically adjusts the fusion result.

[0083] After obtaining the fused features, to enhance the dynamic modeling capability across scales, the module further introduces an attention mechanism to assign different importance weights to features at different scales. Specifically, scale attention weights are obtained through a linear mapping plus Softmax:

[0084]

[0085] Then, these weights are used to sum the features at each scale to obtain the dynamically weighted features F. attn :

[0086]

[0087] Finally, the two information sources (weighted features and nonlinear fusion features) are fused together and residual connections are added:

[0088]

[0089] This approach ensures feature trade-offs across different time scales, allowing the model to dynamically adjust its focus based on the input.

[0090] S7: Multi-scale inverse Fourier attention branch, which performs inverse Fourier attention modeling at each scale by dividing time segments of different lengths.

[0091] Let the input time series be To model frequency dependencies at different time scales, this module first divides the time dimension into multiple segments with different proportions. For each segment length s i The corresponding subsequences are extracted and processed through a dedicated inverse Fourier attention branch. Each inverse Fourier attention branch integrates inverse Fourier transform filtering and a full attention mechanism. Specifically, for each segmented tensor... Reshape it into a three-dimensional form to fit the standard attention structure:

[0092]

[0093] It is then input into the attention encoding layer:

[0094]

[0095] The output is reshaped and multiple scales of output are stitched together to obtain a fused representation:

[0096]

[0097] The fusion results were then stabilized using a layer normalization operation:

[0098]

[0099] This structure can simultaneously capture local high-frequency disturbances and global low-frequency trends, enabling frequency-dependent modeling across time scales.

[0100] S8: Frequency Perturbation Residual Module. Based on the frequency domain output, a lightweight feedforward network is introduced to learn subtle perturbation components in the spectral space and perform frequency correction in the form of residuals to enhance spectral reconstruction capability and robustness. Specifically, it includes:

[0101] While inverse Fourier attention performs well in modeling major frequency components, it may still overlook small local perturbations in the frequency space. Therefore, this invention introduces a frequency residual learning module to explicitly learn the fine-grained variation Δf of each variable in the frequency direction. For the normalized output... Perturbation estimation is performed using a lightweight feedforward neural network:

[0102]

[0103] in, h represents the hidden layer dimension, σ(·) denotes the ReLU activation function, and b1 and b2 are the bias terms of the linear layers, corresponding to the bias parameters of the first and second layers, respectively. This structure can effectively capture local frequency perturbation information with low computational cost.

[0104] The final output is:

[0105]

[0106] This residual perturbation modeling method preserves the core feature information while introducing fine-grained corrections in the frequency direction, further enhancing the model's representational capabilities.

[0107] The time series datasets were predicted using the present invention and other methods respectively. The prediction accuracy results are compared in Table 1.

[0108] Table 1 Comparison of prediction accuracy of the present invention and other methods on time series datasets.

[0109]

[0110] This invention achieves the highest prediction accuracy compared to other models by processing time-series datasets and inputting them into the constructed model. The invention proposes three innovations in time-series prediction: 1. A prototype learning module is introduced, which introduces a set of learnable prototype vectors and dynamically establishes the association between variables and prototypes through a soft assignment mechanism, thereby modeling the semantic structural relationships between variables. This mechanism effectively introduces structural priors and improves the model's ability to model interactions between variables. 2. Multiple convolutional branches are designed to extract temporal features under different receptive fields, and multi-scale features are adaptively weighted and fused through an attention mechanism. This allows the model to dynamically select the most suitable scale and effectively cope with multi-granularity temporal changes. 3. A multi-scale frequency modeling mechanism is introduced, combining inverse Fourier attention and frequency domain residual compensation to effectively model local frequency dynamics and weak perturbations, significantly improving the frequency response capability of long-term sequences.

[0111] Compared with existing technologies, this invention has the following advantages: 1. Existing prototype mechanisms often statically define prototypes without adaptive optimization mechanisms. Many methods use predefined category centers or fixed vectors as prototypes, lacking a self-supervised learning approach consistent with the task objective, resulting in limited prototype quality. Secondly, prototype selection and feature fusion processes are disconnected. Most current methods treat prototype selection as a post-processing or additional module, ignoring the deep semantic interaction between prototypes and variables. Moreover, current prototypes have poor sharing and weak generalization ability. This invention proposes a prototype learning module with a simple structure and efficient mechanism. While retaining the ability to express multivariate information, this module constructs a semantic space through learnable prototype vectors, making up for the structural modeling defects of traditional attention mechanisms at the variable level, thereby improving the model's modeling efficiency, generalization ability, and interpretability. 2. Existing technologies either rely on frequency domain processing, losing temporal domain intuitiveness, or only construct multi-scale at the input partitioning level without explicit fusion in the feature space, resulting in insufficient information coupling between scales and a lack of consistent modeling mechanisms. The multi-scale feature fusion module proposed in this invention employs parallel convolutional branches with multiple kernel scales to capture temporal dynamics within different receptive fields, thereby expanding the breadth of feature representation. 3. Addressing the issue that current mainstream frequency domain methods cannot effectively model frequency perturbations, this invention introduces a residual channel to explicitly model the perturbation Δf in the spectral output, capturing fine-grained changes that the original frequency components cannot express. The frequency domain modeling module of this invention effectively compensates for the shortcomings of existing methods in local frequency modeling and high-order perturbation representation, providing stronger frequency awareness and modeling accuracy for long-term, multivariate time series prediction.

[0112] In one exemplary embodiment, the present invention provides a computer device including a memory and a processor, wherein the memory stores a computer program, and the processor, when executing the computer program, is able to implement the relevant steps in the above-described method embodiments.

[0113] Furthermore, in another exemplary embodiment, the present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, can perform the corresponding operations in the above-described method embodiments.

[0114] Furthermore, the present invention also provides a computer program product comprising a computer program, wherein when a processor executes the computer program, it can implement the specific steps in the above method embodiments.

[0115] It should be emphasized that the user information (such as user device information, user personal information, etc.) and data (including but not limited to data used for analysis, storage and display) involved in this invention can only be used after being authorized by the user or fully authorized by the relevant parties, and the collection, use and processing of all data must comply with the requirements of applicable laws and regulations.

[0116] Those skilled in the art will understand that some or all of the processes in the above method embodiments can be implemented by a computer program instructing related hardware. This computer program can be stored in a non-volatile computer-readable storage medium and, when executed, implements the functions of the above method embodiments. It should be noted that the memory, database, or other storage medium involved in this invention can include both non-volatile and volatile memory types. Examples of non-volatile memory include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric memory (FRAM), phase-change memory (PCM), and graphene memory, etc.; while volatile memory includes random access memory (RAM) and external cache memory, such as static random access memory (SRAM) and dynamic random access memory (DRAM).

[0117] It is worth noting that the technical features of the above embodiments can be combined in different ways. Although this specification does not describe all possible combinations of technical features one by one, as long as these combinations do not have logical contradictions, they should all be considered to be within the scope of the present invention.

[0118] This invention has been described in detail with specific examples illustrating its basic principles and implementation methods. These embodiments are only for helping to understand the core ideas of the invention and are not intended to limit the invention. Those skilled in the art can make appropriate adjustments and optimizations based on this invention to make it applicable to different application scenarios. Therefore, the content of this specification should not be construed as limiting the scope of the invention, but should be determined by the scope of the appended claims.

Claims

1. A multivariate time series prediction method, characterized by, Comprise: Prototype initialization and learnable modeling, a set of learnable prototype vectors are defined to form the clustering centers of latent variable clusters; Prototype assignment, the similarity between each variable and all prototypes is calculated using a feedforward network, and soft clustering is achieved through Softmax weights; Prototype feature fusion, the aggregated prototype features are fused with the original variable features to generate enhanced variable representations; Multi-scale convolution extraction, different convolution kernel sizes (e.g. 3, 5, 7) are used to extract short-term, medium-term and long-term features respectively; Feature concatenation and fusion, the outputs of each convolution branch are concatenated and then mapped and integrated through a feedforward layer; Adaptive weighted fusion, the importance weights of each scale in the current context are calculated to dynamically adjust the fusion results; Multi-scale inverse Fourier attention branch: by dividing different lengths of time segments, inverse Fourier attention modeling is performed on each scale to capture hierarchical local and global frequency features; Frequency perturbation residual module: a lightweight feedforward network is introduced based on the frequency domain output to learn the subtle perturbation components in the frequency spectrum space, and the frequency is corrected in residual form to enhance the frequency reconstruction ability and robustness.

2. The multivariate time series prediction method of claim 1, wherein, Prototype initialization and learnable modeling, a set of learnable prototype vectors are defined to form the clustering centers of latent variable clusters, specifically including: Before initialization, all variables are defined as K learnable prototype vectors, composing a prototype matrix: where each p k represents a latent cluster center in the feature space; at the beginning of model training, the prototype matrix P is randomly initialized by a normal distribution: where σ controls the range of initial distribution, ensuring that each prototype uniformly covers the latent feature space in the initial stage; during backpropagation, the prototype matrix P is updated as a trainable parameter; the gradient is propagated to each prototype center through the prototype assignment stage, prompting p k to adaptively approach the variable cluster with high similarity, thereby automatically learning the optimal cluster center during training: where η is the learning rate, is the prediction loss function.

3. The multivariate time series prediction method of claim 1, wherein, Prototype assignment, the similarity between each variable and all prototypes is calculated using a feedforward network, and soft clustering is achieved through Softmax weights, specifically including: Given variable represents Through a feed-forward mapping function f assign (·) computes its similarity score with the prototype matrix P: s i，k = f assign (x i ) = W2σ(W1x i + b1) + b2 where W1, W2 are linear layer parameters, σ(·) is a non-linear activation function (ReLU), b1, b2 are bias terms for linear layers, corresponding to bias parameters for the first and second layers, respectively; subsequently a temperature-scaled Softmax function is used to generate the prototype weight distribution: Where τ>0 controls the smoothness of the assignment; when τ→0, the variable tends to be assigned to a single prototype, forming a "hard clustering"; when τ is larger, the assignment is smoother, and the variable tends to be assigned to multiple prototypes, which helps to preserve the multi-prototype feature information.

4. The multivariate time series prediction method of claim 1, wherein, Prototype feature fusion, the aggregated prototype features are fused with the original variable features to generate enhanced variable representations, specifically including: Obtaining an assignment weight matrix After that, the prototype aggregation feature can be obtained: Z = WP, where represents the feature of each variable after being weighted and fused based on the semantic prototype; subsequently, it is spliced with the original variable feature and processed through a fusion layer: X' = FFN([X; Z]), where [X; Z] is a splicing operation and FFN is a fusion function composed of a linear layer and a normalization layer; this process realizes the dual-domain fusion of the original variable feature and the clustered semantic representation, thereby enhancing the structural sharing and high-level abstract expression between variables.

5. The multivariate time series prediction method of claim 1, wherein, Multi-scale convolution extraction, different convolution kernel sizes (e.g. 3, 5, 7) are used to extract short-term, medium-term and long-term features respectively, specifically including: The model captures multi-level features of time series from different scales through multiple sets of parallel convolutional layers; given input features where B is the batch size, T is the sequence length, and D is the feature dimension; first extract features through convolution kernels with multiple different receptive fields: F (k) = Conv k (X), k e {3, 5, 7}, each represents the local representation extracted at the corresponding scale; smaller convolution kernels can capture short-term patterns such as rapid fluctuations, while larger convolution kernels can identify periodic or trend features, thus forming a complement between local and global modeling.

6. The multivariate time series prediction method of claim 1, wherein, Feature concatenation and fusion, the outputs of each convolution branch are concatenated and then mapped and integrated through a feedforward layer, specifically including: The features outputted by the multi-scale convolutional layers are spliced in the feature dimension to obtain combined features: Nonlinear transformation and information fusion are then performed by two feedforward networks: The core role of this step is to compress features of different scales to a unified dimension space, eliminate distribution differences between scales, and enhance the global expression ability of the features.

7. The multivariate time series prediction method of claim 1, wherein, Adaptive weighted fusion, the importance weights of each scale in the current context are calculated to dynamically adjust the fusion results, specifically including: After obtaining the fused features, to improve the dynamic modeling ability between scales, the module further introduces an attention mechanism to assign different importance weights to features of different scales. Specifically, a linear mapping and Softmax are used to obtain scale attention weights: Then, these weights are used to weight and sum the features of each scale to obtain dynamic weighted features F attn : Finally, the two information sources (weighted features and nonlinear fused features) are fused and a residual connection is added: This approach ensures that the model can balance features between different time scales and dynamically adjust its focus according to the input.

8. The multivariate time series prediction method of claim 1, wherein, Multi-scale inverse Fourier attention branch, by dividing different lengths of time segments, inverse Fourier attention modeling is performed on each scale to capture hierarchical local and global frequency features, specifically including: First, the time dimension is divided into multiple segments of different proportions For each segment length s i , the corresponding sub-sequence is extracted and processed by a dedicated inverse Fourier attention branch. Each inverse Fourier attention branch incorporates an inverse Fourier transform filter with a full attention mechanism. Specifically, for each segment of the divided tensor It is reshaped into a three-dimensional form to fit the standard attention structure: The input is then fed into the attention encoding layer: The output is reshaped and concatenated across multiple scales to obtain the fused representation: The fusion result is then stabilized by a layer normalization operation: This structure can capture both local high-frequency disturbances and global low-frequency trends, enabling frequency-dependent modeling across time scales.

9. The multivariate time series prediction method of claim 1, wherein, Frequency perturbation residual module, a lightweight feedforward network is introduced based on the frequency domain output to learn the subtle perturbation components in the frequency spectrum space, and the frequency is corrected in residual form to enhance the frequency reconstruction ability and robustness, specifically including: The present application introduces a frequency residual learning module for explicitly learning the fine-grained change Δf of each variable in the frequency direction; for the normalized output The disturbance estimation is performed through a light feedforward neural network wherein h is the hidden layer dimension, σ(·) represents the ReLU activation function, b1 and b2 are the bias terms of the linear layer, and correspond to the bias parameters of the first layer and the second layer respectively; this structure can effectively capture the local frequency disturbance information at a lower computational cost; and the final output is: This residual disturbance modeling method retains the main feature information while introducing fine-grained correction in the frequency direction, further improving the representation ability of the model.

10. A computer device comprising: Memory, processor, and computer program stored in memory and executable on processor, characterized in that the processor executes the computer program to implement the multivariate time series prediction method of any one of claims 1-9.

Images