Cross-time-space conversion method with Fourier filtering operator for multivariable time sequence classification

Abstract

The invention provides a cross-time-space conversion method with a Fourier filtering operator for multivariable time sequence classification. The cross-time-space conversion method comprises the following steps: mapping an input time sequence to a target variable; mapping a time sequence data set to a group of C category labels Y; after training, the CSTFormer model outputs a vector containing C values so as to estimate the probability that the time sequence belongs to each category; in the CSTFormer model, data input by a multivariable time sequence firstly passes through a LayerNorm layer and then enters a global adaptive Fourier neural operator module, noise is adaptively filtered, and important frequency information is enhanced; the output passes through a LayerNorm layer and then enters a time-space consistency learning module; the time-space consistency learning module is composed of three sub-modules, namely a dynamic multi-scale time attention module, a dynamic space attention module and a dynamic cross-distribution fusion module; respectively learning a time context and a space context, and fusing to obtain a learned time-space consistency context; and finally, outputting a classification result by the classification head.

Description

Technical Field

[0001] The present invention relates to the technical field of multivariate time series classification, and in particular to a cross-time and space conversion method with a Fourier filter operator for multivariate time series classification. Background Art

[0002] Time series analysis is one of the most challenging research topics in the field of data mining. In this field, time series classification (TSC) plays a vital role. The goal of TSC is to predict the classification labels of serialized real-valued observations by building machine learning models.

[0003] Distance-based methods, especially the use of Euclidean distance and dynamic time warping (DTW), provide an early practical approach to raw time series classification. As the complexity and diversity of time series data increase, a single similarity measure and classification method may no longer suffice. To address these challenges, existing techniques explore time series classification from various feature space perspectives, such as distance-based methods, shapelet methods, and recursive methods. Deep learning, especially convolutional neural networks (CNNs) and recursive neural networks (RNNs), has gained widespread attention in the field of time series due to its excellent performance in automatic feature extraction and capturing long-term dependencies. Despite these advances, deep learning still faces challenges in ensuring stability and accuracy.

[0004] However, spatial consistency is still a relatively neglected aspect. Spatial consistency, especially considering the stability of individual sensors and the potential correlation between different sensors, is crucial. For example, a small disturbance of any single sensor may seriously affect the classification of the overall MTS data, where amplitude disturbance may lead to the misclassification of certain activities. In addition, the mutual relationship between sensors plays a crucial role. From the perspective of spatiotemporal consistency, the main challenges faced by Multivariate Time Series Classification (MTSC) come from the complex dynamics of time series data and the interdependence between multiple variables. For example, in Human Activity Recognition (see Appendix Figure 1 As shown in Figure 2, the data recorded by the sensors during different activities show obvious changes in spatiotemporal characteristics, which poses the following challenges to the classification model:

[0005] 1. Temporal dynamics: Sensor data from activities change over time, for example, the vibration frequency is higher when running than when walking. The length of the sequence and the speed of change are different in different activities, which makes accurate classification challenging.

[0006] 2. Spatial multivariate dependencies: In multivariate time series, just like sensors located in different parts of the human body, the data are not independent but influence each other. For example, leg movements can affect the readings of waist-mounted sensors, and these spatial dependencies need to be understood and modeled correctly.

[0007] 3. Blurred boundaries between categories: The distinction between some activities, such as brisk walking and jogging, may not be obvious in sensor data, making it difficult for learning algorithms to distinguish these subtle differences

[0008] 4. Noise interference: Noise is an important issue in multivariate time series classification because it can seriously affect the ability of the classification model to extract and learn effective spatiotemporal features. In the data collected by sensors, noise can come from various sources, including environmental interference, inaccuracies in the equipment itself, and errors in sensor placement and operation. Summary of the invention

[0009] Multivariate time series classification (MTSC) is considered to be of great significance in the fields of human activity recognition, medical diagnosis and system monitoring. Traditional classification methods often emphasize temporal dynamics or use similarity measures such as dynamic time warping (DTW) to classify data. However, the challenge is to effectively capture the spatiotemporal consistency of multivariate data, which is crucial for accurate classification as the system state evolves. The present invention provides a cross-spatiotemporal transformation method with Fourier filter operator for multivariate time series classification, which mainly utilizes the converter-based CSTFormer model designed specifically for MTSC tasks, and adopts a dynamic adaptive time-frequency filter to process noise and capture global patterns, thereby improving the clarity and relevance of extracted features.

[0010] The present invention specifically adopts the following technical solutions:

[0011] A cross-temporal and spatial transformation method with Fourier filter operator for multivariate time series classification:

[0012] The CSTFormer model, a cross-temporal transformer with a Fourier filter operator obtained through training, maps the input time series to the target variable: by mapping the time series dataset to a set of C category labels Y; after training, the CSTFormer model outputs a vector containing C values, thereby estimating the probability that the time series belongs to each category;

[0013] In the CSTFormer model, the multivariate time series input data first passes through the LayerNorm layer and then enters a global adaptive Fourier neural operator module to adaptively filter noise and enhance important frequency information; the output then passes through the LayerNorm layer and enters the spatiotemporal consistency learning module; the spatiotemporal consistency learning module consists of three sub-modules: a dynamic multi-scale temporal attention module, a dynamic spatial attention module, and a dynamic cross-distribution fusion module; the temporal context and spatial context are learned separately, and the learned spatiotemporal consistency context is obtained by fusion; finally, the classification head outputs the classification result.

[0014] Furthermore, the dynamic multi-scale temporal attention module combines multi-scale feature extraction with a dynamic adaptive attention mechanism to capture dependencies in time series;

[0015] The dynamic spatial attention module uses channel embedding to achieve cross-channel interaction without changing the channel dimension, and dynamically learns the spatial consistency changes between multiple sequences;

[0016] The dynamic multi-scale temporal attention module and the dynamic spatial attention module capture the spatiotemporal dependencies by learning the spatiotemporal background across channels and time dimensions; and then use dynamic cross-distribution fusion to merge the distribution of the spatiotemporal background; the dynamic cross-distribution fusion merges the temporal and spatial information by exchanging the mean and variance of the context, and then further combines the features through an adaptive fusion mechanism to enhance the model's ability to perceive changes in spatiotemporal consistency and improve classification accuracy.

[0017] Furthermore, in the adaptive Fourier neural operator module:

[0018] In the adaptive Fourier neural operator module:

[0019] For each batch and each channel of the input signal, the one-dimensional fast Fourier transform FFT is first used for processing, and then the real and imaginary parts of the output are processed by the FC layer respectively; then for the real and imaginary parts of the complex output, the einsum function is used to implement matrix multiplication to apply the weight ω1 and bias b1 to the real part X real and the imaginary part X imag ,Then, after nonlinear activation, the real and imaginary parts are combined to form a new complex output;

[0020] O1 real =GELU(einsum(′bnhw,nhm→bnmw′,X real ,ω1)+b1)

[0021] O1 imag =GELU(einsum(′bnhw,nhm→nmw′,X imag,ω1)+b1)

[0022] Then, a soft threshold sofishrink is used to enhance the sparsity of the signal; then, the modified real and imaginary parts are recombined into a complex number, and this complex number is multiplied element by element by the result of the initial Fourier transform, and finally the signal is converted from the frequency domain back to the time domain.

[0023] Furthermore, in the dynamic multi-scale temporal attention module:

[0024] For a given input time series data, it is processed through a series of parallel convolutional layers to capture information at different time scales; among them, n convolutional layers L1, L2, ..., L n , each L i Using different sizes of k i and convolution kernel with stride si; the output of each convolution layer Where B is the batch size of the input time series data, T′ i and C′ i Is affected by k i and i the temporal and characteristic dimensions of impact;

[0025] Then the fusion weights are dynamically calculated and merged according to the input data: the outputs of all convolutional layers are connected along the channel dimension to obtain T is the sequence length of the input time series data. Each channel is globally averaged pooled and input to the fully connected layer FC, and finally normalized by the Softmax function:

[0026] W dynamic =Softmax(FC(GlobalAvgPool(H concat )))

[0027] Use dynamically generated weights W dynamic The outputs of different convolutional layers are weighted and merged to obtain the original multivariate time series H fusion Multi-scale contextual representation of:

[0028] H fusion =W dynamic ·H concat

[0029] A dynamic self-attention mechanism with fused time encoding is used to dynamically adjust the attention weight according to the change of time steps:

[0030] First, the input data is transposed to prioritize the time dimension. Time encoding is used to provide a unique encoding for each time step to capture the temporal information in the sequence. The encoding is implemented through learnable parameters:

[0031] set up is a learnable temporal embedding matrix that uses the input features and temporal embeddings to compute dynamic attention weights for each time step; the weights ω(t) are adjusted based on the feature content and temporal context of each time step:

[0032] ω(t)=σ(W f F(t)+W f TE(t)+b)

[0033] Where F(t) represents the feature at time t, σ is a nonlinear activation function, and W f , W t are the weight matrices for feature and time encoding, respectively, and b is the bias term;

[0034] Then, the query vector Q, key vector K and value vector V are generated through linear transformation;

[0035] Then calculate the dot product between the query vector and the key vector and add the dynamic weight to calculate the attention score; finally, calculate the attention-weighted feature output V':

[0036] W is the dynamic weight matrix

[0037] Attention = softmax(Scores)

[0038] V'=Attention·V

[0039] By incorporating dynamic weights at each time step, the model is made aware of the importance of information at different moments.

[0040] Furthermore, the convolution layer uses a depth-separable one-dimensional convolution, applies convolution to the channel corresponding to the sequence number C, and keeps the length of the sequence before and after the convolution consistent; the fully connected layer FC is composed of a learnable parameter matrix And the nonlinear activation function ReLU.

[0041] Furthermore, in the dynamic spatial attention module:

[0042] Using the direct channel embedding method, the input data is Where C represents the number of variables; convert the data into

[0043] Then, we embed the sequence using a parameter matrix of shape [1, D], and get

[0044]

[0045] Thus, for each channel i∈[1,...,C], a two-dimensional feature matrix of shape [T,D] is obtained; thus, the cross-channel feature interactions are represented in a high-dimensional space without changing the number of channels;

[0046] After encoding each time point, the interaction between channels is equivalent to interacting in C' two-dimensional spaces of size T, D', while considering the impact of T on channel information; for a single sample, the initial input feature map Where D' is the number of feature channels and T' is the time step. The global information aggregation of each channel is to use global average pooling to calculate the multi-scale statistical features of each channel, extract global feature information from each channel, and each pooling operation generates a channel descriptor.

[0047]

[0048] in, is the feature map of the i-th channel; the results of the three global average pooling operations are concatenated to form the final channel descriptor z, which is then transformed through two fully connected layers to generate the channel attention weights The Sigmoid activation function is used to normalize the weights to the range [0, 1]:

[0049] a j =σ(W2·ReLU(W1·z j ))

[0050] Where W1 and W2 are learnable weight matrices, σ is the Sigmoid function; the calculated channel attention weight a is applied to the input feature map Get the weighted feature map

[0051]

[0052] Among them, ⊙ represents element-wise multiplication and allows weighting according to the importance of the feature map of each channel;

[0053] because On each channel, it is a 2D tensor, using depthwise separable convolution from To extract features from the kernel, the deep convolution first performs an independent convolution operation on each input channel:

[0054] For each channel c, there is Y c ∈X' c *K c

[0055] After the depthwise convolution, the point-by-point convolution uses a 1×1 convolution kernel to linearly combine the outputs of all channels at each position:

[0056] Z=Y*K 1×1

[0057] Finally, Z is averaged in the last dimension to restore to a one-dimensional format;

[0058] To ensure that the model synchronously perceives temporal and spatial changes, in the dynamic multi-scale temporal attention module, Q and K are retained, transposed and recalculated as Q and K, and the spatial scores are calculated by the following formula, and then the spatial attention scores are obtained by the softmax function, and finally multiplied by the linear transformation of Z to obtain the spatial context S containing temporal information:

[0059]

[0060] Attention=softmax(Spatial Scores)

[0061] S = Attention (ZW Z )in,

[0062] Furthermore, in the dynamic cross-distribution fusion module:

[0063] For a given time context and the transposed spatial context Where T' represents the number of channels, d represents the sequence count, and the mean μ and variance σ of each context are calculated as follows:

[0064]

[0065] These statistics are then processed through two linear transformations, where FC represents the fully connected layer and W' represents the learnable parameter matrix;

[0066] μ V =ReLU(FC(μ*V,W'*μ V ));μ s =ReLU(FC(μ*S,W'*μ s ))

[0067] σ V =ReLU(FC(μ*V,W'*σ V )); s =ReLU(FC(μ*S,W'*σ s ))

[0068] Then swap the mean and variance of the temporal and spatial contexts:

[0069]

[0070] Continue to use the adaptive fusion mechanism to further merge the two tensors into the spatiotemporal context; first, concatenate the features along the dimension to form an input feature matrix Z V,S =[X 1 , X 2 , ..., X d ; F 1 , F 2 , ..., F d ;], where X / F i represents the i-th channel, d is the total number of channels; then, the concatenated feature tensor is converted into a feature vector through global average pooling, Z = [z 1 , z 2 , ..., z n ], n = 2d; the attention mechanism calculates the weight of each feature, A = [a 1 , a 2 ,...,a n ], where A represents the weighted attention weight:

[0071]

[0072] Among them, Softmax is used to normalize the weights to a probability distribution with a sum of 1; the attention weight A is multiplied by the original feature matrix Z*V,S to obtain the fused feature output:

[0073]

[0074] Among them A i is the i-th weight in A; then linear transformation and softmax are applied to produce the final classification result,

[0075] And, a cross-temporal and spatial transformation system with Fourier filter operator for multivariate time series classification, comprising:

[0076] The CSTFormer model, a cross-temporal transformer with a Fourier filter operator, maps the input time series to the target variable by mapping the time series dataset to a set of C category labels Y; the trained CSTFormer model outputs a vector of C values, estimating the probability that the time series belongs to each category;

[0077] In the CSTFormer model, the multivariate time series input data first passes through the LayerNorm layer and then enters a global adaptive Fourier neural operator module to adaptively filter noise and enhance important frequency information; the output then passes through the LayerNorm layer and enters the spatiotemporal consistency learning module; the spatiotemporal consistency learning module consists of three sub-modules: a dynamic multi-scale temporal attention module, a dynamic spatial attention module, and a dynamic cross-distribution fusion module; the temporal context and spatial context are learned separately, and the learned spatiotemporal consistency context is obtained by fusion; finally, the classification head outputs the classification result.

[0078] An electronic device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of the cross-temporal and spatial conversion method with Fourier filter operator for multivariate time series classification as described above are implemented.

[0079] A non-transitory computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the cross-temporal and spatial transformation method with Fourier filter operator for multivariate time series classification as described above.

[0080] Compared with the prior art, the CSTFormer model provided by the present invention and its preferred embodiment adopts a dynamic multi-scale temporal attention module and a dynamic spatial attention module to capture complex spatiotemporal dependencies by learning spatiotemporal contexts across channel and time dimensions. In addition, in order to integrate the spatiotemporal context more coherently, the present invention proposes dynamic cross-distribution fusion, which merges the distributions of the spatiotemporal contexts together instead of simply connecting them at the feature level. This approach helps to maintain the continuity of spatiotemporal features throughout the learning process and ensures the effective adaptation of the model in various classification scenarios. Extensive evaluations on multiple datasets (including UCR and UEA databases) show that CSTFormer significantly outperforms existing models, with an average accuracy improvement of 23.5% in challenging scenarios. Erosion experiments confirm the effectiveness of the key components of CSTFormer, highlighting its reliability and the key role of its modules in achieving high classification accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0081] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments:

[0082] Figure 1 Schematic diagram of sensor signal characteristics at different spatial positions of the human body in walking and running states;

[0083] Figure 2 A schematic diagram of the overall architecture of CSTFormer provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0084] In order to make the features and advantages of this patent more obvious and easy to understand, the following embodiments are specifically described in detail with the accompanying drawings as follows:

[0085] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present application belongs.

[0086] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0087] The embodiment of the present invention provides a Fourier transformer (CSTFormer) specially customized for cross-temporal and spatial feature learning. CSTFormer uses a dynamic adaptive time-frequency filter to filter and identify potential noise in multivariate time series. Unlike direct denoising, CSTFormer uses Fourier neural operators to selectively select clearer signal features in the frequency domain; at the same time, the parallel convolution module in the time domain recognizes and maps multi-scale patterns in the data, making it easier to learn spatiotemporal patterns in multivariate time series.

[0088] Unlike typical Transformers in time series, CSTFormer aims to learn cohesive spatiotemporal representations, so both the spatial relations and temporal structures of Multivariate Time Series (MTS) are considered. To this end, the present invention develops a dynamic multi-scale temporal attention (DMTA) module, which combines multi-scale feature extraction with a dynamic adaptive attention mechanism, aiming to capture complex dependencies in time series. In addition, the present invention designs a dynamic spatial attention mechanism (DSA), which uses channel embedding to achieve cross-channel interaction without changing the channel dimension, and dynamically learns spatial consistency changes between multiple sequences to improve classification accuracy. Finally, the present invention proposes a dynamic cross-distribution fusion method to merge temporal and spatial information by exchanging the mean and variance of the context. In addition, the present invention designs an adaptive fusion mechanism to further combine these features, enhance the model's ability to perceive spatiotemporal consistency changes and improve classification accuracy. The following is a more specific display and introduction:

[0089] First, the following definition:

[0090] Definition 1 (Time Series (TS)): Time series A = (a1, a2, ..., a n ) is an ordered sequence of n data points, where a i Represents the i-th value of A.

[0091] Definition 2 (Multivariate Time Series (MTS)): Multivariate time series A = (a1, a2, ..., a n )∈R (d×n) is a list of n vectors, each a i are all vectors with d channels (sometimes called dimensions). k,i ∈R represents the i-th observation in the k-th channel. Assume a i All observations in are observed at the same time or space point, so MTS can also be viewed as a set of d time series.

[0092] Definition 3 (data set): Data set D = (X, Y) = (A (i) ,y (i) ) i∈[1,2,...,m] A is a set of m time series and a set of predefined discrete class labels C. Here, A (i) It can be a multivariate time series with labels y (i) ∈C.

[0093] Definition 4 (Multivariate Time Series Classification (MTSC)): MTSC is a supervised learning task that aims to learn the (i) ,y (i) ) i∈[1,2,...,m] The relationship between the target variable and a set of time series, where y (i) Indicates that each A (i) MTSC predicts the categorical value of a time series and classifies the time series data into a limited set of categories. Typically, for MTSC, the target variable y (i) is a one-hot encoded vector.

[0094] The model of the present invention performs the following functions to map the input time series to the target variable:

[0095] f L (θ L ,X)=f L-1 (θ L-1 ,f L-2 (θ L-2 ,…,f1(θ1,X)))

[0096] where f lrepresents the nonlinear activation function of the lth layer, l∈[1,2,...,L]. The goal of time series classification TSC is to train a neural network model to map the time series dataset D to a set of C category labels Y. After training, the neural network outputs a vector containing C values, estimating the probability that the time series X belongs to each category.

[0097] Figure 2 This is the overall architecture of CSTFormer provided by an embodiment of the present invention. The multivariate time series input first passes through a global adaptive Fourier neural operator module to adaptively filter noise and enhance important frequency information. The output passes through the LayerNorm layer and enters the spatiotemporal consistency learning module. This module consists of three submodules: a dynamic multi-scale time attention module, a dynamic spatial attention module, and a dynamic cross-distribution fusion module. These three submodules learn the temporal context and spatial context respectively, and then fuse them to obtain the learned spatiotemporal consistency context. Finally, the classification head outputs the classification result.

[0098] 1 Adaptive Fourier Neural Operator Module

[0099] The embodiment of the present invention proposes an adaptive Fourier neural operator module, which can not only capture the basic frequency components of the time series, but also adaptively adjust the feature representation in the frequency domain through neural network learning to improve the classification accuracy.

[0100] For each batch and each channel of the input signal x, a one-dimensional Fast Fourier Transform (FFT) is applied. The dimensions of x are B×T×C, where B is the batch size, T is the sequence length, and C is the number of sequences, corresponding to the number of channels. The FFT is applied along the sequence length dimension (dim=1) and uses "orthogonal" normalization.

[0101]

[0102] Here, x[n] is the input signal at time point n, and X[K] is the output at frequency k. Since the input to FFT is real, the output dimension is Because only non-negative frequency components are calculated.

[0103] In order to further process the block data, Among them, M is the number of blocks divided by channels, and H is the number of channels in each block, that is,

[0104] After the FFT, the real and imaginary parts of the complex output are further processed and the weights w1 and bias b1 are applied to the real and imaginary parts using matrix multiplication and broadcasting mechanisms:

[0105] O1 real=GELU(einsum('bnhw,nhm→bnmw′,X real ,ω1)+b1)

[0106] O1 imag =GELU(einsum('bnhw,nhm→bnmw′,X imag ,ω1)+b1)

[0107] Then, the real and imaginary parts are combined after nonlinear activation to form a new complex output. Here, X is the newly reorganized complex signal. We then use soft thresholding to enhance the sparsity of the signal. This operation helps to reduce small amplitude components in the frequency domain, which are usually associated with noise.

[0108] X=softshrink(X)

[0109] Here, the softshrink function is applied to each element x i Do the following:

[0110]

[0111] This process effectively sets the small amplitude components in the frequency domain to zero while retaining the larger components. This step helps retain those frequency domain features that are more valuable for time series classification. We then recombine the modified real and imaginary parts into a complex number and multiply it element-wise by the result of the initial Fourier transform.

[0112] X=X·(original FFT output)

[0113] This step aligns the frequency domain representation of the signal with the original frequency components by multiplying with the original FFT output. This helps preserve more of the original signal's features in the subsequent inverse transform.

[0114] The final step is to convert the signal from the frequency domain back to the time domain.

[0115] x=IFFT(X,dim=1,norm="ortho")

[0116] The method proposed in the present invention is different from the traditional method in that it introduces an adaptive learning mechanism and neural network parameters, which enables the feature extraction in the frequency domain to be dynamically adjusted to adapt to different data characteristics.

[0118] 2 Dynamic Multi-Scale Temporal Attention Module

[0119] When processing multivariate time series data, the importance of features in the data stream often fluctuates over time. This requires the model to adapt to this dynamic change in order to effectively capture key information. This paper proposes a Dynamic Multiscale Temporal Attention (DMTA) module, which integrates multi-scale feature extraction using a dynamic adaptive attention mechanism, aiming to capture complex dependencies in time series. The following is a detailed description of the DMTA module: Given an input time series data The convolutional layers are processed in parallel to capture information at different time scales. Specifically, the convolutional layers L1, L2, ..., L n , each L i Using different sizes of k i and stride length s i This design effectively extracts features in different time windows and simulates the dynamics of different frequencies. The output of each convolution layer is Where T' i and C' i Is affected by k i and i Influenced time and feature dimensions. In an embodiment of the present invention, a depth-wise separable one-dimensional convolution is used to apply convolution on the C channel while maintaining the length of the sequence before and after the convolution.

[0120] H i =ReLU(BatchNorm(Conv(X,k i ,s i )))

[0121] The present invention introduces an additional small network to dynamically calculate and merge fusion weights based on the input data. First, the outputs of all convolutional layers are concatenated along the channel dimension to obtain Then, each channel undergoes global average pooling and is input to a fully connected layer (FC) with a learnable parameter matrix It is composed of a nonlinear activation function ReLU and is finally normalized by the Softmax function.

[0122] W dynamic =Softmax(FC(GlobalAvgPool(H concat )))

[0123] Use dynamically generated weights W dynamic The outputs of different convolutional layers are weighted and merged to obtain the original multivariate time series H fusion Multi-scale contextual representation of:

[0124] H fusion =W dynamic ·H concat

[0125] Traditional self-attention mechanisms excel in capturing long-term dependencies, but often ignore the dynamic characteristics of time series. To address this problem, this paper designs a dynamic self-attention mechanism that integrates time encoding, which enables it to dynamically adjust the attention weight according to the change of time steps. The mechanism consists of three modules: time encoding module, dynamic attention weight module and self-attention processing module. The calculation process is as follows: First, transpose the data to prioritize the time dimension:

[0126] here

[0127] Temporal encoding aims to provide a unique encoding for each time step to capture the temporal information in the sequence. These encodings are implemented via learnable parameters:

[0128] TE=W te here is a learnable temporal embedding matrix. Next, the dynamic attention weights for each time step are calculated using the input features and temporal embeddings. This weight is adjusted based on the feature content and temporal context of each time step:

[0129] ω(t)=σ(W f F(t)+W f TF(t)+b)

[0130] Where F(t) represents the feature at time t, σ is a nonlinear activation function (such as sigmoid), and W f , W t are the weight matrices of feature and time encoding, respectively, and b is the bias term. Subsequently, the query vector (Query), key vector (Key), and value vector (Value) are generated through linear transformation:

[0131]

[0132] Then, the dot product between the query and the key is calculated and the dynamic weight is added to calculate the attention score. Finally, the attention-weighted feature output V' is calculated, which has the size of B×T×d.

[0133] W is the dynamic weight matrix

[0134] Attention = softmax(Scores)

[0135] V'=Attention·V

[0136] By incorporating dynamic weights into each time step, the present invention enables the model to perceive the importance of information at different moments. Combined with the self-attention mechanism, the model can not only capture static features but also adapt to dynamic changes, thereby improving classification performance.

[0137] 3 Dynamic Spatial Attention Based on Channel Embedding

[0138] The goal of this invention is to achieve cross-channel interaction without destroying the channel dimension, allowing the model to dynamically learn the spatial consistency changes between multiple sequences, thereby improving the classification accuracy.

[0139] First, the present invention designs a direct channel embedding method, the input data is Where B represents the batch size, C represents the number of variables, and T represents the sequence length. The data is converted to

[0140] X′=reshape(X,[B,C,T,1])

[0141] Then, the sequence is embedded using a parameter matrix W of shape [1, D], resulting in

[0142]

[0143] Through the above steps, for each channel i∈[1,...,C], a two-dimensional feature matrix of shape [T,D] is obtained. This method can represent the cross-channel feature interaction in a high-dimensional space without changing the number of channels, thereby improving the classification accuracy.

[0144] After encoding each time point, the interaction between channels is equivalent to interacting in C' two-dimensional spaces of size T, D, while considering the impact of T on channel information; for a single sample, the initial input feature map Where D is the number of feature channels and T is the time step. The global information aggregation of each channel uses global average pooling to calculate the multi-scale statistical features of each channel. The purpose of this step is to extract global feature information from each channel. Each pooling operation generates a channel descriptor.

[0145] z i =Pooling(X i )fori=1,2,…,C

[0146] Here, Xi is the feature map of the i-th channel. The results of the three global average pooling operations are concatenated to form the final channel descriptor z, which is then transformed through two fully connected layers to generate channel attention weights The Sigmoid activation function is used to normalize the weights to the range [0, 1]:

[0147] a j =σ(W2·ReLU(W1·z j ))

[0148] Where W1 and W2 are learnable weight matrices, and σ is the Sigmoid function. Finally, the calculated channel attention weight a is applied to the input feature map X to obtain the weighted feature map X′:

[0149] X'∈a⊙X

[0150] Here, ⊙ represents element-wise multiplication (Hadamard product), allowing the feature maps of each channel to be weighted according to their importance. Subsequently, since X′ is a 2D tensor on each channel, features are extracted from X′ using depthwise separable convolution, where the depthwise convolution first performs an independent convolution operation on each input channel:

[0151] For each channel c, there is Y c ∈X' c *K c

[0152] Following the depthwise convolution, the point-wise convolution uses a 1×1 convolution kernel to linearly combine the outputs of all channels at each position:

[0153] Z=Y*K 1×1

[0154] Finally, Z is averaged in the last dimension and restored to a one-dimensional format. To ensure that the model perceives temporal and spatial changes synchronously, in the DMTA module, Q and K are retained, transposed, and recalculated as Q and K, and the spatial attention score is calculated and multiplied by the linear transformation of Z to obtain a spatial context containing temporal information:

[0155]

[0156] Attention=softmax(Spatial Scores)

[0157] S = Attention (ZW Z )here

[0158] 4 Dynamic cross-distribution integration

[0159] A key issue in integrating temporal and spatial contextual information is how to effectively merge these sources. Common feature fusion techniques include concatenation, summation, and global average pooling (GAP). The concatenation method simply connects multiple feature matrices along the specified dimension, retaining all feature information, but failing to model the relationship between features. Summation may ignore the different scales and importance of feature matrices, resulting in information loss or dominance of certain features, thus failing to highlight useful unique features. GAP reduces the feature dimension by averaging each feature matrix into a single vector. Although this reduces the dimensionality, it loses important spatial information and fails to capture the complex relationship between feature matrices. In particular, it does not consider the relative scale between features, resulting in a lack of diversity in the fused features.

[0160] It is worth noting that these methods do not model the connection between the temporal and spatial context distributions in the feature space. To address this issue, we propose a dynamic cross-distribution fusion method to merge the two context tensors. and the transposed spatial context Where T represents the number of channels, d represents the sequence count, and the present invention calculates the mean μ and variance σ of each context as follows:

[0161]

[0162] These statistics are then processed through two linear transformations, where FC represents the fully connected layer and W represents the learnable parameter matrix.

[0163] μ V =ReLU(FC(μ*V,W*μ V ));μ s =ReLU(FC(μ*S,W*μ s ))

[0164] σ V =ReLU(FC(μ*V,W*σ V )); s =ReLU(FC(μ*S,W*σ s ))

[0165] Subsequently, the mean and variance of the temporal and spatial contexts are swapped:

[0166]

[0167] This dynamic calculation and adjustment of feature distribution for temporal and spatial contexts fuses the statistical information of each other, so that the model can better perceive the temporal and spatial consistency transformation and improve the generalization ability. Next, an adaptive fusion mechanism is designed to further merge the two tensors into the temporal and spatial context. First, the invention concatenates the features along the dimension to form an input feature matrix Z V,S =[X 1 , X 2 , ..., X d ; F 1 , F 2 , ..., F d ;], where X / F i represents the i-th channel, and d is the total number of channels. Then, the concatenated feature tensor is converted into a feature vector through global average pooling, Z = [z 1 , z 2 , ..., z n ], N = 2d. The attention mechanism calculates the weight of each feature, A = [a 1 , a 2 ,...,a n ], where A represents the weighted attention weight:

[0168]

[0169] Here, Softmax normalizes the weights to a probability distribution, and the sum is 1. The attention weight A is multiplied by the original feature matrix Z*V,S to obtain the fused feature output:

[0170]

[0171] Among them A i is the i-th weight in A. Then linear transformation and softmax are applied to produce the final classification result,

[0172] Experiments and Results

[0173] In order to demonstrate the effectiveness of the CSTFormer proposed in the present invention in the multivariate time series classification task, this example conducts extensive experiments on a large number of well-known datasets and compares its performance with other advanced models.

[0174] Experimental Datasets: In this work, we use the UEA Multivariate Time Series Classification Archive as the collective dataset. The UEA is a comprehensive collection of 30 different datasets and serves as a standardized testing ground for multivariate time series classification.

[0175] Model training settings: The experimental environment is Windows 10 operating system, and it was repeated five times on two NVIDIA RTX 3060 12GB GPUs. The entire model was built using PyTorch, with a maximum training cycle of 10, an initial learning rate of 0.0001, and AdamW as the optimizer.

[0176] Table 1 Classification accuracy of CSTFormer and 5 MSTC models on 30 datasets (test sets)

[0177]

[0178] The experimental results in Table 1 show that the CSTFormer provided by the present invention achieves classification accuracy that is superior to or comparable to existing models on most data sets. By comparing the performance of five different models on multiple data sets, it can be found that the average accuracy of other models is 67.38%, while the average accuracy of the model proposed by the present invention on the same data set is 83.21%, an increase of about 23.50%.

[0179] Specifically, for the atrial fibrillation dataset, the proposed model significantly outperforms other models, with an accuracy improvement of more than 20%. On the basic action and character trajectory datasets, all models showed high accuracy, but the new model slightly improved the accuracy while maintaining stability. For the cricket and DuckDuckGeese datasets, the new model also demonstrated its strong generalization ability, especially when dealing with more complex time series data, its performance has been significantly improved.

[0180] These results show that the proposed model not only has an accuracy advantage in solving multivariate time series classification problems, but also maintains high performance and stability in different types of time series data. Therefore, the model provides an effective solution to the multivariate time series classification problem, especially when dealing with data with complex dynamic characteristics.

[0181] This example also conducts an ablation study on the attention mechanism, using the attention calculation methods of famous time series models such as Informer, Autoformer, Crossformer, and LogTransformer. In the experiment, we also try to replace the time and space attention with a simple multi-layer perceptron (MLP) to see if the performance can be improved. The experimental results are detailed in Table 2.

[0182]

[0183] 1 Dimension

[0184] 2Max Series Length

[0185] The results show that, in addition to the original model, the version with Crossformer attention mechanism has the highest average accuracy, followed by Autoformer. In contrast, the versions using Informer and LogTransformer attention mechanisms perform poorly, probably because these mechanisms tend to discard some information. The worst performance is the version using only MLP, further confirming the importance of capturing spatiotemporal consistency features in multivariate time series classification tasks.

[0186] In summary, the embodiment of the present invention introduces CSTFormer for multivariate time series classification tasks. CSTFormer effectively learns the spatiotemporal consistency representation in the data through its key components. Extensive experiments were conducted on 30 datasets, and CSTFormer surpassed other models with an average accuracy of 83.21%. Ablation studies further show that the key components proposed in the model of the present invention significantly improve its reliability and stability.

[0187] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0188] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0189] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0190] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0191] This patent is not limited to the above-mentioned optimal implementation mode. Anyone can derive other various forms of cross-temporal and spatial conversion methods with Fourier filter operators for multivariate time series classification under the inspiration of this patent. All equal changes and modifications made according to the scope of the patent application of the present invention should fall within the scope of this patent.

Claims

1. A method for cross-temporal and spatial transformation with Fourier filter operator for multivariate time series classification, characterized by: The CSTFormer model, a cross-temporal transformer with a Fourier filter operator obtained through training, maps the input time series to the target variable: by mapping the time series dataset to a set of C category labels Y; after training, the CSTFormer model outputs a vector containing C values, thereby estimating the probability that the time series belongs to each category; In the CSTFormer model, the multivariate time series input data first passes through the LayerNorm layer and then enters a global adaptive Fourier neural operator module to adaptively filter noise and enhance important frequency information; the output then passes through the LayerNorm layer and enters the spatiotemporal consistency learning module; the spatiotemporal consistency learning module consists of three sub-modules: a dynamic multi-scale temporal attention module, a dynamic spatial attention module, and a dynamic cross-distribution fusion module; the temporal context and spatial context are learned separately, and the learned spatiotemporal consistency context is obtained by fusion; finally, the classification head outputs the classification result.

2. The method for cross-temporal and spatial transformation with Fourier filter operator for multivariate time series classification according to claim 1, characterized in that: The dynamic multi-scale temporal attention module combines multi-scale feature extraction with a dynamic adaptive attention mechanism to capture dependencies in time series; The dynamic spatial attention module uses channel embedding to achieve cross-channel interaction without changing the channel dimension, and dynamically learns the spatial consistency changes between multiple sequences; The dynamic multi-scale temporal attention module and the dynamic spatial attention module capture the spatiotemporal dependencies by learning the spatiotemporal background across channels and time dimensions; and then use dynamic cross-distribution fusion to merge the distribution of the spatiotemporal background; the dynamic cross-distribution fusion merges the temporal and spatial information by exchanging the mean and variance of the context, and then further combines the features through an adaptive fusion mechanism to enhance the model's ability to perceive changes in spatiotemporal consistency and improve classification accuracy.

3. The method for cross-temporal and spatial transformation with Fourier filter operator for multivariate time series classification according to claim 1, characterized in that: In the adaptive Fourier neural operator module: For each batch and each channel of the input signal, the one-dimensional fast Fourier transform FFT is first used for processing, and then the real and imaginary parts of the output are processed by the FC layer respectively; then for the real and imaginary parts of the complex output, the einsum function is used to implement matrix multiplication to apply the weight ω1 and bias b1 to the real part X real and the imaginary part X imag ,Then, after nonlinear activation, the real and imaginary parts are combined to form a new complex output; O1 real =GELU(einsum(′bnhw,nhm→bnmw′,X real ,ω1)+b1) 01 imag =GELU(einsum(′bnhw,nhm→bnmw′,X imag ,ω1)+b1) Then, a soft threshold softshrink is used to enhance the sparsity of the signal; then, the modified real and imaginary parts are recombined into a complex number, and this complex number is multiplied element by element by the result of the initial Fourier transform, and finally the signal is converted from the frequency domain back to the time domain.

4. The method for cross-temporal and spatial transformation with Fourier filter operator for multivariate time series classification according to claim 1, characterized in that: In the dynamic multi-scale temporal attention module: For a given input time series data, it is processed through a series of parallel convolutional layers to capture information at different time scales; among them, n convolutional layers L1, L2, ..., L n , each of which L i Using different sizes of k i and stride length s i The convolution kernel of each convolution layer is Where B is the batch size of the input time series data, T′ i and C′ i Is affected by k i and i the temporal and characteristic dimensions of impact; Then the fusion weights are dynamically calculated and merged according to the input data: the outputs of all convolutional layers are connected along the channel dimension to obtain T is the sequence length of the input time series data. Each channel is globally averaged pooled and input to the fully connected layer FC, and finally normalized by the Softmax function: W dynamic =Softmax(FC(GlobalAvgPool(H concat ))) Use dynamically generated weights W dynanic The outputs of different convolutional layers are weighted and merged to obtain the original multivariate time series H fusion Multi-scale contextual representation of: H fusion =W dynamic ·H conca t A dynamic self-attention mechanism with fused time encoding is used to dynamically adjust the attention weight according to the change of time steps: First, the input data is transposed to prioritize the time dimension. Time encoding is used to provide a unique encoding for each time step to capture the temporal information in the sequence. The encoding is implemented through learnable parameters: Let TE = W te , is a learnable temporal embedding matrix that uses the input features and temporal embeddings to compute dynamic attention weights for each time step; the weights ω(t) are adjusted based on the feature content and temporal context of each time step: ω(t)=σ(W f F(t)+W f TE(t)+b) Where F(t) represents the feature at time t, σ is a nonlinear activation function, and W f , W t are the weight matrices for feature and time encoding, respectively, and b is the bias term; Then, the query vector Q, key vector K and value vector V are generated through linear transformation; Then calculate the dot product between the query vector and the key vector and add the dynamic weight to calculate the attention score; finally, calculate the attention weighted feature output V′: W is the dynamic weight matrix Attention = softmax(Scores) V′=Attention·V By incorporating dynamic weights into each time step, the model is made aware of the importance of information at different moments.

5. The method for cross-temporal and spatial transformation with Fourier filter operator for multivariate time series classification according to claim 4, characterized in that: The convolution layer uses a depth-separable one-dimensional convolution, applies convolution to the channel corresponding to the sequence number C, and keeps the length of the sequence before and after the convolution consistent; the fully connected layer FC is composed of a learnable parameter matrix And the nonlinear activation function ReLU.

6. The method for cross-temporal and spatial transformation with Fourier filter operator for multivariate time series classification according to claim 4, characterized in that: In the dynamic spatial attention module: Using the direct channel embedding method, the input data is Where C represents the number of variables; convert the data into X′=reshape(X, [B, C, T, 1]) Then, the sequence is embedded using the parameter matrix W′ of shape [1, D], resulting in Thus, for each channel i∈[1,...,C], a two-dimensional feature matrix of shape [T,D] is obtained; thus, the cross-channel feature interactions are represented in a high-dimensional space without changing the number of channels; After encoding each time point, the interaction between channels is equivalent to interacting in C′ two-dimensional spaces of size T, D′, while considering the influence of T on channel information; for a single sample, the initial input feature map Where D″ is the number of feature channels and T′ is the time step. The global information aggregation of each channel is to use global average pooling to calculate the multi-scale statistical features of each channel, extract global feature information from each channel, and each pooling operation generates a channel descriptor. in, is the feature map of the i-th channel; the results of the three global average pooling operations are concatenated to form the final channel descriptor z, which is then transformed through two fully connected layers to generate the channel attention weights The weights are normalized to the range [0, 1] using the Sigmoid activation function: and j =σ(W2 ReLU(W1 z j )) Where W1 and W2 are learnable weight matrices, σ is the Sigmoid function; the calculated channel attention weight a is applied to the input feature map Get the weighted feature map Among them, ⊙ represents element-wise multiplication and allows weighting according to the importance of the feature map of each channel; because On each channel, it is a 2D tensor, using depthwise separable convolution from To extract features from the kernel, the deep convolution first performs an independent convolution operation on each input channel: For each channel c, there is Y c ∈X′ c *K c After the depthwise convolution, the point-by-point convolution uses a 1×1 convolution kernel to linearly combine the outputs of all channels at each position: Z=Y*K 1×1 Finally, Z is averaged in the last dimension to restore to a one-dimensional format; To ensure that the model synchronously perceives temporal and spatial changes, in the dynamic multi-scale temporal attention module, Q and K are retained, transposed and recalculated as Q and K, and the SpatialScores are calculated by the following formula, and then the spatial attention score is obtained by the softmax function, and finally multiplied by the linear transformation of Z to obtain the spatial context S containing temporal information: Attention=softmax(Spatial Scores) S = Attention (ZW Z )in, 7. The method for cross-temporal and spatial transformation with Fourier filter operator for multivariate time series classification according to claim 6, characterized in that: In the dynamic cross-distribution fusion module: For a given time context and the transposed spatial context Where T′ represents the number of channels, d represents the sequence count, and the mean μ and variance σ of each context are calculated as follows: These statistics are then processed through two linear transformations, where FC represents the fully connected layer and W′ represents the learnable parameter matrix; μ V =ReLU(FC(μ*V,W′*μ V ));μ s =ReLU(FC(μ*S,W′*μ s )) s V =ReLU(FC(μ*V,W′*σ V ));s s =ReLU(FC(μ*S,W′*σ s )) Then swap the mean and variance of the temporal and spatial contexts: Continue to use the adaptive fusion mechanism to further merge the two tensors into the spatiotemporal context; first, concatenate the features along the dimension to form an input feature matrix Z V，S =[X 1 , X 2 , ..., X d ; F 1 , F 2 , ..., F d ;], where X / F i represents the i-th channel, d is the total number of channels; then, the concatenated feature tensor is converted into a feature vector through global average pooling, Z = [z 1 , z 2 , ..., z n ], n = 2d; the attention mechanism calculates the weight of each feature, A = [a 1 , a 2 , ..., a n ], where A represents the weighted attention weight: Among them, Softmax is used to normalize the weights to a probability distribution with a sum of 1; the attention weight A is multiplied by the original feature matrix Z*V,S to obtain the fused feature output: Among them A i is the i-th weight in A; then linear transformation and softmax are applied to produce the final classification result, CLS = Softmax (FC (Z)), 8. A cross-temporal and spatial conversion system with Fourier filter operator for multivariate time series classification, characterized in that: include: The CSTFormer model, a cross-temporal transformer with a Fourier filter operator, maps the input time series to the target variable by mapping the time series dataset to a set of C category labels Y; the trained CSTFormer model outputs a vector of C values, estimating the probability that the time series belongs to each category; In the CSTFormer model, the multivariate time series input data first passes through the LayerNorm layer and then enters a global adaptive Fourier neural operator module to adaptively filter noise and enhance important frequency information; the output then passes through the LayerNorm layer and enters the spatiotemporal consistency learning module; the spatiotemporal consistency learning module consists of three sub-modules: a dynamic multi-scale temporal attention module, a dynamic spatial attention module, and a dynamic cross-distribution fusion module; the temporal context and spatial context are learned separately, and the learned spatiotemporal consistency context is obtained by fusion; finally, the classification head outputs the classification result.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the cross-temporal and spatial conversion method with Fourier filter operator for multivariate time series classification as described in any one of claims 1 to 7 are implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the cross-temporal and spatial transformation method with Fourier filter operator for multivariate time series classification as described in any one of claims 1 to 7 are implemented.

Images