Time sequence prediction method and system for KAN enabling graph attention parallel loop network

Abstract

The invention provides a time sequence prediction method and system for a KAN enabling graph attention parallel loop network, and relates to the technical field of time sequence prediction, and the method comprises the steps: obtaining time sequence input data; inputting the time sequence input data into a feature extraction layer based on fast Fourier transform and discrete wavelet transform, and preprocessing the time sequence input data by the feature extraction layer to obtain spliced feature extraction data; inputting the spliced feature extraction data into a feature aggregation layer of graph attention based on KAN, and preprocessing the spliced feature extraction data by the feature aggregation layer to obtain feature aggregation data; and inputting the feature aggregation data into a time sequence prediction model of the parallel gating circulation unit network based on KAN and attention, and outputting time sequence prediction data by the time sequence prediction model. According to the method, the calculation complexity of time sequence prediction is reduced, and the time sequence prediction efficiency and accuracy are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of time series prediction, and in particular to a time series prediction method and system of a KAN-enabled graph attention parallel recurrent network. Background Art

[0002] With the continuous development of big data and artificial intelligence algorithms, and the continuous enrichment and improvement of the functions of deep learning frameworks such as PyTorch, TensorFlow and PaddlePaddle, deep learning has achieved great success and has been widely used in different scenarios.

[0003] At present, deep learning is mainly used in time series prediction. In time series prediction, due to the complex nonlinear characteristics of data, it is often difficult to obtain ideal prediction accuracy by directly applying raw data for prediction. Therefore, it is necessary to first extract features from the data in order to reveal the inherent laws and characteristics of the data. This includes but is not limited to the decomposition of time series and the extraction of internal information of data through different mathematical transformations. However, since most data often present highly complex nonlinear characteristics, including the volatility of time series and the interaction between multiple features, it is difficult to achieve ideal prediction results when using these feature extraction methods. Therefore, it is of great theoretical and practical value to study the method of extracting features from data and designing more effective methods to learn internal features.

[0004] Existing research on time series prediction mainly includes feature extraction, feature aggregation and time series prediction. In terms of feature extraction, the data is first transformed nonlinearly through a graph neural network with k-hop information transmission, and then the original data is converted into frequency domain data through fast Fourier transform, so as to capture the information of the data at different time scales, and further mine and extract high-dimensional potential features in multiple periods. Although this scheme can realize feature extraction of data at the periodic scale, the ability of this feature extraction is limited, especially when facing non-stationary data, which affects the incomplete information of subsequent feature aggregation, resulting in reduced prediction accuracy. In terms of feature aggregation, with the help of graph convolutional networks, through its unique graph structure processing capabilities, the features between nodes inside the data can be aggregated. Although this scheme can use graph convolutional networks to gradually aggregate local information, it will also restrict the ability to aggregate deep information of complex and multi-dimensional data due to its inherent network architecture limitations. In terms of time series prediction, time series modeling is performed by extending long short-term memory neural networks to realize multivariate data prediction. Although this scheme can realize time series modeling, it is limited by its own structure. This time series modeling cannot be well predicted when facing more complex and nonlinear data, and its parameters are also large. Especially when it is necessary to process large-scale data sets, the problems of significant increase in model parameters, high computational cost, large memory usage and difficulty in parallel computing become increasingly prominent. Summary of the invention

[0005] In order to solve the problems of high computational complexity, low prediction efficiency and poor accuracy of time series prediction in the above-mentioned prior art, the present invention proposes a time series prediction method and system based on KAN-enabled graph attention parallel recurrent network, which reduces the computational complexity of time series prediction and improves the efficiency and accuracy of time series prediction.

[0006] In order to achieve the above technical effects, the technical solution of the present invention is as follows:

[0007] A time series prediction method for a KAN-enabled graph attention parallel recurrent network, the method comprising the following steps:

[0008] S1. Obtain timing input data;

[0009] S2. Inputting the time series input data into a feature extraction layer based on fast Fourier transform and discrete wavelet transform, wherein the feature extraction layer preprocesses the time series input data to obtain spliced ​​feature extraction data;

[0010] S3. Inputting the spliced ​​feature extraction data into a feature aggregation layer based on KAN graph attention, wherein the feature aggregation layer preprocesses the spliced ​​feature extraction data to obtain feature aggregation data;

[0011] S4. Input the feature aggregation data into a timing prediction model based on a parallel gated recurrent unit network of KAN and attention, and the timing prediction model outputs timing prediction data.

[0012] Preferably, the preprocessing of the time series input data includes:

[0013] S21. Performing a fast Fourier transform on the time series input data to obtain stationary feature information; and performing a discrete wavelet transform on the time series input data to obtain non-stationary feature information;

[0014] S22. Perform vector splicing on the stationary feature information and the non-stationary feature information to obtain the spliced ​​feature extraction data.

[0015] Preferably, performing a fast Fourier transform on the time series input data to obtain stationary feature information includes:

[0016] S211. Perform fast Fourier transform on the time series input data using the fast Fourier transform formula as follows:

[0017] F = Avg(Amp(FFT(X)))

[0018]

[0019] Wherein, X represents the time series input data, FFT(·) represents the fast Fourier transform calculation, Amp(·) represents the function of calculating the amplitude value, Avg(·) represents the average value calculation function, vector F represents the amplitude value of each frequency, L represents the total length of the time series input data, and f a Indicates frequency, s a represents the number of cycles selected, a∈{1,...,k}, and *argTopk(F) represents the frequency corresponding to the first k largest average amplitude values ​​selected from F, f1,…,f k Indicates the frequency corresponding to the first k largest average amplitude values, f * represents f1,…,f k one of the;

[0020] S212. Based on the selected number of cycles s a , the time series input data X is reshaped into a four-dimensional tensor of the reconstructed time series as follows:

[0021]

[0022] Among them, Padding(·) means extending the time series by 0 along the time dimension. represents the reshaping function, Represents the reconstructed time series;

[0023] S213. Input the reconstructed time series into a linear layer for learning to obtain different time scale information as follows:

[0024]

[0025] in, represents the first weight matrix, represents the first bias;

[0026] S214. Information on different time scales Aggregate to obtain stable feature information The calculation expression is as follows:

[0027]

[0028] in, Weights on different time scales.

[0029] Preferably, the performing discrete wavelet transform on the time series input data to obtain non-stationary feature information includes:

[0030] S221. Perform wavelet decomposition on the time series input data along the time dimension to obtain a decomposition coefficient matrix Y consisting of low-frequency approximation coefficients and high-frequency detail coefficients as follows:

[0031] Y={A J ,D J ,...,D1}

[0032] Among them, A J represents the low-frequency approximation coefficient of the Jth layer, D J Represents the high-frequency detail coefficient of the Jth layer;

[0033] S222. Perform a linear transformation on the decomposition coefficient matrix Y to obtain a new decomposition coefficient matrix as follows:

[0034]

[0035] Among them, W Y represents the second weight matrix, b Y represents the second bias, Represents the new decomposition coefficient matrix The low-frequency approximation coefficient of Represents the new decomposition coefficient matrix The high-frequency detail coefficient of

[0036] S223. Decomposing the new coefficient matrix Perform inverse wavelet transform to obtain the non-stationary vital signs information as follows:

[0037]

[0038] Among them, j∈(1,J), express The jth element in express The jth element in .

[0039] Preferably, the stationary feature information and the non-stationary characteristic information The calculation expression for vector concatenation is as follows:

[0040]

[0041] in, Represents concatenated feature extraction data.

[0042] Preferably, the preprocessing of the splicing feature extraction data includes:

[0043] S31. Constructing the splicing feature extraction data into a graph structure;

[0044] S32. Performing sustainable and adaptive feature fusion learning on the graph structure through a KAN-based graph attention network to obtain feature enhancement information;

[0045] S33. Based on the feature enhancement information, calculating the attention coefficient between the adjacent nodes of the graph structure and the target node;

[0046] S34. Perform weighted summation on the attention coefficients to obtain the feature aggregation data.

[0047] Preferably, the nodes of the graph structure are feature dimensions of the concatenated feature extraction data, and the node features of the graph structure are dimensions of time steps.

[0048] Preferably, the calculation expression of the attention coefficient is as follows:

[0049]

[0050] Among them, α ij represents the attention coefficient, e ij represents the attention score between the target node i and a certain adjacent node j, e ik represents the attention score of the target node i and all adjacent nodes k, represents the set of all adjacent nodes k.

[0051] Preferably, the time series prediction model includes a KAN network and a parallel gated recurrent unit network, and the time series prediction model processes the splicing feature extraction data, including:

[0052] S41. At each time step, the spliced ​​feature extraction data is used as the current input, and the hidden state of the previous time step and the current input are subjected to a KAN network operation to obtain a new hidden state of the previous time step;

[0053] S42. Using the new hidden state of the previous time step as input data of the parallel gated recurrent unit network, normalizing the input data to obtain normalized data, performing bilateral upward projection on the normalized data, extracting timing information through the parallel gated recurrent unit after the upward projection on one side, and processing the other side through the activation function after the upward projection to obtain nonlinear information;

[0054] S43. Performing vector multiplication of the timing information and the nonlinear information to obtain mixed information;

[0055] S44. Down-project the mixed information to obtain down-projected data, and connect the down-projected data with the input data through residual connection to obtain time series prediction data.

[0056] Preferably, the system comprises:

[0057] An acquisition module, used to acquire time series input data;

[0058] A feature extraction module, used for inputting the time series input data into a feature extraction layer based on fast Fourier transform and discrete wavelet transform, wherein the feature extraction layer pre-processes the time series input data to obtain spliced ​​feature extraction data;

[0059] A feature aggregation module, used for inputting the spliced ​​feature extraction data into a feature aggregation layer based on KAN graph attention, wherein the feature aggregation layer pre-processes the spliced ​​feature extraction data to obtain feature aggregation data;

[0060] A timing prediction module is used to input the feature aggregation data into a timing prediction model of a parallel gated recurrent unit network based on KAN and attention, and the timing prediction model outputs timing prediction data.

[0061] Compared with the prior art, the technical solution of the present invention has the following beneficial effects:

[0062] The present invention proposes a time series prediction method and system of a KAN-enabled graph attention parallel recurrent network. First, feature extraction based on fast Fourier transform and discrete wavelet transform is proposed. By making full use of the advantages of fast Fourier transform and discrete wavelet transform in frequency domain analysis and multi-scale time-frequency analysis respectively, the periodic change trend and local non-stationary features in the time series input data are extracted at the same time, thereby improving the feature extraction capability in the face of dynamic changes and variable data environments; then, by integrating the advantages of KAN's efficient and sustainable learning ability and the graph attention network's adaptive feature aggregation, the mapping of node features is realized, the feature expression effect is significantly improved, and a feature aggregation algorithm based on KAN embedded in the graph attention network is designed to effectively solve the fusion learning problem of high-dimensional nonlinear data features. , without increasing the computational cost and complexity too much, improve its feature aggregation ability, and provide a data basis for subsequent time series modeling; further, by integrating KAN and attention mechanism and gated recurrent unit network, a time series prediction model based on KAN and attention parallel gated recurrent unit network is designed. The efficient dynamic nonlinear modeling ability of KAN is used to effectively capture the relationship between historical information and current input, significantly improving the prediction accuracy of the time series prediction model. At the same time, the fast parallel computing capability of the attention mechanism is embedded in the gated recurrent unit, optimizing the network's long-term dependency modeling ability in large-scale time series data, achieving larger-scale long-term information memory on the basis of reducing the number of parameters, reducing the computational complexity of time series prediction, and improving the efficiency and accuracy of time series prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] Figure 1 A flowchart of a time series prediction method using a KAN-enabled graph attention parallel recurrent network proposed in an embodiment of the present invention is shown;

[0064] Figure 2 A schematic diagram showing the principle of a time series prediction method using a KAN-enabled graph attention parallel recurrent network proposed in an embodiment of the present invention;

[0065] Figure 3 The internal structure of the parallel gated recurrent unit network based on KAN and attention proposed in the embodiment of the present invention is shown in FIG.

[0066] Figure 4 A structural block diagram of a time series prediction system of a KAN-enabled graph attention parallel recurrent network proposed in an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0067] The drawings are for illustrative purposes only and should not be construed as limiting the present patent;

[0068] It is understandable to those skilled in the art that some well-known contents may be omitted in the drawings;

[0069] The technical solution of the present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0070] Example 1

[0071] like Figure 1 and Figure 2 As shown, this embodiment proposes a time series prediction method for a KAN-enabled graph attention parallel recurrent network, and the method includes the following steps:

[0072] S1. Obtain timing input data;

[0073] S2. Inputting the time series input data into a feature extraction layer based on fast Fourier transform and discrete wavelet transform, wherein the feature extraction layer preprocesses the time series input data to obtain spliced ​​feature extraction data;

[0074] In S2, the time series input data X is subjected to feature extraction by using fast Fourier transform (FFT) and discrete wavelet transform (DWT), respectively, so as to comprehensively characterize the information inside the data. Since the feature extraction method proposed in this embodiment is for multivariate input data, it has a wide range of applications, and this processing of input data of other models also falls within the scope of intellectual property protection of the present invention. The preprocessing of the time series input data includes:

[0075] S21. Performing a fast Fourier transform on the time series input data to obtain stationary feature information; and performing a discrete wavelet transform on the time series input data to obtain non-stationary feature information;

[0076] The step of performing a fast Fourier transform on the time series input data to obtain stationary feature information includes:

[0077] S211. Perform fast Fourier transform on the time series input data using the fast Fourier transform formula as follows:

[0078] F = Avg(Amp(FFT(X)))

[0079]

[0080] Wherein, X represents the time series input data, FFT(·) represents the fast Fourier transform calculation, Amp(·) represents the function of calculating the amplitude value, which is calculated by the function Avg(·) in d model dimension, we assume that the intrinsic correlation of this periodicity over time remains stable, which allows us to observe the dynamic properties of the intra-series correlation learned by our model; Avg(·) represents the average calculation function, and the vector represents the amplitude value of each frequency, L represents the total length of the time series input data, and f aIndicates frequency, s a represents the number of cycles selected, a∈{1,...,k}, and *argTopk(F) represents the frequency corresponding to the first k largest average amplitude values ​​selected from F, f1,…,f k Indicates the frequency corresponding to the first k largest average amplitude values, f * represents f1,…,f k one of the;

[0081] S212. Based on the selected number of cycles s a , the time series input data X is reshaped into a four-dimensional tensor of the reconstructed time series as follows:

[0082]

[0083] Among them, Padding(·) means expanding the time series by 0 along the time dimension to make it consistent with compatible, represents the reshaping function, Represents the reconstructed time series; since the reconstructed time series reshapes the input into dimensions with periods and numbers of periods, this conversion method can represent the frequency domain information of the data.

[0084] S213. Input the reconstructed time series into a linear layer for learning to obtain different time scale information as follows:

[0085]

[0086] in, represents the first weight matrix, represents the first bias;

[0087] S214. Reshape back to d model ×L dimensions, according to their magnitudes, for different time scale information Aggregate to obtain stable feature information The calculation expression is as follows:

[0088]

[0089] in, Weights on different time scales, SoftMax represents the activation function, F fk represents f k The corresponding amplitude; in this way, the feature extraction of the fast Fourier transform part is completed.

[0090] The step of performing discrete wavelet transform on the time series input data X to obtain non-stationary feature information includes:

[0091] S221. Perform wavelet decomposition on the time series input data X along the time dimension as follows:

[0092]

[0093] Among them, ψ j,k and φ j,k is the wavelet basis function of the jth layer, A j [n] is the low-frequency approximation coefficient of the jth layer, D j [n] is the high-frequency detail coefficient of the jth layer, and the low-frequency approximation coefficient A j Indicates the low-frequency component of the signal, reflecting the overall trend; high-frequency detail coefficient D j Represents the high-frequency components of the signal and captures local changes. Assuming that the decomposition is to the Jth layer, a set of decomposition coefficient matrices Y consisting of low-frequency approximation coefficients and high-frequency detail coefficients are obtained for each feature sequence as follows:

[0094] Y={A J ,D J ,...,D1}

[0095] Among them, A J Represents the low-frequency approximation coefficient of the Jth layer, which is the approximation coefficient of the lowest layer, D J Represents the high-frequency detail coefficient of the Jth layer;

[0096] S222. Perform a linear transformation on the decomposition coefficient matrix Y to further learn and extract the decomposed features to obtain a new decomposition coefficient matrix as follows:

[0097]

[0098] Among them, W Y represents the second weight matrix, b Y represents the second bias, Represents the new decomposition coefficient matrix The low-frequency approximation coefficient of Represents the new decomposition coefficient matrix The high-frequency detail coefficient of

[0099] S223. Decomposing the new coefficient matrix Perform inverse wavelet transform to restore the original data dimension and obtain the non-stationary vital sign information as follows:

[0100]

[0101] The reconstruction process is performed layer by layer until the length of the original signal is restored, j∈(1,J). express The jth element in express The jth element in .

[0102] S22. Perform vector splicing on the stationary feature information and the non-stationary feature information to obtain the spliced ​​feature extraction data.

[0103] In S22, the stationary feature information and the non-stationary characteristic information The calculation expression for vector concatenation is as follows:

[0104]

[0105] in, Represents concatenated feature extraction data.

[0106] S3. Inputting the spliced ​​feature extraction data into a feature aggregation layer based on KAN graph attention, wherein the feature aggregation layer preprocesses the spliced ​​feature extraction data to obtain feature aggregation data;

[0107] In S3, for the KAN-based graph attention feature aggregation algorithm, the key to the algorithm design is the application of KAN and the graph attention network, which can effectively learn more complete information between different variables in the data. The preprocessing of the spliced ​​feature extraction data includes:

[0108] S31. Constructing the spliced ​​feature extraction data into a graph structure; the nodes of the graph structure are feature dimensions of the spliced ​​feature extraction data, and the node features of the graph structure are dimensions of time steps.

[0109] In S31, for , whose nodes are variables and node features are the dimensions of the time steps {s1,...,s L}, the adjacency matrix here is the same as the previous one, which is an adaptive fully connected matrix. In order to further aggregate the time series information between nodes, a KAN-based graph attention network is used for subsequent feature aggregation.

[0110] S32. Performing sustainable and adaptive feature fusion learning on the graph structure through a KAN-based graph attention network to obtain feature enhancement information;

[0111] In S32, for the univariate function of KAN, the Fourier function is selected here:

[0112]

[0113] Among them, d is the characteristic dimension, and the Fourier coefficient a ik and b ikis trainable, and the hyperparameter g is the grid size, which plays a key role in controlling the number of terms (frequencies) used in the Fourier series expansion. Specifically, g determines how many different sine and cosine terms are included in the Fourier coefficients corresponding to each input dimension. Fourier coefficients have significant advantages in computational efficiency and solve the training difficulties caused by B-spline functions. With KAN, we replace the linear transformation in the original image attention network with the KAN layer, making This can further express the characteristic information of the input data, which is beneficial to the subsequent node feature learning.

[0114] S33. Based on the feature enhancement information, calculating the attention coefficient between the adjacent nodes of the graph structure and the target node;

[0115] In S33, for a pair of nodes i and j, the attention coefficient between them is given by the following formula:

[0116]

[0117] Among them, LeakyReLU represents the nonlinear activation function, h i represents the feature vector of node i, h j The feature vector of node j;

[0118]

[0119] Among them, α ij represents the attention coefficient, e ij represents the attention score between the target node i and a certain adjacent node j, e ik represents the attention score of the target node i and all adjacent nodes k, represents the set of all adjacent nodes k;

[0120] S34. The attention coefficient α ij By performing weighted summation, the feature aggregation data is obtained as follows:

[0121]

[0122] S4. Input the feature aggregation data into a timing prediction model based on a parallel gated recurrent unit network of KAN and attention, and the timing prediction model outputs timing prediction data.

[0123] In S4, given the feature aggregation data that has completed feature learning, the feature aggregation data is input into the time series prediction model to achieve time series prediction. Due to the complexity of the structure of the extended long short-term memory neural network and the limited time series information extraction capability, it is necessary to optimize its model prediction efficiency and capability. This step proposes that the time series prediction model includes a KAN network and a parallel gated recurrent unit network. The time series prediction model processes the spliced ​​feature extraction data. First, the KAN part uses the nonlinear expression capability of KAN's efficient and sustainable learning to make the current input x and the hidden state of the previous time step pass through the KAN layer in parallel, specifically including:

[0124] S41. At each time step, the spliced ​​feature extraction data is used as the current input, and the hidden state of the previous time step and the current input are subjected to a KAN network operation to obtain a new hidden state of the previous time step;

[0125] In S41, the KAN network operation process is as follows: the initial or previous hidden state h obtained by the parallel gated recurrent unit and the current input x are passed through S KAN sub-layers in parallel to obtain a new transition hidden variable Then, the L2 norm is used to calculate the weight of the sub-layer, and the weight is normalized and used to aggregate the sub-layer output, and finally the new hidden state of the previous step is obtained; the above KAN network operation process is expressed by the following formula:

[0126]

[0127]

[0128] At each time step of S41, the hidden state of the previous time step and the current input are subjected to KAN operation to generate a new hidden state. This fusion strategy can adaptively balance the importance of the previous time step information and the current input, thereby optimizing the information flow of the parallel recurrent unit network, enabling the model to capture more complex temporal relationships and further improve the modeling accuracy of long-term dependencies. This combination can help subsequent parallel gated recurrent units better identify the importance of the current time step input and the previous step hidden state, thereby achieving better temporal information extraction.

[0129] S42. Using the initial or new hidden state of the previous time step as input data of the parallel gated recurrent unit network, normalizing the input data to obtain normalized data, performing bilateral upward projection on the normalized data, extracting timing information through the parallel gated recurrent unit after the upward projection on one side, and processing the other side through the activation function after the upward projection to obtain nonlinear information;

[0130] In S42, the parallel gated recurrent unit network structure is as follows Figure 3 As shown, the specific calculation formula is as follows:

[0131] z t =exp(W z x t +b z )

[0132] r t =exp(W r x t +b r )

[0133] q t =W q x t +b q

[0134]

[0135] v t =W v x t +b v

[0136]

[0137] n t =W n x t +U n (r t ⊙n t-1 )+k t

[0138]

[0139]

[0140] Among them, Z t represents the update gate, r t Represents the reset gate. In the figure, K, q, and v represent the key, query, and value of the corresponding attention mechanism, respectively. t represents the normalization gate, C t represents the cell state with the attention mechanism added, represents the candidate hidden state, h t Represents the hidden state of the output;

[0141] S43. Performing vector multiplication of the timing information and the nonlinear information to obtain mixed information;

[0142] S44. Down-project the mixed information to obtain down-projected data, and connect the down-projected data with the input data through residual connection to obtain time series prediction data.

[0143] It should be noted that, compared with the ordinary extended long short-term memory network (xLSTM), the parallel gated recurrent unit network retains the more critical attention mechanism and covariance update rule to enhance the model storage capacity and related information retrieval ability; at the same time, the three gates are reduced to two reset gates and update gates, which simplifies some calculations and reduces the number of parameters of the model. This improvement enables the model to achieve fast and accurate predictions. In order to enable the model to adapt to a variety of application scenarios, the specific structure of the parallel gated recurrent unit network is designed in this embodiment. The data first passes through the normalization layer to stabilize the training process and accelerate convergence; then the data is bilaterally projected to map its features to a higher dimension, extracting timing information through the parallel gated recurrent unit on one side and adding nonlinear information through the activation function Gelu on the other side; then vector multiplication is performed to enhance the interaction between features; finally, it is projected back and output after the residual connection is used to alleviate the information loss problem in the deep network.

[0144] For the external structure of the parallel gated recurrent unit network, the input data is first batch normalized to stabilize the training process and accelerate convergence. Then two up-projections are performed to map the dimensions to a higher space. One of the projections passes through the parallel gated recurrent unit to capture its internal timing information; the other projection passes through the activation function Gelu to express the internal complex nonlinear information. The data after the two processings is multiplied by vectors to mix the information together. Finally, the dimension is mapped back through down-projection, and then through the residual connection to alleviate the gradient vanishing problem, and finally the data is output. The entire algorithm flow is as follows:

[0145] 1) Initialize the prediction model parameters;

[0146] 2) The input data of each node is aggregated and divided according to the time step dimension;

[0147] 3) The input data at the first time point passes through the KAN layer in parallel with the initial hidden state to obtain a new transition hidden state;

[0148] 4) The new transition hidden state and the input are sequentially extracted through a parallel gated recurrent unit to obtain a new hidden state;

[0149] 5) Repeat steps 1-5 to complete the extraction of time series information for multiple time steps;

[0150] 6) Output the predicted value.

[0151] In this embodiment, a feature extraction method based on fast Fourier transform and discrete wavelet transform is first proposed. By making full use of the advantages of fast Fourier transform and discrete wavelet transform in frequency domain analysis and multi-scale time-frequency analysis respectively, the periodic change trend and local non-stationary features in the time series input data are extracted at the same time, thereby improving the feature extraction ability in the face of dynamic changes and variable data environments; then, by integrating the advantages of KAN's efficient and sustainable learning ability and the adaptive feature aggregation of the graph attention network, the mapping of node features is realized, the feature expression effect is significantly improved, and a feature aggregation algorithm based on KAN embedded in the graph attention network is designed to effectively solve the fusion learning problem of high-dimensional nonlinear data features without increasing the computational cost and complexity too much. Under the premise of reducing the complexity, its feature aggregation ability is improved to provide a data basis for subsequent time series modeling; further, by integrating KAN and attention mechanism and gated recurrent unit network, a time series prediction model based on KAN and attention parallel gated recurrent unit network is designed. The efficient dynamic nonlinear modeling ability of KAN is used to effectively capture the relationship between historical information and current input, which significantly improves the prediction accuracy of the time series prediction model. At the same time, the fast parallel computing capability of the attention mechanism is embedded in the gated recurrent unit, which optimizes the network's long-term dependency modeling ability in large-scale time series data, and realizes larger-scale long-term information memory on the basis of reducing the number of parameters, reducing the computational complexity of time series prediction and improving the efficiency and accuracy of time series prediction.

[0152] Example 2

[0153] This embodiment further illustrates a timing prediction method of a KAN-enabled graph attention parallel recurrent network. The above embodiment proposes a timing prediction method of a KAN-enabled graph attention parallel recurrent network, which mainly focuses on three aspects: feature extraction, feature aggregation, and timing modeling, including:

[0154] First: To solve the problem of internal information representation of complex nonlinear data, a feature extraction method based on fast Fourier transform and discrete wavelet transform is proposed;

[0155] Second: After extracting features from the data, in order to further solve the problem of high-dimensional nonlinear data feature fusion and transfer information between nodes, a feature aggregation layer algorithm based on KAN graph attention is designed;

[0156] Third: In order to solve the problem of difficulty in modeling complex time series data, a time series prediction model based on KAN and attention-based parallel gated recurrent unit network is established to provide a model basis for achieving accurate prediction.

[0157] Here, firstly, the fast Fourier transform and discrete wavelet transform are used to simultaneously extract the periodic change trend and local non-stationary features in the time series input data to ensure that the feature representation has wide adaptability and stability; then, the KAN-based graph attention feature aggregation algorithm is used to efficiently map the node features to the high-dimensional space and fuse the relevant information between different nodes; further, after completing the feature aggregation, a time series prediction model based on the parallel gated recurrent unit network of KAN and attention is constructed to achieve efficient and accurate time series modeling and prediction. The specific technical solutions for the above three steps are as follows:

[0158] The first is feature extraction based on fast Fourier transform and discrete wavelet transform. The present invention proposes a feature extraction strategy based on fast Fourier transform and discrete wavelet transform, which aims to transform time series input data into frequency domain and time-frequency domain features, and deeply reveal the intrinsic information of the data. Specifically, the fast Fourier transform is used to extract the frequency domain information of the data and capture the periodic change characteristics; while the discrete wavelet transform provides multi-scale time-frequency analysis capabilities for identifying local non-stationary features. By combining these two transformation methods, the potential dynamic change characteristics in multivariate data can be fully characterized, and the adaptability of the model in different application scenarios can be enhanced. To achieve this feature extraction, the feature extraction strategy is as follows:

[0159] The time series input data will be divided into two branches, which are processed by Fast Fourier Transform and Discrete Wavelet Transform respectively. Fast Fourier Transform is used to process data with obvious frequency characteristics and relatively stable from the frequency view. We assume that the correlation inherent in this periodic view of time evolution remains stable in the time step dimension. This view guides us to observe the dynamic properties of the inter-sequence and intra-sequence correlations learned by the model. Fast Fourier Transform will convert the data from the time domain to the frequency domain, find the top k frequencies with the largest amplitude, and calculate the period with larger energy, which means that the data contains more information in this period. Based on the selected period, we can also get the number of periods, so as to reshape the time step dimension of the input data in the time domain into the dimension of period * number of periods, thereby representing the frequency domain information of the data. After the data is mapped to different time scales in a reasonable way, a linear layer is used to learn the information of this high-dimensional data. Finally, based on the information of different time scales obtained at the beginning, the amplitude is converted into weights on the corresponding time scale, and the data is reshaped back to the original dimension by weight, so as to achieve complete feature extraction on different time scales.

[0160] Discrete wavelet transform is used to analyze data with time-varying characteristics, especially in the case of non-stationary data and denoising. First, the input data is wavelet decomposed along the time dimension seq to obtain multi-scale feature information. Specifically, for each sample and feature sequence, the time series is decomposed into a set of low-frequency approximation coefficients and high-frequency detail coefficients through wavelet transform. These two coefficients reflect the global trend and local changes of the signal respectively. If multi-layer wavelet decomposition is performed, multi-level features can be obtained, including the lowest frequency approximation coefficients and higher frequency detail coefficients. Usually, the length of each level of decomposition coefficients is kept unchanged by zero padding or symmetric padding, so that the decomposed coefficients can maintain the same time step seq as the original sequence. After completing the wavelet decomposition, these decomposition coefficients are linearly transformed to further learn and extract features. For each set of approximation and detail coefficients, a linear layer is applied to adjust the feature dimension, thereby enhancing the features of the wavelet decomposition. This step aims to enable the model to learn and optimize multi-scale features at a deeper level. Finally, an inverse wavelet transform is performed on the linearly transformed coefficients to reconstruct the adjusted features to the same dimension as the input. Through the inverse transformation, the signal is reconstructed layer by layer starting from the lowest frequency layer. The final reconstruction result can accurately match the dimension of the original input, ensuring that no structural information is lost during the feature extraction process. Finally, the data obtained by the two different transformations are vectorized and spliced ​​to obtain the output of the feature extraction.

[0161] Here, the present invention constructs a feature extraction strategy based on fast Fourier transform and discrete wavelet transform, which can comprehensively characterize the periodic change trend and local non-stationary characteristics in the data. By combining the frequency domain transformation of fast Fourier transform and the multi-scale time-frequency analysis of discrete wavelet transform, it shows excellent feature extraction capabilities in the face of dynamic changes and variable data environments, laying a solid data foundation for the subsequent feature aggregation process.

[0162] Secondly, the feature aggregation algorithm design of KAN-based graph attention is proposed. After feature extraction, the present invention aims to further learn and aggregate the correlation information between different variables, and proposes a feature aggregation algorithm based on KAN-based graph attention. This algorithm combines the sustainable and efficient nonlinear learning ability of KAN and the adaptive feature aggregation characteristics of the graph attention network, and can effectively map node features to high-dimensional space, and transmit and integrate information between different nodes, thereby efficiently solving the feature fusion problem of high-dimensional nonlinear data. The steps of the feature aggregation algorithm proposed in this embodiment are as follows:

[0163] The first step is to input the data that has been feature extracted into the feature aggregation algorithm of graph attention learning. We need to construct the converted data into a graph. Here, the nodes of our graph are the feature dimensions of the data, and the node features of the graph are the dimensions of the time step, so that the relevant temporal properties are included in the calculation of the graph. We assume that the nodes are fully connected, so the initial adjacency matrix is ​​a matrix of all 1s. We then multiply this initial adjacency matrix by a weight matrix to obtain a new adjacency matrix, which represents the connection strength between nodes. This adjacency matrix can be updated by gradient descent, which allows the model to learn the relationship between features by itself and achieve adaptive perception of the interaction strength between nodes.

[0164] In the second step, we learn this graph structure through a graph attention network based on KAN. Based on the Kolmogorov-Arnold representation theorem, KAN can decompose any multivariate function into a series of single variable functions and sum operations, and achieve efficient and sustainable network learning by introducing a learnable activation function. The graph attention network solves the limitations of traditional graph neural networks in terms of neighbor feature node weighting and dynamic feature updates by introducing an attention mechanism. By embedding KAN into the graph attention network, the advantages of both can be combined to achieve sustainable and adaptive feature fusion learning. The form of KAN is as follows:

[0165]

[0166] Among them, φ q,p is to map each input variable x p It is a univariate function of , which is a trainable function, and generally uses B-spline function. Different from the original graph attention network that uses linear matrix to change the dimension, here we use the nonlinear expression ability of KAN to map node features to high dimensions and achieve feature enhancement.

[0167] Then, the weights between adjacent nodes and target nodes are calculated according to the same graph attention calculation, and new node features are calculated by weighted aggregation. This improved graph attention based on KAN has stronger nonlinear expression ability and can better adaptively adjust weights to capture more complex relationships between nodes.

[0168] In summary, the graph attention feature aggregation algorithm based on KAN embedding proposed in this paper successfully solves the problem of complex information and difficulty in fusion after feature extraction. The algorithm effectively combines the attention mechanism between nodes and the sustainable and efficient learning ability of KAN, realizes the efficient aggregation of internal features of the data, and provides stable feature support for the subsequent time series modeling link.

[0169] Finally, a timing prediction model based on a parallel gated recurrent unit network of KAN and attention is presented. In multivariable timing prediction, it is particularly important to build a model that can accurately and efficiently capture the dynamic changes of data. To this end, the present invention proposes a timing prediction model based on a parallel gated recurrent unit network of KAN and attention. The timing prediction model utilizes the structural simplicity of the gated recurrent unit and the efficient nonlinear modeling capability of KAN to fully learn the long-term dependencies in the data, significantly improving the prediction accuracy and operating efficiency of the timing prediction model, thereby achieving fast and accurate timing prediction. The principles of the timing prediction model proposed in the present invention are summarized as follows:

[0170] The original extended long short-term memory neural network (xLSTM) is composed of multiple sLSTM (scalar long short-term memory network) and mLSTM (matrix long short-term memory network) modules stacked together. Specifically, by introducing an exponential gating mechanism, xLSTM can more effectively modify and update its memory when processing new information, improving the model's ability to adapt to dynamically changing information. Using the attention mechanism, xLSTM can perform parallel computing, greatly shortening the time for model training and inference, and enhancing the model's ability to capture key information. At the same time, scalar memory, scalar update, and memory hybrid functions further improve the flexibility and efficiency of xLSTM in memory management and updating, giving it a strong advantage in processing complex sequence data and large-scale data sets.

[0171] In the structural design of the time series prediction model, we retained the key structures in mLSTM: exponential gating and attention mechanism, and embedded them into the gated recurrent unit to obtain a parallel gated recurrent unit network. This improvement enables the model to have fewer parameters while also being able to manage long-term memory well and achieve efficient prediction.

[0172] In addition, in order to further enhance the nonlinear expression ability and long-term dependency modeling ability of the model, the present invention introduces KAN into the parallel recurrent unit network structure.

[0173] At each time step, the hidden state of the previous time step is KAN-operated with the current input to generate a new hidden state. This fusion strategy can adaptively balance the importance of the previous time step information and the current input, thereby optimizing the information flow of the parallel recurrent unit network, enabling the model to capture more complex temporal relationships and further improve the modeling accuracy of long-term dependencies.

[0174] Finally, in order to make the model adaptable to a variety of application scenarios, the present invention designs a specific structure of a parallel gated recurrent unit network. The data first passes through a normalization layer to stabilize the training process and accelerate convergence; then the data is bilaterally projected to map its features to a higher dimension, extracting timing information through a parallel gated recurrent unit on one side and adding nonlinear information through the activation function Gelu on the other side; then vector multiplication is performed to enhance the interaction between features; finally, the data is projected back and output after a residual connection to alleviate the problem of information loss in the deep network.

[0175] This parallel gated recurrent unit network structure is more suitable for specific prediction applications and can achieve more stable and accurate timing modeling. In summary, the timing prediction model based on the parallel gated recurrent unit network of KAN and attention achieves efficient timing modeling through parallel gated recurrent units, and integrates KAN's learning of timing relationships to effectively solve the problems of insufficient model prediction accuracy and low operating efficiency. With the parallel computing characteristics of the attention mechanism and the efficient and continuous learning ability of KAN, the algorithm successfully achieves high-precision and high-efficiency timing prediction.

[0176] Compared with the prior art, the time series prediction method of a KAN-enabled graph attention parallel recurrent network proposed in this embodiment has the following advantages:

[0177] 1) Feature extraction is performed based on the fast Fourier transform (FFT) and discrete wavelet transform (DWT) methods, which fully integrates the advantages of frequency domain analysis and multi-scale time-frequency analysis. This enables the model to show excellent characterization capabilities in extracting periodic change trends and local non-stationary features of data, thereby significantly improving the generalization ability and applicability of the model.

[0178] 2) We fully utilize KAN’s efficient and sustainable learning expression capabilities, realize node feature mapping through the KAN layer, and significantly improve the feature expression effect. At the same time, with the flexibility of the graph structure, we construct the data into a graph structure, and use the graph attention mechanism to deeply learn the feature correlation between nodes, thereby achieving adaptive feature aggregation and effectively solving the problem of insufficient data feature aggregation.

[0179] 3) This paper uses KAN's efficient dynamic nonlinear modeling capabilities to effectively capture the relationship between historical information and current input, significantly improving the model's prediction accuracy. At the same time, the fast parallel computing capabilities of the attention mechanism are embedded in the gated recurrent unit (GRU), optimizing the network's long-term dependency modeling capabilities in large-scale time series data. On the basis of reducing the number of parameters, a larger-scale long-term information memory is achieved to ensure that the model can achieve fast and accurate predictions.

[0180] Example 3

[0181] See also Figure 4 This embodiment also proposes a KAN-enabled graph attention parallel recurrent network time series prediction system, the system comprising:

[0182] An acquisition module, used to acquire time series input data;

[0183] A feature extraction module, used for inputting the time series input data into a feature extraction layer based on fast Fourier transform and discrete wavelet transform, wherein the feature extraction layer pre-processes the time series input data to obtain spliced ​​feature extraction data;

[0184] A feature aggregation module, used for inputting the spliced ​​feature extraction data into a feature aggregation layer based on KAN graph attention, wherein the feature aggregation layer pre-processes the spliced ​​feature extraction data to obtain feature aggregation data;

[0185] A timing prediction module is used to input the feature aggregation data into a timing prediction model of a parallel gated recurrent unit network based on KAN and attention, and the timing prediction model outputs timing prediction data.

[0186] In this embodiment, a feature extraction method based on fast Fourier transform and discrete wavelet transform is first proposed. By making full use of the advantages of fast Fourier transform and discrete wavelet transform in frequency domain analysis and multi-scale time-frequency analysis respectively, the periodic change trend and local non-stationary features in the time series input data are extracted at the same time, thereby improving the feature extraction ability in the face of dynamic changes and variable data environments; then, by integrating the advantages of KAN's efficient and sustainable learning ability and the adaptive feature aggregation of the graph attention network, the mapping of node features is realized, the feature expression effect is significantly improved, and a feature aggregation algorithm based on KAN embedded in the graph attention network is designed to effectively solve the fusion learning problem of high-dimensional nonlinear data features without increasing the computational cost and complexity too much. Under the premise of reducing the complexity, its feature aggregation ability is improved to provide a data basis for subsequent time series modeling; further, by integrating KAN and attention mechanism and gated recurrent unit network, a time series prediction model based on KAN and attention parallel gated recurrent unit network is designed. The efficient dynamic nonlinear modeling ability of KAN is used to effectively capture the relationship between historical information and current input, which significantly improves the prediction accuracy of the time series prediction model. At the same time, the fast parallel computing capability of the attention mechanism is embedded in the gated recurrent unit, which optimizes the network's long-term dependency modeling ability in large-scale time series data, and realizes larger-scale long-term information memory on the basis of reducing the number of parameters, reducing the computational complexity of time series prediction and improving the efficiency and accuracy of time series prediction.

[0187] Obviously, the above embodiments of the present invention are only examples for clearly explaining the present invention, and are not intended to limit the implementation methods of the present invention. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all implementation methods here. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the claims of the present invention.

Claims

1. A time series prediction method based on KAN-enabled graph attention parallel recurrent network, characterized in that: The method comprises the following steps: S1. Obtain timing input data; S2. Inputting the time series input data into a feature extraction layer based on fast Fourier transform and discrete wavelet transform, wherein the feature extraction layer preprocesses the time series input data to obtain spliced ​​feature extraction data; S3. Inputting the spliced ​​feature extraction data into a feature aggregation layer based on KAN graph attention, wherein the feature aggregation layer preprocesses the spliced ​​feature extraction data to obtain feature aggregation data; S4. Input the feature aggregation data into a timing prediction model based on a parallel gated recurrent unit network of KAN and attention, and the timing prediction model outputs timing prediction data.

2. According to claim 1, the KAN-enabled graph attention parallel recurrent network timing prediction method is characterized in that: The preprocessing of the time series input data includes: S21. Performing a fast Fourier transform on the time series input data to obtain stationary feature information; and performing a discrete wavelet transform on the time series input data to obtain non-stationary feature information; S22. Perform vector splicing on the stationary feature information and the non-stationary feature information to obtain the spliced ​​feature extraction data.

3. The timing prediction method of KAN-enabled graph attention parallel recurrent network according to claim 2 is characterized in that: The step of performing a fast Fourier transform on the time series input data to obtain stationary feature information includes: S211. Perform fast Fourier transform on the time series input data using the fast Fourier transform formula as follows: F = Avg(Amp(FFT(X))) Wherein, X represents the time series input data, FFT(·) represents the fast Fourier transform calculation, Amp(·) represents the function of calculating the amplitude value, Avg(·) represents the average value calculation function, vector F represents the amplitude value of each frequency, L represents the total length of the time series input data, and f a Indicates frequency, s a represents the number of cycles selected, a∈{1,...,k}, and *argTopk(F) represents the frequency corresponding to the first k largest average amplitude values ​​selected from F, f1,…,f k Indicates the frequency corresponding to the first k largest average amplitude values, f * represents f1,…,f k one of the; S212. Based on the selected number of cycles s a , the time series input data X is reshaped into a four-dimensional tensor of the reconstructed time series as follows: Among them, Padding(·) means extending the time series by 0 along the time dimension. represents the reshaping function, Represents the reconstructed time series; S213. Input the reconstructed time series into a linear layer for learning to obtain different time scale information as follows: in, represents the first weight matrix, represents the first bias; S214. Information on different time scales Aggregate to obtain stable feature information The calculation expression is as follows: in, Weights on different time scales.

4. The timing prediction method of KAN-enabled graph attention parallel recurrent network according to claim 2 is characterized in that: The step of performing discrete wavelet transform on the time series input data to obtain non-stationary feature information includes: S221. Perform wavelet decomposition on the time series input data along the time dimension to obtain a decomposition coefficient matrix Y consisting of low-frequency approximation coefficients and high-frequency detail coefficients as follows: Y={A J ,D J ,...,D1} Among them, A J represents the low-frequency approximation coefficient of the Jth layer, D J Represents the high-frequency detail coefficient of the Jth layer; S222. Perform a linear transformation on the decomposition coefficient matrix Y to obtain a new decomposition coefficient matrix as follows: Among them, W Y represents the second weight matrix, b Y represents the second bias, Represents the new decomposition coefficient matrix The low-frequency approximation coefficient of Represents the new decomposition coefficient matrix The high-frequency detail coefficient of S223. Decomposition of the new coefficient matrix Perform inverse wavelet transform to obtain the non-stationary vital signs information as follows: Among them, j∈(1,J), express The jth element in express The jth element in .

5. The timing prediction method of KAN-enabled graph attention parallel recurrent network according to claim 4 is characterized in that: The stationary feature information and the non-stationary characteristic information The calculation expression for vector concatenation is as follows: in, Represents concatenated feature extraction data.

6. The timing prediction method of KAN-enabled graph attention parallel recurrent network according to claim 1 is characterized in that: The preprocessing of the splicing feature extraction data comprises: S31. Constructing the splicing feature extraction data into a graph structure; S32. Performing sustainable and adaptive feature fusion learning on the graph structure through a KAN-based graph attention network to obtain feature enhancement information; S33. Based on the feature enhancement information, calculating the attention coefficient between the adjacent nodes of the graph structure and the target node; S34. Perform weighted summation on the attention coefficients to obtain the feature aggregation data.

7. The timing prediction method of KAN-enabled graph attention parallel recurrent network according to claim 6 is characterized in that: The nodes of the graph structure are feature dimensions of the concatenated feature extraction data, and the node features of the graph structure are dimensions of time steps.

8. The timing prediction method of KAN-enabled graph attention parallel recurrent network according to claim 6 is characterized in that: The calculation expression of the attention coefficient is as follows: Among them, α ij represents the attention coefficient, e ij represents the attention score between the target node i and a certain adjacent node j, e ik represents the attention score of the target node i and all adjacent nodes k, represents the set of all adjacent nodes k.

9. The timing prediction method of KAN-enabled graph attention parallel recurrent network according to claim 1 is characterized in that: The time series prediction model includes a KAN network and a parallel gated recurrent unit network, and the time series prediction model processes the splicing feature extraction data, including: S41. At each time step, the spliced ​​feature extraction data is used as the current input, and the hidden state of the previous time step and the current input are subjected to a KAN network operation to obtain a new hidden state of the previous time step; S42. Using the new hidden state of the previous time step as input data of the parallel gated recurrent unit network, normalizing the input data to obtain normalized data, performing bilateral upward projection on the normalized data, extracting timing information through the parallel gated recurrent unit after the upward projection on one side, and processing the other side through the activation function after the upward projection to obtain nonlinear information; S43. Performing vector multiplication of the timing information and the nonlinear information to obtain mixed information; S44. Down-project the mixed information to obtain down-projected data, and connect the down-projected data with the input data through residual connection to obtain time series prediction data.

10. A KAN-enabled graph attention parallel recurrent network time series prediction system, characterized in that: The system comprises: An acquisition module, used to acquire time series input data; A feature extraction module, used for inputting the time series input data into a feature extraction layer based on fast Fourier transform and discrete wavelet transform, wherein the feature extraction layer pre-processes the time series input data to obtain spliced ​​feature extraction data; A feature aggregation module, used for inputting the spliced ​​feature extraction data into a feature aggregation layer based on KAN graph attention, wherein the feature aggregation layer pre-processes the spliced ​​feature extraction data to obtain feature aggregation data; A timing prediction module is used to input the feature aggregation data into a timing prediction model of a parallel gated recurrent unit network based on KAN and attention, and the timing prediction model outputs timing prediction data.

Images