A large model time series prediction method and system based on Kolmogorov-Arnold Networks enhancement

Abstract

The present application relates to the technical field of large model prediction, in particular to a large model time series prediction method and system based on Kolmogorov-Arnold Networks enhancement. It comprises: constructing an external knowledge base and a time series dataset; constructing a large model based on Kolmogorov-Arnold Networks enhancement of MLP multilayer perceptron and KAN neural network parallel architecture; training the large model through the training set based on the low-rank tuning fine-tuning technology; extracting domain knowledge and scenario knowledge through the GraphRAG architecture and the KAN neural network; forming the unified representation of the task scenario specific knowledge and the original sequence information, and inputting it into the trained large model for the final time series prediction. The present application realizes the parallel integration of KAN and MLP in the hidden layer of LLM, constructs a parallel coupling mechanism, fully utilizes the advantage of KAN mathematical representation ability, and enhances the processing capacity of the model for complex multi-element time series.

Description

Technical Field

[0001] This invention relates to the field of large model prediction technology, and in particular to a method and system for time series prediction of large models based on Kolmogorov-Arnold Networks enhancement. Background Technology

[0002] Time series forecasting, as a crucial support for a wide range of applications, has been an important and indispensable research hotspot in academia over the past decade. From modeling and forecasting trends in financial markets and economic indicators to predicting logistics demand, monitoring healthcare indicators, meteorological and hydrological analysis, and forecasting electricity consumption and pricing, time series forecasting technology has demonstrated tremendous practical value.

[0003] With the rapid development of Large Language Models (LLMs), research utilizing these models to assist time series forecasting has achieved significant results in various application fields. This progress has introduced new perspectives and methodological innovations to the field of time series forecasting. Currently, common methods are mainly based on the analysis of auxiliary information related to the forecasting task, including dataset metadata and supplementary descriptions, semantic alignment, and other techniques. These techniques can leverage the excellent text processing capabilities of large oracle models to steadily improve the performance of time series forecasting. However, existing methods lack in-depth descriptions of the multivariate causal relationships in the current forecasting scenario and specialized knowledge of the physical nature of the current task, making it difficult for LLMs to achieve deep adaptation to time series forecasting tasks and limiting further breakthroughs in overall system performance.

[0004] Traditional time series forecasting methods include statistical models and deep learning-based model architectures. These methods typically rely on fitting and extrapolating the statistical properties and spatiotemporal dependencies of historical numerical sequences. This purely numerically driven paradigm has significant limitations: it treats time series as isolated streams of numbers, lacks an understanding of the rich semantic context and underlying physical generation mechanisms behind the data, and struggles to capture the deep causal logic among multiple variables in complex systems.

[0005] Existing time series prediction methods based on large language models can be divided into three technical approaches. The first is to transform discrete time series values ​​into natural language descriptions that large language models can understand through prompt engineering. In this process, many methods incorporate professional knowledge related to the prediction task, embedding it as text within the prompt. However, this discretized representation disrupts the inherent numerical continuity and mathematical logic of the time series, making it difficult for the model to capture subtle numerical trends. Furthermore, converting high-frequency values ​​into lengthy natural language descriptions leads to a sharp decrease in information density, making it difficult for the model to handle long-term historical data. This exposes LLM (Large Language Model) outputs, as generative models, to the risk of insufficient accuracy and numerical illusions.

[0006] Second, a strategy of temporal decomposition and semantic abstraction is employed. This type of model identifies global and local attributes of time series using techniques such as classifiers, aligns these statistical features with natural language descriptions, and inputs them into the LLM to assist in understanding the macroscopic behavior of the data, thereby achieving effective prediction of future trends. However, existing LLM architectures are almost all based on the Multilayer Perceptron (MLP) architecture, which lacks sensitivity to time series and is not suitable for modeling time dependencies. Furthermore, MLPs struggle to simulate long-term time patterns, such as delay effects and periodic trends spanning extended time ranges. In scenarios involving high-dimensional and multivariate time series, MLPs are also prone to overfitting, thus affecting generalization performance.

[0007] Third, a strategy of cross-modal alignment and feature reprogramming is employed. This type of method constructs a shared embedding space, aligning and fusing external knowledge such as the numerical features of time series data with spatial topology. By inputting the fused multimodal vectors into the LLM, its powerful multi-head self-attention mechanism is used to capture complex long-range dependencies and dynamic spatial associations, obtaining a high-dimensional hidden layer representation of the model. This representation is then mapped back to the numerical space through a lightweight linear projection layer, resulting in accurate predictions. However, the external knowledge introduced by this method is usually a static structure, ignoring the dynamic evolution of causal dependencies between variables. Static alignment alone is insufficient to capture such time-varying spatial correlations, limiting the upper limit of inference in dynamic and complex systems.

[0008] While existing research in the field of large-scale model time series forecasting has endowed LLMs with context-aware capabilities by integrating dataset metadata and supplementary descriptions, this supplementary information based on shallow text often remains superficial. Existing integration paradigms fail to explicitly model complex multivariate dynamic causal networks and lack a deep representation of the physical laws governing data evolution. This lack of mechanistic knowledge makes it difficult for LLMs to construct prediction paths that conform to physical laws when dealing with non-stationary and complex systems, due to their reliance on shallow reasoning based on statistical correlations. This severely limits the full realization of their general reasoning potential in the time series domain.

[0009] Furthermore, existing research indicates that traditional MLPs rely on pre-defined and fixed node activation functions. This point-like nonlinear transformation is frozen after training, causing the model to attempt to approximate a dynamic system that may fluctuate dramatically over time using a static set of basis functions. However, real-world time series often exhibit significant non-stationarity and distribution drift. The single, fixed activation pattern of MLPs lacks adaptive inductive bias and cannot dynamically adjust its response function based on the real-time distribution characteristics of the input data. This results in the model only capturing averaged statistical patterns when faced with time series data exhibiting multimodal distributions or abrupt changes, failing to accurately characterize the subtle dynamic evolution processes within the data. Simultaneously, in the globally-based fully connected architecture of MLPs, the inherent temporal dependencies and local topological structures of time series are disrupted, causing strong correlations between adjacent time points and weak correlations between distant time points to be mixed in the same weight matrix, making it difficult for the model to effectively distinguish between signal and noise. Therefore, the inherent limitations of MLP, such as structural rigidity and lack of local perception, lead to a decline in its performance when facing increasingly complex modern time-series prediction tasks. A new architecture that can balance expressiveness, sparsity, and interpretability is needed to compensate for these deficiencies. Summary of the Invention

[0010] To address the technical problems of performance waste, semantic information loss, and lack of interpretability in existing technologies, this invention provides a large-model temporal prediction method and system based on Kolmogorov-Arnold Networks enhancement. The technical solution is as follows: On the one hand, a large-model time series prediction method based on Kolmogorov-Arnold Networks enhancement is provided, characterized in that the method includes: S1. Construct an external knowledge base; introduce a general time series dataset; divide the time series dataset into a training set, a validation set, and a test set; S2. Construct a large model enhanced by Kolmogorov-Arnold Networks based on the parallel architecture of MLP multilayer perceptron and KAN neural network; train the large model using the training set based on low-rank parameter tuning fine-tuning technology; S3. Evaluate the performance of the trained large model using the validation set; S4. Extract domain and scenario knowledge from the test set data using the GraphRAG architecture (Graph-based Retrieval-Augmented Generation, a knowledge graph retrieval and generation framework) and the KAN neural network; S5. The domain knowledge and scenario knowledge are combined to form a unified representation of the task scenario-specific knowledge and the original sequence information, which is then input into the large model after evaluation and verification for the final time series prediction.

[0011] Optionally, in S1, an external knowledge base is constructed; a general time-series dataset is introduced, including: An external knowledge base is constructed by collecting text data from academic papers, technical reports, and expert documents from the Internet and professional databases. The text data includes descriptions of complex causal relationships between variables, the impact of different temperatures and times on the target variable, the time series data characteristics of different datasets, and data for short-term time series prediction. Introduce a general time series dataset.

[0012] Optionally, in S2, a large model enhanced with Kolmogorov-Arnold Networks based on a parallel architecture of MLP multilayer perceptron and KAN neural network is constructed, including: Integrate KAN and MLP in parallel into the hidden layers of an LLM large language model; Make the input of KAN and its intermediate hierarchical structure fit the input and intermediate structure of the LLM internal architecture; After receiving input data, LLM will perform training iterations. This architecture integrates KAN into the hidden layer of Transformer and runs in parallel with traditional MLP modules to jointly extract temporal features. A large model enhanced by Kolmogorov-Arnold Networks based on the parallel architecture of MLP multilayer perceptron and KAN neural network was obtained.

[0013] Optionally, in S2, a large model is trained using a training set based on low-rank parameter tuning techniques, including: Based on low-rank parameter tuning and fine-tuning techniques, the pre-trained backbone network is frozen, and a small trainable low-rank matrix is ​​introduced. The large model is then trained using the training set.

[0014] Optionally, in S3, the performance of the trained large model is evaluated using a validation set, including: Obtain validation set data and evaluate and verify the model using the validation set after each round of large model training. If the performance evaluation passes, time series prediction is performed using the test set; if the performance evaluation fails, the model is retrained until the evaluation passes.

[0015] Optionally, in S4, domain knowledge and scenario knowledge are extracted from the test set data using the GraphRAG architecture and the KAN neural network, including: The test set data was processed using the GraphRAG architecture to perform graph-based processing and knowledge extraction on these unstructured texts; By employing a knowledge graph-based retrieval strategy, high-confidence task-specific knowledge is filtered out. The input historical data is standardized and segmented, and each segment of data is processed by KAN to output a function expression relationship under a specific time series scenario.

[0016] Optionally, in S5, domain knowledge and scenario knowledge are combined to form a unified representation of task-scenario-specific knowledge and original sequence information, which is then input into the large model after evaluation and verification for final time-series prediction, including: We acquire task-specific knowledge with high confidence, functional relationships extracted by KAN, and a segmented piece of time series data, and use all three types of data as input to the large model. The hidden state of the last layer of the large model is used as a high-dimensional feature representation, and the high-dimensional feature representation is input into the linear prediction head. Through feature mapping, a high-precision time series prediction result is finally produced.

[0017] On the other hand, a large model time series prediction system based on Kolmogorov-Arnold Networks is provided. This system is applied to the large model time series prediction method based on Kolmogorov-Arnold Networks and includes: The dataset building module is used to build an external knowledge base; it introduces a general time series dataset; and it divides the time series dataset into a training set, a validation set, and a test set. The model building and training module is used to build a large model enhanced by Kolmogorov-Arnold Networks based on the parallel architecture of MLP multilayer perceptron and KAN neural network; the large model is trained on the training set based on low-rank parameter tuning and fine-tuning techniques. The performance evaluation module is used to evaluate the performance of a large trained model using a validation set. The data input module is used to extract domain knowledge and scenario knowledge from the test set data through the GraphRAG architecture and the KAN neural network. The prediction module is used to combine domain knowledge and scenario knowledge into a unified representation of task-scenario-specific knowledge and original sequence information, which is then input into the large model after evaluation and verification for final time series prediction.

[0018] On the other hand, a large model time series prediction device based on Kolmogorov-Arnold Networks is provided. The large model time series prediction device based on Kolmogorov-Arnold Networks includes: a processor; a memory, wherein computer-readable instructions are stored in the memory, and when the computer-readable instructions are executed by the processor, any one of the methods in the large model time series prediction method based on Kolmogorov-Arnold Networks described above is implemented.

[0019] On the other hand, a computer-readable storage medium is provided, wherein at least one instruction is stored therein, the at least one instruction being loaded and executed by a processor to implement any of the above-described methods of large model time series prediction based on Kolmogorov-Arnold Networks enhancement.

[0020] The beneficial effects of the technical solutions provided in the embodiments of the present invention include at least the following: This invention proposes an enhanced LLM framework for time series prediction based on a Kolmogorov-Arnold network. By combining domain-specific expertise extracted by GraphRAG with scene-specific intrinsic functional relationships extracted by a KAN network, a two-layer knowledge architecture with rich representational relationships is formed, significantly improving the understanding depth of large language models regarding complex external temporal patterns. Furthermore, a parallel coupling mechanism is introduced at the feedforward network level of the LLM to combine KAN with MLP. This allows the model to maintain the large-scale parameter processing capabilities of MLP while leveraging the mathematical representation capabilities of KAN to compensate for the shortcomings of traditional MLP in processing time series data.

[0021] Unlike traditional large-scale model prediction methods, this invention leverages the superiority of KAN in processing time-series data to endow large models with more refined feature representation capabilities and better time-series prediction performance. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 This is a flowchart of a large model time series prediction method based on Kolmogorov-Arnold Networks enhancement provided by an embodiment of the present invention; Figure 2This is a block diagram of a large model time series prediction system based on Kolmogorov-Arnold Networks enhancement provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the structure of the electronic device provided in an embodiment of the present invention. Detailed Implementation

[0024] The technical solution of the present invention will now be described with reference to the accompanying drawings.

[0025] In embodiments of the present invention, words such as "exemplarily," "for example," etc., are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" in the present invention should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the word "exemplary" is intended to present the concept in a concrete manner. Furthermore, in embodiments of the present invention, the meaning expressed by "and / or" can be both, or either one.

[0026] In this embodiment of the invention, sometimes a subscript such as W1 may be written in a non-subscript form such as W1. When the difference is not emphasized, the meaning they express is the same.

[0027] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0028] This invention provides a large model time series prediction method based on Kolmogorov-Arnold Networks enhancement. This method can be implemented by a large model time series prediction device based on Kolmogorov-Arnold Networks enhancement, which can be a terminal or a server. Figure 1 The flowchart shown is for a large model time series prediction method enhanced by Kolmogorov-Arnold Networks. The processing flow of this method may include the following steps: S1. Construct an external knowledge base; introduce a general time series dataset; divide the time series dataset into a training set, a validation set, and a test set; In one feasible implementation, in S1, an external knowledge base is constructed; a general time-series dataset is introduced; and the time-series dataset is divided into a training set, a validation set, and a test set, including: An external knowledge base is constructed by collecting text data from academic papers, technical reports, and expert documents from the Internet and professional databases. The text data includes descriptions of complex causal relationships between variables, the impact of different temperatures and times on the target variable, the time series data characteristics of different datasets, and data for short-term time series prediction. A general time-series dataset is introduced; the general time-series dataset in this invention is a publicly available time-series dataset in the prior art (such as the ETT dataset, Traffic dataset, etc.); the time-series dataset is normally processed and standardized in sequence to obtain standardized sequences; the standardized sequences are segmented using a block-based strategy; The processed time series dataset is divided into training set, validation set and test set.

[0029] In one feasible implementation, the raw data required by the present invention mainly includes an external knowledge base and a time series dataset. The external knowledge base consists of academic papers, technical reports, and expert documents collected extensively from the Internet and professional databases. It includes descriptions of complex causal relationships between variables, the impact of different temperatures and times on the target variable, the time series data characteristics of different datasets, and the characteristics of short-term time series prediction.

[0030] Training set: mainly includes the ETT time series dataset and the Traffic dataset, used to train model performance.

[0031] Validation set: Contains some time-series data that the model has not seen before, used to evaluate model performance.

[0032] Test set: Independent of the training and validation sets, it is used specifically for final performance testing.

[0033] S2. Construct a large model enhanced by Kolmogorov-Arnold Networks based on the parallel architecture of MLP multilayer perceptron and KAN neural network; train the large model using the training set based on low-rank parameter tuning fine-tuning technology; In one feasible implementation, in S2, a large model enhanced with Kolmogorov-Arnold Networks based on a parallel architecture of MLP multilayer perceptron and KAN neural network is constructed, including: Integrate KAN and MLP in parallel into the hidden layers of an LLM large language model; Make the input of KAN and its intermediate hierarchical structure fit the input and intermediate structure of the LLM internal architecture; After receiving input data, LLM will perform training iterations. This architecture integrates KAN into the hidden layer of Transformer and runs in parallel with traditional MLP modules to jointly extract temporal features. A large model enhanced by Kolmogorov-Arnold Networks based on the parallel architecture of MLP multilayer perceptron and KAN neural network was obtained.

[0034] One feasible implementation proposes introducing a KAN architecture parallel to MLP into large language models. Existing research shows that Multilayer Perceptrons (MLPs) lack sensitivity to temporal order and are not the optimal network structure for modeling temporal dependencies. Compared to traditional MLPs, KANs exhibit superior function approximation capabilities with fewer parameters. Furthermore, KANs, combining explicit sparsity induction and pruning mechanisms, possess better generalization ability and interpretability. The superior function fitting capabilities of KANs can be leveraged to enhance the effective capture of complex temporal patterns (such as long-term dependencies and periodic trends) by LLMs, thereby improving the temporal modeling capabilities of the original LLMs without compromising their general reasoning abilities. The challenge of this parallel architecture lies in adapting the input and intermediate layer structure of the KAN network to the input and intermediate structure of the LLM internal architecture. This requires introducing fully connected layers as adaptation modules to linearly map the high-dimensional input data of the hidden layers of the large model, transforming it into an input dimension that KAN can directly process. Simultaneously, at the parallel output of the MLP and KAN, fully connected layers are used again to dimensionally align the inter-layer features extracted by KAN, transforming them into a tensor format that can be added and fused with the data from the MLP hidden layers. To enable KAN to run in a large model environment, some of the KAN's initialization structure needs to be transformed into a post-processing method to address the problem of KAN adaptation to large models. The LLM undergoes training iterations after receiving input data. This architecture integrates KAN into the hidden layers of the Transformer, running in parallel with the traditional MLP module to jointly extract temporal features. In each Transformer block, the KAN module is fused with the MLP, which can be represented as:

[0035] in, This represents the input to each hidden layer. This represents the output of each hidden layer. This represents a fully connected layer that performs relation mapping on the last hidden layer of the KAN to match the MLP. This represents a learnable weight parameter used to balance the weight ratios of MLP and KAN. This represents layer normalization and residual connections before passing the outputs of KAN and MLP to the next Transformer block.

[0036] In one feasible implementation, in S2, a large model is trained using a training set based on low-rank parameter tuning techniques, including: Based on low-rank parameter tuning and fine-tuning techniques, the pre-trained backbone network is frozen, and a small trainable low-rank matrix is ​​introduced. The large model is then trained using the training set.

[0037] In one feasible implementation, this invention employs a low-rank parameter tuning technique for model training. This method freezes the pre-trained backbone network and introduces a small, trainable low-rank matrix to enhance the model's ability to capture complex temporal dependencies.

[0038] S3. Evaluate the performance of the trained large model using the validation set; In one feasible implementation, in S3, the performance of the trained large model is evaluated using a validation set, including: Obtain validation set data and evaluate and verify the model using the validation set after each round of large model training. If the performance evaluation passes, time series prediction is performed using the test set; if the performance evaluation fails, the model is retrained until the evaluation passes.

[0039] In one feasible implementation, evaluation using the validation set typically involves comparing the loss curves of the validation set and the training set to determine whether the training is overfitting and whether training should be prematurely terminated if the loss curve of the validation set fails to decrease further.

[0040] S4. Extract domain knowledge and scenario knowledge from the test set data using the GraphRAG architecture and the KAN neural network; In one feasible implementation, in S4, domain knowledge and scenario knowledge are extracted from the test set data using the GraphRAG architecture and the KAN neural network, including: The test set data was processed using the GraphRAG architecture to perform graph-based processing and knowledge extraction on these unstructured texts; By employing a knowledge graph-based retrieval strategy, high-confidence task-specific knowledge is filtered out. The input historical data is standardized and segmented, and each segment of data is processed by KAN to output a function expression relationship under a specific time series scenario.

[0041] In one feasible implementation, before the model begins operation, this invention constructs an external knowledge base for the target domain. The target domain can be understood as follows: since time series forecasting can be subdivided into different domains such as wind power forecasting and electricity price forecasting, the specific domain here is the category that the large model is currently predicting. For example, if the dataset is Traffic, then relevant content on traffic flow forecasting will be collected. First, technical reports, academic papers, and expert documents related to the time series forecasting task are extensively collected from the internet and professional databases, forming an original corpus. Subsequently, the GraphRAG architecture is used to perform graph-based processing and knowledge extraction on these unstructured texts. Through a knowledge graph-based retrieval strategy, high-confidence task-specific knowledge is selected. This process can be represented as:

[0042] in, This represents the original corpus that the present invention found on the internet. This invention represents the success of the invention. Domain knowledge is retrieved and filtered. When the model starts operating, the system first standardizes and partitions the input time-series dataset, then uses KAN to output a function expression for each data block under a specific time-series scenario. To enhance the model's robustness to non-stationary time series, this invention first performs instance normalization on the input time-series dataset to alleviate the common distribution shift problem in time-series prediction. By standardizing each input sample, this invention maps the original values ​​to a stable probability distribution space, ensuring that the model maintains consistency in feature expression when facing data of different magnitudes. Simultaneously, a partitioning strategy is used to segment the standardized sequence, cutting long sequences into sub-sequence blocks containing local semantic information. This helps the model capture local trend and fluctuation features, reducing the model's computational complexity while preserving key temporal context information.

[0043] In this embodiment of the invention, regarding data processing: To enhance the robustness of the model to non-stationary time series, the input time series is first normalized to mitigate the common distribution shift problem in time series prediction. By standardizing each input sample, the original values ​​are mapped to a stable probability distribution space, ensuring that the model maintains consistency in feature representation when facing data of different magnitudes. Simultaneously, a block-based strategy is employed to segment the standardized sequence, cutting long sequences into sub-sequence blocks containing local semantic information. This helps the model capture local trend and fluctuation features, reducing computational complexity while preserving key temporal context information.

[0044] S5. Combine domain knowledge and scenario knowledge to form a unified representation of task scenario-specific knowledge and original sequence information, and input it into the trained large model for final time series prediction.

[0045] In one feasible implementation, in step S5, domain knowledge and scenario knowledge are combined to form a unified representation of task-scenario-specific knowledge and original sequence information, which is then input into the large model after evaluation and verification for final time-series prediction, including: We acquire task-specific knowledge with high confidence, functional relationships extracted by KAN, and a segmented piece of time series data, and use all three types of data as input to the large model. The hidden state of the last layer of the large model is used as a high-dimensional feature representation, and the high-dimensional feature representation is input into the linear prediction head. Through feature mapping, a high-precision time series prediction result is finally produced.

[0046] In one feasible implementation, the functional representation relation under a specific time-series scenario, along with the domain knowledge retrieved and filtered by GraphRAG and the time-series data, are used as input to the large language model:

[0047] in This represents the input to the large model. This represents the functional relationship extracted by KAN. This represents a segmented block of time-series data. This two-layer knowledge mechanism design forms a unified representation of task-specific knowledge and original sequence information, significantly improving the model's performance in complex time-series prediction scenarios.

[0048] In one feasible implementation, during inference, the model processes the input sequence enhanced by the two-layer knowledge mechanism of this invention, which combines task-specific expertise from external sources with scene-specific temporal patterns in the KAN. The enhanced representation is then passed through a modified Transformer backbone network. Finally, the hidden states of the last layer of the large language model are used as high-dimensional feature representations and input into a linear prediction head, ultimately producing high-precision temporal prediction results through feature mapping.

[0049] This invention proposes an LLM framework enhanced by Kolmogorov Arnold Networks. This framework employs a two-layer knowledge extraction mechanism: it utilizes KAN to capture scene-specific intrinsic functional logic and combines it with GraphRAG to retrieve domain-specific structured professional knowledge. Together with time-series data, feature fusion enhances the LLM's ability to understand and deconstruct complex external knowledge. Furthermore, this invention constructs a parallel coupling mechanism by implementing parallel integration of KAN and MLP in the hidden layers of the LLM, fully leveraging the advantages of KAN's mathematical representation capabilities to enhance the model's ability to process complex multivariate time series data. This invention demonstrates significant innovation in real-time performance, cost control, and adaptability to complex scenarios.

[0050] This invention first uses domain-specific expertise extracted by GraphRAG and scene-specific intrinsic functional relationships extracted by KAN network to form a two-layer knowledge architecture with rich representational relationships, which significantly improves the understanding depth of large language models of complex external temporal patterns; Secondly, a parallel coupling mechanism is introduced at the feedforward network layer of LLM, combining KAN with MLP. This allows the model to maintain the large-scale parameter processing capability of MLP while leveraging the mathematical representation capability of KAN to compensate for the shortcomings of traditional MLP in processing time-series data. Finally, unlike traditional large-scale model prediction methods, KAN's superior ability to process time-series data endows large models with more refined feature representation capabilities and better time-series prediction performance.

[0051] Figure 2 This is a block diagram of a large model time series prediction system 300 based on Kolmogorov-Arnold Networks enhancement, illustrated according to an exemplary embodiment. The system 300 is used for a large model time series prediction method based on Kolmogorov-Arnold Networks enhancement. (Refer to...) Figure 2 The system includes a dataset construction module 310, a model construction and training module 320, a performance evaluation module 330, a data input module 340, and a prediction module 360. Among them: The dataset construction module 310 is used to build an external knowledge base; it introduces a general time series dataset; and it divides the time series dataset into a training set, a validation set, and a test set. The model building and training module 320 is used to build a large model enhanced by Kolmogorov-Arnold Networks based on the parallel architecture of MLP multilayer perceptron and KAN neural network; the large model is trained on the training set based on low-rank parameter tuning and fine-tuning techniques. The performance evaluation module 330 is used to evaluate the performance of the trained large model using a validation set. The data input module 340 is used to extract domain knowledge and scenario knowledge from the test set data through the GraphRAG architecture and the KAN neural network. The prediction module 350 is used to combine domain knowledge and scenario knowledge into a unified representation of task scenario-specific knowledge and original sequence information, and input it into the large model after evaluation and verification for final time series prediction.

[0052] Optionally, the dataset construction module 310 is used to construct an external knowledge base by collecting text data from academic papers, technical reports and expert documents from the Internet and professional databases; wherein, the text data includes descriptions of complex causal relationships between variables, the effects of different temperatures and times on the target variable, the time series data characteristics of different datasets and data for short-term time series prediction. Introduce a general time series dataset.

[0053] Optionally, the model building training module 320 is used to integrate KAN and MLP in parallel into the hidden layers of the LLM large language model; Make the input of KAN and its intermediate hierarchical structure fit the input and intermediate structure of the LLM internal architecture; After receiving input data, LLM will perform training iterations. This architecture integrates KAN into the hidden layer of Transformer and runs in parallel with traditional MLP modules to jointly extract temporal features. A large model enhanced by Kolmogorov-Arnold Networks based on the parallel architecture of MLP multilayer perceptron and KAN neural network was obtained.

[0054] Optionally, the model building training module 320 is used to freeze the pre-trained backbone network based on low-rank parameter tuning and fine-tuning techniques, introduce a small trainable low-rank matrix, and train the large model using the training set.

[0055] Optionally, the performance evaluation module 330 is used to obtain the data of the validation set and input the data of the validation set into the trained large model for performance evaluation. If the performance evaluation passes, time series prediction is performed using the test set; if the performance evaluation fails, the model is retrained until the evaluation passes.

[0056] Optionally, the data input module 340 is used to perform graphing and knowledge extraction on these unstructured texts using the GraphRAG architecture; By employing a knowledge graph-based retrieval strategy, high-confidence task-specific knowledge is filtered out. The input historical data is standardized and segmented, and each segment of data is processed by KAN to output a function expression relationship under a specific time series scenario.

[0057] Optionally, the prediction module 350 is used to acquire the task-specific knowledge with high confidence, the functional relationship extracted by KAN, and a segmented piece of time series data, and use the three types of data as input to the large model simultaneously. The hidden state of the last layer of the large model is used as a high-dimensional feature representation, and the high-dimensional feature representation is input into the linear prediction head. Through feature mapping, a high-precision time series prediction result is finally produced.

[0058] This invention proposes an LLM framework enhanced by Kolmogorov Arnold Networks. This framework employs a two-layer knowledge extraction mechanism: it utilizes KAN to capture scene-specific intrinsic functional logic and combines it with GraphRAG to retrieve domain-specific structured professional knowledge. Together with time-series data, feature fusion enhances the LLM's ability to understand and deconstruct complex external knowledge. Furthermore, this invention constructs a parallel coupling mechanism by implementing parallel integration of KAN and MLP in the hidden layers of the LLM, fully leveraging the advantages of KAN's mathematical representation capabilities to enhance the model's ability to process complex multivariate time series data. This invention demonstrates significant innovation in real-time performance, cost control, and adaptability to complex scenarios.

[0059] This invention first uses domain-specific expertise extracted by GraphRAG and scene-specific intrinsic functional relationships extracted by KAN network to form a two-layer knowledge architecture with rich representational relationships, which significantly improves the understanding depth of large language models of complex external temporal patterns; Secondly, a parallel coupling mechanism is introduced at the feedforward network layer of LLM, combining KAN with MLP. This allows the model to maintain the large-scale parameter processing capability of MLP while leveraging the mathematical representation capability of KAN to compensate for the shortcomings of traditional MLP in processing time-series data. Finally, unlike traditional large-scale model prediction methods, KAN's superior ability to process time-series data endows large models with more refined feature representation capabilities and better time-series prediction performance.

[0060] Figure 3 This is a schematic diagram of the structure of a large model time series prediction device based on Kolmogorov-Arnold Networks enhancement provided in an embodiment of the present invention, as shown below. Figure 3 As shown, a large model time series prediction device based on Kolmogorov-Arnold Networks enhancements may include the above-mentioned... Figure 2 The diagram illustrates a large model time series prediction system based on Kolmogorov-Arnold Networks enhancement. Optionally, a large model time series prediction device 410 based on Kolmogorov-Arnold Networks enhancement may include a first processor 2001.

[0061] Optionally, a large model time series prediction device 410 based on Kolmogorov-Arnold Networks enhancement may also include a memory 2002 and a transceiver 2003.

[0062] The first processor 2001, memory 2002, and transceiver 2003 can be connected via a communication bus.

[0063] The following is combined Figure 3 A detailed description of the various components of a large model time series prediction device 410 enhanced with Kolmogorov-Arnold Networks is provided below: The first processor 2001 is the control center of a large model timing prediction device 410 enhanced by Kolmogorov-Arnold Networks. It can be a single processor or a collective term for multiple processing elements. For example, the first processor 2001 can be one or more central processing units (CPUs), application-specific integrated circuits (ASICs), or one or more integrated circuits configured to implement embodiments of the present invention, such as one or more digital signal processors (DSPs), or one or more field-programmable gate arrays (FPGAs).

[0064] Optionally, the first processor 2001 can perform various functions of a large model time series prediction device 410 based on Kolmogorov-Arnold Networks by running or executing software programs stored in memory 2002 and calling data stored in memory 2002.

[0065] In a specific implementation, as one example, the first processor 2001 may include one or more CPUs, for example... Figure 3 CPU0 and CPU1 are shown in the diagram.

[0066] In a specific implementation, as one example, a large model time series prediction device 410 based on Kolmogorov-Arnold Networks enhancements may also include multiple processors, for example... Figure 3 The first processor 2001 and the second processor 2004 are shown in the diagram. Each of these processors can be a single-core processor or a multi-core processor. Here, a processor can refer to one or more devices, circuits, and / or processing cores used to process data (such as computer program instructions).

[0067] The memory 2002 is used to store the software program that executes the present invention, and is controlled by the first processor 2001 to execute it. The specific implementation method can be referred to the above method embodiment, and will not be repeated here.

[0068] Optionally, the memory 2002 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. The memory 2002 may be integrated with the first processor 2001 or may exist independently, and may be interfaced through an interface circuit of a large model timing prediction device 410 enhanced with Kolmogorov-Arnold Networks. Figure 3 (Not shown in the image) is coupled to the first processor 2001, and this embodiment of the invention does not specifically limit this.

[0069] The transceiver 2003 is used to communicate with network devices or with terminal devices.

[0070] Alternatively, transceiver 2003 may include a receiver and a transmitter. Figure 3 (Not shown separately). The receiver is used to implement the receiving function, and the transmitter is used to implement the transmitting function.

[0071] Alternatively, the transceiver 2003 can be integrated with the first processor 2001 or exist independently, and can be interfaced through an interface circuit of a large model timing prediction device 410 enhanced with Kolmogorov-Arnold Networks. Figure 3 (Not shown in the image) is coupled to the first processor 2001, and this embodiment of the invention does not specifically limit this.

[0072] It should be noted that, Figure 3The structure of a large model time-series prediction device 410 based on Kolmogorov-Arnold Networks enhancement shown in the figure does not constitute a limitation on the router. Actual knowledge structure identification devices may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0073] Furthermore, the technical effect of a large model time series prediction device 410 enhanced by Kolmogorov-Arnold Networks can be referred to the technical effect of a large model time series prediction method enhanced by Kolmogorov-Arnold Networks described in the above method embodiments, and will not be repeated here.

[0074] It should be understood that the first processor 2001 in the embodiments of the present invention may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0075] It should also be understood that the memory in the embodiments of the present invention can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous DRAM (DDR SDRAM), enhanced synchronous DRAM (ESDRAM), synchronous linked DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0076] The above embodiments can be implemented, in whole or in part, by software, hardware (such as circuits), firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable sensors. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.

[0077] It should be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. A and B can be singular or plural. Additionally, the character " / " in this article generally indicates an "or" relationship between the preceding and following related objects, but it can also represent an "and / or" relationship. Please refer to the context for a more accurate understanding.

[0078] It should be understood that, in various embodiments of the present invention, the order of the above-mentioned process numbers does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0079] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0080] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0081] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0082] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention.

[0083] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A large model time series prediction method based on Kolmogorov-Arnold Networks enhancement, characterized in that, The term includes: S1. Construct an external knowledge base; introduce a general time-series dataset; The time series dataset is divided into a training set, a validation set, and a test set; S2. Construct a large model enhanced with Kolmogorov-Arnold Networks based on the parallel architecture of MLP multilayer perceptron and KAN neural network; train the large model using the training set based on low-rank parameter tuning fine-tuning technology; S3. Evaluate the performance of the trained large model using the validation set; S4. Extract domain knowledge and scenario knowledge from the test set data using the GraphRAG architecture and the KAN neural network; S5. The domain knowledge and scenario knowledge are combined to form a unified representation of the task scenario-specific knowledge and the original sequence information, which is then input into the large model after evaluation and verification for the final time series prediction.

2. The large model time series forecasting method enhanced based on Kolmogorov-Arnold Networks according to claim 1, characterized in that, In step S1, an external knowledge base is constructed; a general time-series dataset is introduced, including: An external knowledge base is constructed by collecting text data from academic papers, technical reports, and expert documents from the Internet and professional databases. The text data includes descriptions of complex causal relationships between variables, the impact of different temperatures and times on the target variable, the time series data characteristics of different datasets, and data for short-term time series prediction. Introduce a general time series dataset.

3. The large model time series forecasting method enhanced based on Kolmogorov-Arnold Networks according to claim 2, characterized in that, In S2, a large model enhanced with Kolmogorov-Arnold Networks based on a parallel architecture of MLP multilayer perceptron and KAN neural network is constructed, including: Integrate KAN and MLP in parallel into the hidden layers of an LLM large language model; Make the input of KAN and its intermediate hierarchical structure fit the input and intermediate structure of the LLM internal architecture; After receiving input data, LLM will perform training iterations. This architecture integrates KAN into the hidden layer of Transformer and runs in parallel with traditional MLP modules to jointly extract temporal features. A large model enhanced by Kolmogorov-Arnold Networks based on the parallel architecture of MLP multilayer perceptron and KAN neural network was obtained.

4. The large model time series forecasting method enhanced based on Kolmogorov-Arnold Networks according to claim 3, characterized in that, In S2, a large model is trained using a training set based on low-rank parameter tuning techniques, including: Based on low-rank parameter tuning and fine-tuning techniques, the pre-trained backbone network is frozen, and a small trainable low-rank matrix is ​​introduced. The large model is then trained using the training set.

5. The large model time series forecasting method enhanced based on Kolmogorov-Arnold Networks according to claim 4, characterized in that, In S3, the performance of the trained large model is evaluated using a validation set, including: Obtain validation set data and evaluate and verify the model using the validation set after each round of large model training. If the performance evaluation passes, time series prediction is performed using the test set; if the performance evaluation fails, the model is retrained until the evaluation passes.

6. The large model time series prediction method based on Kolmogorov-Arnold Networks enhancement according to claim 5, characterized in that, In step S4, domain knowledge and scenario knowledge are extracted from the test set data using the GraphRAG architecture and the KAN neural network, including: The test set data was processed using the GraphRAG architecture to perform graph-based processing and knowledge extraction on these unstructured texts; By employing a knowledge graph-based retrieval strategy, high-confidence task-specific knowledge is filtered out. The input historical data is standardized and segmented, and each segment of data is processed by KAN to output a function expression relationship under a specific time series scenario.

7. The large model time series forecasting method enhanced based on Kolmogorov-Arnold Networks according to claim 6, characterized in that, In step S5, domain knowledge and scenario knowledge are combined to form a unified representation of task-scenario-specific knowledge and original sequence information, which is then input into the large model after evaluation and verification for final time-series prediction, including: We acquire task-specific knowledge with high confidence, functional relationships extracted by KAN, and a segmented piece of time series data, and use all three types of data as input to the large model. The hidden state of the last layer of the large model is used as a high-dimensional feature representation, and the high-dimensional feature representation is input into the linear prediction head. Through feature mapping, a high-precision time series prediction result is finally produced.

8. A large model time series prediction system based on Kolmogorov-Arnold Networks enhancement, wherein the large model time series prediction system based on Kolmogorov-Arnold Networks enhancement is used to implement the large model time series prediction method based on Kolmogorov-Arnold Networks enhancement as described in any one of claims 1-7, characterized in that, The system includes: The dataset building module is used to build an external knowledge base; it introduces a general time series dataset; and it divides the time series dataset into a training set, a validation set, and a test set. The model building and training module is used to build a large model enhanced by Kolmogorov-Arnold Networks based on the parallel architecture of MLP multilayer perceptron and KAN neural network; the large model is trained on the training set based on low-rank parameter tuning and fine-tuning techniques. The performance evaluation module is used to evaluate the performance of a large trained model using a validation set. The data input module is used to extract domain knowledge and scenario knowledge from the test set data through the GraphRAG architecture and the KAN neural network. The prediction module is used to combine domain knowledge and scenario knowledge into a unified representation of task-scenario-specific knowledge and original sequence information, which is then input into the large model after evaluation and verification for final time series prediction.

9. A large model time series prediction device based on Kolmogorov-Arnold Networks enhancement, the large model time series prediction device based on Kolmogorov-Arnold Networks enhancement comprising: processor; A memory storing computer-readable instructions, which, when executed by the processor, implement any one of the methods in the large model time series prediction method based on Kolmogorov-Arnold Networks enhancement as described in any one of claims 1-7.

10. A computer-readable storage medium storing at least one instruction, said at least one instruction being loaded and executed by a processor to implement any one of the methods in the large model time series prediction method based on Kolmogorov-Arnold Networks enhancement as described in any one of claims 1-7.

Images