Time sequence prediction method and device based on wavelet transform and large language model

Abstract

The invention belongs to the technical field of artificial intelligence and data analysis, and discloses a time sequence prediction method and device based on wavelet transform and a large language model. The time sequence prediction method comprises the following steps: obtaining known historical time sequence data; based on the obtained known historical time sequence data, performing time sequence prediction in a future time period by using the trained time sequence prediction model to obtain predicted values of three frequency domain components of low frequency, intermediate frequency and high frequency; adding the obtained predicted values of the low frequency domain component, the intermediate frequency domain component and the high frequency domain component, and carrying out inverse normalization processing on the added result to obtain a final time sequence prediction result; the time sequence prediction model comprises a wavelet decomposition module, a backbone network of a pre-training large language model and a prediction head. According to the technical scheme disclosed by the invention, the accuracy of time sequence prediction can be remarkably improved.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence and data analysis technology, and in particular relates to a time series prediction method and device based on wavelet transform and large language model. Background Art

[0002] Time series forecasting is a core task in data science, widely used in fields such as financial market analysis, energy load forecasting, weather forecasting, and health monitoring. In recent years, with the rise of pre-trained models for natural language processing and computer vision, researchers have begun to focus on the application of pre-trained models to time series tasks. However, how to effectively apply pre-trained large language models to time series forecasting remains an open problem.

[0003] Time series often contain complex frequency-domain features, primarily manifested in their temporal patterns, which manifest as a combination of long-term trends, cyclical fluctuations, and random noise. This is a key characteristic that distinguishes time series data from classical sequential data. Existing large-model-based methods have yet to fully exploit the complex patterns in time series data, and the accuracy of time series forecasting needs to be further improved.

[0004] Specifically, existing research can be roughly divided into the following two categories based on the differences in the methods of using pre-trained models. The first category uses the prompt mechanism or text embedding mechanism of a large language model to align time series data into natural language modalities that are easier for the large model to handle through different processing methods, and uses the language knowledge of the pre-trained model to complete the prediction task. The core idea of ​​these methods is to leverage the powerful capabilities of the large language model in natural language understanding to transform time series tasks into text generation or classification problems. The problem with this type of method is that it is usually necessary to manually construct prompt templates based on the characteristics of the dataset, which relies heavily on human experience. Other studies have shown that pre-trained large language models can achieve better accuracy than the baseline in zero-shot reasoning by simply using appropriate embedding methods for numbers. This type of method uses discrete encoding for numerical values, thereby converting continuous regression tasks into classification tasks of predicting tokens. However, this method results in loss of numerical precision and destruction of continuity, making it difficult to use time series-related methods for further optimization. The second type of method no longer uses a large language model as the backbone network. Instead, it draws on the pre-training idea of ​​the large language model to train a specialized time series pre-training model on a large amount of time series data for use in a variety of downstream time series tasks. This method has achieved great success in the fields of computer vision and natural language processing, but it still faces some key problems in the time series field: First, the limitation of data scale. Existing time series datasets are usually small in scale, with only a few thousand or even a few hundred samples, far from the millions or even billions required for pre-training. In addition, the strong heterogeneity of time series data is also a major challenge. Time series data in different fields and scenarios vary significantly. For example, the feature distribution of meteorological data, financial data, and medical data varies greatly, which makes training a general time series pre-training model twice as difficult. Summary of the Invention

[0005] The present invention aims to provide a time series prediction method and apparatus based on wavelet transform and large language model to solve one or more of the above-mentioned technical problems. The technical solution disclosed in the present invention can significantly improve the accuracy of time series prediction.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions: In a first aspect, the present invention provides a time series prediction method based on wavelet transform and large language model, comprising the following steps: Obtain known historical time series data; Based on the acquired known historical time series data, a trained time series prediction model is used to perform time series prediction for a future time period, and predicted values ​​of three frequency domain components, namely, low frequency, medium frequency, and high frequency, are obtained; wherein, the normalization processing method same as that used in the training phase of the time series prediction model is firstly used to obtain a normalization processing result of the known historical time series data; and then, the normalization processing result of the known historical time series data is input into the trained time series prediction model to perform time series prediction for a future time period, and predicted values ​​of the three frequency domain components, namely, low frequency, medium frequency, and high frequency, are obtained; The predicted values ​​of the three frequency domain components of low frequency, medium frequency and high frequency are added together, and the added results are inverse normalized to obtain the final time series prediction results; Wherein, the time series prediction model includes: The wavelet decomposition module is used to input the time series and decompose it into three frequency domain components: low frequency, medium frequency and high frequency. The low frequency, medium frequency and high frequency components correspond to long-term trends, cyclical patterns and short-term fluctuations respectively. The backbone network of the pre-trained large language model is used to extract features based on the decomposition results of the wavelet decomposition module; The prediction head is used to generate prediction values ​​for the three frequency domain components of low frequency, medium frequency, and high frequency based on the features extracted by the backbone network of the large language model.

[0007] A further improvement of the technical solution of the present invention is that the training step of the time series prediction model includes: Perform mapping transformation on the three frequency domain components decomposed by the wavelet decomposition module to obtain the three mapped frequency domain components; divide each mapped frequency domain component into multiple overlapping data blocks, and map each data block to the embedding space of the pre-trained large language model through linear projection; Each frequency domain component is concatenated with its mixed prompt and used as the input of the backbone network. The pre-trained large language model is fine-tuned using low-rank adaptation technology to adapt it to the time series prediction task. The output of the fine-tuned large language model is converted into the predicted values ​​of each frequency domain component through a linear layer. Among them, the mixed prompts of each frequency domain component are obtained by splicing the hard prompts and soft prompts of each frequency domain component; in addition, the loss is calculated and the model parameters are updated according to the dual loss optimization mechanism; the dual loss optimization mechanism simultaneously optimizes the prediction loss used to measure the difference between the model prediction value and the true value, and the decomposition and reconstruction loss used to measure the difference between the frequency domain components of the model decomposition and the components obtained by the standard decomposition method.

[0008] A further improvement of the technical solution of the present invention is that, in the wavelet decomposition module, the wavelet transform uses the db4 wavelet basis function to perform multi-scale decomposition on the time series, with a decomposition level of 3.

[0009] A further improvement of the technical solution of the present invention lies in that, in the step of mapping and transforming the frequency domain components after decomposition by the wavelet decomposition module to obtain the three mapped frequency domain components, the low-frequency frequency domain components are linearly mapped to keep the sequence length unchanged; the medium-frequency and high-frequency frequency domain components are first mapped to a higher-dimensional space through a linear layer, and then mapped back to the original dimension through a ReLU nonlinear activation function and a second linear layer.

[0010] A further improvement of the technical solution of the present invention is that, in the step of dividing the mapped frequency domain components into a plurality of overlapping data blocks, the data blocks are divided in a sliding window manner, and there is overlap between adjacent data blocks.

[0011] A further improvement to the technical solution of the present invention lies in that, in the step of obtaining the mixed prompts of each frequency domain component by splicing the hard prompts and soft prompts of each frequency domain component, the hard prompt mechanism uses predefined text prompts to guide the large language model to process different frequency domain components; and the soft prompt mechanism uses a learnable prompt vector pool to dynamically select the most relevant prompt vector combination based on the input sequence characteristics.

[0012] A further improvement of the technical solution of the present invention is that, in the step of first adopting the same normalization processing method as the time series prediction model training stage to obtain the normalization processing result of the known historical time series data, the input time series is normalized, the mean μ and standard deviation σ of the training set are calculated, and the training set is normalized according to x. _normalized = (x - μ) / σ for normalization; where x is the input time series, x _normalized is the normalized result.

[0013] A second aspect of the present invention provides a time series prediction system based on wavelet transform and large language model, comprising: Data acquisition module, which obtains known historical time series data; A prediction value acquisition module is used to perform time series prediction for future time periods using a trained time series prediction model based on the acquired known historical time series data, and obtain prediction values ​​for the three frequency domain components of low frequency, medium frequency, and high frequency. The module first adopts the same normalization processing method as that used in the training phase of the time series prediction model to obtain the normalization processing results of the known historical time series data; then, the normalization processing results of the known historical time series data are input into the trained time series prediction model to perform time series prediction for future time periods, and obtain prediction values ​​for the three frequency domain components of low frequency, medium frequency, and high frequency. The prediction result acquisition module is used to add the prediction values ​​of the three frequency domain components of low frequency, medium frequency and high frequency, and perform inverse normalization processing on the added results to obtain the final time series prediction result; Wherein, the time series prediction model includes: The wavelet decomposition module is used to input the time series and decompose it into three frequency domain components: low frequency, medium frequency and high frequency. The low frequency, medium frequency and high frequency components correspond to long-term trends, cyclical patterns and short-term fluctuations respectively. The backbone network of the pre-trained large language model is used to extract features based on the decomposition results of the wavelet decomposition module; The prediction head is used to generate prediction values ​​for the three frequency domain components of low frequency, medium frequency, and high frequency based on the features extracted by the backbone network of the large language model.

[0014] In a third aspect of the present invention, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the time series prediction method based on wavelet transform and large language model as described in any one of the first aspects of the present invention is implemented.

[0015] In a fourth aspect, the present invention provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the time series prediction method based on wavelet transform and large language model as described in any one of the first aspects of the present invention is implemented.

[0016] Compared with the prior art, the present invention has the following beneficial effects: The present invention specifically discloses a time series prediction method based on wavelet transform and large language model, which combines the frequency domain decomposition capability of wavelet transform with the representation learning capability of large language model, can effectively capture the multi-scale features in time series, while maintaining the continuity and precision of numerical values, and can significantly improve the accuracy of time series prediction. Specifically, to address the problem that existing methods have poor modeling capabilities for complex time series features, the technical solution of the present invention adopts wavelet transform to introduce frequency domain features. As a powerful time-frequency decomposition tool, wavelet transform can provide frequency domain decomposition while maintaining time domain information. Unlike traditional Fourier transform, wavelet transform uses local support basis functions and can simultaneously capture the time and frequency domain characteristics of the signal. Its mathematical basis can be expressed as the inner product of the signal and a set of wavelet basis functions. This mathematical form gives the wavelet transform multi-resolution analysis capability, enabling it to analyze signals at different scales, provide better time resolution for high-frequency components, and provide better frequency resolution for low-frequency components. Therefore, it is suitable for analyzing time series with complex patterns, and can introduce multi-frequency domain dimensions of time series features and powerful analysis capabilities for time series prediction models.

[0017] To address the modal difference between time series data and language model pre-training domains, WaveLLM introduces a hybrid cue learning strategy. Specifically, the study uses a hybrid cue of hard cues and learnable soft cues for different frequency components, and fuses the hybrid cues with frequency features after encoding, thereby providing richer contextual information for time series prediction tasks, while guiding the pre-trained language model to understand the characteristics of time series data and the goals of the prediction task; finally, transferring language knowledge requires training key weights such as positional encoding. Considering the large parameter scale of large language models, directly training the entire model is impractical in an environment with limited computing resources. WaveLLM adopts low-rank adaptation, namely LoRA fine-tuning technology, which achieves efficient parameter fine-tuning by adding small low-rank matrices in the network layer. From the perspective of matrix decomposition, LoRA can be expressed as the original weight matrix plus the product of two low-rank matrices, thereby reducing computational and storage overhead while retaining most of the pre-training knowledge of the large language model, significantly improving the efficiency of the model in adapting to time series tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below; obviously, the drawings described below are some embodiments of the present invention, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0019] Figure 1 1 is a flow chart of a time series prediction method based on wavelet transform and large language model in an embodiment of the present invention; Figure 2 1 is a schematic diagram of the principle architecture of a time series prediction method based on wavelet transform and large language model in an embodiment of the present invention; Figure 3 is a schematic diagram of the ablation experiment results of the wavelet transform module in an embodiment of the present invention; Figure 4 2 is a schematic diagram of the dimensionality reduction result of the frequency components within the model using T-SNE in an embodiment of the present invention; Figure 5 1 is a schematic diagram showing the contribution of each component to the prediction accuracy in an embodiment of the present invention; Figure 6 This is a schematic diagram showing the effect of the number of GPT-2 backbone layers on prediction accuracy in an embodiment of the present invention; Figure 7 3 is a schematic diagram of a time series prediction system based on wavelet transform and large language model in an embodiment of the present invention. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solutions and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention; it is obvious that the described embodiments and technical solutions are only part of the embodiments of the present invention, not all of the embodiments.

[0021] All other embodiments obtained by persons of ordinary skill in the art based on the technical solutions disclosed in the embodiments of the present invention without creative effort shall fall within the scope of protection of the present invention. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units that are not explicitly listed or that are inherent to these processes, methods, products, or apparatuses.

[0022] See also Figure 1 The embodiment of the present invention provides a time series prediction method based on wavelet transform and large language model. The method has simple process and high computational efficiency. The time series prediction model used has good prediction accuracy and generalization ability. Specifically, the method may include the following steps: Step 1: Obtain known historical time series data; Step 2: Based on the known historical time series data obtained in step 1, the trained time series prediction model is used to perform time series prediction for the future time period, and the predicted values ​​of the three frequency domain components of low frequency, medium frequency, and high frequency are obtained; wherein, the normalization processing method same as that in the time series prediction model training stage is first used to obtain the normalization processing result of the known historical time series data; then, the normalization processing result of the known historical time series data is input into the trained time series prediction model to perform time series prediction for the future time period, and the predicted values ​​of the three frequency domain components of low frequency, medium frequency, and high frequency are obtained; Step 3: Add the predicted values ​​of the low-frequency, medium-frequency, and high-frequency frequency domain components, and perform inverse normalization on the added results to obtain the final time series prediction result; Wherein, the time series prediction model includes: The wavelet decomposition module is used to input the time series and decompose it into three frequency domain components: low frequency, medium frequency and high frequency. The low frequency, medium frequency and high frequency components correspond to long-term trends, cyclical patterns and short-term fluctuations respectively. The backbone network of the pre-trained large language model is used to extract features based on the decomposition results of the wavelet decomposition module; The prediction head is used to generate prediction values ​​for the three frequency domain components of low frequency, medium frequency, and high frequency based on the features extracted by the backbone network of the large language model.

[0023] In an embodiment of the present invention, the training step of the time series prediction model includes: Perform mapping transformation on each frequency domain component decomposed by the wavelet decomposition module to obtain three mapped frequency domain components; divide each mapped frequency domain component into multiple overlapping data blocks, and map each data block to the embedding space of the pre-trained large language model through linear projection; Each frequency domain component and its mixed prompt are concatenated and used as the input of the backbone network. The pre-trained large language model is fine-tuned using parameter-efficient low-rank adaptation techniques to adapt it to the time series prediction task. The output of the fine-tuned large language model is converted into the predicted values ​​of each frequency domain component through a linear layer. The mixed prompt of each frequency domain component is obtained by concatenating the hard prompt and soft prompt of each frequency domain component. The loss is calculated and the model parameters are updated according to the set loss function, which includes prediction loss and decomposition and reconstruction loss.

[0024] As a further improved technical solution of the embodiment of the present invention, in step 2, the wavelet transform uses the db4 wavelet basis function to perform multi-scale decomposition on the input historical time series data, with a decomposition level of 3.

[0025] As a technical solution further improved in an embodiment of the present invention, in step 3, the mapping transformation includes: linearly mapping the low-frequency components to keep the sequence length unchanged; first mapping the intermediate-frequency and high-frequency components to a higher-dimensional space through a linear layer, and then mapping them back to the original dimension through a ReLU nonlinear activation function and a second linear layer.

[0026] As a further improved technical solution of the embodiment of the present invention, in step 4, the data blocks are divided using a sliding window method, there is overlap between adjacent data blocks, the overlapping step is a predefined stride value, and the data block size is patch_size.

[0027] As a further improved technical solution of the embodiment of the present invention, the method also includes a hybrid prompt learning mechanism, which is implemented by one of the following methods: a hard prompt mechanism, which uses predefined text prompts to guide the large language model to process different frequency domain components; a soft prompt mechanism, which uses a learnable prompt vector pool to dynamically select the most relevant prompt vector combination based on the input sequence characteristics.

[0028] As a technical solution further improved by an embodiment of the present invention, the method also includes a dual loss optimization mechanism, which simultaneously optimizes: prediction loss, which measures the difference between the model prediction value and the true value; decomposition and reconstruction loss, which measures the difference between the frequency domain components of the model decomposition and the components obtained by the standard decomposition method.

[0029] In the technical solution disclosed in the embodiment of the present invention, the time series is decomposed into frequency domain components with clear physical meanings through wavelet transform (low-frequency components correspond to long-term trends, medium-frequency components correspond to periodic patterns, and high-frequency components correspond to short-term fluctuations and noise), so that the model can simultaneously focus on multiple time scales of the time series, thereby more comprehensively understanding the intrinsic structure of the data; a special continuous value mapping mechanism is designed, which directly maps the continuous value features after wavelet decomposition to the embedding space of the large language model through learnable linear projection and nonlinear transformation, thereby retaining the continuity and accuracy of the time series data and avoiding the information loss caused by discrete coding; a soft prompt learning strategy is introduced, A set of learnable prompt vectors are designed. These vectors are adaptively adjusted during the training process to guide the pre-trained language model to understand the characteristics of time series data and the goals of the prediction task. A dual-loss optimization mechanism is designed to simultaneously optimize the prediction loss and the decomposition and reconstruction loss. It not only focuses on prediction accuracy, but also ensures that the model can accurately understand the frequency domain characteristics of the time series, thereby improving the interpretability and generalization ability of the model. Low-rank adaptation technology is adopted to achieve efficient parameter fine-tuning by adding small low-rank matrices to key layers, reducing the number of parameters that need to be updated to less than 1% of the original model, greatly reducing computational and storage overhead while retaining most of the knowledge of the pre-trained model.

[0030] The specific exemplary technical solutions of the embodiments of the present invention are applied in wind speed prediction and traffic flow prediction, including: Wind speed dataset: Wind power generation data from actual wind farms, including three characteristic variables: wind speed, wind direction, and power output. The sampling interval is 10 minutes, and the time span is from May 2022 to November 2023, with a total of 79,872 sample points. This dataset was collected from the wind tower data of the Balasu wind farm in Yulin City, Shaanxi Province. It represents actual industrial application scenarios and has highly nonlinear and random characteristics.

[0031] Traffic flow dataset: Traffic flow data from the California transportation network, including records from 862 sensors, spanning the period from 2016 to 2018, with a sampling interval of 1 hour and a total of 17,544 time points. The dataset shows obvious intra-day and intra-week patterns, as well as holiday effects.

[0032] This paper implements a multiresolution analysis method based on wavelet transforms. Using discrete wavelet transforms, the time series is decomposed into low-frequency, medium-frequency, and high-frequency components. The method uses the Daubechies4 (db4) wavelet basis function and a three-level decomposition, with parameters adjustable based on data characteristics. During the decomposition process, approximate and detail coefficients are first obtained, followed by reconstructing components with different frequency characteristics. To convert the time series data into a form suitable for sequence-to-sequence prediction, a sliding window approach is used to generate inputs and outputs. Two widely used evaluation metrics, mean squared error (MSE) and mean absolute error (MAE), are used. The mean and standard deviation of these metrics are reported in the experimental evaluation to comprehensively assess model performance and ensure statistical significance. For each sequence length and prediction length (96, 192, 336, 512, and 720 time steps), the study conducts multiple independent experiments and calculates the average performance to mitigate the effects of random initialization. Lower MSE and MAE values ​​indicate better prediction performance.

[0033] An embodiment of the present invention provides a time series prediction method based on wavelet transform and a large language model. The core idea of ​​this method is to use wavelet transform to accurately decompose the input time series into three frequency domain components: low frequency, medium frequency, and high frequency. Then, a special mapping mechanism and soft prompt strategy are designed to convert these components into a representation that can be processed by the large language model. Finally, a parameter-efficient fine-tuning strategy is used to adapt the pre-trained language model to the time series prediction task.

[0034] like Figure 2 As shown, the WaveLLM model of the present invention includes four key components: a wavelet transform module, a patch representation module, a pre-trained language model adaptation module, and a dual-loss optimization mechanism. These components work together to form a complete processing flow: first, the original time series data is accurately decomposed into three frequency domain representations: high frequency, medium frequency, and low frequency through the wavelet transform module; then, these frequency domain representations are converted into a form suitable for language model processing through the patch representation module; then, the pre-trained language model adaptation module uses the powerful representation capabilities of the large language model to deeply encode the time series features; finally, the dual-loss optimization mechanism simultaneously optimizes prediction accuracy and decomposition quality, ensuring that the model can accurately capture the multi-scale characteristics of the time series.

[0035] In a specific exemplary technical solution, the method of the present invention comprises the following steps: Step 1, wavelet transform decomposition: Decompose the input time series x into three frequency domain components through wavelet transform; the wavelet transform uses the db4 wavelet basis function and the decomposition level is 3; the three components obtained after decomposition are: low-frequency component (trend), corresponding to the long-term trend of the time series; medium-frequency component (season), corresponding to the periodic pattern of the time series; high-frequency component (resid), corresponding to the short-term fluctuation and noise of the time series.

[0036] Step 2, frequency domain component mapping: perform mapping transformation on the three frequency domain components after decomposition: process the low-frequency component through linear mapping; process the intermediate-frequency and high-frequency components through nonlinear mapping; the mapping of the intermediate-frequency and high-frequency components includes a linear layer with dimension expansion, a ReLU activation function, and a linear layer with dimension restoration.

[0037] Step 3, Patch representation generation: Divide the mapped frequency domain components into multiple overlapping data blocks (patches); use a sliding window method with a window size of patch_size and a step size of stride; and map each data block to the embedding space of the pre-trained language model (with a dimension of 768) through linear projection.

[0038] Step 4, prompt learning mechanism: select hard prompt or soft prompt mechanism according to the configuration; Hard prompt mechanism: using predefined text prompts, such as "Predict the future time step given the trend"; Soft hint mechanism: Build a hint pool containing 30 learnable hint vectors. Each hint consists of 3 tokens. Dynamically select the three hint vector combinations with the highest similarity as the hint for the current input.

[0039] Step 5, large language model processing: input the patch representation with prompts into the pre-trained GPT-2 model for processing; among them, the representations of the three frequency domain components are spliced ​​together; the hidden state representation is obtained through the GPT-2 model; and the representations of the three frequency domain components are extracted from the hidden state respectively.

[0040] Step 6, prediction generation: The output of the large language model is converted into the predicted value of each frequency domain component through a linear layer, the predicted values ​​of the three frequency domain components are added together, and inverse normalization is performed to obtain the final time series prediction result.

[0041] This embodiment of the present invention provides an efficient parameter fine-tuning strategy for the WaveLLM model. To effectively utilize the knowledge of the pre-trained language model while reducing computational overhead, the present invention uses low-rank adaptation (LoRA) technology for efficient parameter fine-tuning, including: Set low-rank adaptation parameters: Rank (r) = 16: controls the dimension of the low-rank matrix; Low rank scaling factor (lora_alpha) = 16: controls the impact of low rank updates; Low rank dropout rate (lora_dropout) = 0.1: to prevent overfitting; Selective parameter update strategy: Layer normalization and position embedding parameters are fully fine-tuned to adapt to the characteristics of time series data. Core parameters of the pre-trained model (such as the attention mechanism and feedforward network) are kept frozen and updated indirectly through a low-rank matrix. This parameter-efficient fine-tuning strategy reduces the number of parameters that need to be updated to less than 1% of the original model, significantly reducing computational and storage overhead while retaining most of the knowledge of the pre-trained model.

[0042] The embodiment of the present invention provides a dual-loss optimization mechanism for the WaveLLM model. In order to improve the prediction accuracy and interpretability of the model, the present invention designs a dual-loss optimization mechanism to optimize two losses simultaneously: Prediction loss: Use mean squared error (MSE) to measure the difference between the model's predicted value and the true value; Decomposition and reconstruction loss: measures the difference between the frequency domain components decomposed by the model and the components obtained by the standard decomposition method. This dual loss optimization mechanism not only focuses on prediction accuracy, but also ensures that the model can accurately understand the frequency domain characteristics of the time series, thereby improving the model's interpretability and generalization ability.

[0043] In the embodiment of the present invention, the WaveLLM model training process is shown in Table 1, and is specifically described as follows: Data preprocessing: Normalize the input time series, calculate the mean μ and standard deviation σ of the training set, and normalize all data sets according to x_normalized = (x - μ) / σ; Wavelet decomposition: The normalized time series is decomposed into three levels using the Daubechies 4 (db4) wavelet basis function to obtain the low-frequency trend component, the medium-frequency seasonal component, and the high-frequency residual component; Data blocking: Use the sliding window method to divide the decomposed frequency domain components into multiple overlapping data blocks, with the window size being patch_size and the step size being stride; Model training: Data blocks are mapped to the embedding space of a pre-trained large language model using linear projection; the pre-trained large language model is fine-tuned using parameter-efficient low-rank adaptation techniques; both prediction loss and decomposition and reconstruction loss are optimized; the Adam optimizer is used with an initial learning rate of 0.0005 and a cosine annealing schedule; an early stopping strategy is implemented, stopping training when the validation loss does not improve within 5 consecutive epochs; Model prediction: The test data is processed through the same preprocessing and decomposition steps; the trained model is used to generate predicted values ​​for each frequency domain component; the predicted values ​​are summed and inverse normalized to obtain the final prediction result.

[0044] Table 1. Model training process

[0045] To verify the effectiveness of the WaveLLM model proposed in this paper, a comprehensive evaluation was conducted on multiple public benchmark datasets and an actual wind farm project dataset. The experimental results are analyzed as follows.

[0046] Table 2 shows the prediction accuracy of WaveLLM and various baseline models on a field wind speed dataset. WaveLLM outperforms existing methods, including advanced models such as TIME-LLM, Autoformer, FEDformer, and PatchTST, across all test datasets and forecast lengths. On the ETT dataset, WaveLLM's average prediction MAE is 8.7% lower and MSE is 10.2% lower than the best baseline method.

[0047] Table 2. Prediction accuracy of WaveLLM and various benchmark models on the field wind speed dataset

[0048] Table 3 shows the experimental results of WaveLLM on a public dataset. WaveLLM has excellent long-term prediction capabilities: as the prediction step length increases, WaveLLM's performance advantage over the baseline method gradually expands. When the prediction length extends from 96 to 720 steps, WaveLLM's performance decays significantly slower than other methods, demonstrating its unique advantages in long-term prediction tasks. Table 3. Experimental results of WaveLLM on public datasets

[0049] Model stability: In experiments with different input sequence lengths (48 to 720 time steps), WaveLLM demonstrated stable prediction performance, especially when the input sequence was longer, where its advantages were more obvious.

[0050] Figure 3 Wavelet decomposition effect: Ablation experiments show that the wavelet transform module is crucial to model performance. Compared with not using decomposition (no_decomp) or using other decomposition methods (such as STL decomposition), WaveLLM using wavelet transform has a significant improvement in prediction accuracy.

[0051] Figure 4This visualization shows the results of t-SNE dimensionality reduction. The WaveLLM model effectively decomposes the time series into three distinct components. The low-frequency component (blue) exhibits an arc-shaped distribution, the mid-frequency component (orange) forms a distinctive spiral structure, and the high-frequency component (green) exhibits a relatively concentrated cloud-like distribution. This clear separation demonstrates that the model successfully captures the unique characteristics of each component.

[0052] Figure 5 This visualization analyzes the component contributions to the model's final forecast. In the ETT dataset, low-frequency components dominate, indicating that long-term trend information is the most critical component in time series forecasting. High-frequency components contribute relatively little across all datasets, suggesting that short-term random fluctuations have limited value for long-term forecasting, but still provide some predictive information. These findings have important implications for the design of time series forecasting models: modeling a time series by decomposing it into different frequency components may be more effective than directly modeling the original series, as different frequency components require different processing strategies.

[0053] Table 4 shows the ablation experiment results of different prompt learning strategies. The hybrid prompt strategy (combining hard prompts and soft prompts) achieves the best performance on the ETTm1 dataset, reducing the MSE and MAE by 12.6% and 6.1% respectively compared to the unprompted baseline.

[0054] Table 4. Ablation experiment results of different prompt learning strategies

[0055] Figure 6 The impact of model depth: As the number of Transformer layers increases, the model's overall predictive performance continues to improve. When the number of layers increases from 2 to 12, both the MAE and MSE metrics improve significantly, validating the effectiveness of deep network structures in time series forecasting tasks.

[0056] In summary, the above experimental results fully demonstrate the effectiveness and advancement of the WaveLLM model of the present invention in time series forecasting tasks, especially in long-term forecasting and capturing complex time series patterns.

[0057] In summary, the present invention relates to a time series prediction method and device based on wavelet decomposition and a large language model, belonging to the field of artificial intelligence and data analysis technology. The method first standardizes the input time series, then uses the Daubechies 4 wavelet basis function to perform a three-level decomposition to obtain a low-frequency trend component, a medium-frequency seasonal component, and a high-frequency residual component; then uses a sliding window method to divide the decomposed frequency domain components into data blocks; the data blocks are mapped to the embedding space of a pre-trained large language model through linear projection and fine-tuned using a parameter-efficient low-rank adaptation technique; finally, the predicted values ​​of each frequency domain component are added and inverse normalized to obtain the final prediction result, forming a system with high-precision prediction, strong long-term prediction capability, and good model stability. The method of the present invention can effectively process complex time series data, outperforming existing methods on multiple public benchmark data sets, and has the advantages of high prediction accuracy, strong adaptability, and good computational efficiency. It is suitable for time series prediction applications in energy, finance, meteorology and other fields.

[0058] The following are device embodiments of the present invention, which can be used to perform the method embodiments of the present invention. For details not disclosed in the device embodiments, please refer to the method embodiments of the present invention.

[0059] See also Figure 7 In an embodiment of the present invention, a time series prediction system based on wavelet transform and large language model is provided, comprising: Data acquisition module, which obtains known historical time series data; A prediction value acquisition module is used to perform time series prediction for future time periods using a trained time series prediction model based on the acquired known historical time series data, and obtain prediction values ​​for the three frequency domain components of low frequency, medium frequency, and high frequency. The module first adopts the same normalization processing method as that used in the training phase of the time series prediction model to obtain the normalization processing results of the known historical time series data; then, the normalization processing results of the known historical time series data are input into the trained time series prediction model to perform time series prediction for future time periods, and obtain prediction values ​​for the three frequency domain components of low frequency, medium frequency, and high frequency. The prediction result acquisition module is used to add the prediction values ​​of the three frequency domain components of low frequency, medium frequency and high frequency, and perform inverse normalization processing on the added results to obtain the final time series prediction result; Wherein, the time series prediction model includes: The wavelet decomposition module is used to input the time series and decompose it into three frequency domain components: low frequency, medium frequency and high frequency. The low frequency, medium frequency and high frequency components correspond to long-term trends, cyclical patterns and short-term fluctuations respectively. The backbone network of the pre-trained large language model is used to extract features based on the decomposition results of the wavelet decomposition module; The prediction head is used to generate prediction values ​​for the three frequency domain components of low frequency, medium frequency, and high frequency based on the features extracted by the backbone network of the large language model.

[0060] In one embodiment of the present invention, a computer device is provided, comprising a processor and a memory, wherein the memory is used to store a computer program, the computer program including program instructions, and the processor is used to execute the program instructions stored in the computer storage medium. The processor may be a central processing unit (CPU), or may be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It is the computing core and control core of the terminal, and is suitable for implementing one or more instructions, specifically suitable for loading and executing one or more instructions in a computer storage medium to implement a corresponding method flow or corresponding function; the processor described in the embodiment of the present invention can be used to perform operations of a time series prediction method based on wavelet transform and a large language model.

[0061] In one embodiment of the present invention, a storage medium is provided, specifically a computer-readable storage medium (Memory). The computer-readable storage medium is a memory device in a computer device, used to store programs and data. It is understood that the computer-readable storage medium herein may include both built-in storage media in the computer device and, of course, extended storage media supported by the computer device. The computer-readable storage medium provides storage space, which stores the terminal's operating system. Furthermore, this storage space also stores one or more instructions suitable for being loaded and executed by a processor. These instructions may be one or more computer programs (including program code). It should be noted that the computer-readable storage medium herein may be a high-speed random access memory (RAM) or a non-volatile memory, such as at least one disk storage device. The processor may load and execute the one or more instructions stored in the computer-readable storage medium to implement the corresponding steps of the time series prediction method based on wavelet transform and large language model in the above-mentioned embodiment.

[0062] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, optical storage, etc.) containing computer-usable program code.

[0063] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0064] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0065] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0066] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered by the scope of protection of the claims of the present invention.

Claims

1. A time series prediction method based on wavelet transform and large language model, characterized in that: The following steps are involved: Obtain known historical time series data; Based on the acquired known historical time series data, a trained time series prediction model is used to perform time series prediction for a future time period, and predicted values ​​of three frequency domain components, namely, low frequency, medium frequency, and high frequency, are obtained; wherein, the normalization processing method same as that used in the training phase of the time series prediction model is firstly used to obtain a normalization processing result of the known historical time series data; and then, the normalization processing result of the known historical time series data is input into the trained time series prediction model to perform time series prediction for a future time period, and predicted values ​​of the three frequency domain components, namely, low frequency, medium frequency, and high frequency, are obtained; The predicted values ​​of the three frequency domain components of low frequency, medium frequency and high frequency are added together, and the added results are inverse normalized to obtain the final time series prediction results; Wherein, the time series prediction model includes: The wavelet decomposition module is used to input the time series and decompose it into three frequency domain components: low frequency, medium frequency and high frequency. The low frequency, medium frequency and high frequency components correspond to long-term trends, cyclical patterns and short-term fluctuations respectively. The backbone network of the pre-trained large language model is used to extract features based on the decomposition results of the wavelet decomposition module; The prediction head is used to generate prediction values ​​for the three frequency domain components of low frequency, medium frequency, and high frequency based on the features extracted by the backbone network of the large language model.

2. The time series prediction method based on wavelet transform and large language model according to claim 1, characterized in that: The training steps of the time series prediction model include: Perform mapping transformation on the three frequency domain components decomposed by the wavelet decomposition module to obtain the three mapped frequency domain components; divide each mapped frequency domain component into multiple overlapping data blocks, and map each data block to the embedding space of the pre-trained large language model through linear projection; Each frequency domain component is concatenated with its mixed prompt and used as the input of the backbone network. The pre-trained large language model is fine-tuned using low-rank adaptation technology to adapt it to the time series prediction task. The output of the fine-tuned large language model is converted into the predicted values ​​of each frequency domain component through a linear layer. Among them, the mixed prompts of each frequency domain component are obtained by splicing the hard prompts and soft prompts of each frequency domain component; in addition, the loss is calculated and the model parameters are updated according to the dual loss optimization mechanism; the dual loss optimization mechanism simultaneously optimizes the prediction loss used to measure the difference between the model prediction value and the true value, and the decomposition and reconstruction loss used to measure the difference between the frequency domain components of the model decomposition and the components obtained by the standard decomposition method.

3. The time series prediction method based on wavelet transform and large language model according to claim 2, characterized in that: In the wavelet decomposition module, the wavelet transform uses the db4 wavelet basis function to perform multi-scale decomposition on the time series, with a decomposition level of 3.

4. The time series prediction method based on wavelet transform and large language model according to claim 2, characterized in that: In the step of performing mapping transformation on each frequency domain component decomposed by the wavelet decomposition module to obtain the three mapped frequency domain components, linear mapping is performed on the low-frequency frequency domain component to keep the sequence length unchanged; The mid-frequency and high-frequency frequency domain components are first mapped to a higher-dimensional space through a linear layer, and then mapped back to the original dimension through the ReLU nonlinear activation function and a second linear layer.

5. The time series prediction method based on wavelet transform and large language model according to claim 2, characterized in that: In the step of dividing the mapped frequency domain components into a plurality of overlapping data blocks, the data blocks are divided in a sliding window manner, and there is overlap between adjacent data blocks.

6. The time series prediction method based on wavelet transform and large language model according to claim 2, characterized in that: In the step of obtaining the mixed prompts of the frequency domain components by concatenating the hard prompts and soft prompts of the frequency domain components, the hard prompt mechanism uses predefined text prompts to guide the large language model to process different frequency domain components; The soft hint mechanism uses a learnable hint vector pool to dynamically select the most relevant hint vector combination based on the input sequence characteristics.

7. The time series prediction method based on wavelet transform and large language model according to claim 1, characterized in that: In the step of first obtaining the normalization processing result of the known historical time series data by adopting the same normalization processing method as the time series prediction model training stage, the input time series is normalized, the mean μ and standard deviation σ of the training set are calculated, and the training set is normalized according to x. _normalized = (x - μ) / σ for normalization; where x is the input time series, x _normalized is the normalized result.

8. A time series prediction system based on wavelet transform and large language model, characterized in that: include: Data acquisition module, which obtains known historical time series data; A prediction value acquisition module is used to perform time series prediction for future time periods using a trained time series prediction model based on the acquired known historical time series data, and obtain prediction values ​​for the three frequency domain components of low frequency, medium frequency, and high frequency. The module first adopts the same normalization processing method as that used in the training phase of the time series prediction model to obtain the normalization processing results of the known historical time series data; then, the normalization processing results of the known historical time series data are input into the trained time series prediction model to perform time series prediction for future time periods, and obtain prediction values ​​for the three frequency domain components of low frequency, medium frequency, and high frequency. The prediction result acquisition module is used to add the prediction values ​​of the three frequency domain components of low frequency, medium frequency and high frequency, and perform inverse normalization processing on the added results to obtain the final time series prediction result; Wherein, the time series prediction model includes: The wavelet decomposition module is used to input the time series and decompose it into three frequency domain components: low frequency, medium frequency and high frequency. The low frequency, medium frequency and high frequency components correspond to long-term trends, cyclical patterns and short-term fluctuations respectively. The backbone network of the pre-trained large language model is used to extract features based on the decomposition results of the wavelet decomposition module; The prediction head is used to generate prediction values ​​for the three frequency domain components of low frequency, medium frequency, and high frequency based on the features extracted by the backbone network of the large language model.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the time series prediction method based on wavelet transform and large language model is implemented according to any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the time series prediction method based on wavelet transform and large language model according to any one of claims 1 to 7 is implemented.

Images