Adaptation method and device for online test of time sequence prediction

Abstract

The invention relates to the technical field of artificial intelligence, in particular to an online test adaptation method and device for time sequence prediction. Comprising the following steps: establishing and maintaining a historical sample memory bank, storing historical time sequence data, and updating the memory bank through a first-in first-out strategy; screening a historical sample set in a historical sample memory bank, wherein the similarity between the historical sample set and the test sample in the potential space meets a preset condition; performing frequency domain-based mixed data enhancement on the test sample and the historical sample set to generate an enhanced sample set; inputting the enhanced sample set into a time sequence prediction model for batch training, dynamically adjusting model parameters to adapt to distribution offset, and outputting a final prediction result generated through fusion; and evaluating the performance of the model through the loss function. According to the method, the adaptability of the model to new distribution can be enhanced, the robustness of the model is improved, the noise sensitivity is suppressed, data enhancement is performed on the time sequence, and damage to a time domain of the time sequence is avoided.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and particularly relates to an online test-time adaptation method and device for time series prediction. Background Art

[0002] Online Time Series Forecasting technology is a dynamic modeling method for streaming data. Its core feature is to continuously receive real-time data streams, synchronously update model parameters, and output prediction results. Compared with the traditional batch training mode, this technology effectively meets the high requirements for low-latency decision-making in scenarios such as financial high-frequency trading, industrial equipment status monitoring, and intelligent traffic flow prediction by integrating mechanisms such as incremental learning and real-time feedback correction. However, in the actual application process, the inherent non-stationary characteristics of streaming data (such as the statistical property drift of sensor-acquired signals and the pattern mutation of market trading behaviors) will cause the dynamic evolution of data distribution, specifically manifested as the change in the feature space distribution caused by Covariate Shift, and the reconstruction of the input-output relationship caused by Concept Drift. Such distribution shift phenomena will violate the independent and identically distributed assumption preset in the model training stage, resulting in problems such as response lag and error accumulation in the prediction model statically trained based on historical data, and ultimately leading to a continuous decline in prediction accuracy as the device state changes.

[0003] In recent years, Online Test-Time Adaptation (OTTA), as a cutting-edge technology for dealing with streaming data distribution shift, has gradually become a research hotspot. Its innovation lies in breaking through the strict boundary between the traditional model's "training - inference" stages, allowing the model to perform unsupervised or self-supervised incremental parameter adjustment based on real-time input data streams during the inference process. Specifically, OTTA realizes the real-time alignment of the input data distribution and the internal representation of the model by designing lightweight adaptive modules (such as dynamically calibrating the statistics of the batch normalization layer and optimizing the feature projection entropy minimization objective) without the need to access the original training data or perform global model retraining.

[0004] However, applying Online Test-Time Adaptation (OTTA) to the field of time series prediction still faces core challenges. First, the strong temporal dependence of time series makes lightweight parameter updates vulnerable to noise interference. High-frequency fine-tuning may cause model parameter oscillations, disrupting the memory ability for historical patterns. Additionally, time series prediction models usually rely on hidden states (such as the state maintenance of recurrent neural networks or attention mechanisms). If the incremental update is incompatible with the state evolution mechanism, it may affect the stability of long-term dependence modeling. Second, existing OTTA methods rely on labels or pseudo-labels of classification tasks to guide updates, but time series prediction, as a regression task, lacks future true values as supervision signals. Finally, existing methods do not fully exploit the cross-channel correlations of time series. The dynamic associations between channels in multivariate time series (such as the coupling relationships between physical device variables) need to be explicitly modeled, while the current OTTA framework mainly deals with spatial local correlations and lacks the design of cross-channel spatio-temporal interaction mechanisms.

[0005] Currently, deep learning methods for streaming data mainly include Online Test-Time Adaptation and Online Time Series Prediction.

[0006] Online Test-Time Adaptation updates the model in real-time according to the received data during the test process to ensure rapid adaptation to the changing data distribution without changing the original training process. In the existing technology, entropy loss minimization is used during the test process to dynamically adjust the parameters of the batch normalization layer to cope with distribution shift, and a method of selectively optimizing the unsupervised proxy loss is further introduced, focusing on reliable and information-rich data points. Knowledge distillation is incorporated into test-time adaptation, and the prediction and enhanced input from the teacher model are used to supervise the output of the student. In addition, high / low-rank adapters are introduced, which are continuously adjusted as the online test-time adaptation continues to update to adapt to the data distribution. Zeng et al. [Zeng L, Han J, Du L, et al. Rethinking precision ofpseudo label: Test-time adaptation via complementary learning[J]. PatternRecognition Letters, 2024, 177: 96-102.] integrated a memory bank containing output distributions and established thresholds for complementary labels. This innovative method ensures the continuous relevance and effectiveness of the memory bank by regularly updating the latest model parameters.

[0007] Online Time Series Prediction also focuses on streaming data. For each received sample \(x_i\) containing \(N\) variables, the model constructs a retrospective window \(X\) of length \(L\) and outputs a prediction window \(Y\) of length \(H\), and then uses the true value to improve the prediction performance of the model for the next sample.

[0008] Due to the natural sequential characteristics of time series data, online time series has a wide range of application scenarios in the real world. Existing technologies solve regression problems through the idea of Bayesian continual learning. However, these methods follow the Bayesian framework, allowing the forgetting of past knowledge and being unable to quickly learn information about the current distribution. In addition, these methods are not implemented using deep neural networks and are difficult to extend to existing research. Inspired by the complementary learning theory in the dual learning device, data is explicitly modeled through the mechanism of experience replay. The historical data is quickly adapted through an adapter module, and progressive learning is carried out on newly collected samples with the help of the temporal convolutional network architecture, achieving good results in online time series prediction. The necessity of modeling the inter-channel dependence relationship is further discussed in the existing technology. By balancing the cross-time dependence and cross-variable dependence through an online convex programming module, the performance of the method on multi-variable datasets has been further improved.

[0009] Although online time series prediction and online test-time adaptation methods have achieved certain results on the streaming data of time series prediction, there are still obvious deficiencies in domain adaptability and anti-noise robustness for existing methods.

[0010] First of all, existing OTTA technologies highly rely on the supervised paradigm of image classification tasks. Its core assumption is that there exist discrete class labels or constructible pseudo-labels (such as clustering labels, entropy minimization objectives) to drive parameter updates. However, as a typical continuous-value regression task, time series prediction cannot obtain the true values of future time steps in real time as supervision signals during the online inference stage, and traditional label-dependent adaptive mechanisms completely fail. In addition, the classification task aims at aligning decision boundaries, while the regression task needs to model the continuous mapping relationship between input and output. Directly migrating strategies such as adversarial training and class centroid alignment in classification OTTA will lead to deviation in the optimization direction.

[0011] Secondly, existing online time series prediction methods generally ignore the time-varying evolution characteristics of data distribution, resulting in continuous degradation of model performance in the streaming scenario. In addition, existing methods cannot calibrate the distribution shift of the input feature space in real time (such as the scale change of statistics caused by sensor range drift), resulting in a gradually mismatched representation of the input data by the feature encoder.

[0012] Finally, existing methods do not take into account the impulse noise, measurement errors, and outliers commonly existing in streaming data, resulting in the deviation of the parameter update path of existing methods from the theoretical optimal solution. Existing methods often adopt lightweight update strategies such as gradient fine-tuning and lack robust constraints on noise. The strong temporal dependence of time series makes the single-step update error accumulate with time error, triggering model oscillation.

[0013] In summary, the existing online test-time adaptation and online time series prediction methods still face many challenges in time series prediction for streaming data. There is an urgent need to introduce new mechanisms such as time series adaptation and distribution shift mitigation to comprehensively improve the robustness of the model to cope with complex and changing real-world scenarios. Only by solving the above problems can online time series prediction be better implemented in real-world scenarios, providing more reliable and practical technical support for applications such as financial high-frequency trading, industrial equipment status monitoring, and intelligent traffic flow prediction. Summary of the Invention

[0014] To solve the technical problem that the existing technology still faces many challenges in time series prediction for streaming data, an embodiment of the present invention provides an online test-time adaptation method and device for time series prediction. The technical solution is as follows: On the one hand, an online test-time adaptation method for time series prediction is provided, and the method includes: S1. Establish and maintain a historical sample memory bank, store historical time series data, and update the memory bank through a first-in-first-out strategy; S2. Receive a test sample; screen a set of historical samples in the historical sample memory bank whose similarity with the test sample in the latent space meets a preset condition; S3. Perform frequency-domain-based hybrid data augmentation on the test sample and the set of historical samples to generate an augmented sample set; S4. Input the augmented sample set into a pre-trained time series prediction model for batch training, dynamically adjust the model parameters to adapt to the distribution shift, and output the final prediction result through fusion; evaluate the performance of the model through a loss function.

[0015] Optionally, in S2, receiving a test sample; screening a set of historical samples in the historical sample memory bank whose similarity with the test sample in the latent space meets a preset condition includes: Mapping the test sample and the historical samples in the memory bank to the latent space through a pre-trained variational autoencoder (VAE); Calculating the cosine similarity between the latent vector of the test sample and the latent vectors of each historical sample; Sorting according to the cosine similarity and selecting the top n historical samples with the highest similarity to form the set of historical samples.

[0016] Optionally, in S3, performing frequency-domain-based hybrid data augmentation on the test sample and the set of historical samples to generate an augmented sample set includes: Performing a fast Fourier transform on the test sample and the set of historical samples, and respectively extracting the amplitude component and the phase component; Dynamically determining the mixing coefficients of the amplitude and the phase according to the latent space distance between the historical samples and the test sample; Interpolate and mix the amplitude component and the phase component respectively, and reconstruct the enhanced time series through inverse Fourier transform.

[0017] Optionally, the determination rule of the mixing coefficient includes: If the latent space distance is less than or equal to the preset threshold τ, the mixing coefficient is sampled from the uniform distribution U( β , 1.0); If the latent space distance is greater than τ, the mixing coefficient is sampled from the truncated normal distribution N( μ , θ ).

[0018] Optionally, in S4, input the enhanced sample set into a pre-trained time series prediction model for batch training, dynamically adjust the model parameters to adapt to the distribution shift, and output the final prediction result through fusion, including: The time series prediction model adopts a two-stream predictor architecture; decompose the input time series into a long-term trend component and a periodic seasonal component; Input the trend component and the seasonal component into independent prediction sub-models respectively, and fuse the output results into the final prediction value.

[0019] Optionally, decomposing the input time series into a long-term trend component and a periodic seasonal component includes: The sequence decomposition of the input time series is realized through moving average kernel and difference calculation: ; where is the trend component, is the seasonal component, is the enhanced sample.

[0020] Optionally, inputting the trend component and the seasonal component into independent prediction sub-models respectively, and fusing the output results into the final prediction value includes: Input the decomposed trend component and the periodic component into the same-structured two-stream predictor; the output of the two-stream predictor is fused to generate the final prediction result : .

[0021] Optionally, in S4, inputting the enhanced sample set into a pre-trained time series prediction model for batch training includes: Adopt to optimize the model parameters with loss, and the loss function is defined as: ; where N represents the number of channels in the time series; represents the true value; H represents the prediction length; k represents the index of the channel; During the adaptation process of online testing, the true values of all samples and the mean squared error and mean absolute error between the model prediction output Y are used as the final evaluation metrics, as shown below: ; .

[0022] On the other hand, an online testing adaptation device for time series prediction, characterized in that the online testing adaptation device for time series prediction is used to implement the above-mentioned online testing adaptation method for time series prediction, and the device includes: A transferable historical sample selection module, used to establish and maintain a historical sample memory bank, store historical time series data, and update the memory bank through a first-in-first-out strategy; Receive test samples; screen a set of historical samples in the historical sample memory bank whose similarity with the test samples in the latent space meets the preset conditions; A transferable online enhancement module, used to perform frequency-domain-based hybrid data enhancement on the test samples and the set of historical samples to generate a set of enhanced samples; A prediction and evaluation module, used to input the set of enhanced samples into a pre-trained time series prediction model for batch training, dynamically adjust the model parameters to adapt to the distribution shift, and output the final prediction result through fusion; evaluate the performance of the model through a loss function.

[0023] On the other hand, an online testing adaptation device for time series prediction is provided, and the online testing adaptation device for time series prediction includes: a processor; a memory, and a computer-readable instruction is stored on the memory. When the computer-readable instruction is executed by the processor, any one of the methods in the above-mentioned online testing adaptation method for time series prediction is implemented.

[0024] On the other hand, a computer-readable storage medium is provided, and at least one instruction is stored in the storage medium. The at least one instruction is loaded and executed by a processor to implement any one of the methods in the above-mentioned online testing adaptation method for time series prediction.

[0025] The beneficial effects brought by the technical solutions provided by the embodiments of the present invention at least include: In the embodiments of the present invention, by making full use of historical distribution information, a transferable historical sample selection module is designed to select historical samples close to the test samples. By intelligently selecting historical distribution information similar to the current data pattern, the adaptability of the model to new distributions is enhanced.

[0026] The original online learning paradigm of updating with a single sample is transformed, and a batch training module is introduced to mitigate the negative impact of potential noise, improve the robustness of the model, and suppress noise sensitivity.

[0027] Data augmentation is performed on the time series based on the frequency domain. From the two perspectives of amplitude and phase, data augmentation is respectively performed on the time series, avoiding destructive interference in the time domain of the time series.

[0028] The present invention performs excellently in real scenarios. Experiments show that the present invention can still achieve good prediction accuracy on datasets with severe distribution shifts, verifying the practical value of the present invention. Description of the Drawings

[0029] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0030] Figure 1 It is a flowchart of an online test adaptation method for time series prediction provided by an embodiment of the present invention; Figure 2 It is a detailed flowchart of an online test adaptation method for time series prediction provided by an embodiment of the present invention; Figure 3 It is a data graph showing the performance of the method provided by an embodiment of the present invention in a real scenario; Figure 4 It is a block diagram of an online test adaptation device for time series prediction provided by an embodiment of the present invention; Figure 5 It is a schematic structural diagram of an electronic device provided by an embodiment of the present invention. Detailed Embodiments

[0031] The following will describe the technical solutions in the present invention with reference to the drawings.

[0032] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to give examples, illustrations or explanations. Any embodiment or design solution described as an "example" in the present invention should not be construed as being more preferred or having more advantages than other embodiments or design solutions. Rather, the use of the word "example" is intended to present concepts in a specific way. In addition, in the embodiments of the present invention, the meaning expressed by "and / or" can be both, or either of the two can be selected.

[0033] In the embodiments of the present invention, sometimes a subscript such as W1 may be written in a non-subscript form such as W1. When the difference is not emphasized, the meanings they express are the same.

[0034] To make the technical problems, technical solutions and advantages to be solved by the present invention clearer, the following will be described in detail with reference to the drawings and specific embodiments.

[0035] Figure 1 is a schematic diagram of an online test adaptation method for time series prediction provided by an embodiment of the present invention, as Figure 1 shown, an embodiment of the present invention provides an online test adaptation method for time series prediction, as Figure 2 shown is a detailed flowchart of the online test adaptation method for time series prediction provided by an embodiment of the present invention; this method can be implemented by an online test adaptation device for time series prediction, and this online test adaptation device for time series prediction can be a terminal or a server. The online test adaptation method for time series prediction proposed by the present invention, the processing flow of this method can include the following steps: S1. Establish and maintain a historical sample memory bank, store historical time series data, and update the memory bank through a first-in, first-out strategy; S2. Receive test samples; screen a set of historical samples in the historical sample memory bank whose similarity with the test samples in the latent space meets a preset condition.

[0036] In a feasible implementation, first, problem definition is performed, and a time series prediction model pre-trained on a training set and a set of unlabeled time series segments are given. The goal of online test adaptation is to use the prediction model embedded in a labeled knowledge in an online manner to infer the future values of samples under distribution shift. In this problem, the learning process is carried out in a series of rounds: the model receives a backtracking window of length , predicts a prediction window of length and updates the parameters according to the received true values to improve the prediction performance of the model in subsequent rounds. The goal of the present invention is to continuously optimize the prediction model , to mitigate the negative impact brought by distribution shift.

[0037] In a feasible implementation, S2 includes: Mapping the test sample and the historical samples in the memory bank to the latent space through a pre-trained variational autoencoder (VAE); Calculating the cosine similarity between the latent vector of the test sample and the latent vectors of each historical sample; Sorting according to the cosine similarity and selecting the top n historical samples with the highest similarity to form the historical sample set.

[0038] In a feasible implementation, the present invention establishes a memory bank (denoted as ), which stores historical sample information and updates historical samples through a first-in-first-out (FIFO) strategy to maintain a fixed capacity. When receiving a test sample (denoted as ), the present invention uses a transferable historical sample selection module to screen historical samples in the memory bank that are semantically similar and distributionally close to the test sample. This selection is achieved through a mapping model, and any model that can maintain the semantic consistency of the original data and the mapped data can be used as the mapping model. Given that the variational autoencoder (VAE) naturally has the ability to maintain the semantic consistency of input and output, the present invention selects VAE as the mapping model. In the initial stage, the VAE model is pre-trained in an unsupervised manner on the training set; in the test stage, its parameters are frozen to remain unchanged. When introducing the test sample , both this sample and the historical samples stored in the memory bank are projected into the latent space. Subsequently, the distances of such samples in the latent space are calculated, and historical samples with smaller differences from the test sample are screened out. These historical samples are semantically similar to the test sample and constitute the selected historical sample set , and the above process can be formalized as the following formula:

[0039]

[0040]

[0041] Among them, represents the encoder of the VAE model, represents the sample element index in; represents the sample in; represents the distance between latent vectors; represents the historical sample set; represents the sorted historical sample element index; is a hyperparameter used to specify the number of historical samples to be utilized. By effectively leveraging historical samples with a smaller distribution difference from the test samples, the present invention can effectively alleviate the distribution shift problem and improve the prediction performance.

[0042] S3. Perform frequency-domain-based hybrid data augmentation on the test samples and the historical sample set to generate an augmented sample set; In a feasible implementation manner, S3 includes: performing a fast Fourier transform on the test samples and the historical sample set, and respectively extracting the amplitude component and the phase component; Dynamically determine the mixing coefficients of the amplitude and the phase according to the latent space distance between the historical samples and the test samples; Perform interpolation mixing on the amplitude component and the phase component respectively, and reconstruct the enhanced time series through inverse Fourier transform.

[0043] In a feasible implementation manner, the determination rule of the mixing coefficient includes: If the latent space distance is less than or equal to the preset threshold τ, then the mixing coefficient is sampled from the uniform distribution ( , 1.0), [[ID=--]] represents the lower bound of the uniform distribution; If the latent space distance is greater than τ, then the mixing coefficient is sampled from the truncated normal distribution ( , ), represents the mean of the truncated normal distribution, represents the variance of the truncated normal distribution.

[0044] S4. Input the augmented sample set into the pre-trained time series prediction model for batch training, dynamically adjust the model parameters to adapt to the distribution shift, and output the final prediction result through fusion; evaluate the performance of the model through a loss function.

[0045] In a feasible implementation manner, inputting the augmented sample set into the pre-trained time series prediction model for batch training, dynamically adjusting the model parameters to adapt to the distribution shift, and outputting the final prediction result through fusion includes: The time series prediction model adopts a two-stream predictor architecture; decompose the input time series into a long-term trend component and a periodic seasonal component; Input the trend component and the seasonal component into independent prediction sub-models respectively, and fuse the output results into the final predicted value.

[0046] In a feasible implementation manner, decomposing the input time series into a long-term trend component and a periodic seasonal component includes: The sequence decomposition of the input time series is achieved through moving average kernel and difference calculation:

[0047] wherein, is the trend component, is the seasonal component, is the enhanced sample.

[0048] In a feasible implementation, the trend component and the seasonal component are respectively input into independent prediction sub-models, and the output results are fused into the final prediction value, including: Input the decomposed trend component and the periodic component into the same-structured two-stream predictors; the outputs of the two-stream predictors are fused to generate the final prediction result : .

[0049] In a feasible implementation, during the adaptation process of online testing, the present invention alleviates the negative impact of distribution shift by introducing batch training, which can be achieved through data augmentation techniques. However, existing data augmentation methods, such as linear mixing and cutmix, focus on mixing time series in the time domain, which will destroy the frequency domain information crucial for prediction. Since the distribution shift is more significant in the adaptation of online testing, it becomes particularly crucial to retain the frequency domain information of the time series.

[0050] To better retain the frequency domain information, the present invention proposes a two-stream enhancement method focusing on the frequency domain amplitude and phase, and selects a set of historical samples with a closer distribution during testing as the enhancement source. This measure ensures that the amplitude and phase of the data augmentation samples are reasonably interpolated based on the test samples, avoiding causing destructive interference in the frequency domain. First, apply the fast Fourier transform to the test samples and the historical samples in, and decompose them into amplitude components and phase components, as shown in the following formula:

[0051] wherein, represents the fast Fourier transform, and The sub-table represents amplitude and phase, e represents the base of the natural logarithm, and j represents the imaginary number. Subsequently, the amplitudes and phases of the test samples and historical samples are mixed to enhance the data through relevant historical patterns while retaining the frequency domain information. To better utilize the historical sample information, when the distance between the historical sample and the test sample in the latent space is small, which means their distributions are close, data augmentation dominated by the historical sample is adopted at this time; conversely, when the distance between the two is large, which means the distribution gap is large, data augmentation based on the test sample is prioritized at this time. The specific mixing process is shown in the following formula:

[0052]

[0053] Among them, represents the set of samples after data augmentation, and are the mixing coefficients of amplitude and phase respectively. When the distance between the latent vectors corresponding to the test sample and the historical sample is lower than the threshold, the mixing coefficient is sampled from the uniform distribution indicating that data augmentation is dominated by the historical sample. If the distance is higher than the threshold, it is sampled from the truncated normal distribution with a high mean and a low standard deviation indicating that data augmentation is dominated by the test sample. The sampling distribution determination of the mixing coefficient is shown in the following formula:

[0054] Among them, is the distance between the latent vectors, is the pre-set threshold hyperparameter. Finally, the inverse fast Fourier transform is applied to the mixed components to obtain the enhanced sample set For better understanding, the present invention uses to represent the elements in , and the specific process is shown in the following formula:

[0055] Among them, represents the inverse fast Fourier transform. After obtaining the enhanced sample , the enhanced sample and the test sample are concatenated into a batch and input into the next module for training. Compared with the method of only using a single test sample to update the model, the transferable online enhancement module significantly reduces the negative impact of noise on model optimization.

[0056] In a feasible implementation, in order to effectively learn complex time patterns in time series prediction, the present invention adopts a sequence decomposition technique. This technique simplifies the complexity of the original data, enabling the model to make better predictions. Specifically, the present invention applies a moving average kernel to the input sequence to extract the trend component of the time series, and takes the difference between the trend component and the original sequence as the seasonal component. The two respectively reflect the long-term trend and periodic relationship of the time series. The sequence decomposition process is as follows:

[0057] Wherein, and respectively represent the extracted long-term trend term and periodic term. The present invention maintains the length of the original sequence through a padding operation, and then uses an average pooling layer (AvgPool) for moving average calculation. The decomposed trend component and the periodic component will be input into a two-stream predictor with the same structure. The output of the two-stream predictor is fused to generate the final prediction result , and the specific process is as follows: .

[0058] In a feasible implementation, in S4, inputting the enhanced sample set into a pre-trained time series prediction model for batch training includes: Using to optimize the model parameters, and the loss function is defined as:

[0059] Wherein, N represents the number of channels of the time series; represents the true value; H represents the prediction length; k represents the index of the channel;

[0060] During the adaptation process of online testing, the mean squared error and mean absolute error between the true values of all samples and the model prediction output Y are used as the final evaluation metrics to compare the model performance, as follows: ; .

[0061] In a feasible implementation manner, the batch training combined with online transferable data augmentation method proposed by the present invention is a general module, aiming to alleviate the distribution shift problem in the online learning process. This adaptability enables it to be applied to any online time series prediction model. The flexibility of the batch training combined with online transferable data augmentation method allows it to be seamlessly integrated with various models, enhancing the robustness of the model to data distribution changes. In addition, the batch training combined with online transferable data augmentation method is not limited to a specific algorithm or framework, which means that with the development of more advanced deep models, the batch training combined with online transferable data augmentation method can be embedded therein to further improve performance.

[0062] In the embodiment of the present invention, by making full use of historical distribution information, a transferable historical sample selection module is designed to select historical samples close to the test samples. By intelligently selecting historical distribution information similar to the current data pattern, the adaptability of the model to the new distribution is enhanced.

[0063] The paradigm of the original online learning that updates with a single sample is changed, and a batch training module is introduced to alleviate the negative impact of potential noise, improve the robustness of the model, and suppress noise sensitivity.

[0064] Data augmentation is performed on the time series based on the frequency domain. From the two perspectives of amplitude and phase, data augmentation is respectively performed on the time series, avoiding destructive interference on the time domain of the time series.

[0065] The present invention performs excellently in real scenarios as Figure 3 shown Figure 3 showing the prediction effects of three methods on the Weather dataset, where Figure 3 a in Figure 3 is the effect of the method PatchTST, Figure 3 b in

[0066] In the embodiment of the present invention, there is a 1. Transferable historical sample selection module. In traditional online time series prediction, historical distribution information is often obtained by directly using the most recent historical samples over a time span. The present invention innovatively proposes an intelligent historical sample selection method. Specifically, the present invention first trains an unsupervised VAE model on the training set, learns the latent space of the exclusive data through the VAE model, and selects historical samples close to the test distribution for data augmentation by measuring the distance between historical samples and test samples in the latent space. Without this key module of transferable historical sample selection, it would be difficult for the present invention to make full use of historical distribution information, and the prediction accuracy would also be greatly reduced.

[0067] 2. Transferable online augmentation module. The introduction of noise will cause the model to deviate from the original optimization path and reduce the model performance. To address this problem, the present invention cleverly uses historical distribution information to neutralize the noise information carried by test samples. Since batch training has more trainable samples and the model is more robust and less susceptible to noise interference compared to single-sample training, the present invention concatenates the data-augmented samples and test samples and inputs them into the model for training to mitigate the interference of noise. Without this key module of transferable online augmentation, it would be difficult for the present invention to reduce the negative impact of noise, and the adaptation effect of the model during testing would also be weakened.

[0068] 3. Frequency-domain-based data augmentation technique. Traditional time-domain-based data augmentation techniques often simply mix two time series in the time domain, which will lose the key information of the time series and cause destructive interference to the time series. For this reason, the present invention innovatively introduces a frequency-domain-based data augmentation technique. Data augmentation is performed separately from the two aspects of amplitude and phase, which avoids the interpolation interference of directly mixing in the time domain and effectively improves the quality of the augmented time series data. Without the frequency-domain-based data augmentation technique, it would be difficult for the model to make full use of historical distribution information and achieve batch training.

[0069] In summary, the transferable historical sample selection module, the transferable online augmentation module, and the frequency-domain-based data augmentation method are the three key technologies of the present invention. They enhance the robustness of the model from the three aspects of sample selection, batch training, and data augmentation respectively. These three key technologies are closely linked and complement each other, and none of them can be missing, jointly constituting the core competitiveness of the present invention. Through the detailed description in the technical solution part, it can be seen that if any one of the key technologies is missing, it would be difficult for the present invention to achieve the desired anti-noise performance. Therefore, these three key innovation points are not only the key points of patent protection but also the necessary conditions for achieving the expected goals of the present invention.

[0070] Figure 4 is a schematic diagram of an online test-time adaptation device for time series prediction provided by an embodiment of the present invention, as Figure 4As shown in the figure, an online test adaptation device for time series prediction is provided in an embodiment of the present invention. This device can be implemented by an online test adaptation device for time series prediction, and the online test adaptation device for time series prediction can be a terminal or a server. The online test adaptation device for time series prediction proposed by the present invention includes: a transferable historical sample selection module 310, a transferable online enhancement module 320, and a prediction and evaluation module 330; The transferable historical sample selection module 310 is configured to establish and maintain a historical sample memory bank, store historical time series data, and update the memory bank through a first-in-first-out strategy; Receive test samples; screen a set of historical samples in the historical sample memory bank whose similarity in the latent space with the test samples meets a preset condition; The transferable online enhancement module 320 is configured to perform frequency-domain-based hybrid data enhancement on the test samples and the set of historical samples to generate an enhanced sample set; The prediction and evaluation module 330 is configured to input the enhanced sample set into a pre-trained time series prediction model for batch training, dynamically adjust the model parameters to adapt to distribution shift, and output a final prediction result through fusion; evaluate the performance of the model through a loss function.

[0071] Optionally, the transferable historical sample selection module 310 is configured to map the test samples and the historical samples in the memory bank to the latent space through a pre-trained variational autoencoder (VAE); Calculate the cosine similarity between the latent vector of the test sample and the latent vectors of each historical sample; Sort according to the cosine similarity, and select the top n historical samples with the highest similarity to form the set of historical samples.

[0072] Optionally, the transferable online enhancement module 320 is configured to perform a fast Fourier transform on the test samples and the set of historical samples, and extract the amplitude component and the phase component respectively; Dynamically determine the mixing coefficients of the amplitude and the phase according to the latent space distance between the historical samples and the test samples; Perform interpolation mixing on the amplitude component and the phase component respectively, and reconstruct the enhanced time series through an inverse Fourier transform.

[0073] Optionally, the determination rule of the mixing coefficient includes: If the latent space distance is less than or equal to a preset threshold τ, the mixing coefficient is sampled from a uniform distribution ( , 1.0); If the latent space distance is greater than τ, the mixing coefficient is sampled from a truncated normal distribution ( , ) for sampling.

[0074] Optionally, the prediction and evaluation module 330 is used for the time series prediction model to adopt a two-stream predictor architecture; decompose the input time series into a long-term trend component and a periodic seasonal component; Input the trend component and the seasonal component into independent prediction sub-models respectively, and fuse the output results into the final prediction value.

[0075] Optionally, decomposing the input time series into a long-term trend component and a periodic seasonal component includes: The sequence decomposition of the input time series is realized through a moving average kernel and difference calculation:

[0076] Among them, is the trend component, is the seasonal component, is the enhanced sample.

[0077] Optionally, inputting the trend component and the seasonal component into independent prediction sub-models respectively, and fusing the output results into the final prediction value includes: Input the decomposed trend component and the periodic component into the two-stream predictors with the same structure; the output of the two-stream predictors is fused to generate the final prediction result : .

[0078] Optionally, the prediction and evaluation module 330 is used to adopt to optimize the model parameters with the loss function defined as:

[0079] Among them, N represents the number of channels of the time series; represents the true value; H represents the prediction length; k represents the index of the channel;

[0080] During the adaptation process in online testing, the mean squared error and mean absolute error between the true values of all samples and the model prediction output Y are used as the final evaluation metrics, as follows:

[0081] .

[0082] In the embodiments of the present invention, by making full use of historical distribution information, a transferable historical sample selection module is designed to select historical samples close to the test samples. By intelligently selecting historical distribution information similar to the current data pattern, the adaptability of the model to new distributions is enhanced.

[0083] The original online learning paradigm of updating with a single sample is transformed, and a batch training module is introduced to mitigate the negative impact of potential noise, improve the robustness of the model, and suppress noise sensitivity.

[0084] Data augmentation is performed on the time series based on the frequency domain. Data augmentation is performed on the time series from two perspectives: amplitude and phase, avoiding destructive interference in the time domain of the time series.

[0085] The present invention performs excellently in real scenarios. Experiments show that the present invention can still achieve good prediction accuracy on datasets with severe distribution shifts, verifying the practical value of the present invention.

[0086] Figure 5 It is a schematic structural diagram of an online test adaptation device for time series prediction provided by an embodiment of the present invention. As Figure 5 shown, the online test adaptation device for time series prediction may include the above Figure 4 shown online test adaptation device for time series prediction. Optionally, the online test adaptation device 410 for time series prediction may include a first processor 2001.

[0087] Optionally, the online test adaptation device 410 for time series prediction may further include a memory 2002 and a transceiver 2003.

[0088] Among them, the first processor 2001 is connected to the memory 2002 and the transceiver 2003, such as through a communication bus.

[0089] Next, in combination with Figure 5 each component of the online test adaptation device 410 for time series prediction will be specifically introduced: Among them, the first processor 2001 is the control center of the online test-time adaptation device 410 for time series prediction, which can be a single processor or a collective term for multiple processing elements. For example, the first processor 2001 can be one or more central processing units (CPUs), or can be an application specific integrated circuit (ASIC), or can be one or more integrated circuits configured to implement the embodiments of the present invention, such as: one or more digital signal processors (DSPs), or one or more field programmable gate arrays (FPGAs).

[0090] Optionally, the first processor 2001 can execute various functions of the online test-time adaptation device 410 for time series prediction by running or executing software programs stored in the memory 2002 and calling data stored in the memory 2002.

[0091] In a specific implementation, as an embodiment, the first processor 2001 can include one or more CPUs, such as Figure 5 the CPU0 and CPU1 shown in

[0092] In a specific implementation, as an embodiment, the online test-time adaptation device 410 for time series prediction can also include multiple processors, such as Figure 5 the first processor 2001 and the second processor 2004 shown in. Each of these processors can be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). Here, the processor can refer to one or more devices, circuits, and / or processing cores for processing data (such as computer program instructions).

[0093] Among them, the memory 2002 is used to store software programs for implementing the solution of the present invention and is controlled by the first processor 2001 for execution. The specific implementation manner can refer to the above method embodiments and will not be elaborated here.

[0094] Optionally, the memory 2002 may be a read-only memory (ROM) or other type of static storage device that can store static information and instructions, a random access memory (RAM) or other type of dynamic storage device that can store information and instructions, or may also be an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM), or other optical disc storage, optical disc storage (including compact discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), magnetic disk storage media, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 2002 may be integrated with the first processor 2001 or may exist independently and is coupled to the first processor 2001 through an interface circuit of the online test adaptation device 410 for time series prediction ( Figure 5 not shown in the figure), and the embodiments of the present invention do not make specific limitations thereto.

[0095] The transceiver 2003 is used to communicate with a network device or communicate with a terminal device.

[0096] Optionally, the transceiver 2003 may include a receiver and a transmitter ( Figure 5 not shown separately in the figure). Among them, the receiver is used to implement the receiving function, and the transmitter is used to implement the sending function.

[0097] Optionally, the transceiver 2003 may be integrated with the first processor 2001 or may exist independently and is coupled to the first processor 2001 through an interface circuit of the online test adaptation device 410 for time series prediction ( Figure 5 not shown in the figure), and the embodiments of the present invention do not make specific limitations thereto.

[0098] It should be noted that Figure 5 the structure of the online test adaptation device 410 for time series prediction shown in the figure does not constitute a limitation to the router. The actual knowledge structure recognition device may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.

[0099] In addition, the technical effects of the online test adaptation device 410 for time series prediction may refer to the technical effects of the online test adaptation method for time series prediction described in the above method embodiments, and will not be elaborated here.

[0100] It should be understood that the first processor 2001 in the embodiments of the present invention may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0101] It should also be understood that the memory in the embodiments of the present invention may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM) or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchlink DRAM (SLDRAM) and direct rambus RAM (DR RAM).

[0102] The above embodiments can be implemented in whole or in part by software, hardware (such as circuits), firmware, or any combination thereof. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the processes or functions described in the embodiments of the present invention are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable sensors. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by wired (such as infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or a data center that contains one or more collections of available media. The available media can be magnetic media (such as floppy disks, hard disks, magnetic tapes), optical media (such as DVDs), or semiconductor media. The semiconductor media can be a solid-state drive.

[0103] It should be understood that the term "and / or" in this document is merely a description of the association relationship between associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. Here, A and B can be singular or plural. In addition, the character " / " in this document generally represents an "or" relationship between the associated objects before and after, but it may also represent an "and / or" relationship, which can be specifically understood by referring to the context before and after.

[0104] It should be understood that in various embodiments of the present invention, the order numbers of the above processes do not mean the order of execution. The order of execution of each process should be determined by its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present invention.

[0105] Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed in this document can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Professionals can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present invention.

[0106] The unit described as a separation component may or may not be physically separated. The component shown as a unit may or may not be a physical unit, that is, it may be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0107] In addition, each functional unit in various embodiments of the present invention may be integrated into a processing unit, may exist physically separately for each unit, or two or more units may be integrated into one unit.

[0108] If the above functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or a part of this technical solution can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention.

[0109] As described above, the above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention can easily think of changes or substitutions, which should all be covered by the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.

Claims

1. An online test-time adaptation method for time series prediction, characterized in that, Including: S1. Establish and maintain a historical sample memory bank, store historical time series data, and update the memory bank through a first-in-first-out strategy; S2. Receive a test sample; screen a set of historical samples in the historical sample memory bank whose similarity with the test sample in the latent space meets a preset condition; S3. Perform frequency-domain-based hybrid data augmentation on the test sample and the set of historical samples to generate a set of augmented samples; S4. Input the set of augmented samples into a pre-trained time series prediction model for batch training, dynamically adjust the model parameters to adapt to distribution shift, and output a final prediction result through fusion; evaluate the performance of the model through a loss function.

2. The method according to claim 1, characterized in that, In S2, receive a test sample; screen a set of historical samples in the historical sample memory bank whose similarity with the test sample in the latent space meets a preset condition, including: Map the test sample and the historical samples in the memory bank to the latent space through a pre-trained variational autoencoder (VAE); Calculate the cosine similarity between the latent vector of the test sample and the latent vectors of each historical sample; Sort according to the cosine similarity, and select the top n historical samples with the highest similarity to form a set of historical samples.

3. The method according to claim 1, wherein In S3, perform frequency-domain-based hybrid data augmentation on the test sample and the set of historical samples to generate a set of augmented samples, including: Perform fast Fourier transform on the test sample and the set of historical samples, and extract the amplitude component and the phase component respectively; Dynamically determine the mixing coefficients of the amplitude and the phase according to the latent space distance between the historical samples and the test sample; Perform interpolation mixing on the amplitude component and the phase component respectively, and reconstruct the enhanced time series through inverse Fourier transform.

4. The method according to claim 3, wherein The determination rule of the mixing coefficient includes: If the latent space distance is less than or equal to a preset threshold τ, the mixing coefficient is sampled from a uniform distribution U( β , 1.0); If the latent space distance is greater than τ, the mixing coefficient is sampled from a truncated normal distribution N( μ , θ ).

5. The method according to claim 1, wherein In S4, input the set of augmented samples into a pre-trained time series prediction model for batch training, dynamically adjust the model parameters to adapt to distribution shift, and output a final prediction result through fusion, including: The time series prediction model adopts a two-stream predictor architecture; decompose the input time series into a long-term trend component and a periodic seasonal component; Input the trend component and the seasonal component into independent prediction sub-models respectively, and fuse the output results into a final predicted value.

6. The method according to claim 5, wherein Decompose the input time series into a long-term trend component and a periodic seasonal component, including: The sequence decomposition of the input time series is realized through a moving average kernel and difference calculation: ； Among them, is the trend component, is the seasonal component, is the enhanced sample.

7. The method according to claim 6, wherein Input the trend component and the seasonal component into independent prediction sub-models respectively, and fuse the output results into a final predicted value, including: The decomposed trend component and the periodic component are input into two-stream predictors with the same structure; the outputs of the two-stream predictors are fused to generate the final prediction result : 。 8. The method according to claim 7, characterized in that, In S4, input the set of augmented samples into a pre-trained time series prediction model for batch training, including: Adopt Loss optimization model parameters, and the loss function is defined as: ； Among them, N represents the number of channels of the time series; represents the true value; H represents the prediction length; k represents the index of the channel; During the adaptation process in online testing, the true values of all samples and the mean squared error and mean absolute error between the model prediction output Y are used as the final evaluation metrics, as shown below: ； 。 9. An online test-time adaptation device for time series prediction, characterized in that, The online test-time adaptation device for time series prediction is used to implement the online test-time adaptation method for time series prediction according to any one of claims 1-8, wherein the device includes: A transferable historical sample selection module, used to establish and maintain a historical sample memory bank, store historical time series data, and update the memory bank through a first-in-first-out strategy; Receive a test sample; screen a set of historical samples in the historical sample memory library whose similarity with the test sample in the latent space meets a preset condition; A transferable online enhancement module for performing frequency-domain-based hybrid data enhancement on the test sample and the set of historical samples to generate a set of enhanced samples; A prediction and evaluation module for inputting the set of enhanced samples into a pre-trained time series prediction model for batch training, dynamically adjusting model parameters to adapt to distribution shift, and outputting a final prediction result through fusion; evaluating the performance of the model through a loss function.

10. An online test-time adaptation device for time series prediction, characterized in that, The online test-time adaptation device for time series prediction includes: A processor; a memory, on which computer-readable instructions are stored, and when the computer-readable instructions are executed by the processor, any one of the methods in the online test-time adaptation method for time series prediction according to any one of claims 1-8 is implemented.

Images