Time sequence pre-training large model key characteristic rapid test method and related device

Abstract

The invention belongs to the field of time sequence pre-training large model characteristic testing, and provides a time sequence pre-training large model key characteristic rapid testing method and a related device, and the method comprises the steps: constructing a test set, inputting the test set into a to-be-tested time sequence pre-training model, and determining and outputting a model with basic and composite time pattern recognition capability; sequentially adding different types of external features into the original target sequence, respectively calculating baseline prediction performance of only using the original target sequence and prediction performance after adding each feature, and outputting a feature redundancy evaluation result; and based on a feature redundancy evaluation result, observing a performance curve of prediction performance of the model changing along with the context length, and based on the performance curve, judging an optimal memory window of the model and sensitivity of the optimal memory window to the long context. The test process has rapidness and systematicness, the basic portrait of the model can be obtained through rapid diagnosis for several minutes, and direction guidance is provided for deep development of model tuning.

Description

Technical Field

[0001] This invention belongs to the field of feature testing of time-series pre-trained large models, and specifically relates to a rapid testing method and related apparatus for key features of time-series pre-trained large models. Background Technology

[0002] Pre-trained time series models (such as the Moirai series) represent a significant breakthrough in the field of time series forecasting. These models achieve powerful zero-shot prediction capabilities through pre-training on large-scale, multi-domain time series data. However, unlike traditional machine learning models, the internal mechanisms of pre-trained models are more complex. Their prediction and decision-making processes are "black boxes," making it difficult for users to understand the model's preferences, capability boundaries, and optimal usage.

[0003] In practical applications of large-scale time series models, we often face the following dilemmas: which time patterns is the model most sensitive to, whether external features (covariates) truly contribute to prediction, and how to choose the optimal context length. Traditional trial-and-error methods are not only inefficient but may also lead to wasted time and computational resources in uncertain or even incorrect directions. More importantly, pre-trained time series models already have a large amount of prior knowledge about time patterns built in, meaning that some seemingly useful feature engineering may actually be redundant or even have negative effects. For example, adding time-encoded features may conflict with the model's built-in temporal understanding, leading to a decrease in prediction performance. Similarly, excessively long historical context may introduce noise rather than provide effective and useful information. Summary of the Invention

[0004] The purpose of this invention is to provide a rapid testing method and related apparatus for key characteristics of time-series pre-trained large models, so as to solve the problems of low efficiency and inaccurate judgment of traditional methods.

[0005] To achieve the above objectives, the present invention adopts the following technical solution: In a first aspect, the present invention provides a method for rapid testing of key features of a time-series pre-trained large model, including: Construct a synthetic time series test set containing different time patterns, input the test set into the time series pre-trained model to be tested, calculate the prediction mean square error and prediction correlation coefficient of the model for each pattern, and determine whether the output model has the ability to recognize basic and composite time patterns. Based on a model with basic and composite temporal pattern recognition capabilities, different types of external features are added sequentially to the original target sequence. The baseline prediction performance using only the original target sequence and the prediction performance after adding each feature are calculated. The feature effectiveness index is calculated based on the performance change, and the feature redundancy evaluation result is output. Based on the feature redundancy evaluation results, the historical context length of the input model is varied, and the performance curve of the model's prediction performance as a function of the context length is observed. Based on the performance curve, the optimal memory window of the model and its sensitivity to long contexts are determined.

[0006] Furthermore, the construction includes a synthetic time series test set comprising different time patterns, including: Construct a synthetic time series test set containing daily cycle patterns, weekly cycle patterns, trend patterns, stochastic patterns, and mixed patterns: Daily cycle pattern: p1(t) = μ_target + (σ_target × β1) × sin(2π × t / T_day) + ε1(t) (1) Weekly cycle pattern: p2(t)=μ_target + (σ_target × β2) × sin(2π × t / T_week) + ε2(t) (2) Trend Pattern: p3(t) = μ_target × α3+ (σ_target / T_context × β3) × t + ε3(t) (3) Random mode: p4(t) = μ_target + ε4(t)(4) Mixed Mode: p5(t) = p1(t) + p2(t) + p3(t) + ε5(t)(5) Where t is the time step index, μ_target is the empirical mean of the target sequence, σ_target is the empirical standard deviation of the target sequence, T_context is the context length; α3 is the trend pattern baseline adjustment coefficient; β1, β2, β3 are pattern feature adjustment coefficients; T_day is the daily cycle length, T_week is the weekly cycle length; ε i (t) represents the noise term, γ i This is the noise level adjustment coefficient.

[0007] Furthermore, the step of inputting the test set into the time series pre-trained model to be tested and calculating the model's prediction mean square error and prediction correlation coefficient for each mode includes: For each model, calculate the predictive performance metric. Mean square error: MSE(p i ) = (1 / n) × Σ j (y j - j ) 2 (6) Correlation coefficient: r(p i ) = Corr(y, ) (7) where y j is the true value, j is the predicted value, n is the number of prediction points; j is the time - point index, j = 1, 2,..., n; Corr(y, ) is the Pearson correlation coefficient, which measures the linear correlation degree between the predicted value and the true value.

[0008] Further, the model for judging the output with basic and composite time - pattern recognition capabilities includes: For the single - calculation metrics p1, p2, p3, p4, when MSE(p1) < MSE(p4) and r(p1) > 0.7, the model has the ability to understand daily cycles; when MSE(p2) < MSE(p4) and r(p2) > 0.6, the model has the ability to understand weekly cycles; when MSE(p3) < MSE(p4), the model has the ability to identify trends; when MSE(p4) is the smallest, the model is conservative and tends to predict the average value.

[0009] For the mixed pattern p5, when MSE(p5) < MSE(p4) and r(p5) > max(r(p1), r(p2), r(p3)), the discriminant model has the ability to recognize and process composite patterns.

[0010] Further, based on the model with basic and composite time - pattern recognition capabilities, different types of external features are sequentially added to the original target sequence, and the baseline prediction performance using only the original target sequence and the prediction performance after adding each feature are calculated respectively. According to the performance change, the feature effectiveness index includes: Define the baseline prediction performance as MSE0, and use the original target sequence {x1, x2,..., x t} for prediction; The prediction performance after adding the feature fi is MSE i , and use the enhanced sequence {(x1, fi1), (x2, fi2),...,(x t , fi t )} for prediction; fi includes the following four categories, where f1 is the time - derivative feature: f1(t) = sin(2π × (t mod T) / T) (8) f2 is the noise feature: f 2(t) ~ N(0,1)(9) f3 represents the trend characteristic: f3(t) = t(10) f4 is a lag characteristic: f 4(t) = x_{tk}(11) t mod T is the modulo operation, which calculates the remainder when t is divided by T, mapping continuous time to the interval [0, T-1] to achieve periodic repetition; T is the time encoding period; N(0,1) is the standard normal distribution; k is the number of lag steps; f1 tests the degree of conflict with the model's built-in time understanding; f2 serves as a negative control, testing the model's resistance to irrelevant information and its tendency to overfit; f3 tests the redundancy of linear trend information; and f4 tests the value of autoregressive information. The feature redundancy evaluation criteria are set as follows: Effectiveness (f) i ) = (MSE0 - MSE i ) / MSE0×100%(12) MSE0 is the mean squared error when only the original target sequence is used. i is the mean squared error after adding feature fi, and Effectiveness is the effectiveness of the feature, with positive values ​​indicating benefits and negative values ​​indicating harm or redundancy.

[0011] Furthermore, the output feature redundancy evaluation result includes: When Effectiveness(f i When -2% > 2%, features help increase predictive performance; when -2% ≤ Effectiveness(f) i If the effectiveness (f) is ≤ 2%, the feature has a limited or no impact on the model's performance improvement; when the effectiveness (f) is ≤ 2%, the feature has a limited or no impact on the model's performance improvement. i <-2%, feature redundancy, performance improvement of the damage model.

[0012] Furthermore, based on the feature redundancy evaluation results, the historical context length of the input model is varied, the performance curve of the model's prediction performance changes with the context length is observed, and the optimal memory window of the model and its sensitivity to long contexts are determined based on the performance curve, including: Define a context length set C = {c1, c2, ..., c m}, for each length c i Using historical sequence {x_{tc i +1}, x_{tc i +2}, ..., xt Predict the future sequence {x_{t+1}, x_{t+2}, ..., x_{t+h}}; In the above definition, C represents the set of test context lengths; c i x_i represents the length of the i-th context; t represents the current time point, serving as the prediction boundary; h represents the prediction field of view length; x_{tc} represents the length of the i-th context. i +1} represents the value of the starting time point of the history window, x t The value at the current time point t; By experimentally observing the variation pattern of MSE under different context lengths c, computational performance curves were constructed, and data points were recorded: {(c1,MSE1), (c2,MSE2), ..., (c m MSE m )}; when MSE / c<0, longer context is beneficial; when MSE(c*) = min{MSE(c)}, then c* is the optimal context length parameter; when MSE / If c>0, then long context information is harmful; when MSE remains unchanged, the model is not sensitive to context length.

[0013] Secondly, the present invention provides a rapid testing system for key characteristics of a time-series pre-trained large model, characterized in that it includes: The dataset construction module is used to construct a synthetic time series test set containing different time patterns. The test set is input into the time series pre-trained model to be tested, the mean square error of the model's prediction for each pattern and the prediction correlation coefficient are calculated, and the output model is judged to have the ability to recognize basic and composite time patterns. The redundancy evaluation module is used to add different types of external features to the original target sequence in sequence based on a model with basic and composite temporal pattern recognition capabilities. It calculates the baseline prediction performance using only the original target sequence and the prediction performance after adding each feature. Based on the performance change, it calculates the feature effectiveness index and outputs the feature redundancy evaluation results. The sensitivity assessment module is used to determine the optimal memory window of the model and its sensitivity to long contexts based on the feature redundancy assessment results, by varying the historical context length of the input model, observing the performance curve of the model's prediction performance as a function of the context length, and then determining the model's sensitivity to long contexts based on the performance curve.

[0014] Furthermore, the construction includes a synthetic time series test set comprising different time patterns, including: Construct a synthetic time series test set containing daily cycle patterns, weekly cycle patterns, trend patterns, stochastic patterns, and mixed patterns: Daily cycle pattern: p1(t) = μ_target + (σ_target × β1) × sin(2π × t / T_day) + ε1(t) (1) Weekly cycle pattern: p2(t)=μ_target + (σ_target × β2) × sin(2π × t / T_week) + ε2(t) (2) Trend Pattern: p3(t) = μ_target × α3+ (σ_target / T_context × β3) × t + ε3(t) (3) Random mode: p4(t) = μ_target + ε4(t)(4) Mixed Mode: p5(t) = p1(t) + p2(t) + p3(t) + ε5(t)(5) Where t is the time step index, μ_target is the empirical mean of the target sequence, σ_target is the empirical standard deviation of the target sequence, T_context is the context length; α3 is the trend pattern baseline adjustment coefficient; β1, β2, β3 are pattern feature adjustment coefficients; T_day is the daily cycle length, T_week is the weekly cycle length; ε i (t) represents the noise term, γ i This is the noise level adjustment coefficient.

[0015] Furthermore, the step of inputting the test set into the time series pre-trained model to be tested and calculating the model's prediction mean square error and prediction correlation coefficient for each mode includes: For each model, calculate the predictive performance metric. Mean square error: MSE(p i ) = (1 / n) × Σ j (y j - j ) 2 (6) Correlation coefficient: r(p i = Corr(y, (7) Where y j For the true value, j Here, is the predicted value, n is the number of prediction points, and j is the time point index, j = 1, 2, ..., n; Corr(y, ) is the Pearson correlation coefficient, which measures the linear correlation degree between the predicted value and the true value.

[0016] Furthermore, the model for judging and outputting with basic and composite time pattern recognition capabilities includes: For the single calculation indicators p1, p2, p3, p4, when MSE(p1) < MSE(p4) and r(p1) > 0.7, the model has the ability to understand daily cycles; when MSE(p2) < MSE(p4) and r(p2) > 0.6, the model has the ability to understand weekly cycles; when MSE(p3) < MSE(p4), the model has the ability to identify trends; when MSE(p4) is the smallest, the model is conservative and tends to predict the average value.

[0017] For the mixed mode p5, when MSE(p5) < MSE(p4) and r(p5) > max(r(p1), r(p2), r(p3)), the discriminant model has the ability to recognize and process composite patterns.

[0018] Furthermore, based on the model with basic and composite time pattern recognition capabilities, different types of external features are sequentially added to the original target sequence, and the baseline prediction performance using only the original target sequence and the prediction performance after adding each feature are calculated respectively. According to the performance change, the feature effectiveness index is calculated, including: Define the baseline prediction performance as MSE0, and use the original target sequence {x1, x2,..., x t} for prediction; The prediction performance after adding the feature fi is MSE i , and use the enhanced sequence {(x1, fi1), (x2, fi2),...,(x t , fi t )} for prediction; fi includes the following four categories, where f1 is a time-derived feature: f1(t) = sin(2π × (t mod T) / T) (8) f2 is a noise feature: f2(t) ~ N(0,1) (9) f3 is a trend feature: f3(t) = t (10) f4 is a lag feature: f4(t) = x_{t-k} (11) t mod T is the modulo operation, which calculates the remainder when t is divided by T, mapping continuous time to the interval [0, T-1] to achieve periodic repetition; T is the time encoding period; N(0,1) is the standard normal distribution; k is the number of lag steps; f1 tests the degree of conflict with the model's built-in time understanding; f2 serves as a negative control, testing the model's resistance to irrelevant information and its tendency to overfit; f3 tests the redundancy of linear trend information; and f4 tests the value of autoregressive information. The feature redundancy evaluation criteria are set as follows: Effectiveness (f) i ) = (MSE0 - MSE i ) / MSE0×100%(12) MSE0 is the mean squared error when only the original target sequence is used. i is the mean squared error after adding feature fi, and Effectiveness is the effectiveness of the feature, with positive values ​​indicating benefits and negative values ​​indicating harm or redundancy.

[0019] Furthermore, the output feature redundancy evaluation result includes: When Effectiveness(f i When -2% > 2%, features help increase predictive performance; when -2% ≤ Effectiveness(f) i If the effectiveness (f) is ≤ 2%, the feature has a limited or no impact on the model's performance improvement; when the effectiveness (f) is ≤ 2%, the feature has a limited or no impact on the model's performance improvement. i <-2%, feature redundancy, performance improvement of the damage model.

[0020] Furthermore, based on the feature redundancy evaluation results, the historical context length of the input model is varied, the performance curve of the model's prediction performance changes with the context length is observed, and the optimal memory window of the model and its sensitivity to long contexts are determined based on the performance curve, including: Define a context length set C = {c1, c2, ..., c m}, for each length c i Using historical sequence {x_{tc i +1}, x_{tc i +2}, ..., x t Predict the future sequence {x_{t+1}, x_{t+2}, ..., x_{t+h}}; In the above definition, C represents the set of test context lengths; c i x_i represents the length of the i-th context; t represents the current time point, serving as the prediction boundary; h represents the prediction field of view length; x_{tc} represents the length of the i-th context.i +1} represents the value of the starting time point of the history window, x t The value at the current time point t; By experimentally observing the variation pattern of MSE under different context lengths c, computational performance curves were constructed, and data points were recorded: {(c1,MSE1), (c2,MSE2), ..., (c m MSE m )}; when MSE / c<0, longer context is beneficial; when MSE(c*) = min{MSE(c)}, then c* is the optimal context length parameter; when MSE / If c>0, then long context information is harmful; when MSE remains unchanged, the model is not sensitive to context length.

[0021] Thirdly, the present invention provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the rapid testing method for key features of the time-series pre-trained large model.

[0022] Fourthly, the present invention provides a computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the steps of the method for rapid testing of key features of a time-series pre-trained large model.

[0023] Compared with the prior art, the present invention has the following technical effects: This invention's testing method constructs comparative experiments, combining synthetic data with known answers and real data, to observe the model's response patterns under different input conditions. It uses mathematically generated standard time patterns to test the model's pattern recognition ability; it assesses feature redundancy by adding different types of external features and observing performance changes; and it systematically varies the context length to determine the model's optimal memory window. The testing process is rapid and systematic, providing a basic profile of the model within minutes of quick diagnosis, offering guidance for in-depth model optimization. Attached Figure Description

[0024] Figure 1 This is a flowchart of the present invention.

[0025] Figure 2 This is a logic block diagram of the present invention. Detailed Implementation

[0026] The present invention will be further described below with reference to the accompanying drawings: Example 1, please refer to Figure 1This invention provides a fast testing method for key features of time-series pre-trained large models, including: Construct a synthetic time series test set containing different time patterns, input the test set into the time series pre-trained model to be tested, calculate the prediction mean square error and prediction correlation coefficient of the model for each pattern, and determine whether the output model has the ability to recognize basic and composite time patterns. Based on a model with basic and composite temporal pattern recognition capabilities, different types of external features are added sequentially to the original target sequence. The baseline prediction performance using only the original target sequence and the prediction performance after adding each feature are calculated. The feature effectiveness index is calculated based on the performance change, and the feature redundancy evaluation result is output. Based on the feature redundancy evaluation results, the historical context length of the input model is varied, and the performance curve of the model's prediction performance as a function of the context length is observed. Based on the performance curve, the optimal memory window of the model and its sensitivity to long contexts are determined.

[0027] This invention effectively addresses the critical challenge of unclear intrinsic preferences and capability boundaries in large time-series pre-trained models due to their "black box" nature by constructing a rapid and systematic "comparative experiment" testing system. Its core technological advantage lies in generating a comprehensive capability "profile" for the model within minutes, significantly improving the efficiency and targeting of model optimization and deployment. Specifically, this solution employs three logically progressive steps: time-series pattern recognition, feature redundancy assessment, and context sensitivity assessment. These steps respectively achieve quantitative diagnosis of the model's basic cognitive capabilities, internal knowledge redundancy, and adaptability to external input conditions. This not only transforms model evaluation from the traditional "black box" trial-and-error approach relying on downstream task performance to interpretable and attributable "white box" analysis, but also provides direct and precise data support and directional guidance for subsequent model fine-tuning (such as specifically strengthening the learning of weak patterns), feature engineering (avoiding the introduction of redundant features), and deployment parameter configuration (setting the optimal context length). It fundamentally changes the current situation of blind optimization and empirical use, achieving maximum efficiency and resource optimization in the model application process.

[0028] Example 2, please refer to Figure 2 This invention provides a fast testing method for key features of time-series pre-trained large models, including: (1) Temporal pattern recognition By constructing synthetic time series with known mathematical characteristics, the model's prediction accuracy for different time patterns is tested, thereby inferring the model's inherent time understanding ability.

[0029] Time pattern definition Define a test pattern set P = {p1, p2, ..., p...} n}, where each pattern p i(t) is a function of time t: Daily Cycle Pattern: p1(t) = μ_target + (σ_target × β1) × sin(2π × t / T_day) + ε1(t) (1) Weekly cycle pattern: p2(t)=μ_target + (σ_target × β2) × sin(2π × t / T_week) + ε2(t) (2) Trend Pattern: p3(t) = μ_target × α3+ (σ_target / T_context × β3) × t + ε3(t) (3) Random mode: p4(t) = μ_target + ε4(t)(4) Blending mode: p5(t) = p1(t) + p2(t) + p3(t) + ε5(t)(5) Where t is the time step index (incrementing from 0), μ_target is the empirical mean of the target sequence, σ_target is the empirical standard deviation of the target sequence, T_context is the context length (number of time steps); α3 is the trend pattern baseline adjustment coefficient, which is set to 1.0 in this invention; β1, β2, β3 are pattern feature adjustment coefficients, where β1∈ [1.5,3.0], β2∈ [0.8,2.0], and β3∈ [0.1,0.5] in this invention; T_day is the daily cycle length, set to 96 (15 minutes × 96 = 24 hours), and T_week is the weekly cycle length, set to 672 (15 minutes × 96 × 7 = 7 days); ε i (t) represents the noise term, ε i (t) ~ N(0, (σ_target × γ i ) 2 ), γ i For noise level adjustment coefficients, in this invention, γ1=γ2=0.1, γ3=0.15, γ4=0.25, γ5=0.08.

[0030] In addition, it should be noted that since capturing linear trends is a fundamental capability of time-based large models, p3(t) in the trend pattern of equation (3) only introduces time-based linear trends, with the aim of quickly verifying the basic ability to detect trends.

[0031] Indicator Setting Calculate the prediction performance metrics for each pattern. Mean squared error: MSE(p i ) = (1 / n) × Σ j (y j - j ) 2 (6) Correlation coefficient: r(p i ) = Corr(y, )(7) where y j is the true value, j is the predicted value, n is the number of prediction points (prediction window size); j is the time point index, j = 1, 2,..., n; Corr(y, ) is the Pearson correlation coefficient, measuring the linear correlation degree between the predicted value and the true value.

[0032] 3) Result evaluation For single calculation metrics such as p1, p2, p3, p4, etc., when MSE(p1) < MSE(p4) and r(p1) > 0.7, the model has the ability to understand daily cycles; when MSE(p2) < MSE(p4) and r(p2) > 0.6, the model has the ability to understand weekly cycles; when MSE(p3) < MSE(p4), the model has the ability to identify trends; when MSE(p4) is the smallest, the model may be overly conservative and tend to predict the average value.

[0033] For the mixed pattern p5, when MSE(p5) < MSE(p4) and r(p5) > max(r(p1), r(p2), r(p3)), it is judged that the model has the ability to recognize and process composite patterns.

[0034] (2) Feature redundancy evaluation By adding different types of external features, observe the change in the model's prediction performance to judge the effectiveness and redundancy of the features.

[0035] 1) Analysis method Define the baseline prediction performance as MSE0, and use the original target sequence {x1, x2,..., x t} for prediction.

[0036] The prediction performance after adding the feature fi is MSE i , and use the enhanced sequence {(x1,fi1), (x2,fi2),..., (x t ,fit )} Make predictions. fi includes the following four categories, among which, f1 is a time-derived feature: f 1(t) = sin(2π × (t mod T) / T)(8) f2 represents the noise characteristics: f 2(t) ~ N(0,1)(9) f3 represents the trend characteristic: f3(t) = t(10) f4 is a lag characteristic: f 4(t) = x_{tk}(11) In the above formula, t mod T is the modulo operation, which calculates the remainder when t is divided by T, mapping continuous time to the interval [0, T-1] to achieve periodic repetition; T is the time encoding period, for example, it can be set to T_day=96 or T_week=672; N(0,1) is the standard normal distribution; k is the lag step, for example, k=1 or 96, which can correspond to the previous step and the previous day, respectively.

[0037] f1 tests the degree of conflict with the model's built-in time understanding; f2 serves as a negative control, testing the model's resistance to irrelevant information and its tendency to overfit; f3 tests the redundancy of linear trend information; and f4 tests the value of autoregressive information.

[0038] 2) Indicator Setting The feature redundancy evaluation criteria are set as follows: Effectiveness (f) i ) = (MSE0 - MSE i ) / MSE0×100%(12) In the above formula, MSE0 is the mean squared error when only the original target sequence is used, and MSE i is the mean squared error after adding feature fi, and Effectiveness is the effectiveness of the feature, with positive values ​​indicating benefits and negative values ​​indicating harm or redundancy.

[0039] 3) Result Evaluation When Effectiveness(f i When -2% > 2%, features help increase predictive performance; when -2% ≤ Effectiveness(f) i If the effectiveness (f) is ≤ 2%, the feature has a limited or no significant impact on the model's performance improvement; when the effectiveness (f) is ≤ 2%, the feature has a limited or no significant impact on the model's performance improvement. i <-2%, feature redundancy, performance improvement of the damage model.

[0040] (3) Context sensitivity assessment The evaluation method involves systematically changing the model's context length (history window size), observing how predictive performance changes with context length, and determining the optimal memory window.

[0041] 1) Evaluation Methods Define a context length set C = {c1, c2, ..., c m}, for each length c i Using historical sequence {x_{tc i +1}, x_{tc i +2}, ..., x t Predict the future sequence {x_{t+1}, x_{t+2}, ..., x_{t+h}}.

[0042] In the above definition, C represents the set of test context lengths, which can typically take values ​​such as 96, 128, 192, 256, 512, 1024, etc.; c i x_{tc} is the length of the i-th context (historical window size); t is the current time point, serving as the prediction boundary; h is the prediction field of view length (the number of steps to be predicted in the future), such as h can take values ​​like 96 (1 day), 672 (7 days), 960 (10 days), etc.; i +1} represents the value of the starting time point of the history window, x t This is the value at the current time point t (the end point of the history window).

[0043] 2) Indicator setting and result evaluation By experimentally observing the variation pattern of MSE under different context lengths c, computational performance curves were constructed, and data points were recorded: {(c1,MSE1), (c2,MSE2), ..., (c m MSE m )}.

[0044] when MSE / c < 0, longer context is beneficial; when MSE(c*) = min{MSE(c)}, then c* is the optimal context length parameter; MSE / If c>0, then long contextual information is harmful; when MSE remains basically unchanged, the model is not sensitive to the length of the context.

[0045] This invention addresses the problem that existing large-scale time-series models are difficult to investigate due to their "black-box" construction and usage methods. It proposes to use "comparative experiments" to combine synthetic data with known answers and real data to gain in-depth insights into the model's response patterns under different input conditions, thereby providing support and guidance for model optimization and deployment.

[0046] The temporal pre-trained model testing system comprises three core dimensions: temporal pattern recognition, feature redundancy analysis, and context sensitivity assessment. Each dimension employs quantitative evaluation metrics, and the intrinsic characteristics of the model are inferred by comparing baseline performance with performance differences after conditional changes.

[0047] In another embodiment of the present invention, a rapid testing system for key features of a time-series pre-trained large model is provided, which can be used to implement the above-mentioned rapid testing method for key features of a time-series pre-trained large model. Specifically, the system includes: The dataset construction module is used to construct a synthetic time series test set containing different time patterns. The test set is input into the time series pre-trained model to be tested, the mean square error of the model's prediction for each pattern and the prediction correlation coefficient are calculated, and the output model is judged to have the ability to recognize basic and composite time patterns. The redundancy evaluation module is used to add different types of external features to the original target sequence in sequence based on a model with basic and composite temporal pattern recognition capabilities. It calculates the baseline prediction performance using only the original target sequence and the prediction performance after adding each feature. Based on the performance change, it calculates the feature effectiveness index and outputs the feature redundancy evaluation results. The sensitivity assessment module is used to determine the optimal memory window of the model and its sensitivity to long contexts based on the feature redundancy assessment results, by varying the historical context length of the input model, observing the performance curve of the model's prediction performance as a function of the context length, and then determining the model's sensitivity to long contexts based on the performance curve.

[0048] The module division in this embodiment of the invention is illustrative and represents only one logical functional division. In actual implementation, other division methods may be used. Furthermore, the functional modules in the various embodiments of the invention can be integrated into a single processor, exist as separate physical entities, or be integrated into a single module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0049] In another embodiment of the present invention, a computer device is provided, comprising a processor and a memory. The memory stores a computer program, which includes program instructions. The processor executes the program instructions stored in the computer storage medium. The processor may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It is the computing and control core of the terminal, suitable for implementing one or more instructions, specifically suitable for loading and executing one or more instructions from the computer storage medium to achieve a corresponding method flow or corresponding function. The processor described in this embodiment of the present invention can be used for the operation of a method for rapid testing of key characteristics of a time-series pre-trained large model.

[0050] In another embodiment of the present invention, a storage medium is provided, specifically a computer-readable storage medium (Memory), which is a memory device in a computer device used to store programs and data. It is understood that the computer-readable storage medium here can include both the built-in storage medium in the computer device and extended storage media supported by the computer device. The computer-readable storage medium provides storage space that stores the terminal's operating system. Furthermore, the storage space also stores one or more instructions suitable for loading and execution by a processor. These instructions can be one or more computer programs (including program code). It should be noted that the computer-readable storage medium here can be high-speed RAM or non-volatile memory, such as at least one disk storage device. The processor can load and execute one or more instructions stored in the computer-readable storage medium to implement the corresponding steps of the rapid testing method for key characteristics of time-series pre-trained large models in the above embodiments.

[0051] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0052] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0053] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0054] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0055] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A rapid testing method for key features of a time-series pre-trained large model, characterized in that: include: Construct a synthetic time series test set containing different time patterns, input the test set into the time series pre-trained model to be tested, calculate the prediction mean square error and prediction correlation coefficient of the model for each pattern, and determine whether the output model has the ability to recognize basic and composite time patterns. Based on a model with basic and composite temporal pattern recognition capabilities, different types of external features are sequentially added to the original target sequence. The baseline prediction performance using only the original target sequence and the prediction performance after adding each feature are calculated separately. The feature effectiveness index is calculated based on the performance changes. Output feature redundancy evaluation results; Based on the feature redundancy evaluation results, the historical context length of the input model is varied, and the performance curve of the model's prediction performance as a function of the context length is observed. Based on the performance curve, the optimal memory window of the model and its sensitivity to long contexts are determined.

2. The method for rapid testing of key characteristics of a time-series pre-trained large model according to claim 1, characterized in that, The construction comprises a synthetic time series test set with different time patterns, including: Construct a synthetic time series test set containing daily cycle patterns, weekly cycle patterns, trend patterns, stochastic patterns, and mixed patterns: Daily cycle pattern: p1(t) = μ_target + (σ_target × β1) × sin(2π × t / T_day) + ε1(t) (1) Weekly cycle pattern: p2(t)=μ_target + (σ_target × β2) × sin(2π × t / T_week) + ε2(t) (2) Trend Pattern: p3(t) = μ_target × α3 + (σ_target / T_context × β3) × t + ε3(t) (3) Random mode: p4(t) = μ_target + ε4(t)(4) Mixed Mode: p5(t) = p1(t) + p2(t) + p3(t) + ε5(t)(5) Where t is the time step index, μ_target is the empirical mean of the target sequence, σ_target is the empirical standard deviation of the target sequence, T_context is the context length; α3 is the trend pattern baseline adjustment coefficient; β1, β2, β3 are pattern feature adjustment coefficients; T_day is the daily cycle length, T_week is the weekly cycle length; ε i (t) represents the noise term, γ i This is the noise level adjustment coefficient.

3. The method for rapid testing of key characteristics of a time-series pre-trained large model according to claim 2, characterized in that, The step of inputting the test set into the time series pre-trained model to be tested and calculating the prediction mean square error and prediction correlation coefficient of the model for each mode includes: For each model, calculate the predictive performance metric. Mean square error: MSE(p i ) = (1 / n) × Σ j (y j - j ) 2 (6) Correlation coefficient: r(p i ) = Corr(y, )(7) Where y j For the true value, j Here, is the predicted value, n is the number of prediction points, and j is the time point index, j = 1, 2, ..., n; Corr(y, ) is the Pearson correlation coefficient, which measures the degree of linear correlation between predicted and actual values.

4. The method for rapid testing of key characteristics of a time-series pre-trained large model according to claim 3, characterized in that, The model whose output is determined to have basic and composite time pattern recognition capabilities includes: For the single calculated indicators p1, p2, p3, and p4, when MSE(p1) < MSE(p4) and r(p1) > 0.7, the model has the ability to understand daily cycles; when MSE(p2) < MSE(p4) and r(p2) > 0.6, the model has the ability to understand weekly cycles; when MSE(p3) < MSE(p4), the model has the ability to identify trends; when MSE(p4) is the smallest, the model is conservative and tends to predict the average value. For the mixed pattern p5, when MSE(p5) < MSE(p4) and r(p5) > max(r(p1), r(p2), r(p3)), the discriminant model has the ability to recognize and process the mixed pattern.

5. The method for rapid testing of key characteristics of a large time-series pre-trained model according to claim 1, characterized in that, The model, based on basic and composite temporal pattern recognition capabilities, sequentially adds different types of external features to the original target sequence. It then calculates the baseline prediction performance using only the original target sequence and the prediction performance after adding each feature. Based on the performance changes, it calculates a feature effectiveness index, including: Define the baseline prediction performance as MSE0, using the original target sequence {x1, x2, ..., x...} t Make predictions; The prediction performance after adding feature fi is MSE i Using the enhanced sequence {(x1, f i1), (x2, f i2), ..., (x t fi t To make predictions; fi includes the following four categories, among which, f1 is a time-derived feature: f 1(t) = sin(2π × (t mod T) / T)(8) f2 represents the noise characteristics: f 2(t) ~ N(0,1)(9) f3 represents the trend characteristic: f3(t) = t(10) f4 is a lag characteristic: f 4(t) = x_{tk}(11) t mod T is the modulo operation, which calculates the remainder when t is divided by T, mapping continuous time to the interval [0, T-1] to achieve periodic repetition; T is the time encoding period; N(0,1) is the standard normal distribution; k is the number of lag steps; f1 tests the degree of conflict with the model's built-in time understanding; f2 serves as a negative control, testing the model's resistance to irrelevant information and its tendency to overfit; f3 tests the redundancy of linear trend information; and f4 tests the value of autoregressive information. The feature redundancy evaluation criteria are set as follows: Effectiveness(f i ) = (MSE0 - MSE i ) / MSE0×100%(12) MSE0 is the mean squared error when only the original target sequence is used. i is the mean squared error after adding feature fi, and Effectiveness is the effectiveness of the feature, with positive values ​​indicating benefits and negative values ​​indicating harm or redundancy.

6. The method for rapid testing of key characteristics of a time-series pre-trained large model according to claim 5, characterized in that, The output feature redundancy evaluation results include: When Effectiveness(f i When -2% > 2%, features help increase predictive performance; when -2% ≤ Effectiveness(f) i If the effectiveness (f) is ≤ 2%, the feature has a limited or no impact on the model's performance improvement; when the effectiveness (f) is ≤ 2%, the feature has a limited or no impact on the model's performance improvement. i < -2%, feature redundancy, performance improvement of the damage model.

7. The method for rapid testing of key characteristics of a time-series pre-trained large model according to claim 1, characterized in that, The process of evaluating feature redundancy, varying the historical context length of the input model, observing the model's prediction performance as a function of context length, and determining the model's optimal memory window and its sensitivity to long contexts based on the performance curve includes: Define a context length set C = {c1, c2, ..., c m }, for each length c i Using historical sequence {x_{tc i +1}, x_{tc i +2}, ..., x t Predict the future sequence {x_{t+1}, x_{t+2}, ..., x_{t+h}}; In the above definition, C represents the set of test context lengths; c i x_i represents the length of the i-th context; t represents the current time point, serving as the prediction boundary; h represents the prediction field of view length; x_{tc} represents the length of the i-th context. i +1} represents the value of the starting time point of the history window, x t The value at the current time point t; By experimentally observing the variation pattern of MSE under different context lengths c, computational performance curves were constructed, and data points were recorded: {(c1,MSE1), (c2,MSE2), ..., (c m MSE m )}; when MSE / c < 0, longer context is beneficial; when MSE(c*) = min{MSE(c)}, then c* is the optimal context length parameter; when MSE / If c > 0, then long context information is harmful; when MSE remains unchanged, the model is not sensitive to context length.

8. A rapid testing system for key characteristics of time-series pre-trained large models, characterized in that: include: The dataset construction module is used to construct a synthetic time series test set containing different time patterns. The test set is input into the time series pre-trained model to be tested, the mean square error of the model's prediction for each pattern and the prediction correlation coefficient are calculated, and the output model is judged to have the ability to recognize basic and composite time patterns. The redundancy evaluation module is used to add different types of external features sequentially to the original target sequence based on a model with basic and composite temporal pattern recognition capabilities. It calculates the baseline prediction performance using only the original target sequence and the prediction performance after adding each feature, and calculates the feature effectiveness index based on the performance changes. Output feature redundancy evaluation results; The sensitivity assessment module is used to determine the optimal memory window of the model and its sensitivity to long contexts based on the feature redundancy assessment results, by varying the historical context length of the input model, observing the performance curve of the model's prediction performance as a function of the context length, and then determining the model's sensitivity to long contexts based on the performance curve.

9. The rapid testing system for key characteristics of a time-series pre-trained large model according to claim 8, characterized in that, The construction comprises a synthetic time series test set with different time patterns, including: Construct a synthetic time series test set containing daily cycle patterns, weekly cycle patterns, trend patterns, stochastic patterns, and mixed patterns: Daily cycle pattern: p1(t) = μ_target + (σ_target × β1) × sin(2π × t / T_day) + ε1(t) (1) Weekly cycle pattern: p2(t)=μ_target + (σ_target × β2) × sin(2π × t / T_week) + ε2(t) (2) Trend Pattern: p3(t) = μ_target × α3 + (σ_target / T_context × β3) × t + ε3(t) (3) Random mode: p4(t) = μ_target + ε4(t)(4) Mixed Mode: p5(t) = p1(t) + p2(t) + p3(t) + ε5(t)(5) Where t is the time step index, μ_target is the empirical mean of the target sequence, σ_target is the empirical standard deviation of the target sequence, T_context is the context length; α3 is the trend pattern baseline adjustment coefficient; β1, β2, β3 are pattern feature adjustment coefficients; T_day is the daily cycle length, T_week is the weekly cycle length; ε i (t) represents the noise term, γ i This is the noise level adjustment coefficient.

10. The rapid testing system for key characteristics of a time-series pre-trained large model according to claim 9, characterized in that, The step of inputting the test set into the time series pre-trained model to be tested and calculating the prediction mean square error and prediction correlation coefficient of the model for each mode includes: For each model, calculate the predictive performance metric. Mean square error: MSE(p i ) = (1 / n) × Σ j (y j - j ) 2 (6) Correlation coefficient: r(p i ) = Corr(y, )(7) Where y j For the true value, j Here, is the predicted value, n is the number of prediction points, and j is the time point index, j = 1, 2, ..., n; Corr(y, ) is the Pearson correlation coefficient, which measures the degree of linear correlation between predicted and actual values.

11. The rapid testing system for key characteristics of a time-series pre-trained large model according to claim 10, characterized in that, The model whose output is determined to have basic and composite time pattern recognition capabilities includes: For the single calculated indicators p1, p2, p3, and p4, when MSE(p1) < MSE(p4) and r(p1) > 0.7, the model has the ability to understand daily cycles; when MSE(p2) < MSE(p4) and r(p2) > 0.6, the model has the ability to understand weekly cycles; when MSE(p3) < MSE(p4), the model has the ability to identify trends; when MSE(p4) is the smallest, the model is conservative and tends to predict the average value. For the mixed pattern p5, when MSE(p5) < MSE(p4) and r(p5) > max(r(p1), r(p2), r(p3)), the discriminant model has the ability to recognize and process the mixed pattern.

12. The rapid testing system for key characteristics of a time-series pre-trained large model according to claim 8, characterized in that, The model, based on basic and composite temporal pattern recognition capabilities, sequentially adds different types of external features to the original target sequence. It then calculates the baseline prediction performance using only the original target sequence and the prediction performance after adding each feature. Based on the performance changes, it calculates a feature effectiveness index, including: Define the baseline prediction performance as MSE0, using the original target sequence {x1, x2, ..., x...} t Make predictions; The prediction performance after adding feature fi is MSE i Using the enhanced sequence {(x1, f i1), (x2, f i2), ..., (x t fi t To make predictions; fi includes the following four categories, among which, f1 is a time-derived feature: f 1(t) = sin(2π × (t mod T) / T)(8) f2 represents the noise characteristics: f 2(t) ~ N(0,1)(9) f3 represents the trend characteristic: f3(t) = t(10) f4 is a lag characteristic: f 4(t) = x_{tk}(11) t mod T is the modulo operation, which calculates the remainder when t is divided by T, mapping continuous time to the interval [0, T-1] to achieve periodic repetition; T is the time encoding period; N(0,1) is the standard normal distribution; k is the number of lag steps; f1 tests the degree of conflict with the model's built-in time understanding; f2 serves as a negative control, testing the model's resistance to irrelevant information and its tendency to overfit; f3 tests the redundancy of linear trend information; and f4 tests the value of autoregressive information. The feature redundancy evaluation criteria are set as follows: Effectiveness(f i ) = (MSE0 - MSE i ) / MSE0×100%(12) MSE0 is the mean squared error when only the original target sequence is used. i is the mean squared error after adding feature fi, and Effectiveness is the effectiveness of the feature, with positive values ​​indicating benefits and negative values ​​indicating harm or redundancy.

13. The rapid testing system for key characteristics of a time-series pre-trained large model according to claim 12, characterized in that, The output feature redundancy evaluation results include: When Effectiveness(f i When -2% > 2%, features help increase predictive performance; when -2% ≤ Effectiveness(f) i If the effectiveness (f) is ≤ 2%, the feature has a limited or no impact on the model's performance improvement; when the effectiveness (f) is ≤ 2%, the feature has a limited or no impact on the model's performance improvement. i < -2%, feature redundancy, performance improvement of the damage model.

14. The rapid testing system for key characteristics of a time-series pre-trained large model according to claim 8, characterized in that, The process of evaluating feature redundancy, varying the historical context length of the input model, observing the model's prediction performance as a function of context length, and determining the model's optimal memory window and its sensitivity to long contexts based on the performance curve includes: Define a context length set C = {c1, c2, ..., c m }, for each length c i Using historical sequence {x_{tc i +1}, x_{tc i +2}, ..., x t Predict the future sequence {x_{t+1}, x_{t+2}, ..., x_{t+h}}; In the above definition, C represents the set of test context lengths; c i x_i represents the length of the i-th context; t represents the current time point, serving as the prediction boundary; h represents the prediction field of view length; x_{tc} represents the length of the i-th context. i +1} represents the value of the starting time point of the history window, x t The value at the current time point t; By experimentally observing the variation pattern of MSE under different context lengths c, computational performance curves were constructed, and data points were recorded: {(c1,MSE1), (c2,MSE2), ..., (c m MSE m )}; when MSE / c < 0, longer context is beneficial; when MSE(c*) = min{MSE(c)}, then c* is the optimal context length parameter; when MSE / If c > 0, then long context information is harmful; when MSE remains unchanged, the model is not sensitive to context length.

15. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method for rapid testing of key features of a time-series pre-trained large model as described in any one of claims 1 to 7.

16. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the method for rapid testing of key features of a time-series pre-trained large model as described in any one of claims 1 to 7.

Images