Self-adaptive time sequence analysis method based on dynamic prompt mechanism and related device

Abstract

The invention belongs to the technical field of time sequence data analysis, and discloses a self-adaptive time sequence analysis method based on a dynamic prompt mechanism and a related device. The adaptive time sequence analysis method comprises the following steps: acquiring time sequence data to be analyzed, and constructing a time sequence sub-sequence by using a sliding window method; extracting the feature representation of each time sequence sub-sequence, and generating the dynamic prompt of each time sequence sub-sequence based on the dynamic prompt pool; combining the feature representation of the selected time sequence sub-sequence with the dynamic prompt based on the analysis task type of the time sequence data to be analyzed, inputting a combination result into a large language model, and obtaining an analysis result of the time sequence data through the large language model; the large language model is provided with an adaptive window length optimization mechanism. The method not only can flexibly adapt to different time scales and complex modes, but also can efficiently utilize historical knowledge, and meanwhile, the calculation efficiency and the model interpretability are kept.

Description

Technical Field

[0001] The present invention belongs to the technical field of time series data analysis, and in particular relates to an adaptive time series analysis method based on a dynamic prompt mechanism and a related device. Background Art

[0002] In the field of modern data science and artificial intelligence, time series analysis plays a vital role and is widely used in many fields such as weather forecasting, intelligent question answering, industrial production monitoring, etc. With the advent of the big data era and the popularization of Internet of Things technology, the scale, complexity and diversity of time series data are increasing dramatically, posing a huge challenge to the existing traditional time series analysis methods.

[0003] At present, the existing traditional time series analysis methods, such as the Autoregressive Integrated Moving Average (ARIMA) model, perform well in processing simple and stable time series data; however, the above methods are often unable to cope with non-stationary, nonlinear, and high-dimensional complex time series data. In recent years, deep learning methods, especially recurrent neural networks (RNN) and long short-term memory networks (LSTM), have made significant progress in time series analysis. Deep learning methods can capture complex temporal dependencies, but they still face challenges in processing long series and multivariate time series, especially in terms of computational efficiency and model interpretability. Recently, a method based on the Transformer architecture model has been proposed to achieve efficient processing of long-series time series data through the self-attention mechanism. This type of method performs well in capturing long-term dependencies, but still relies on a fixed-length input window and is difficult to flexibly adapt to time patterns of different scales. In addition, graph neural networks (GNNs) have shown potential in processing data with complex spatial-temporal dependencies. The use of graph structures can effectively capture spatial-temporal dependencies in time series data, but such methods usually require predefined graph structures and have limitations when dealing with dynamically changing time series relationships. In addition, hybrid models such as N-BEATS (Neural Basis Expansion Analysis for Interpretable Time Series Forecasting) attempt to combine deep learning and classical time series decomposition techniques to improve model interpretability while maintaining forecast accuracy, but such methods still face challenges when dealing with highly nonlinear and non-stationary time series. In summary, the existing traditional time series analysis methods still have some defects. In order to flexibly adapt to different time scales and complex patterns, efficiently utilize historical knowledge, and maintain good computational efficiency and model interpretability, it is urgent to design new time series analysis methods.

[0004] Recently, Large Language Models (LLMs) have made breakthrough progress in various natural language processing tasks, prompting researchers to explore their application in time series analysis. Studies have shown that LLMs have zero-sample time series prediction capabilities, but directly applying large language models to time series analysis faces challenges such as large computing resource requirements and difficulty in adapting to specific domain knowledge. Summary of the invention

[0005] The purpose of the present invention is to provide an adaptive time series analysis method and related devices based on a dynamic prompt mechanism to solve one or more of the above-mentioned technical problems. The adaptive time series analysis scheme based on a dynamic prompt mechanism disclosed in the present invention is a new scheme based on a large language model. The large language model used has good adaptability, robustness, generalization ability and prediction accuracy. It can flexibly adapt to different time scales and complex patterns, and can efficiently use historical knowledge, while maintaining computational efficiency and model interpretability.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions:

[0007] In a first aspect, the present invention provides an adaptive time series analysis method based on a dynamic prompt mechanism, comprising the following steps:

[0008] Obtain the time series data to be analyzed and construct the time series subsequence using the sliding window method;

[0009] Extracting feature representations of each time series subsequence, and generating dynamic prompts for each time series subsequence based on a dynamic prompt pool;

[0010] Based on the analysis task type of the time series data to be analyzed, the feature representation of the selected time series subsequence is combined with the dynamic prompt, and the combined result is input into the large language model, and the analysis result of the time series data is obtained through the large language model;

[0011] Wherein, the large language model is provided with an adaptive window length optimization mechanism.

[0012] A further improvement of the adaptive time series analysis method of the present invention is that in the step of generating dynamic prompts for each time series subsequence based on the dynamic prompt pool,

[0013] The dynamic prompt pool is an initialized dynamic prompt pool or a dynamically updated dynamic prompt pool; wherein,

[0014] The step of obtaining the initialized dynamic prompt pool includes: encoding the training data using a pre-trained Transformer model to generate an initial key-value pair set, and then optimizing the prompt pool structure by clustering to obtain the initialized dynamic prompt pool; wherein each prompt is represented as a key-value pair (k, v); k∈R d is the key vector, R d is a d-dimensional real vector; v∈R l ×d is the value matrix, d is the embedding dimension, l is the hint length, R l×d is a real matrix with l rows and d columns;

[0015] After dynamically updating the initialized dynamic prompt pool based on the analysis result of the time series data, a dynamically updated dynamic prompt pool is obtained.

[0016] A further improvement of the adaptive time series analysis method of the present invention is that in the step of generating dynamic prompts for each time series subsequence based on the dynamic prompt pool,

[0017] For the time series subsequence S t , calculate the time series subsequence S t The characteristic representation f t With each key vector k in the dynamic hint pool m The cosine similarity sim(f t ,k m ), the calculation expression is:

[0018] sim(f t ,k m )=(f t ·k m ) / (‖f t ‖‖k m ‖);

[0019] In the formula, f t ·k m represents the dot product of the vectors, ‖f t ‖‖k m ‖ represents the multiplication of vector norms;

[0020] Based on the cosine similarity sorting, select the K prompts with the highest similarity to form a prompt value matrix;

[0021] Applying the attention mechanism to the prompt value matrix generates the final dynamic prompt, which is expressed as:

[0022] p t =∑ i α i v i ;

[0023]

[0024] In the formula, p t represents the final dynamic prompt generated; α i represents the attention weight of the i-th prompt; represents f t The transpose of v i represents the value vector of the i-th prompt; W q represents the learnable query matrix parameters; k i A key vector representing the i-th prompt.

[0025] A further improvement of the adaptive time series analysis method of the present invention is that, in the step of combining the feature representation of the selected time series subsequence with the dynamic prompt based on the analysis task type of the time series data to be analyzed,

[0026] For the prediction task, a feed-forward neural network is used to combine the feature representation of the selected time series subsequence with dynamic cues;

[0027] For missing data interpolation, a gated recurrent unit is used to combine the feature representation of the selected time series subsequence with dynamic cues.

[0028] A further improvement of the adaptive time series analysis method of the present invention is that in the adaptive window length optimization mechanism, a window length candidate set is preset, a time series subsequence is constructed for each candidate window length and features are extracted, and then a comprehensive score of each window length is calculated, and based on the comprehensive scores of each candidate window length, the optimal window length is dynamically selected; wherein,

[0029] The calculation expression of the comprehensive score is:

[0030]

[0031] In the formula, τ i represents the length of the i-th candidate window; k j represents the j-th key vector; Indicates the use of window length τ i The mean square error of prediction when ; Represents the window length τ i The extracted features and the hint key vector k j similarity; Represents the window length τ i The information entropy of the extracted features; α, β, and γ represent the weight coefficients of each indicator respectively; N represents the number of prompt key vectors evaluated;

[0032] Dynamically selected optimal window length τ opt It is expressed as:

[0033]

[0034] A further improvement of the adaptive time series analysis method of the present invention is that a smoothing mechanism is also introduced into the adaptive window length optimization mechanism; wherein,

[0035] τ t =(1-λ)τ t-1 +λτ opt ;

[0036] Where λ is the smoothing factor.

[0037] A further improvement of the adaptive time series analysis method of the present invention is that the large language model is also provided with a feedback optimization mechanism based on reinforcement learning, and a policy gradient method is used to optimize the decision strategy.

[0038] A further improvement of the adaptive time series analysis method of the present invention is that the large language model also uses a SelfExtend method to process the input sequence.

[0039] A further improvement of the adaptive time series analysis method of the present invention is that the large language model is also provided with an interpretability module; wherein,

[0040] The interpretability module is used to calculate the contribution of each hint to the final prediction using the SHAP value, computing the expression:

[0041]

[0042] In the formula, φ i represents the SHAP value of the i-th feature; F represents the set of all features; S represents the feature subset that does not contain feature i; |S| represents the number of features in subset S; |F| represents the total number of features; f S (x S ) represents the model output using only the feature subset S for prediction; f S∪{i} (x S∪{i} ) represents the model output after adding the i-th feature; is the combination weight, which is used to balance the impact of different feature combinations;

[0043] The interpretability module is also used to visualize the attention weights α during the dynamic cue generation process i , and use the integrated gradient method to calculate the importance score of the input feature. The calculation expression is:

[0044]

[0045] In the formula, IG i (x) represents the integrated gradient value of the i-th feature x; x represents the actual feature value vector of the current input, x i represents the actual input value of the ith feature; x′ represents the baseline input, which is a reference point value, i represents the baseline input value of the i-th feature; f represents the prediction model function; α represents the integral path parameter; Represents the partial derivative of the model with respect to the i-th feature.

[0046] In a second aspect, the present invention provides an adaptive time series analysis system based on a dynamic prompt mechanism, comprising:

[0047] The time series subsequence construction module is used to obtain the time series data to be analyzed and construct the time series subsequence using the sliding window method;

[0048] A dynamic prompt generation module is used to extract the feature representation of each time series subsequence and generate a dynamic prompt for each time series subsequence based on the dynamic prompt pool;

[0049] An analysis module, used for combining the feature representation of the selected time series subsequence with the dynamic prompt based on the analysis task type of the time series data to be analyzed, and inputting the combined result into the large language model, and obtaining the analysis result of the time series data through the large language model;

[0050] Wherein, the large language model is provided with an adaptive window length optimization mechanism.

[0051] Compared with the prior art, the present invention has the following beneficial effects:

[0052] The present invention discloses an adaptive time series analysis method based on a dynamic prompt mechanism, which focuses on solving the limitations of fixed window length in traditional time series analysis methods. In complex time series data, different data may require historical information of different lengths to accurately model; fixed window length is difficult to adapt to such diversity, which may lead to information loss or the introduction of irrelevant information; the technical solution disclosed by the present invention is provided with an adaptive window length optimization mechanism, which enables the model to automatically adjust the length of historical information of interest, adapt to patterns of different time scales, and effectively solve the limitations of fixed window length. In addition, the present invention provides a method for flexibly capturing and utilizing relevant historical information through a dynamic prompt mechanism, which can improve the adaptability and prediction accuracy of large language models.

[0053] The present invention is dedicated to more effectively capturing and processing complex patterns and trends in time series data. Time series data in practical applications often contain complex seasonality, cyclical changes and outliers. These patterns may vary on different time scales, and traditional methods are difficult to fully capture these complex patterns. The dynamic prompt mechanism of the present invention enhances the recognition and understanding capabilities of the large language model for these complex patterns by retrieving historical prompts similar to the current data pattern. The present invention aims to improve the generalization capabilities of large language models for time series analysis, especially in the face of data distribution shifts. In practical applications, there are often differences in the statistical characteristics of training data and test data, which poses a challenge to the generalization capabilities of large language models. The dynamic prompt mechanism of the present invention enables the model to better adapt to new and unseen data distributions by retrieving relevant historical experience, thereby improving the robustness and generalization capabilities of large language models. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below; obviously, the drawings described below are some embodiments of the present invention, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0055] Figure 1 is a schematic diagram of an adaptive time series analysis method based on a dynamic prompt mechanism in an embodiment of the present invention;

[0056] Figure 2 It is a schematic diagram of an adaptive time series analysis system based on a dynamic prompt mechanism in an embodiment of the present invention. DETAILED DESCRIPTION

[0057] In order to make the purpose, technical solutions and advantages of the present invention clearer, the technical solutions in the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention; it is obvious that the described embodiments and technical solutions are only part of the embodiments of the present invention, not all of the embodiments.

[0058] All other embodiments obtained by those of ordinary skill in the art without creative work based on the technical solutions disclosed in the embodiments of the present invention belong to the scope of protection of the present invention. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device including a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0059] See also Figure 1 In an embodiment of the present invention, an adaptive time series analysis method based on a dynamic prompt mechanism is disclosed, which specifically includes the following steps:

[0060] Step 1: Obtain the time series data to be analyzed and construct the time series subsequence using the sliding window method;

[0061] In an optional technical solution, after obtaining the time series data to be analyzed, the data is first standardized, and then the time series subsequence is constructed using a sliding window method;

[0062] In a specific exemplary technical solution, the PEMS-BAY dataset is taken as an example. The dataset contains traffic flow data of 325 sensors at 52116 time points. The data of one of the sensors (for example, the sensor numbered 100) is selected; for the selected sensor, the traffic flow data recorded every 5 minutes for 30 consecutive days is obtained, and the data is z-score standardized, and then a sliding window of 1 hour (12 time steps) is used to construct a time series subsequence;

[0063] Specifically, the following phenomena can be observed:

[0064] 1) The traffic flow on weekdays shows a clear bimodal pattern, with a significant increase in traffic during the morning peak (7:00-9:00) and evening peak (17:00-19:00) periods;

[0065] 2) Traffic flow is relatively slow on weekends, but there is a small peak in the afternoon (14:00-16:00);

[0066] 3) The overall traffic flow on rainy days is about 15% lower than on sunny days, and the peak during rush hour is delayed by about 30 minutes.

[0067] Step 2, extracting the feature representation of each time series subsequence, and generating dynamic prompts for each time series subsequence based on the initialized dynamic prompt pool;

[0068] In an optional technical solution, the step of obtaining the initialized dynamic prompt pool includes: encoding the training data using a pre-trained Transformer model to generate an initial set of key-value pairs, and then optimizing the prompt pool structure by clustering to obtain the initialized dynamic prompt pool;

[0069] In a specific exemplary technical solution, based on the observation results of step 1, the traffic mode is encoded into the following typical modes:

[0070] 1) “Double peak mode on working days”;

[0071] 2) “weekend single peak mode”;

[0072] 3) “Rainy day delayed peak mode”;

[0073] 4) “Late-night low-traffic mode”;

[0074] The above four modes are encoded into key-value pairs and organized into a structured prompt pool through a clustering method.

[0075] Based on the above four typical patterns, in a further exemplary technical solution, assume that a 1-hour subsequence at 8:30 am on Tuesday is analyzed: first, the feature representation of this 1-hour subsequence is extracted, such as the average flow rate (for example, 1,200 vehicles / hour), the flow rate growth rate (for example, a 40% increase compared to the previous hour), etc.; by comparing these extracted features with the patterns in the prompt pool, the "weekday double peak pattern" may be retrieved as the most relevant dynamic prompt.

[0076] Step 3, based on the analysis task type of the time series data to be analyzed, the feature representation of the selected time series subsequence is combined with the dynamic prompt and input into the large language model, and the analysis result is obtained through the large language model;

[0077] In an optional technical solution, a corresponding neural network structure is first determined based on the analysis task type of the time series data to be analyzed, and then the determined neural network is used to combine the feature representation of the selected time series subsequence with the dynamic prompt;

[0078] In a specific exemplary technical solution, the analysis task type is to predict the traffic flow one hour into the future, and the determined neural network structure is a feedforward neural network. The traffic data features of the current one-hour subsequence and the dynamic prompt "weekday double peak mode" retrieved from the prompt pool are combined as the input of the large language model, and the traffic flow one hour into the future is predicted through the large language model; for example, for the traffic flow prediction task, a feedforward neural network is used to combine the feature representation of the current one-hour traffic data with the "weekday double peak mode" retrieved from the prompt pool and use it as the input for subsequent predictions, and the preliminary prediction of the traffic flow in the next hour is 1,350 vehicles / hour.

[0079] In the embodiment of the present invention, in the step of obtaining the analysis result through the large language model, the large language model is provided with a multi-dimensional adaptive optimization mechanism;

[0080] In the exemplary technical solution, the traffic flow prediction during the Tuesday morning peak period (8:30-9:30) is taken as an example; based on the prompt of the "weekday double peak mode" and the current observation data, the preliminary prediction of the traffic flow in the next hour is 1350 vehicles / hour; in order to improve the prediction accuracy, the present invention optimizes this prediction result through a multi-dimensional adaptive optimization mechanism;

[0081] First, in the adaptive analysis of the time dimension, the system observed that the traffic flow in the period of 8:15-8:30 was significantly faster than that in the period of 8:00-8:15 (increasing from 1,100 vehicles / hour to 1,200 vehicles / hour). Taking into account this rapid change trend, the adaptive window mechanism dynamically adjusted the analysis window from 1 hour to 30 minutes to more sensitively capture the characteristics of traffic changes, and recalculated based on the adjusted time window, and the predicted value was adjusted to 1,380 vehicles / hour.

[0082] Secondly, the patterns in the prompt pool are dynamically updated, including: by analyzing the data of the same period in the past three weeks, it was found that the average traffic flow in this period was about 1,400 vehicles / hour, and the peak characteristics of the "weekday double peak mode" were updated accordingly. This optimization further adjusted the predicted value to 1,390 vehicles / hour, which better reflects the historical statistical laws.

[0083] Next, the prediction strategy was optimized through reinforcement learning, including: taking into account the rapid growth trend currently observed, the system adjusted the feature weight distribution, increasing the weight of recent trends to 0.6 and adjusting the weight of historical patterns to 0.4. This optimization finally determined the predicted value to be 1,395 vehicles per hour.

[0084] Finally, the interpretability module of the present invention performs a detailed attribution analysis on the final predicted value of 1395 vehicles / hour. The results show that the recent rapid growth trend of traffic contributed 45% of the impact, the historical weekday pattern contributed 40% of the impact, the sunny weather of the day contributed 10% of the impact, and other factors contributed 5% of the impact. This clear attribution analysis not only improves the credibility of the prediction results, but also provides a basis for subsequent optimization.

[0085] Through this series of adaptive optimization, the confidence interval of the prediction results is reduced from ±100 vehicles / hour to ±50 vehicles / hour, and the confidence level is increased from 85% to 93%. This significant performance improvement fully demonstrates the adaptive analysis capability of the present invention based on the dynamic prompt mechanism, which can dynamically adjust the analysis strategy according to real-time data, continuously optimize the prediction performance, and maintain a high degree of interpretability. This adaptive optimization mechanism makes the present invention particularly suitable for handling complex and changeable time series analysis tasks.

[0086] In a specific embodiment of the present invention, an adaptive time series analysis method based on a dynamic prompt mechanism is provided, and the specific process is as follows:

[0087] Step S1, obtain the time series data to be analyzed, and construct the time series subsequence using the sliding window method; wherein,

[0088] The original time series data to be analyzed is represented by X∈R N×T, X is a data matrix, representing a dataset consisting of N univariate time series, and T represents the number of timestamps;

[0089] Each time series in the data matrix X is z-score standardized to ensure that data of different scales are comparable. The standardized calculation expression is:

[0090]

[0091] Where, X norm is the standardized time series; μ is the mean vector; σ is the standard deviation vector;

[0092] The expression for constructing a time series subsequence using the sliding window method is:

[0093]

[0094] In the formula, S t is a time series subsequence; τ is a predefined window size; is the window time series data with a time length of τ;

[0095] Explanatorily, the steps of temporal subsequence construction here are used to provide a basis for subsequent dynamic cue generation.

[0096] Step S2, based on the initialized dynamic prompt pool, generating dynamic prompts for each time series subsequence; wherein,

[0097] The initialization of the dynamic prompt pool is the core component of the technical solution of the present invention to deal with complex patterns and trend capture; wherein each prompt is represented as a key-value pair (k, v), k∈R d is the key vector, R d is a d-dimensional real vector; v∈R l×d is the value matrix, d is the embedding dimension, l is the hint length, R l×d is a real matrix with l rows and d columns;

[0098] In the embodiment of the present invention, the training data is encoded using a pre-trained Transformer model to obtain an initial key-value pair set, which is expressed as:

[0099] P={(k1,v1),(k2,v2),…,(k M ,v M )};

[0100] Where P is the initial key-value pair set; (k M ,v M ) is the Mth key-value pair, k M is the key vector in the Mth key-value pair, v Mis the value matrix in the Mth key-value pair;

[0101] Furthermore, in order to improve retrieval efficiency and reduce redundancy, K-means clustering is performed on the initialized key vector, which is expressed as:

[0102]

[0103] In the formula, represents the clustering solution C that minimizes the objective function; k represents the key vector in the prompt pool; μ i represents the center vector of the i-th cluster; C i represents the i-th cluster, which contains all key vectors assigned to the cluster; M represents the preset number of clusters;

[0104] Explanatory, the steps here not only optimize the structure of the prompt pool, but also lay the foundation for subsequent rapid retrieval; this prompt pool not only stores historical patterns and trend information, but also provides the possibility for knowledge reuse and transfer.

[0105] In the dynamic prompt generation and retrieval stage, for each input time series subsequence S t , first use a multi-scale convolutional neural network to extract its feature representation:

[0106]

[0107] In the formula, f t Represents a time series subsequence S t Feature representation; CNN i Represents convolutional layers of different scales;

[0108] This multi-scale feature extraction helps capture patterns and trends at different time scales, enhancing the ability to understand complex temporal dynamics;

[0109] Then, calculate f t With each key vector k in the prompt pool m The cosine similarity sim(f t ,k m ), the calculation expression is:

[0110] sim(f t ,k m )=(f t ·k m ) / (‖f t ‖‖k m ‖);

[0111] In the formula, f t ·k m represents the dot product of the vectors, ‖f t ‖‖km ‖Multiplication of vector norms;

[0112] Based on the cosine similarity ranking, select the K tips with the highest similarity;

[0113] Applying the attention mechanism to the retrieved prompt value matrix generates the final dynamic prompt, which is expressed as:

[0114] p t =∑ i α i v i ;

[0115]

[0116] In the formula, p t represents the final dynamic prompt generated; α i represents the attention weight of the i-th prompt; represents f t The transpose of v i represents the value vector of the i-th prompt; W q represents the learnable query matrix parameters; k i represents the key vector of the i-th prompt;

[0117] Explanation: Select the K tips with the highest similarity, and perform a weighted combination of the retrieved tip value matrix through the attention mechanism to generate the final dynamic tip. The specific calculation process is as follows: First, calculate the attention weight α of each tip i , it passes through f t With the hint key vector k i The similarity is obtained by introducing the learnable query matrix W q To enhance the flexibility of similarity calculation; then all retrieved prompt value vectors are weighted and summed according to their corresponding attention weights to obtain the final dynamic prompt. The weighted combination method based on the attention mechanism of the present invention enables the present invention to adaptively select and fuse the most relevant historical knowledge according to the characteristics of the input data, thereby improving the recognition ability and generalization performance of complex time series patterns;

[0118] Step S3, based on the analysis task type of the time series data to be analyzed, the feature representation of the selected time series subsequence is combined with the dynamic prompt and input into the large language model, and the analysis result is obtained through the large language model; wherein,

[0119] In order to adapt to different time series analysis tasks, this paper designs a task-specific dynamic prompt application mechanism. According to the task type (such as prediction, missing data interpolation, anomaly detection or classification), different neural network structures are used to transform the dynamic prompt p t With the feature representation ft Combine;

[0120] For example, for the prediction task, a feedforward neural network is used, which is expressed as:

[0121] h t =FFN([f t ;p t ]); where [f t ;p t ] indicates f t With dynamic prompt p t splicing operation;

[0122] For example, for missing data interpolation, a gated recurrent unit is used, which is expressed as:

[0123] h t =GRU(f t ,p t );

[0124] In summary, this flexible task adaptation mechanism can be summarized as:

[0125] h t =TaskNetwork(f t ,p t ); TaskNetwork varies according to the specific task type;

[0126] In the formula, h t Represents task-specific output feature representation; FFN represents feedforward neural network; GRU represents gated recurrent unit; TaskNetwork represents a task-specific neural network structure.

[0127] The task-specific dynamic prompt application mechanism of the present invention has the following advantages:

[0128] 1) Targetedness: By designing specialized network structures for different types of tasks, the feature fusion process is more adaptable to task requirements;

[0129] 2) Flexibility: The most suitable network structure can be selected or designed according to the characteristics of the specific task;

[0130] 3) Versatility: It maintains a unified framework form, making it easy to expand to new task types;

[0131] Through this mechanism, the present invention can achieve efficient feature fusion and processing for different types of time series analysis tasks while maintaining the unity of the framework, thereby obtaining good task-specific performance.

[0132] In a specific embodiment of the present invention, in order to further improve the adaptability and generalization ability of the model, an adaptive window length optimization mechanism is adopted; wherein a window length candidate set W = {τ1, τ2, ..., τ n}, construct a time series subsequence for each candidate window length and extract features, then calculate a comprehensive score for each window length, the calculation expression is:

[0133]

[0134] In the formula, τ i represents the length of the i-th candidate window; k j represents the j-th key vector; Indicates the use of window length τ i The mean square error of prediction when ; Represents the window length τ i The extracted features and the hint key vector k j similarity; Represents the window length τ i The information entropy of the extracted features; α, β, γ represent the weight coefficients of each indicator respectively; N represents the number of prompt key vectors evaluated.

[0135] Based on the scores calculated above, the optimal window length is dynamically selected, which is expressed as:

[0136]

[0137] Furthermore, in order to avoid frequent changes in window length, a smoothing mechanism is introduced, which is expressed as:

[0138] τ t =(1-λ)τ t-1 +λτ opt ;

[0139] Where λ is the smoothing factor;

[0140] This mechanism enables the model to automatically adjust the length of historical information of interest and adapt to patterns at different time scales, effectively solving the limitations of fixed window length.

[0141] Regarding a specific embodiment of the adaptive window length mechanism, the window length candidate set W is set to {30 minutes, 1 hour, 2 hours};

[0142] During the morning peak period (e.g. 7:00-9:00 a.m.), the length score of each window is calculated;

[0143] 30-minute window: low MSE, high similarity, and high information entropy;

[0144] 1 hour window: moderate MSE, moderate similarity, moderate information entropy;

[0145] 2-hour window: high MSE, low similarity, and low information entropy;

[0146] In summary, a 30-minute window is selected as the optimal window length, and it is gradually adjusted through a smoothing mechanism (λ=0.3) to avoid sudden changes.

[0147] In one embodiment of the present invention, a feedback optimization mechanism based on reinforcement learning is also designed to further enhance the adaptive learning ability of the model; wherein,

[0148] Define the state space S (including current data features, selected prompts and task types), action space A (including operations such as prompt selection and window length adjustment), and reward function R (based on the performance of the model on the validation set);

[0149] The decision strategy is optimized using the policy gradient method, expressed as:

[0150]

[0151] In the formula, π θ is the parameterized policy function, R t is the discounted cumulative reward starting from time t;

[0152] The technical means of the embodiments of the present invention can continuously adjust its behavior according to real-time feedback and improve its performance in a dynamically changing environment.

[0153] In the embodiment of the reinforcement learning optimization mechanism, the state is: the current traffic volume is 1,200 vehicles per hour, the time is the morning rush hour on a weekday, and the weather is clear; the action is: select a 30-minute window and adopt the "weekday rush hour" prediction strategy; the reward is: give positive feedback based on the prediction accuracy.

[0154] In one embodiment of the present invention, in order to improve computational efficiency, especially when processing long sequences, the SelfExtend technology is used. This technology enables the model to effectively process long sequence inputs through group attention and neighborhood attention mechanisms; wherein,

[0155] The input sequence is divided into G groups, expressed as:

[0156] X=[X1,…,X G ];

[0157] Apply self-attention mechanism within each group:

[0158]

[0159] Apply neighborhood attention between adjacent groups:

[0160] O g =Attention(A g ,[A g-1 ,A g ,A g+1 ],[A g-1 ,A g ,A g+1 ]);

[0161] Among them, W Q , W K , W V is a learnable weight matrix;

[0162] The technical means of the embodiment of the present invention not only improves the efficiency of processing long sequences, but also maintains the ability to capture long-term dependencies.

[0163] In an embodiment of an efficient sequence processing mechanism, 24-hour data is divided into eight 3-hour groups for processing; self-attention is calculated within the group to capture local traffic patterns; and attention is calculated between adjacent groups to maintain temporal continuity.

[0164] In a specific embodiment of the present invention, in order to enhance the interpretability of the model, an interpretability module is used, which uses SHAP (SHapley Additive exPlanations) value to calculate the contribution of each hint to the final prediction, and the calculation expression is:

[0165]

[0166] In the formula, φ i represents the SHAP value of the i-th feature, indicating the contribution of the feature to the prediction result; F represents the set of all features; S represents the feature subset that does not contain feature i; |S| represents the number of features in subset S; |F| represents the total number of features; f S (x S ) represents the model output using only the feature subset S for prediction; f S∪{i} (x S∪{i} ) represents the model output after adding feature i; is the combination weight, which is used to balance the impact of different feature combinations;

[0167] The present invention also visualizes the attention weight α in the dynamic prompt generation process i , and use the integrated gradient method to calculate the importance score of the input feature. The calculation expression is:

[0168]

[0169] In the formula, IG i(x) represents the integrated gradient value of the i-th feature; x i represents the actual input value of the i-th feature; x′ i represents the baseline input value of the i-th feature (usually the mean value or zero value of the feature is selected); f represents the prediction model function; α represents the integral path parameter, ranging from 0 to 1; Represents the partial derivative of the model with respect to the i-th feature.

[0170] The technical means of the embodiments of the present invention not only improve the transparency of model decisions, but also provide insights for further optimization.

[0171] In the embodiment of the application of the explainability mechanism,

[0172] Feature importance analysis includes:

[0173] Historical traffic flow: SHAP value = 0.45 (positive contribution);

[0174] Time feature: SHAP value = 0.25 (positive contribution);

[0175] Weather conditions: SHAP value = -0.15 (negative contribution);

[0176] The forecast interpretation report includes:

[0177] Prediction result: Traffic will increase by 20%;

[0178] The main influencing factors include:

[0179] Historical similarity pattern matching (contribution 45%);

[0180] Characteristics of the current time period (contribution 25%);

[0181] Weather influence (contribution 15%);

[0182] Other factors (contribution 15%);

[0183] Through this explainability mechanism, the present invention can provide a quantitative explanation of the prediction results, identify key influencing factors, assist decision makers in understanding the basis of the prediction, and provide guidance for model optimization.

[0184] In summary, firstly, the present invention focuses on solving the limitation of fixed window length in traditional time series analysis methods. In complex time series data, different data may require historical information of different lengths to accurately model; fixed window length is difficult to adapt to this diversity, which may lead to information loss or the introduction of irrelevant information. The present invention provides a method for flexibly capturing and utilizing relevant historical information through a dynamic prompt mechanism, thereby improving the adaptability and prediction accuracy of large language models. Secondly, the present invention is committed to more effectively capturing and processing complex patterns and trends in time series data. Time series data in practical applications often contain complex seasonality, cyclical changes and outliers. These patterns may vary on different time scales, and traditional methods are difficult to fully capture these complex patterns; the dynamic prompt mechanism of the present invention enhances the recognition and understanding capabilities of large language models of these complex patterns by retrieving historical prompts similar to the current data pattern. The present invention aims to improve the generalization ability of a large language model for time series analysis, especially in the face of data distribution deviation. In practical applications, there are often differences in the statistical characteristics of training data and test data, which poses a challenge to the generalization ability of large language models. The dynamic prompt mechanism of the present invention enables the model to better adapt to new and unseen data distributions by retrieving relevant historical experience, thereby improving the robustness and generalization ability of the large language model. The present invention solves the problem of knowledge reuse and migration in time series analysis. In different time periods or different data sets, certain patterns may recur. Effective use of these repeated patterns can significantly improve analysis efficiency and accuracy. The present invention realizes the encoding, storage and retrieval of knowledge by establishing and maintaining a prompt pool, so that the model can effectively reuse and migrate learned knowledge, thereby improving the prediction and analysis ability of new data. Finally, the present invention is committed to realizing adaptive learning of time series analysis. Time series data is essentially dynamically changing, requiring the model to continuously adapt to new data features and patterns. The dynamic prompt mechanism of the present invention enables the large language model to dynamically adjust its learning process according to new input data, adapt to data changes by retrieving the most relevant prompts, thereby achieving continuous performance improvement.

[0185] In general, the present invention provides a flexible, efficient and adaptive solution for the analysis of complex time series data through an innovative dynamic prompt mechanism, which effectively solves multiple key technical challenges faced by traditional methods in processing complex time series data.

[0186] Further specifically, in a specific embodiment of the present invention, an adaptive window length mechanism is implemented, and the decision strategy is optimized through reinforcement learning, while improving computational efficiency and model interpretability; wherein,

[0187] Regarding the adaptive window: when a rapid change in traffic flow is detected (such as during the morning rush hour of 7:00-9:00), the adaptive window length mechanism of the present invention will dynamically adjust the sliding window length from 1 hour to 30 minutes based on the calculation result of the scoring function score(τ). This adjustment can more accurately capture the drastic change characteristics of traffic flow;

[0188] Regarding reinforcement learning optimization: The reinforcement learning optimization mechanism of the present invention learns the optimal decision-making strategy by continuously accumulating state-action-reward experience; for example, when rainy weather is detected, based on the results of historical data analysis, the reinforcement learning optimizer will automatically adjust the prediction parameters and use the "rainy day delay peak mode" for prediction, thereby adapting to the special changing rules of traffic flow on rainy days;

[0189] Regarding interpretability: The interpretability module of the present invention generates a forecast basis analysis report based on SHAP value calculation and integrated gradient method; for example, the basis for the 30% increase in traffic flow forecast for the current period (8:30 a.m. on Tuesday) is that the contribution of the double peak mode characteristics on weekdays is 50%, the contribution of the current period growth rate is 30%, and the contribution of historical data for the same period is 20%. This quantitative explanation provides a clear forecast basis;

[0190] Through the synergistic effect of the above-mentioned technical features, the present invention realizes dynamic adaptive analysis of complex time series data, efficient extraction and utilization of historical knowledge, optimal allocation of computing resources and interpretable analysis of prediction results. The above-mentioned technical means enable the present invention to accurately grasp the dynamic change laws of time series data (such as traffic flow) and provide high-precision prediction results and clear prediction basis.

[0191] The following are device embodiments of the present invention, which can be used to implement the method embodiments of the present invention. For details not disclosed in the device embodiments, please refer to the method embodiments of the present invention.

[0192] See also Figure 2 In an embodiment of the present invention, an adaptive time series analysis system based on a dynamic prompt mechanism is provided, comprising:

[0193] The time series subsequence construction module is used to obtain the time series data to be analyzed and construct the time series subsequence using the sliding window method;

[0194] A dynamic prompt generation module is used to extract the feature representation of each time series subsequence and generate a dynamic prompt for each time series subsequence based on the dynamic prompt pool;

[0195] An analysis module, used for combining the feature representation of the selected time series subsequence with the dynamic prompt based on the analysis task type of the time series data to be analyzed, and inputting the combined result into the large language model, and obtaining the analysis result of the time series data through the large language model;

[0196] Wherein, the large language model is provided with an adaptive window length optimization mechanism.

[0197] In one embodiment of the present invention, a computer device is provided, the computer device comprising a processor and a memory, the memory is used to store a computer program, the computer program includes program instructions, and the processor is used to execute the program instructions stored in the computer storage medium. The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc., which are the computing core and control core of the terminal, which are suitable for implementing one or more instructions, and are specifically suitable for loading and executing one or more instructions in the computer storage medium to implement the corresponding method flow or corresponding function; the processor described in the embodiment of the present invention can be used to perform the operation of the adaptive time series analysis method based on the dynamic prompt mechanism.

[0198] In one embodiment of the present invention, a storage medium is provided, specifically a computer-readable storage medium (Memory), which is a memory device in a computer device for storing programs and data. It is understandable that the computer-readable storage medium here can include both built-in storage media in the computer device and, of course, extended storage media supported by the computer device. The computer-readable storage medium provides a storage space, which stores the operating system of the terminal. In addition, one or more instructions suitable for being loaded and executed by the processor are also stored in the storage space, and these instructions can be one or more computer programs (including program codes). It should be noted that the computer-readable storage medium here can be a high-speed RAM (Random Access Memory) memory, or a non-volatile memory (non-volatile memory), such as at least one disk memory. The processor can load and execute one or more instructions stored in the computer-readable storage medium to implement the corresponding steps of the adaptive time series analysis method based on the dynamic prompt mechanism in the above embodiment.

[0199] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, optical storage, etc.) that contain computer-usable program code.

[0200] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0201] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0202] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0203] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. An adaptive time series analysis method based on a dynamic prompt mechanism, characterized in that: The following steps are involved: Obtain the time series data to be analyzed and construct the time series subsequence using the sliding window method; Extracting feature representations of each time series subsequence, and generating dynamic prompts for each time series subsequence based on a dynamic prompt pool; Based on the analysis task type of the time series data to be analyzed, the feature representation of the selected time series subsequence is combined with the dynamic prompt, and the combined result is input into the large language model, and the analysis result of the time series data is obtained through the large language model; Wherein, the large language model is provided with an adaptive window length optimization mechanism.

2. According to claim 1, the adaptive time series analysis method based on dynamic prompt mechanism is characterized in that: In the step of generating dynamic prompts for each time series subsequence based on the dynamic prompt pool, The dynamic prompt pool is an initialized dynamic prompt pool or a dynamically updated dynamic prompt pool; wherein, The step of obtaining the initialized dynamic prompt pool includes: encoding the training data using a pre-trained Transformer model to generate an initial key-value pair set, and then optimizing the prompt pool structure by clustering to obtain the initialized dynamic prompt pool; wherein each prompt is represented as a key-value pair (k, v); k∈R d is the key vector, R d is a d-dimensional real vector; v∈R l×d is the value matrix, d is the embedding dimension, l is the hint length, R l×d is a real matrix with l rows and d columns; After dynamically updating the initialized dynamic prompt pool based on the analysis result of the time series data, a dynamically updated dynamic prompt pool is obtained.

3. The adaptive time series analysis method based on dynamic prompt mechanism according to claim 1 is characterized in that: In the step of generating dynamic prompts for each time series subsequence based on the dynamic prompt pool, For the time series subsequence S t , calculate the time series subsequence S t The characteristic representation f t With each key vector k in the dynamic hint pool m The cosine similarity sim(f t ,k m ), the calculation expression is: sim(f t ,k m )=(f t ·k m ) / (‖f t ‖‖k m ‖); In the formula, f t ·k m represents the dot product of the vectors, ‖f t ‖‖k m ‖ represents the multiplication of vector norms; Based on the cosine similarity sorting, select the K prompts with the highest similarity to form a prompt value matrix; Applying the attention mechanism to the prompt value matrix generates the final dynamic prompt, which is expressed as: p t =∑ i a i v i ; In the formula, p t represents the final dynamic prompt generated; α i represents the attention weight of the i-th prompt; represents f t The transpose of v i represents the value vector of the i-th prompt; W q represents the learnable query matrix parameters; k i A key vector representing the i-th prompt.

4. The adaptive time series analysis method based on dynamic prompt mechanism according to claim 1 is characterized in that: In the step of combining the feature representation of the selected time series subsequence with the dynamic prompt based on the analysis task type of the time series data to be analyzed, For the prediction task, a feed-forward neural network is used to combine the feature representation of the selected time series subsequence with dynamic cues; For missing data interpolation, a gated recurrent unit is used to combine the feature representation of the selected time series subsequence with dynamic cues.

5. The adaptive time series analysis method based on dynamic prompt mechanism according to claim 3 is characterized in that: In the adaptive window length optimization mechanism, a window length candidate set is preset, a time series subsequence is constructed for each candidate window length and features are extracted, and then a comprehensive score of each window length is calculated. Based on the comprehensive scores of each candidate window length, the optimal window length is dynamically selected; wherein, The calculation expression of the comprehensive score is: In the formula, τ i represents the length of the i-th candidate window; k j represents the j-th key vector; Indicates the use of window length τ i The mean square error of prediction when ; Represents the window length τ i The extracted features and the hint key vector k j similarity; Represents the window length τ i The information entropy of the extracted features; α, β, and γ represent the weight coefficients of each indicator respectively; N represents the number of prompt key vectors evaluated; Dynamically selected optimal window length τ opt It is expressed as:

6. The adaptive time series analysis method based on dynamic prompt mechanism according to claim 5 is characterized in that: The adaptive window length optimization mechanism also introduces a smoothing mechanism; wherein, t t =(1-λ)τ t-1 +lt opt ; Where λ is the smoothing factor.

7. The adaptive time series analysis method based on dynamic prompt mechanism according to claim 1 is characterized in that: The large language model is also provided with a feedback optimization mechanism based on reinforcement learning, and a policy gradient method is used to optimize the decision-making strategy.

8. The adaptive time series analysis method based on dynamic prompt mechanism according to claim 1 is characterized in that: The large language model also uses a SelfExtend method to process the input sequence.

9. The adaptive time series analysis method based on dynamic prompt mechanism according to claim 1 is characterized in that: The large language model is also provided with an interpretability module; wherein, The interpretability module is used to calculate the contribution of each hint to the final prediction using the SHAP value, computing the expression: In the formula, φ i represents the SHAP value of the i-th feature; F represents the set of all features; S represents the feature subset that does not contain feature i; |S| represents the number of features in subset S; |F| represents the total number of features; f S (x S ) represents the model output using only the feature subset S for prediction; f S∪{i} (x S∪{i} ) represents the model output after adding the i-th feature; is the combination weight, which is used to balance the impact of different feature combinations; The interpretability module is also used to visualize the attention weights α during the dynamic cue generation process i , and use the integrated gradient method to calculate the importance score of the input feature. The calculation expression is: In the formula, IG i (x) represents the integrated gradient value of the i-th feature x; x represents the actual feature value vector of the current input, x i represents the actual input value of the ith feature; x′ represents the baseline input, which is a reference point value, i represents the baseline input value of the i-th feature; f represents the prediction model function; α represents the integral path parameter; Represents the partial derivative of the model with respect to the i-th feature.

10. An adaptive time series analysis system based on a dynamic prompt mechanism, characterized in that: include: The time series subsequence construction module is used to obtain the time series data to be analyzed and construct the time series subsequence using the sliding window method; A dynamic prompt generation module is used to extract the feature representation of each time series subsequence and generate a dynamic prompt for each time series subsequence based on the dynamic prompt pool; An analysis module, used for combining the feature representation of the selected time series subsequence with the dynamic prompt based on the analysis task type of the time series data to be analyzed, and inputting the combined result into the large language model, and obtaining the analysis result of the time series data through the large language model; Wherein, the large language model is provided with an adaptive window length optimization mechanism.

Images