Method for dynamically tracking sparse high-information-content variable characteristics in time sequence

Abstract

The invention provides a sparse high information variable (SIVs) dynamic tracking method for time sequence prediction, which can automatically detect, separate and effectively utilize SIVs so as to improve prediction precision and model stability. According to the method, firstly, an automatic detection mechanism is used for identifying SIVs, a potential influence curve is generated based on a probability model, and a long-term dynamic dependency relationship is captured through integral operation; and then, introducing an adversarial progressive mode generation mechanism, optimizing long-term influence modeling of SIV, and adopting a multi-scale comparison fusion method to enhance feature extraction capability under different time scales. In order to further optimize the prediction effect, the method decomposes a time sequence into a trend term and a periodic term, and performs feature coding through a long and short term memory network. On the basis, the trend aggregation module integrates a plurality of influence factors to improve the local trend modeling capability. And finally, predicting the trend item and the season item by using a decoder, and obtaining a final prediction result through weighted fusion. The loss function is composed of prediction error loss, adversarial loss and comparison loss, and model parameters are optimized through gradient descent, so that the accuracy and generalization ability of time sequence prediction are improved. The method is suitable for multiple fields of medical health, industrial control, financial analysis and the like, and can effectively improve the time sequence prediction performance under the condition that the sparse variable with high information content exists.

Description

Technical Field:

[0001] The present invention relates to time series analysis and prediction technologies, especially technologies applied to dealing with the impact of Sparse but Informative Variables (SIVs) in time series prediction. Specifically, the present invention proposes a method through dynamic tracking and feature extraction, which can effectively extract variable features that are highly informative but sparse from time series data, thereby improving the prediction accuracy in the presence of sparse auxiliary signals. This method is widely applicable to time series prediction tasks in multiple fields such as healthcare and industrial process optimization. Background Art:

[0002] In time series prediction tasks, the value of the future target variable is usually affected by multiple factors, which can be divided into internal effects and external effects. The internal effect refers to the dynamic changes reflected by the historical data of the target variable itself, while the external effect comes from the influence of external signals or auxiliary variables. In particular, SIVs refer to those auxiliary variables in time series data whose values are zero at most time points, but these variables will change significantly at a few moments and have an important impact on the target signal. For example, in diabetes management, the impact of meal data and insulin injection data on blood glucose is significant, but these signals only appear at specific moments (such as when eating or injecting insulin), and are zero at other times. Therefore, how to extract effective information from these sparse and highly informative variables and integrate it into the time series prediction model has become the key to improving the model performance.

[0003] Existing time series prediction methods usually assume that the change rates of internal and external effects are similar, or process by directly combining all input variables into a unified prediction model. These methods mostly rely on directly modeling multiple variables in the time series, expecting to make accurate predictions by learning the relationship between the input variables and the target variable. Specifically, many traditional multi-input multi-output models, regression models, and deep learning models (such as Recurrent Neural Networks (RNNs)) are designed assuming that the change of the external signal is similar to the change rate of the target variable, or they assume that the frequency of the external signal is the same as the frequency of the target signal, so the external signal is directly used as an additional input for joint prediction. However, traditional methods often fail to effectively process these sparse signals, especially when there is a time delay or discontinuity in the relationship between the target variable and the external signal, it is difficult for the model to capture the true impact of the external signal on the target variable.

[0004] To address this issue, researchers have attempted to compensate for the deficiencies of sparse signals through methods such as data resampling, filling in missing values, or performing continuation processing on SIVs. For example, some methods resample the training data to increase the data samples containing non-zero SIV values, or "carry" the most recent non-zero SIV value to subsequent time points during the prediction process. Although these methods can enhance the model's perception of sparse signals to a certain extent, they do not take into account that the impact of SIVs on the target variable is a dynamic process, and this impact is accompanied by a time delay. "Carrying" the most recent non-zero SIV value to subsequent time points will blur the boundary between the actual occurrence time of the SIV and its subsequent impact, resulting in the model's inability to accurately capture the short-term impact of SIVs and may also lead to the model misjudging its duration. Over-reliance on this method will also increase the complexity of the model, bringing additional costs for training and inference, and affecting the stability and generalization ability of the model in practical applications.

[0005] Another challenge that urgently needs to be addressed is that for the discovery of SIVs, most current methods still rely on manual observation and analysis of data. This means that researchers usually need to judge based on experience which signals are SIVs and manually label them as inputs to the prediction model. Due to the sparsity and variability of SIVs, this manual analysis method is not only inefficient but also vulnerable to subjective judgment, resulting in mislabeling and a decline in model performance. Currently, there is a lack of an effective strategy that can automatically identify and separate SIVs from large-scale datasets, which forms a huge bottleneck in practical applications.

[0006] Therefore, how to design a novel prediction method that can automatically separate SIVs and effectively process the information of these sparse variables has become an important challenge in the current time series prediction field. The ideal solution should be able to isolate the internal effects and external effects, avoid over-reliance on sparse variables, and at the same time reasonably limit the impact of external signals by combining domain knowledge, so as to improve the model's processing ability for these high-information but sparse signals, and thus improve the prediction accuracy. Summary of the Invention:

[0007] The present invention provides a method that can automatically identify and separate sparse high-information variables and dynamically track the characteristics of this variable for time series prediction.

[0008] First, in order to effectively and automatically detect and separate SIVs, the present invention first defines SIVs, making subsequent detection and processing more accurate and efficient. The definition is as follows:

[0009] If the vector X = {x1, x2,..., x N} ∈ R Nis a sparse high-information variable, which can be approximated as Y ∼ B(1, p), where B(1, p) is the Bernoulli distribution. When N → +∞ and p → 0, we have:

[0010]

[0011] where N represents the dimension of the vector X, and X norm is the X after max-min normalization, and δ(·) represents the standard deviation of the variable. After defining the SIVs, an automatic detection mechanism is used to analyze the time series data to identify those SIVs that are zero or sparse at most time points but have a significant impact on the target variable at specific moments, thus avoiding the dependence on manual annotation.

[0012] Then, an influence curve is generated through a probability model and integrated to capture the long-term dynamic dependence information of the data. To enhance the flexibility and diversity of the model, the present invention combines the adversarial idea to further improve the model's adaptability to uncertainty and complexity. Finally, a multi-scale comparison method is adopted to automatically capture the influence range of the data through the comparison of different time scales, so as to accurately extract the influence of SIVs on the target variable. This method effectively solves the problem of dealing with sparse variables and is widely applicable to multiple fields such as medical health, industrial process control, and financial analysis. Description of the Drawings:

[0013] Figure 1 is the overall system framework diagram of the method applied in the present invention;

[0014] Figure 2 is the working flow chart of the method described in the present invention. Detailed Embodiments:

[0015] The present invention provides a method for dynamically tracking the characteristics of sparse high-information variables in a time series. Since this problem widely exists in the field of blood glucose detection, we describe the method using blood glucose detection, including the following steps:

[0016] Step 1: Obtain a blood glucose monitoring data set. The present invention uses the UVa / Padova simulator to generate blood glucose monitoring data for 30 individuals, including 10 children, 10 adolescents, and 10 adults. The simulator generates 10-day simulation data, which contains a total of 28,800 time points. Meal and insulin data are provided on average every 111 time points. During the simulation, the default basal-insulin controller is used for insulin injection, but three-quarters of the insulin injections are randomly delayed by 20 minutes to 2 hours, and the delay time is randomly drawn from a uniform distribution. This delay processing can effectively distinguish the influence of multiple SIVs at the same time point.

[0017] Step 2: After obtaining the blood glucose monitoring data, it is crucial to perform data preprocessing. First, the data will undergo normalization. Through the Max-Min normalization method, the values of each variable are mapped into the interval [0, 1], ensuring consistent scales among different features and avoiding model training biases caused by scale differences. Specifically as follows:

[0018]

[0019] Where X represents the original time series, max(X) and min(X) represent the maximum and minimum values of this feature, and X′ represents the time series data after max-min normalization. Next, to help the model better capture the characteristics of time series data, the present invention introduces the concept of positional encoding. Positional encoding includes absolute positional encoding A t , relative positional encoding R t and periodic encoding Sin t , and the specific manifestations are as follows:

[0020] A t =t,

[0021] R t =t - t0,

[0022]

[0023] Where t represents the time index, t0 represents the moment when SIVs occur, and T represents the cycle length we set. We default to half a day as a cycle. These variables help capture the patterns and regularities that change over time in time series data.

[0024] Step 3: After the data preprocessing is completed, according to the relevant definitions under SIV, the present invention separates sparse high-information variables (SIVs) by calculating the standard deviation ratio. First, for the feature value X norm at each time point and the target variable Y, we calculate the standard deviation δ(X norm ) of the normalized feature value and the standard deviation δ(Y) of the target variable, and then use the following formula to calculate the standard deviation ratio: Next, the upper threshold for SIV separation is controlled by the hyperparameter ∈. When the standard deviation ratio exceeds this threshold ∈, it means that the change degree of this feature compared to the target variable is lower, and thus it is determined as an SIV. Specifically, the SIV separation rule is as follows:

[0025]

[0026] Among them, 1 indicates that the feature is recognized as an SIV, and 0 indicates that the feature is not an ordinary variable. Through this method, the model can automatically identify sparse high-information variables that have a significant impact on the target variable and extract them for subsequent analysis and modeling.

[0027] Step 4: While performing SIV separation, the present invention decomposes the target variable Y into periodic components and trend components through a hybrid decomposition method. Assume that the time series data is represented as X t-L+1：t ∈R L×C , where L represents the number of time steps and C represents the number of channels. When extracting trend features, the present invention applies a moving average pooling method to smooth the seasonal fluctuations in the target sequence and extract the trend component This process is defined as follows:

[0028]

[0029] where AvgPool(·) represents the average pooling operation, and Padding(·) is used to extend the length of the time series to ensure that the size of the time series remains unchanged after the pooling operation is applied. After extracting the trend component , the periodic component is extracted by subtracting the trend feature obtained from the smoothing process from the original data That is:

[0030]

[0031] where Through this decomposition method, the present invention can effectively extract the long-term trend and periodic fluctuations in the time series, providing more accurate features for subsequent prediction modeling. This method can not only better understand the seasonal changes in the data but also improve the accuracy and stability of the prediction by separating the trend component and the periodic component.

[0032] Step 5: After completing the decomposition of the periodic component and the trend component of the target variable, the present invention uses a long short-term memory network (LSTM) to encode the extracted periodic component, trend component, and the remaining non-SIV variables. The LSTM can dynamically adjust its memory at each time step and effectively retain important long-term information when processing time series. In the present invention, the extracted trend component periodic component and non-SIV variables are respectively input into the LSTM network. The LSTM will learn the long-term dependency relationships and short-term fluctuation information in these components, thereby generating rich feature representations for subsequent prediction use. In this way, the LSTM can help the model better understand the temporal dependence of the target variable and improve the prediction ability for future data.

[0033] Step 6: The present invention adopts an adversarial progressive curve generation method to simulate the long-term impact of SIVs on the target variable. Since SIVs remain unchanged at most time points and only change at specific moments, it is difficult for traditional methods to capture their true impact. Therefore, this method uses a probability curve to model the potential impact pattern of SIV, and its mathematical definition is as follows:

[0034]

[0035] Where W ik and W jk are learnable parameters that control the distribution form of the curve. Due to the unimodal characteristic of a single Gaussian curve, the present invention further adopts a multimodal potential impact pattern, and simulates the diversity of SIVs by superimposing multiple probability curves. Its formula definition is as follows:

[0036]

[0037] Where γ is a hyperparameter representing the number of superimposed curves. To prevent mode collapse, the present invention introduces adversarial regularization, and enhances the diversity of SIVs by maximizing the distribution difference between different modes. The loss function L adversarial is as follows:

[0038]

[0039] Through this method, the model can more accurately simulate the dynamic impact of SIVs and improve the prediction stability and generalization ability.

[0040] Step 7: Next, in order to obtain the influence range of SIVs on the target variable, the present invention adopts a multi-scale contrast fusion method. Compared with directly using the original time series data, this method introduces the cumulative value as the input to smooth the short-term fluctuations and capture the overall trend in a longer time range. This method not only reduces the influence of noise, but also can more accurately identify the periodic and cyclic changes in the time series, improving the calculation efficiency and prediction stability. The calculation formula of multi-scale integration is as follows:

[0041]

[0042] Where L w represents the length of the integration region, and S n represents the window size of different scales. Experiments show that the size of L w has a significant impact on the final prediction result. However, due to the lack of prior knowledge about this variable, it is difficult to directly determine the optimal integration region. Therefore, the present invention adopts multiple integration regions for multi-scale embedding and aggregates them at different scales:

[0043]

[0044] This method enables the model to automatically capture the influence range of SIV at multiple time scales, ensuring that information at different time scales can be fully integrated. To enhance the feature discrimination ability between different time scales, the present invention further introduces a contrastive loss to maximize the similarity between features of the same time series while minimizing the feature distance at different time scales. Its mathematical expression is as follows:

[0045]

[0046] Where \(z_1\) and \(z_2\) respectively represent the feature representations of the same time series at different scales, \(z^-\) represents the negative sample (i.e., the encoding results of different time series), and τ is the temperature hyperparameter used to adjust the sensitivity of the contrastive loss.

[0047] Step 8: After completing the multi-scale contrastive fusion, the present invention further adopts a trend aggregation method to integrate multiple trend influencing factors and enhance the ability to model local trends. Since the periodic component mainly reflects the inherent periodic characteristics in the data, and other input variables mainly affect the prediction results through the trend component, multiple variables affecting the trend may have different degrees of variation. If these variables are used separately for modeling, the overall trend may not be fully captured. Therefore, it is necessary to aggregate the trend components to obtain a more comprehensive trend representation. Trend aggregation performs a weighted sum of all variables affecting the trend and uses a non-linear transformation to enhance the expression ability of the trend features. Its calculation formula is as follows:

[0048]

[0049] Where \(C_1\) and \(C_2\) respectively represent the number of ordinary variables and SIV channels, σ is the GELU activation function, \(W\) c and \(b\) c are learnable weight matrices and bias parameters. In this way, the trend aggregation method can comprehensively consider the influence of multiple variables on the trend, enabling the model to more accurately learn the trend changes within the local time range, thereby improving the stability and accuracy of the prediction.

[0050] Step 9: Since the time series has been decomposed into seasonal and trend terms, the present invention predicts them separately and combines their outputs to obtain the final prediction result. Specifically, we use LSTM to decode the seasonal and trend term encodings obtained in Step 5 and Step 8 to obtain the seasonal prediction term and the trend prediction term After that, we add them together to obtain the final prediction result, which is as follows:

[0051]

[0052] Step 10: To evaluate the prediction performance of the model, the present invention uses the root mean square error (RMSE) as the loss function for optimization, and its calculation formula is as follows:

[0053]

[0054] where Y t+i represents the true value, represents the predicted value, and H is the number of prediction time steps. The final loss function of the model consists of three parts, including the prediction error loss the adversarial loss and the contrastive loss and its overall loss function is defined as follows:

[0055]

[0056] where α and β are hyperparameters used to control the weights of the adversarial loss and the contrastive loss in the total loss, ensuring that the model can balance the impacts of different loss terms during the optimization process. During the optimization process, the present invention uses the gradient descent method to minimize the loss function and optimize the model parameters.

Claims

1. A method for defining and automatically separating sparse high-information variables (SIVs), characterized in that SIVs are defined as variables that are zero or sparse at most time points but have a significant impact on the target variable at specific moments and can be approximately modeled as following a Bernoulli distribution \(Y\sim B(1,p)\). When \(N\rightarrow+\infty\) and \(p\rightarrow0\), its standard deviation satisfies: where X norm is a time series variable after normalization, δ(·) represents the standard deviation, and N is the dimension of the variable. Based on this definition, this method uses the SIV separation function for automatic detection and separation of SIVs, and its calculation method is as follows: where \(\varepsilon\) is the threshold for SIV detection. The automatic separation process does not require manual annotation and can accurately identify SIVs from time series data.

2. A SIV feature processing and time series prediction model, characterized in that This model extracts the long-term dependencies of time series through the trend and cycle decomposition method and uses a long short-term memory network (LSTM) to encode the trend and cycle terms. Subsequently, adversarial progressive curves are adopted to enhance the SIV modeling ability, and the feature representation at different time scales is optimized through a multi-scale contrast fusion method. Finally, the trend and cycle terms are predicted separately by the decoder, and the final prediction result is obtained through weighted fusion, improving the stability and accuracy of time series prediction.

Images