Cascade text enhancement diffusion model time series interpolation method

Abstract

This invention discloses a cascaded text enhancement diffusion model time series interpolation method, comprising the following steps: S1: Acquire multimodal text data and perform forward diffusion noise reduction processing; S2: Encode and modal alignment processing of the text data; S3: Perform cascaded filtering on the aligned text embeddings through hierarchical semantic filters to suppress irrelevant semantics and focus on the context most relevant to the missing values; S4: Perform modal fusion and denoising processing through a diffusion model; S5: Introduce a missing mechanism estimator and construct an overall loss function by combining denoising loss, hybrid expert network-assisted loss, and masking loss, train the interpolation model using gradient descent, optimize network parameters, and obtain a trained interpolation model; S6: Perform the interpolation task using the trained time series interpolation model. This invention improves the accuracy of interpolation by jointly modeling data distribution and missing mechanisms.

Description

Technical Field

[0001] This invention relates to the field of time series data processing technology, and in particular to a diffusion model time series interpolation method for cascaded text enhancement. Background Technology

[0002] Time series data is widely used in numerous fields such as medical monitoring, financial transactions, and transportation. However, data loss is unavoidable due to factors such as equipment failure, incomplete data entry, or human error. Research shows that missing values ​​may encode underlying patterns or key information and therefore should not be ignored in time series analysis. The quality of missing data imputation has a critical impact on downstream tasks; low-quality imputation may introduce bias and impair the effectiveness of the analysis.

[0003] Existing deep neural network interpolation methods can be categorized into deterministic and probabilistic interpolation. Deterministic methods cannot quantify the uncertainty of their predictions, which is a significant drawback when dealing with highly volatile and noisy time series data. In recent years, diffusion models, due to their powerful probabilistic modeling capabilities, have been introduced into time series interpolation tasks, flexibly approximating the latent distribution of multivariate time series and achieving significant results. In many real-world applications, missing values ​​in time series are closely related to external factors and contextual information. Relying solely on numerical sequences for interpolation ignores these valuable external signals, leading to performance bottlenecks. Textual information derived from Large Language Models (LLMs) has been shown to significantly enhance the effectiveness of time series analysis.

[0004] However, effectively extracting and integrating textual information still faces the following challenges: Textual information derived from LLM inevitably contains content weakly correlated with the time series imputation task. Without filtering, this noise can mislead model learning, leading to performance degradation. Although cross-attention mechanisms can partially mitigate interference, processing these unnecessary features still consumes significant computational resources. Some studies have attempted to perform text filtering directly at the token level. However, in reality, a time series is often associated with multiple hierarchically organized text segments. Direct token retrieval may select highly correlated tokens from unrelated text segments, imposing incorrect conditional constraints. Furthermore, abnormally high-frequency tokens may "hijack" the filtering process, and calculating similarity across the entire token set still incurs significant overhead. Most existing imputation methods assume that the data is completely random missing (MCAR) or random missing (MAR), ignoring the missing mechanism. However, real-world missing data is often non-random (MNAR), meaning the missing data itself is directly related to unobserved values. Ignoring the MNAR mechanism can lead to biased model estimations.

[0005] Therefore, there is an urgent need for a time-series interpolation method that can effectively filter text noise, reduce computational overhead, and explicitly model the MNAR missing mechanism. Summary of the Invention

[0006] The purpose of this invention is to provide a time series interpolation method for cascaded text enhancement diffusion models, in order to solve the technical problems in the prior art that it is difficult to effectively filter noise in LLM derived text information, has high computational overhead, and ignores non-random missing mechanisms.

[0007] This invention is achieved using the following technical solution: a time series interpolation method for cascaded text enhancement diffusion models, comprising the following steps: S1: Acquire multimodal data, which includes K-dimensional time series data of length L, corresponding observation masks, and text data aligned with the time series in the time window, and perform forward diffusion noise processing on the time series data; S2: Encode and modal align text data; S3: The aligned text embeddings are cascaded through a hierarchical semantic filter to suppress irrelevant semantics and focus on the context most relevant to the missing values; S4: Modal fusion and denoising are performed using a diffusion model; S5: Introduce a missing mechanism estimator and combine it with denoising loss, hybrid expert network auxiliary loss and mask loss to construct the overall loss function. Train the imputation model by gradient descent, optimize the network parameters, and obtain a well-trained imputation model. S6: Perform the interpolation task using the trained time series interpolation model.

[0008] Furthermore, step S1 includes the following sub-steps: S11: Obtain the dataset ,in for Dimensions, Length Time series, To observe the mask, A collection of text segments aligned with the time series on a time window; S12: Further sample a training mask from the original observation mask; S13: Perform the forward diffusion process, adding Gaussian noise at each time step according to the preset noise schedule; S14: Constructing the Denoising Network Input: Input the target part after adding noise Compared with the original observation section By piecing together along the new dimension, we obtain This serves as the input for the subsequent encoder.

[0009] Furthermore, step S2 includes the following sub-steps: S21: Encode the text data using a pre-trained language model to obtain an initial token embedding representation; S22: The initial token embedding representation is projected onto the time series feature space through a hybrid expert network to achieve the alignment of heterogeneous modalities and obtain the text segment embedding representation.

[0010] Furthermore, the hybrid expert network includes a multi-expert router and a set of expert networks. The router calculates the allocation probability based on the input token embedding representation, selects K experts with the highest probability for each token, routes the token to the selected expert networks for feedforward computation, and combines the output results according to probability weights. Finally, an auxiliary load balancing loss is introduced to encourage the router to perform more balanced token allocation.

[0011] Furthermore, step S3 includes the following sub-steps: S31: At the fragment level, use a time-semantic gate to filter and select the most relevant text fragments based on the time-series embedding representation; S32: At the token level, cross-key ordering attention is used to automatically extract the tokens most beneficial for imputation from the selected text fragments and assign them higher weights to generate the final text conditional embedding representation.

[0012] Furthermore, step S31 includes the following sub-steps: S311: Perform average pooling on the time dimension of the time series embedding representation; S312: Input the pooled features into a multilayer perceptron to generate FiLM parameters, namely scaling factors and offset terms; S313: Perform average pooling on the embedded representation of each text segment and modulate it using the FiLM parameter; S314: Map the modulated representation to a scalar score through a linear layer; S315: Combine fragment masking to select the highest score. A text fragment.

[0013] Furthermore, step S32 includes the following sub-steps: S321: The query is represented by a time series embedding, and the key and value are represented by the token embedding of the selected text fragment; S322: Calculate the standard scaled dot product attention between the query and the key to obtain the similarity matrix A; S323: Average the similarity matrix A along the query dimension to obtain the global importance score for each token key; S324: Select the highest-scoring candidate based on global importance score. A token serves as the final key. Sum ; S325: Index each query pair from the similarity matrix A. The attention weights of each key are then renormalized to obtain... ; S326: Calculate the final output .

[0014] Furthermore, step S4 includes the following sub-steps: S41: Modal fusion of the time series embedding representation with the final text conditional embedding representation is performed, specifically: in the residual block of the denoising network, the time series embedding is added element-wise with the text conditional embedding obtained in step S3; the time series embedding representation is obtained through a 2D attention encoder containing time and feature dimensions; S42: Input the fused result into the denoising network of the denoising diffusion model to predict the noise and calculate the difference between the predicted noise and the actual noise. loss.

[0015] Furthermore, step S5 includes the following sub-steps: S51: To explicitly model the MNAR mechanism, a lightweight estimator is introduced to normalize the original time series, generate the missing log odds through a learnable affine transformation, and calculate the binary cross-entropy loss as the mask loss Lmask using the training mask as the supervision signal. S52: Weighted sum of denoising loss, auxiliary loss, and masking loss to obtain the overall loss function. ,in and Hyperparameters are used to balance various losses; S53: Minimize the overall loss function using gradient descent. Optimize all parameters of the model to obtain a well-trained interpolation model.

[0016] Furthermore, step S6 includes the following sub-steps: S61: Input the observed time series data and its corresponding text data; S62: Using the trained interpolation model, perform a backdiffusion sampling process, taking the text conditional embedding representation and observation data as conditions, gradually remove noise from Gaussian noise, and generate interpolation results with missing values.

[0017] The beneficial effects of this invention are as follows: This invention is specifically designed for time series imputation in non-random missing data scenarios. By jointly modeling data distribution and missing mechanisms, it improves the accuracy of imputation. Furthermore, this invention introduces a hybrid expert network to efficiently embed heterogeneous text into the time series feature space, achieving modality alignment (T-homogeneity).

[0018] The hierarchical semantic filter of this invention employs a cascaded filtering strategy from fragments to tokens, effectively suppressing noise interference in LLM-derived text and avoiding the erroneous constraints and high-frequency token "hijacking" problems caused by direct token-level filtering. Furthermore, the cross-key ranking attention mechanism in the hierarchical semantic filter significantly reduces computational complexity, substantially reducing computational and memory overhead while improving performance compared to standard cross-attention. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the structures shown in these drawings without creative effort.

[0020] Figure 1 This is an overall flowchart of the present invention; Figure 2 A schematic diagram of the algorithm for model training; Figure 3 A schematic diagram illustrating the process of time series completion. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0022] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0023] The following detailed description of some embodiments of the present invention is provided in conjunction with the accompanying drawings. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0024] See Figure 1 A time series interpolation method for cascaded text enhancement diffusion models includes the following steps: S1. Obtain time series data, corresponding missing masks, and time-aligned text data, and perform forward diffusion noise processing. S2. Encode the text data using a pre-trained language model, perform modality alignment using a hybrid expert network, and project it onto the time series feature space. S3. Perform cascade filtering on the aligned text embeddings through hierarchical semantic filters. First, filter relevant text segments at the fragment level, and then extract key context information at the token level. S4. Encode the original time series and fuse it with the text conditional embedding obtained in S3, then input it into the denoising network to predict noise; S5. Introduce a missing mechanism estimator and construct an overall loss function by combining denoising loss, hybrid expert network auxiliary loss and masking loss. Train the imputation model using gradient descent and optimize the network parameters. S6. The trained time series interpolation model is used for actual interpolation tasks. Through the backdiffusion sampling process, combined with the observation data and text conditions, noise is gradually removed to generate interpolation results with missing values.

[0025] In this embodiment, step S1 involves acquiring multimodal data and performing forward diffusion processing, specifically including the following sub-steps: S11. Obtain the dataset ,in for Dimensions, Length time series To observe the mask, A collection of text segments aligned with the time series on a time window; S12. To perform self-supervised learning, from the original observation mask... Further sampling of a training mask Define the observation part. and training target part ,in This indicates element-wise multiplication; S13. Perform forward diffusion process (noise addition): according to the preset noise schedule. ,exist Always towards Add Gaussian noise ,get : ,in ; S14. Construct a denoising network. Input: Input the denoised target part... Compared with the original observation section By piecing together along the new dimension, we obtain This serves as the input for the subsequent encoder.

[0026] In this embodiment, step S2 involves performing text encoding and modal alignment processing, specifically including the following sub-steps: S21. Use a pre-trained large language model on all text fragments. Encode to obtain the initial token embedding ,in The total number of tokens for all fragments. For the embedding dimension of large language models; S22, will Enter a hybrid expert network. This network contains a router. and A network of experts The router selects for each token. The most relevant experts are assigned probabilities as follows: ; in For a trainable parameter matrix, It is the bias vector; S23, route to the... Each expert's token is fed into the corresponding feedforward network for projection transformation, dimensional projection onto time series features The dimensions are calculated and weighted according to routing probabilities, resulting in a fragment-organized embedding representation. ,in For the number of segments, The number of tokens per fragment; S24. To encourage expert load balancing, an auxiliary loss function is introduced as follows: in Is it routed to the The number of tokens for each expert. It is the first The token was allocated to the first The probability of an expert.

[0027] In this embodiment, step S3 involves using a hierarchical semantic filter to perform cascading filtering from the fragment level to the token level, specifically including the following sub-steps: S31, Temporal-Semantic Gate (Fragment Filtering): a. First, S14 obtained Generate embedded representations of time series using a time series encoder. .

[0028] b. A time-dimensional average pooling is performed, and FiLM parameters, i.e., scaling factors, are generated through a multilayer perceptron. and offset terms .

[0029] c. For each text segment Average pooling is performed, modulated using FiLM parameters, and then passed through a linear layer. Calculate the score for each segment. : ,in It is a learnable projection matrix.

[0030] d. Combine with fragment mask Choose the one with the highest score. The fragments, their tokens are pieced together. ,in .

[0031] S32, Cross-Key Ranking Attention (Token Filtering): a) Embedded with time series For querying, the text embedding obtained from S31 Use the key as the reference. Calculate the similarity matrix between the two. .

[0032] b. Calculate the global importance score for each key. ,Right now Average value along the query dimension: .

[0033] c. According to Choose the highest score Each key, get .

[0034] d. Extract each query pair from A. The attention weights for each key are assigned and then renormalized along the key dimension to obtain the weight matrix. .

[0035] e. Calculate the final text conditional embedding representation. Its dimensions and time series embedding Alignment: .

[0036] In this embodiment, step S4 involves performing diffusion model fusion and denoising processing, specifically including the following sub-steps: S41, Time Series Encoding: The code obtained in S14... A time-series embedding representation is generated using an encoder that incorporates 2D attention, namely a temporal Transformer and a feature Transformer. ; S42. Modal fusion: Embedding the time series data into the residual blocks of the denoising network. Conditional embedding of text obtained with S32 Perform element-wise addition; S43. Denoising Network: Combines the fused representation with the number of diffusion steps. The encoding, timestamp, and other conditional information are fed into a denoising network with a structure like WaveNet to predict the noise added in S13. The model's output is ; S44. Denoising Loss: Calculates the difference between predicted noise and actual noise. loss.

[0037] In this embodiment, step S5 involves missing mechanism estimation and model training, which mainly includes the following sub-steps: S51, Missing Mechanism Estimator: To explicitly model the MNAR mechanism, a lightweight estimator is introduced: a. For the original time series Normalization yields .

[0038] b. Apply a learnable affine transformation to Z, where Z contains scaling parameters. and bias parameters Generate missing logs : .

[0039] c. Use the training mask in S12 As a monitoring signal, calculate the binary cross-entropy loss: in This is the sigmoid function.

[0040] S52. Overall Loss and Training: The denoising loss from S44, the auxiliary loss from S24, and the masking loss from S51 are weighted and summed to obtain the overall loss function. ,in and Hyperparameters to balance various losses; S53. Minimize the overall loss function using gradient descent. Optimize all parameters of the model to obtain a well-trained interpolation model.

[0041] Specifically, the model training principle in steps S2-S5 above is as follows: Figure 2 As shown.

[0042] In this embodiment, step S6 is a model-based interpolation application, which specifically includes the following sub-steps: S61. Apply the trained time-series interpolation model to the actual interpolation task. This process is as follows: Figure 3 As shown; S62. Input the actual observation data. and corresponding text data ,in This is the actual missing mask; S63, from a standard Gaussian noise Beginning, from arrive Iteratively execute the DDIM backsampling steps. At each step... : a. Using a trained denoising network Predict noise.

[0043] b. Update using DDIM sampling formula The process is based on S is a condition for the hierarchical semantic filter: ; in It is the predicted "clean" data. It is a parameter that controls the randomness of sampling.

[0044] S64, when At that time, the model output That is to Partial interpolation results; S65, Interpolation results Compared with the original observation data By combining these, we obtain a complete and intact time series. .

[0045] Based on the above embodiments, the present invention has at least the following technical effects: the present invention overcomes the shortcomings of existing methods in filtering noise, having high computational overhead, and ignoring non-random missing mechanisms when fusing text information, thus enabling them to better complete the time series interpolation task.

[0046] For the foregoing embodiments, in order to simplify the description, they are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, because according to this application, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions involved are not necessarily essential to this application.

[0047] The above embodiments describe the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Modifications and variations made by those skilled in the art without departing from the spirit and scope of the invention should be within the protection scope of the appended claims.

Claims

1. A time series interpolation method for a diffusion model of cascaded text enhancement, characterized in that, Includes the following steps: S1: Acquire multimodal data, which includes K-dimensional time series data of length L, corresponding observation masks, and text data aligned with the time series in the time window, and perform forward diffusion noise processing on the time series data; S2: Encode and modal align text data; S3: The aligned text embeddings are cascaded through a hierarchical semantic filter to suppress irrelevant semantics and focus on the context most relevant to the missing values; S4: Modal fusion and denoising are performed using a diffusion model; S5: Introduce a missing mechanism estimator and combine it with denoising loss, hybrid expert network auxiliary loss and mask loss to construct the overall loss function. Train the imputation model by gradient descent, optimize the network parameters, and obtain a well-trained imputation model. S6: Perform the interpolation task using the trained time series interpolation model.

2. The time series interpolation method for cascaded text enhancement diffusion models as described in claim 1, characterized in that, Step S1 includes the following sub-steps: S11: Obtain the dataset ,in for Dimensions, Length Time series, To observe the mask, A collection of text segments aligned with the time series on a time window; S12: Further sample a training mask from the original observation mask; S13: Perform the forward diffusion process, adding Gaussian noise at each time step according to the preset noise schedule; S14: Constructing the Denoising Network Input: Input the target part after adding noise Compared with the original observation section By piecing together along the new dimension, we obtain This serves as the input for the subsequent encoder.

3. The time series interpolation method for cascaded text enhancement diffusion models as described in claim 1, characterized in that, Step S2 includes the following sub-steps: S21: Encode the text data using a pre-trained language model to obtain an initial token embedding representation; S22: The initial token embedding representation is projected onto the time series feature space through a hybrid expert network to achieve the alignment of heterogeneous modalities and obtain the text segment embedding representation.

4. The time series interpolation method for cascaded text enhancement diffusion models as described in claim 3, characterized in that, The hybrid expert network includes a multi-expert router and a set of expert networks. The router calculates the allocation probability based on the input token embedding representation, selects K experts with the highest probability for each token, routes the token to the selected expert networks for feedforward computation, and combines the output results according to probability weights. Finally, an auxiliary load balancing loss is introduced to encourage the router to perform more balanced token allocation.

5. The time series interpolation method for cascaded text enhancement diffusion models as described in claim 1, characterized in that, Step S3 includes the following sub-steps: S31: At the fragment level, use a time-semantic gate to filter and select the most relevant text fragments based on the time-series embedding representation; S32: At the token level, cross-key ordering attention is used to automatically extract the tokens most beneficial for imputation from the selected text fragments and assign them higher weights to generate the final text conditional embedding representation.

6. The time series interpolation method for cascaded text enhancement diffusion models as described in claim 5, characterized in that, Step S31 includes the following sub-steps: S311: Perform average pooling on the time dimension of the time series embedding representation; S312: Input the pooled features into a multilayer perceptron to generate FiLM parameters, namely scaling factors and offset terms; S313: Perform average pooling on the embedded representation of each text segment and modulate it using the FiLM parameter; S314: Map the modulated representation to a scalar score through a linear layer; S315: Combine fragment masking to select the highest score. A text fragment.

7. The time series interpolation method for cascaded text enhancement diffusion models as described in claim 5, characterized in that, Step S32 includes the following sub-steps: S321: The query is represented by a time series embedding, and the key and value are represented by the token embedding of the selected text fragment; S322: Calculate the standard scaled dot product attention between the query and the key to obtain the similarity matrix A; S323: Average the similarity matrix A along the query dimension to obtain the global importance score for each token key; S324: Select the highest-scoring candidate based on global importance score. A token serves as the final key. Sum ; S325: Index each query pair from the similarity matrix A. The attention weights of each key are then renormalized to obtain... ; S326: Calculate the final output .

8. The time series interpolation method for cascaded text enhancement diffusion models as described in claim 1, characterized in that, Step S4 includes the following sub-steps: S41: Modal fusion of the time series embedding representation with the final text conditional embedding representation is performed, specifically: in the residual block of the denoising network, the time series embedding is added element-wise with the text conditional embedding obtained in step S3; the time series embedding representation is obtained through a 2D attention encoder containing time and feature dimensions; S42: Input the fused result into the denoising network of the denoising diffusion model to predict the noise and calculate the difference between the predicted noise and the actual noise. loss.

9. The time series interpolation method for cascaded text enhancement diffusion models as described in claim 1, characterized in that, Step S5 includes the following sub-steps: S51: To explicitly model the MNAR mechanism, a lightweight estimator is introduced to normalize the original time series, generate the missing log odds through a learnable affine transformation, and calculate the binary cross-entropy loss as the mask loss Lmask using the training mask as the supervision signal. S52: The denoising loss, auxiliary loss, and masking loss are weighted and summed to obtain the overall loss function. ,in and Hyperparameters to balance various losses; S53: Minimize the overall loss function using gradient descent. Optimize all parameters of the model to obtain a well-trained interpolation model.

10. The time series interpolation method for cascaded text enhancement diffusion models as described in claim 1, characterized in that, Step S6 includes the following sub-steps: S61: Input the observed time series data and its corresponding text data; S62: Using the trained interpolation model, perform a backdiffusion sampling process, taking the text conditional embedding representation and observation data as conditions, gradually remove noise from Gaussian noise, and generate interpolation results with missing values.

Images