A robust light-weight time series forecasting method based on bidirectional padding and geometric attention

Abstract

The application discloses a robust light-weight time series prediction method based on bidirectional imputation and geometric attention, and belongs to the field of missing data filling and prediction of multivariate time series. The problem of missing monitoring data caused by frequent sensor failure in complex geological environment is solved, and the accuracy and reliability of geological disaster prediction are improved. The application proposes a collaborative processing framework that fuses bidirectional recurrent imputation for time series (BRITS) and SimpleTM light-weight prediction model. Firstly, the BRITS model is used to capture the forward and backward time series dependence and multivariate correlation of sensor data, accurately identify the failure mode and achieve efficient data filling; then, combined with the specific stationary wavelet transform multi-scale decomposition and geometric product attention mechanism of the SimpleTM model, the fine-grained time series characteristics of the filled data are deeply mined under low computational overhead, providing reliable support for subsequent disaster prediction. The application significantly improves the prediction accuracy and robustness in the missing data scenario, while taking into account the light-weight deployment requirements in the resource-constrained tunnel construction environment, effectively ensuring the safety of railway tunnel construction.

Description

Technical Field

[0001] This invention belongs to the field of multivariate time series data processing and prediction, and provides a robust and lightweight time series prediction method based on bidirectional imputation and geometric attention. Background Technology

[0002] Geological disasters are a significant natural-anthropogenic coupling phenomenon threatening human life, property, and the ecological environment. Their spatiotemporal distribution is influenced by a combination of multiple geological structures, climatic conditions, and engineering disturbances. Existing geological disaster prediction research largely focuses on the correlation analysis between meteorological factors and disaster evolution trends, paying less attention to prediction problems under conditions of missing data. Currently, data imputation is the most common method for handling missing time series data. Data imputation methods mostly focus on time series completion under a single environment or rely on static data correlation analysis, lacking in-depth consideration of how to construct sensor failure modes. In addition, most of the above methods are based on deep learning, which has certain requirements for computing power and operating hardware; however, it is difficult to configure the computing conditions that meet the environmental requirements of tunnel construction sites, making it difficult for these methods to be applied in practice.

[0003] In summary, the existing technology has the following technical problems:

[0004] Existing methods for filling missing data in multivariate time series data have technical problems such as high requirements for computing power and operating hardware, reliance on static data correlation analysis, lack of sensor failure mode construction, and insufficient model robustness. Summary of the Invention

[0005] The purpose of this invention is to solve the technical problems of existing multivariate time-series data missing data filling methods, such as high computational requirements, reliance on static data correlation analysis, and insufficient prediction robustness caused by the lack of sensor failure mode construction.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] A robust and lightweight time series prediction method based on bidirectional imputation and geometric attention includes the following steps:

[0008] Step S1: Introduce a dynamic random missing pattern, mask the corresponding feature columns according to a certain ratio, and construct the missing scenario under multiple constraints of time dimension, space dimension and device dimension;

[0009] Step S2: The BRITS model captures the forward and backward temporal dependencies and multivariate correlations of sensor data, identifies failure modes, and fills in missing data.

[0010] Step S3: Input the padded data into the SimpleTM model and use its stationary wavelet transform multi-scale decomposition and geometric product attention mechanism to mine fine-grained temporal features.

[0011] Step S4: Based on the processed features, perform geological disaster prediction to improve the prediction robustness in scenarios with missing input.

[0012] Furthermore, the specific steps of step S1 are as follows:

[0013] Step S1.1: Missing values ​​are mainly in the mode of random missing in the feature dimension. That is, for the input data x_enc, some features are randomly selected in the feature dimension feature_dim to simulate missing values. The missing value ratio is controlled by the parameter mask_ratio.

[0014] Step S1.2: Control the maximum duration of continuous missing data through the freq parameter to simulate different failure types. The feature index to be masked is randomly selected by torch.randperm(), and the position of mask_idx is randomly selected on the feature dimension so that the time distribution of missing data conforms to the time sequence characteristics of real sensor data. Different missing probabilities are set for different feature columns (corresponding to different devices) to ensure that the missing scenario not only has randomness, but also reflects the complex reasons for the missing sensor data in the real environment.

[0015] Furthermore, the specific steps of step S2 are as follows:

[0016] Step S2.1: BRITS generates estimates of missing values ​​through an interpolation network and combines these estimates with a masking vector to determine whether to use the true values ​​or the interpolated results. The specific formula is as follows:

[0017] Input value after interpolation: .in, It is the occlusion vector (0 indicates missing, 1 indicates observed). It is the actual value. These are estimates generated by the interpolation network;

[0018] Interpolation networks generate estimates: .in, It is the hidden state of a bidirectional LSTM, and the interpolation result is generated through a regression layer;

[0019] Step S2.2: A bidirectional LSTM is used to capture the forward and backward dependencies of the time series. A gating mechanism is employed to filter and fuse bidirectional information, accurately handling long-interval continuous missing data (such as rock mass deformation in fractured zones) and random point missing data (such as electromagnetic interference from water-rich strata). The state update formula is as follows:

[0020] Forward LSTM hidden state update: .in, It is a time decay factor that adjusts the weight of historical information. It is the interval between the current time step and the previous observation time step. It is a time constant.

[0021] Cell state update: The cell state of the forward LSTM is updated step by step from the start position to the end position of the sequence, carrying historical information while incorporating current information.

[0022]

[0023] Backward LSTM hidden state update:

[0024] Cell state update (similar to forward, direction adjustment): Information on reverse sequences is filtered, transmitted, and updated through a gating mechanism to capture reverse sequence dependencies.

[0025]

[0026] in, , Forward / backward LSTM time steps The hidden state; For time steps Input after interpolation; , , , , The input gate, forget gate, candidate cell state, cell state, and output gate of the forward / backward LSTM; In a forward / backward LSTM, the input... The weight matrices for the input gate, forget gate, candidate cell state, and output gate; This represents the weight matrix from the hidden state of the previous time step to each gate in the forward / backward LSTM. This represents the bias term for each gate in the forward / backward LSTM; This is the sigmoid activation function used for the gated unit output.

[0027] Step S2.3: BRITS further combines the correlation of other features at the same time step to generate an estimate.

[0028] Feature-based prediction: .in, It is a weight matrix (the diagonal is zero to avoid self-dependency). It is a bias.

[0029] Combining historical data with characteristic features for prediction: Weight Due to time decay factor and occlusion vector Dynamic adjustment.

[0030] Step S2.4: The total loss of BRITS consists of interpolation loss, prediction loss, and consistency loss. The specific formula is as follows:

[0031] Interpolation loss (mean absolute error): ;

[0032] Predicted loss (e.g., binary cross-entropy): ;

[0033] Consistency loss: To ensure consistency between two-way forecasts;

[0034] Total loss: .

[0035] in, It is the predicted output. These are real labels, hyperparameters. Balance the contributions of each part.

[0036] Furthermore, the specific steps of step S3 are as follows:

[0037] Step S3.1: Decomposition of stationary wavelet transform. The input multivariate time-series data is decomposed into multiple scales. Learnable low-pass and high-pass filters are used to generate time-frequency features at different scales, covering the full range of time patterns from short-term fluctuations to long-term trends. Initial conditions. ,in, This is the input signal after linear projection. The iterative decomposition formula is:

[0038]

[0039] in, and These are the upsampled versions of the low-pass and high-pass filters at scale s (by inserting 2). s (to achieve this with -1 zero), and These represent the approximation coefficient and detail coefficient for scale s, respectively;

[0040] Step S3.2, Improvement of the Geometric Product Attention Mechanism. Geometric algebraic operations are introduced on top of the standard self-attention mechanism. Dot products capture scalar similarity, and wedge products capture linear independence and spatial orientation, enabling the model to capture higher-order nonlinear relationships while avoiding high computational costs. The specific formula is as follows:

[0041]

[0042] in, It is a query, key, and value matrix (generated by linear projection). It is the wedge product (outer product), which generates a bivector, capturing the linear independence and geometric relationship between channels. It is a dimensionality reduction function (taking the magnitude of a two-vector or a trainable MLP). It refers to the number of channels.

[0043] Step S3.3: Reconstruction using inverse stationary wavelet transform. Starting from the coarsest scale, the tokens processed at each scale are iteratively integrated, incorporating complex dependency information captured by the attention mechanism. Finally, the prediction result is output by the feedforward network and layer normalization. The core formula of inverse stationary wavelet transform is:

[0044]

[0045] in, It is the coarsest approximation coefficient. and It is a learnable reconstruction filter. This represents the approximate coefficients at time point t for the (s-1)th scale (fine scale). This represents the approximate coefficients at time point t+k for the s-th scale (coarse scale). This represents the detail coefficients at time point t+k at the s-th scale (coarse scale). Indicates the filter index. It is a scale parameter.

[0046] Step S3.4: Based on steps S3.1-S3.3, ensure that the model extracts multi-scale features from the original data, models the relationship between features through the attention mechanism, and then restores the processed features to interpretable time series prediction results.

[0047] This invention significantly improves the prediction robustness and practicality in scenarios with missing multivariate temporal data by integrating bidirectional imputation and geometric attention mechanisms, all within a lightweight architecture. Specific beneficial effects are as follows:

[0048] 1. By introducing a dynamic random missing mode (step S1), this invention constructs a missing scenario with multiple constraints in the time, space and device dimensions, which solves the problem of the lack of in-depth consideration of sensor failure modes in the prior art and achieves the effect of accurately simulating the complexity of real missing data.

[0049] Step S1 simulates long-interval continuous missing data (such as rock mass deformation in fractured zones) and random point missing data (such as electromagnetic interference in water-rich strata) by randomly missing feature dimensions (step S1.1), controlling the maximum duration of continuous missing data (step S1.2), and differentiating the missing data probability. This method avoids the limitations of static data correlation analysis. By considering temporal distribution, spatial rationality, and equipment differentiation, it realistically recreates the diverse causes of sensor failure (such as hardware failure or environmental interference), thus providing an input basis that matches the actual scenario for subsequent infilling and ensuring the generalization ability of model training and prediction.

[0050] 2. This invention generates dynamic interpolation by combining the BRITS model (step S2) with the time decay factor of bidirectional LSTM and the correlation of multivariate variables, which solves the problems of existing methods relying on static data correlation and lacking dynamic failure mode processing, and achieves the effect of efficiently filling missing data and preserving temporal dependencies.

[0051] Step S2 generates an estimate using an interpolation network (step S2.1), and combines it with a time decay factor through a bidirectional LSTM (step S2.2). The method dynamically adjusts the weights of historical information while integrating the correlations of other features at the same time step (step S2.3). This approach captures forward and backward temporal dependencies (such as the rock deformation trend during tunnel construction) and generates filled values ​​through implicit correlations of multivariate variables (such as the coupling between displacement and seepage pressure), avoiding information loss during single-variable interpolation. Combining interpolation loss, prediction loss, and consistency loss (step S2.4), this mechanism accurately handles missing patterns under a lightweight architecture, reducing reliance on high computing power.

[0052] 3. This invention transforms time-series data into multi-frequency token sequences through stationary wavelet transform multi-scale decomposition (step S3.1), which solves the problem that existing methods are difficult to capture fine-grained time-series features and achieves the effect of preserving locality and multi-scale dynamic features.

[0053] Step S3.1 iteratively decomposes the input signal using learnable low-pass and high-pass filters to generate approximation and detail coefficients. This method decomposes the original sequence into different scales (from short-term fluctuations to long-term trends), covering a full range of time patterns (such as instantaneous vibrations and long-term settlement in tunnel monitoring). By keeping the signal length constant, this decomposition extracts variable-specific patterns (such as the periodicity of seepage pressure data) with lightweight computation, avoiding feature loss due to data gaps in traditional methods, and providing structured input for subsequent attention mechanisms.

[0054] 4. This invention solves the problems of limited information and high computational cost of traditional self-attention by fusing scalar similarity and geometric relationship through geometric product attention mechanism (step S3.2), and achieves the effect of lightweight modeling of complex dynamics.

[0055] Step S3.2 introduces wedge product based on standard self-attention ( ) capture linear independence and spatial orientation (such as the geometric correlation of data from different sensors), and use a dimensionality reduction function ( This approach integrates high-order nonlinear relationships (GeoProdAttn formula). It captures scalar similarity through dot products and enhances geometric features (such as the directional dependence of displacement vectors) through wedge products. While avoiding high computational burden, it amplifies the complex dynamics between multivariate variables (such as the coupling between rock mass deformation and hydrological changes). Combined with the output of stationary wavelet transform, this mechanism achieves efficient feature modeling and improves prediction robustness in resource-constrained scenarios.

[0056] 5. This invention solves the problems of insufficient model robustness and dependence on data integrity through the collaborative framework of BRITS and SimpleTM (steps S2 and S3), and achieves the effect of maintaining prediction accuracy in missing scenarios.

[0057] The BRITS-filled data (step S2) preserves temporal dependencies and variable correlations, which are further amplified by SimpleTM's geometric attention (step S3.2). Simultaneously, the multi-scale features of the stationary wavelet transform (step S3.1) and the inverse transform reconstruction (step S3.3) ensure feature interpretability. This collaborative mechanism, through the complementarity of dynamic interpolation and geometric attention, can still model fine-grained temporal dynamics (such as the sudden evolution of geological disasters) even with random missing data. Furthermore, the entire framework is lightweight (e.g., SimpleTM's product attention reduces the number of parameters), significantly reducing the demand for hardware computing power and adapting to resource-constrained environments such as tunnel construction. Attached Figure Description

[0058] Figure 1 This is a flowchart of the method steps of the present invention. Detailed Implementation

[0059] The present invention will now be further described in conjunction with the accompanying drawings and specific embodiments.

[0060] Please see Figure 1 The present invention provides a technical solution:

[0061] To address the problems mentioned above, this invention, taking railway tunnel construction safety monitoring as an example, proposes a lightweight time series prediction method that can handle data loss in the face of sensor failure under sudden environmental conditions. By establishing a method for processing missing sensor data, the robustness of tunnel construction monitoring data can be improved.

[0062] Specifically, this invention provides a robust and lightweight time series prediction method based on bidirectional incompleteness and geometric attention, comprising the following steps:

[0063] Step S1: Introduce a dynamic random missing pattern, masking the corresponding feature columns at a ratio of 30%, and construct the missing scenario under multiple constraints in the time, space, and device dimensions. Specifically, this includes the following steps:

[0064] Step 1.1: Missing values ​​are mainly in the mode of random missing in feature dimensions. That is, for the input data, some features are randomly selected to simulate missing values. The missing value ratio is controlled by the parameter mask_ratio.

[0065] The pre-trained random mask module is:

[0066] mask = torch.zeros_like(masked_x)

[0067] for i in range(masked_x.size(0)):

[0068] mask_idx=torch.randperm(masked_x.size(2))[:int(masked_x.size(2)*self.mask_ratio)]

[0069] mask[i, :, mask_idx] = 1

[0070] masked_x = masked_x * (1 - mask)

[0071] Step 1.2: By controlling the maximum proportion of consecutive missing data, different failure types are simulated to ensure that the temporal distribution of missing data conforms to the temporal characteristics of real sensor data; differentiated missing probabilities are set for different feature columns to ensure that the missing scenarios are random. Specifically, mask_feature controls the random missing of certain feature columns.

[0072] The core module for random missing values ​​is:

[0073] if self.args.mask_feature:

[0074] mask = torch.zeros_like(batch_x).to(self.device)

[0075] mask_idx = torch.randperm(batch_x.size(-1))[:int(batch_x.size(-1)* self.args.mask_ratio)]

[0076] mask[:, :, mask_idx] = 1

[0077] batch_x = batch_x * (1 - mask)

[0078] Step S2: The BRITS model captures the forward and backward temporal dependencies and multivariate correlations of sensor data, identifies failure modes, and fills in missing data.

[0079] The specific steps are as follows:

[0080] Step 2.1: BRITS generates estimates of missing values ​​through an interpolation network, calculates the duration of missing values ​​using _get_delta, and captures missing patterns; the interpolated value at each time step is generated from the current hidden state and dynamically adjusted in conjunction with historical information.

[0081] The core function is as follows:

[0082] def _get_delta(self, m):

[0083] seq_len = m.size(1)

[0084] delta = []

[0085] for i in range(seq_len):

[0086] if i == 0:

[0087] delta.append(torch.zeros_like(m[:, i, :]))

[0088] else:

[0089] prev_m = m[:, i-1, :]

[0090] cur_m = m[:, i, :]

[0091] delta_t = (1 - cur_m) * (delta[-1] + 1) + cur_m * 0

[0092] delta.append(delta_t)

[0093] return torch.stack(delta, dim=1)

[0094] Step 2.2: A bidirectional LSTM is used to capture the forward and backward dependencies of the time series, accurately handling long-interval continuous missing data and random point missing data. BRITS combines two RITS methods, improving accuracy by fusing the bidirectional interpolation results. The forward RITS learns the dependencies from the past to the future, while the backward RITS learns the dependencies from the future to the past.

[0095] The core modules of BRITS bidirectional prediction are as follows:

[0096] class BRITS(nn.Module):

[0097] def __init__(self, input_size, hidden_size):

[0098] super(BRITS, self).__init__()

[0099] self.rits_f = RITS(input_size, hidden_size)

[0100] self.rits_b = RITS(input_size, hidden_size)

[0101] def forward(self, x, m):

[0102] delta_f = self._get_delta(m)

[0103] delta_b = self._get_delta(m.flip(1)).flip(1)

[0104] imputations_f = self.rits_f(x, m, delta_f)

[0105] imputations_b = self.rits_b(x.flip(1), m.flip(1), delta_b).flip(1)

[0106] imputations = (imputations_f + imputations_b) / 2

[0107] return imputations

[0108] Step 2.3: BRITS further combines the correlation of other features at the same time step to generate an estimate.

[0109] Feature-based prediction: .in, It is a weight matrix (the diagonal is zero to avoid self-dependency). It is a bias;

[0110] Combining historical data with characteristic features for prediction: Weight Due to time decay factor and occlusion vector Dynamic adjustment;

[0111] Step 2.4: The total loss of BRITS consists of interpolation loss, prediction loss, and consistency loss. MSE loss is calculated only for missing locations to avoid interference with valid data.

[0112] The specific formula is as follows:

[0113] Interpolation loss (mean absolute error):

[0114] Predicted loss (e.g., binary cross-entropy):

[0115] Consistency loss: This ensures consistency between two-way forecasts.

[0116] Total loss:

[0117] in, It is the predicted output. These are real labels, hyperparameters. Balance the contributions of each part.

[0118] The core function is as follows:

[0119] def brits_loss(x, mask, imputations_forward, imputations_backward,

[0120] h_forward, h_backward, lambda_consistency=1.0):

[0121] observed_loss = F.mse_loss(imputations_forward * mask, x * mask,reduction='sum') + \

[0122] F.mse_loss(imputations_backward * mask, x * mask,reduction='sum')

[0123] missing_mask = 1 - mask

[0124] missing_loss = F.mse_loss(imputations_forward * missing_mask,

[0125] imputations_backward * missing_mask,

[0126] reduction='sum')

[0127] consistency_loss = F.mse_loss(h_forward, h_backward, reduction='sum')

[0128] total_obs = 2 * torch.sum(mask)

[0129] total_missing = torch.sum(missing_mask)

[0130] total_hidden = h_forward.numel()

[0131] observed_loss_mean = observed_loss / total_obs if total_obs > 0else 0

[0132] missing_loss_mean = missing_loss / total_missing if total_missing> 0 else 0

[0133] consistency_loss_mean = consistency_loss / total_hidden if total_hidden > 0 else 0

[0134] total_loss = (observed_loss + missing_loss + lambda_consistency *consistency_loss) / \

[0135] (total_obs + total_missing + lambda_consistency *total_hidden)

[0136] return total_loss, {

[0137] 'observed_loss': observed_loss_mean,

[0138] 'missing_loss': missing_loss_mean,

[0139] 'consistency_loss': consistency_loss_mean

[0140] }

[0141] Step S3: Input the padded data into the SimpleTM model and use its stationary wavelet transform multi-scale decomposition and geometric product attention mechanism to mine fine-grained temporal features.

[0142] Step 3.1: Decomposition of stationary wavelet transform. Multi-scale decomposition is performed on the input multivariate time-series data. Learnable low-pass and high-pass filters are used to generate time-frequency features at different scales, covering the full range of time patterns from short-term fluctuations to long-term trends. The multi-scale decomposition of the signal, through filter upsampling and convolution, maintains the signal length and obtains detailed coefficients for each layer and the final approximation coefficients.

[0143] The core function of the stationary wavelet transform is:

[0144] def swt_decompose(signal, wavelet, level):

[0145] wavelet_obj = pywt.Wavelet(wavelet)

[0146] dec_lo = wavelet_obj.dec_lo

[0147] dec_hi = wavelet_obj.dec_hi

[0148] signal_len = len(signal)

[0149] coeffs = []

[0150] current_signal = np.copy(signal)

[0151] for j in range(level):

[0152] upsampling_factor = 2 ** j

[0153] dec_lo_up = np.zeros(len(dec_lo) + (upsampling_factor - 1) *(len(dec_lo) - 1))

[0154] dec_hi_up = np.zeros(len(dec_hi) + (upsampling_factor - 1) *(len(dec_hi) - 1))

[0155] dec_lo_up[::upsampling_factor] = dec_lo

[0156] dec_hi_up[::upsampling_factor] = dec_hi

[0157] pad_len = max(len(dec_lo_up), len(dec_hi_up)) - 1

[0158] padded = np.pad(current_signal, pad_len / / 2, mode='symmetric')

[0159] approx = np.convolve(padded, dec_lo_up, mode='valid')

[0160] detail = np.convolve(padded, dec_hi_up, mode='valid')

[0161] approx = approx[:signal_len]

[0162] detail = detail[:signal_len]

[0163] coeffs.append((approx, detail))

[0164] current_signal = approx

[0165] return coeffs

[0166] Step 3.2, Improvement of the Geometric Product Attention Mechanism. The core logic of the geometric attention mechanism is implemented through GeomAttention and GeomAttentionLayer. The GeomAttention class implements the core calculation of geometric attention, incorporating geometric features when calculating attention weights. The GeomAttentionLayer class encapsulates GeomAttention and, combined with operations such as residual connections and layer normalization, ensures stable model training.

[0167] The core function of the geometric attention mechanism is:

[0168] def forward(self, queries, keys, values, attn_mask=None):

[0169] B, L, H, E = queries.shape

[0170] _, S, _, _ = values.shape

[0171] scale = self.scale or 1. / sqrt(E)

[0172] dot_product = torch.einsum("blhe,bshe->bhls", queries, keys)

[0173] queries_norm2 = torch.sum(queries**2, dim=-1).permute(0, 2,1).unsqueeze(-1)

[0174] keys_norm2 = torch.sum(keys**2, dim=-1).permute(0, 2, 1).unsqueeze(-2)

[0175] wedge_norm2 = queries_norm2 * keys_norm2 - dot_product ** 2

[0176] wedge_norm2 = F.relu(wedge_norm2)

[0177] wedge_norm = torch.sqrt(wedge_norm2 + 1e-8)

[0178] scores = (1 - self.alpha) * dot_product + self.alpha * wedge_norm

[0179] scores = scores * scale

[0180] if self.mask_flag:

[0181] if attn_mask is None:

[0182] attn_mask = torch.tril(torch.ones(L, S)).to(scores.device)

[0183] scores.masked_fill_(attn_mask.unsqueeze(1).unsqueeze(2) == 0, float('-inf'))

[0184] A = self.dropout(torch.softmax(scores, dim=-1))

[0185] V = torch.einsum("bhls,bshd->blhd", A, values)

[0186] if self.output_attention:

[0187] return V.contiguous()

[0188] else:

[0189] return (V.contiguous(), scores.abs().mean())

[0190] Step 3.3: Reconstruction using inverse stationary wavelet transform. Starting from the coarsest scale, the approximation coefficients are gradually merged with the corresponding layer's detail coefficients through convolution using a reconstruction filter.

[0191] The core function of the inverse stationary wavelet transform is:

[0192] def swt_reconstruct(coeffs, wavelet):

[0193] wavelet_obj = pywt.Wavelet(wavelet)

[0194] rec_lo = wavelet_obj.rec_lo

[0195] rec_hi = wavelet_obj.rec_hi

[0196] level = len(coeffs)

[0197] signal_len = len(coeffs[0][0])

[0198] current_signal = coeffs[-1][0]

[0199] for j in reversed(range(level)):

[0200] detail = coeffs[j][1]

[0201] upsampling_factor = 2 ** j

[0202] rec_lo_up = np.zeros(len(rec_lo) + (upsampling_factor - 1) *(len(rec_lo) - 1))

[0203] rec_hi_up = np.zeros(len(rec_hi) + (upsampling_factor - 1) *(len(rec_hi) - 1))

[0204] rec_lo_up[::upsampling_factor] = rec_lo

[0205] rec_hi_up[::upsampling_factor] = rec_hi

[0206] pad_len = max(len(rec_lo_up), len(rec_hi_up)) - 1

[0207] padded_approx = np.pad(current_signal, pad_len / / 2, mode='symmetric')

[0208] padded_detail = np.pad(detail, pad_len / / 2, mode='symmetric')

[0209] rec_approx = np.convolve(padded_approx, rec_lo_up, mode='valid')

[0210] rec_detail = np.convolve(padded_detail, rec_hi_up, mode='valid')

[0211] current_signal = (rec_approx + rec_detail)[:signal_len]

[0212] return current_signal

[0213] Step S4: Based on the processed features, perform geological disaster prediction to improve the prediction robustness in scenarios with missing input.

[0214] Example 1

[0215] A robust and lightweight time series prediction method based on bidirectional imputation and geometric attention includes the following steps:

[0216] Step S1: Introduce a dynamic random missing pattern, mask the corresponding feature columns according to a certain ratio, and construct the missing scenario under multiple constraints of time dimension, space dimension and device dimension;

[0217] Step S2: The BRITS model captures the forward and backward temporal dependencies and multivariate correlations of sensor data, identifies failure modes, and fills in missing data.

[0218] Step S3: Input the padded data into the SimpleTM model and use its stationary wavelet transform multi-scale decomposition and geometric product attention mechanism to mine fine-grained temporal features.

[0219] Step S4: Based on the processed features, perform geological disaster prediction to improve the prediction robustness in scenarios with missing input.

[0220] Furthermore, the specific steps of step S1 are as follows:

[0221] Step S1.1: Missing values ​​are mainly in the mode of random missing in the feature dimension. That is, for the input data x_enc, some features are randomly selected in the feature dimension feature_dim to simulate missing values. The missing value ratio is controlled by the parameter mask_ratio.

[0222] Step S1.2: Control the maximum duration of continuous missing data through the freq parameter to simulate different failure types. The feature index to be masked is randomly selected by torch.randperm(), and the position of mask_idx is randomly selected on the feature dimension so that the time distribution of missing data conforms to the time sequence characteristics of real sensor data. Different missing probabilities are set for different feature columns (corresponding to different devices) to ensure that the missing scenario not only has randomness, but also reflects the complex reasons for the missing sensor data in the real environment.

[0223] Furthermore, the specific steps of step S2 are as follows:

[0224] Step S2.1: BRITS generates estimates of missing values ​​through an interpolation network and combines these estimates with a masking vector to determine whether to use the true values ​​or the interpolated results. The specific formula is as follows:

[0225] Input value after interpolation: .in, It is the occlusion vector (0 indicates missing, 1 indicates observed). It is the actual value. These are estimates generated by the interpolation network;

[0226] Interpolation networks generate estimates: .in, It is the hidden state of a bidirectional LSTM, and the interpolation result is generated through a regression layer;

[0227] Step S2.2: A bidirectional LSTM is used to capture the forward and backward dependencies of the time series. A gating mechanism is employed to filter and fuse bidirectional information, accurately handling long-interval continuous missing data (such as rock mass deformation in fractured zones) and random point missing data (such as electromagnetic interference from water-rich strata). The state update formula is as follows:

[0228] Forward LSTM hidden state update: .in, It is a time decay factor that adjusts the weight of historical information. It is the interval between the current time step and the previous observation time step. It is a time constant.

[0229] Cell state update: The cell state of the forward LSTM is updated step by step from the start position to the end position of the sequence, carrying historical information while incorporating current information.

[0230]

[0231] Backward LSTM hidden state update:

[0232] Cell state update (similar to forward, direction adjustment): Information on reverse sequences is filtered, transmitted, and updated through a gating mechanism to capture reverse sequence dependencies.

[0233]

[0234] in, , Forward / backward LSTM time steps The hidden state; For time steps Input after interpolation; , , , , The input gate, forget gate, candidate cell state, cell state, and output gate of the forward / backward LSTM; In a forward / backward LSTM, the input... The weight matrices for the input gate, forget gate, candidate cell state, and output gate; This represents the weight matrix from the hidden state of the previous time step to each gate in the forward / backward LSTM. This represents the bias term for each gate in the forward / backward LSTM; This is the sigmoid activation function used for the gated unit output.

[0235] Step S2.3: BRITS further combines the correlation of other features at the same time step to generate an estimate.

[0236] Feature-based prediction: .in, It is a weight matrix (the diagonal is zero to avoid self-dependency). It is a bias.

[0237] Combining historical data with characteristic features for prediction: Weight Due to time decay factor and occlusion vector Dynamic adjustment.

[0238] Step S2.4: The total loss of BRITS consists of interpolation loss, prediction loss, and consistency loss. The specific formula is as follows:

[0239] Interpolation loss (mean absolute error): ;

[0240] Predicted loss (e.g., binary cross-entropy): ;

[0241] Consistency loss: To ensure consistency between two-way forecasts;

[0242] Total loss: .

[0243] in, It is the predicted output. These are real labels, hyperparameters. Balance the contributions of each part.

[0244] Furthermore, the specific steps of step S3 are as follows:

[0245] Step S3.1: Decomposition of stationary wavelet transform. The input multivariate time-series data is decomposed into multiple scales. Learnable low-pass and high-pass filters are used to generate time-frequency features at different scales, covering the full range of time patterns from short-term fluctuations to long-term trends. Initial conditions. ,in, This is the input signal after linear projection. The iterative decomposition formula is:

[0246]

[0247] in, and These are the upsampled versions of the low-pass and high-pass filters at scale s (by inserting 2). s (to achieve this with -1 zero), and These represent the approximation coefficient and detail coefficient for scale s, respectively;

[0248] Step S3.2, Improvement of the Geometric Product Attention Mechanism. Geometric algebraic operations are introduced on top of the standard self-attention mechanism. Dot products capture scalar similarity, and wedge products capture linear independence and spatial orientation, enabling the model to capture higher-order nonlinear relationships while avoiding high computational costs. The specific formula is as follows:

[0249]

[0250] in, It is a query, key, and value matrix (generated by linear projection). It is the wedge product (outer product), which generates a bivector, capturing the linear independence and geometric relationship between channels. It is a dimensionality reduction function (taking the magnitude of a two-vector or a trainable MLP). It refers to the number of channels.

[0251] Step S3.3: Reconstruction using inverse stationary wavelet transform. Starting from the coarsest scale, the tokens processed at each scale are iteratively integrated, incorporating complex dependency information captured by the attention mechanism. Finally, the prediction result is output by the feedforward network and layer normalization. The core formula of inverse stationary wavelet transform is:

[0252]

[0253] in, It is the coarsest approximation coefficient. and It is a learnable reconstruction filter (low-pass and high-pass).

[0254] Step S3.4: Based on steps S3.1-S3.3, ensure that the model extracts multi-scale features from the original data, models the relationship between features through the attention mechanism, and then restores the processed features to interpretable time series prediction results.

[0255] This invention has the following characteristics:

[0256] This invention utilizes a missing data collaborative processing framework that integrates BRITS bidirectional cyclic interpolation time series and the SimpleTM lightweight prediction model. This framework effectively improves prediction robustness in scenarios with random missing data, captures the temporal dependencies and multivariate sequence correlations of sensor data, and efficiently fills in missing data. Specifically, this is manifested in:

[0257] I. This invention uses a dynamic interpolation mechanism, BRITS, combined with a time decay factor, to adjust the weight of historical information and utilize the implicit correlation between multiple variables to generate filler values, thus avoiding information loss caused by single variable interpolation.

[0258] Second, this invention transforms time series into token sequences of different frequencies through stationary wavelet transform multi-scale decomposition, preserving temporal locality and multi-scale dynamic features, and effectively capturing the unique patterns of variables.

[0259] Third, this invention utilizes a geometric attention mechanism, which integrates scalar similarity and geometric relationships to achieve accurate modeling of complex dynamics of multivariate time series under a lightweight architecture. This overcomes the information limitations of traditional self-attention and reduces the computational burden of the model.

[0260] Fourth, this invention reduces the dependence of the SimpleTM algorithm on the integrity of the data input. The temporal and variable correlations preserved by BRITS can be further amplified by the geometric attention mechanism of SimpleTM, effectively improving the prediction robustness in random missing scenarios and enabling efficient operation even in resource-limited scenarios.

[0261] The above are merely representative embodiments among the many specific applications of this invention, and do not constitute any limitation on the scope of protection of this invention. All technical solutions formed by transformation or equivalent substitution fall within the scope of protection of this invention.

Claims

1. A robust light-weight time series forecasting method based on bidirectional padding and geometric attention, characterized in that, Includes the following steps: Step S1: Introduce a dynamic random missing mode. Input the corresponding feature columns of the data according to the preset ratio mask. Construct missing scenarios under multiple constraints in the time dimension, spatial dimension and device dimension to generate missing data that conforms to the real sensor failure mode. Step S2: Capture the forward and backward temporal dependencies and multivariate correlations of sensor data using the BRITS model, identify failure modes, and fill in missing data; Step S3: Input the padded data into the SimpleTM model, use stationary wavelet transform for multi-scale decomposition and reconstruction, and mine fine-grained temporal features through geometric product attention mechanism. Step S4: Based on the temporal features output in Step 3, perform geological disaster prediction to improve the prediction robustness in scenarios with missing input. The specific steps of step S2 are as follows: Step S2.1: The BRITS model generates estimates of missing values ​​through an interpolation network. And combined with the occlusion vector Decide whether to use the real value or interpolation result The specific formula is as follows: Interpolated input value: ; The interpolation network generates the estimate value: , is the hidden state of the bidirectional LSTM, which generates the interpolation result through a regression layer, where denotes the regression layer operation; Step S2.2: A bidirectional LSTM is used to capture the forward and backward dependencies of the time series. Through forward and backward processing layers, the sequence data is learned from both the forward and backward time dimensions, while integrating contextual features of historical and future information. Forward LSTM hidden state update: in, It is a time decay factor that adjusts the weight of historical information. It is the interval between the current time step and the previous observation time step. It is a time constant; Forward cell state update: Updated according to the natural temporal order of the sequence, through the regulation of forget gates, input gates, and output gates, historical information in the sequence is gradually passed to subsequent positions, recording the forward context of the temporal order, and is responsible for capturing the dependencies between the past and the present. Backward LSTM hidden state update: Backward cell state update: Similar to forward, but the direction is adjusted. Through a gating mechanism, future information in the sequence is passed backward to the previous position, and updates are performed in reverse chronological order of the sequence. It is responsible for capturing the dependencies between the future and the present. in, , Forward / backward LSTM time steps The hidden state; For time steps Input after interpolation; , , , , The input gate, forget gate, candidate cell state, cell state, and output gate of the forward / backward LSTM; In a forward / backward LSTM, the input... The weight matrices for the input gate, forget gate, candidate cell state, and output gate; This represents the weight matrix from the hidden state of the previous time step to each gate in the forward / backward LSTM. This represents the bias term for each gate in the forward / backward LSTM; The sigmoid activation function is used for the output of the gated unit; Step S2.3: The BRITS module combines the correlation of other features at the same time step to generate an estimate. The system dynamically integrates historical data and feature predictions to obtain the final interpolated estimate. : weights by a temporal decay factor and a masking vector are dynamically adjusted, is a weight matrix, is a bias; Step S2.4, Total Loss of BRITS Due to interpolation loss Predicting losses Composed of consistency loss The specific formula is as follows: Interpolation loss: Predicted loss: Consistency loss: Total loss: in, It is the predicted output. These are real labels, hyperparameters. Balancing the contributions of each part The binary cross-entropy loss function is... This indicates that the forward prediction is at time step The output, This indicates backward prediction at time step The output of .

2. The robust light-weight time series forecasting method based on bidirectional padding and geometric attention according to claim 1, characterized in that, The specific steps of step S1 are as follows: Step S1.1: Missing values ​​are mainly in the mode of random missing in the feature dimension. For the input data x_enc, some features are randomly selected in the feature dimension feature_dim to simulate missing values. The missing value ratio is controlled by the parameter mask_ratio. Step S1.2: By controlling the maximum duration of continuous missing data, different failure types are simulated, and the missing probability of each feature column is set differently so that the missing data distribution conforms to the temporal characteristics, spatial rationality, and equipment differences of the sensor data.

3. The robust light-weight time series forecasting method based on bidirectional padding and geometric attention according to claim 2, characterized in that, The specific steps of step S3 are as follows: Step S3.1, performing a stationary wavelet transform multi-scale decomposition on the padded data performing a stationary wavelet transform multi-scale decomposition: in, and These are learnable low-pass / high-pass filters at scale s, representing low-pass and high-pass filtering, respectively. and These represent the approximation coefficient and detail coefficient for scale s, respectively; Step S3.2: Fuse multi-scale features through a geometric product attention mechanism: wherein, is a query, key, value matrix, is a wedge product, generating a bi-vector, capturing linear independence and geometric relations across channels, is a dimensionality reduction function, is a number of channels; Step S3.3: Reconstruct time series features using inverse stationary wavelet transform: Starting from the coarsest scale, the tokens processed at each scale are iteratively integrated, and the complex dependency information captured by the attention mechanism is fused. Finally, the prediction result is output by the feedforward network and layer normalization. The core formula of the inverse stationary wavelet transform is: in, It is the coarsest approximation coefficient. and It is a learnable reconstruction filter. This represents the approximate coefficients at time point t for the (s-1)th scale. This represents the approximate coefficients at time point t+k for the s-th scale. This represents the detail coefficients at time point t+k at the s-th scale. Indicates the filter index. It is a scale parameter; Step S3.4: Based on steps S3.1-S3.3, ensure that the model extracts multi-scale features from the original data, models the relationship between features through the attention mechanism, and then restores the processed features to interpretable time series prediction results.

Images