Time series data trend prediction method based on attention mechanism
By extracting feature sets from time-series data for prediction and segmenting them based on logical coherence rules, combined with sparse Bayesian learning and attention weight optimization, the prediction error problem of incomplete time-series data is solved, and highly robust trend prediction is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- DEDE SHIJIE (NINGBO) TECHNOLOGY CO LTD
- Filing Date
- 2026-02-26
- Publication Date
- 2026-05-19
AI Technical Summary
Existing attention-based time series prediction techniques suffer from several problems when faced with incomplete time series data. These problems include the introduction of human assumption bias during pre-completion operations, the inability of traditional segmentation methods to distinguish logical coherence, and the susceptibility of self-attention mechanisms to being misled by spurious related data, leading to the accumulation of prediction errors and poor robustness.
By extracting the feature set of the original time series data, segmenting it based on logical coherence rules, using a sparse Bayesian learning algorithm to quantify the logical matching degree, constructing an attention weight constraint mask, and iteratively optimizing the weights in the attention mechanism, the pre-completion operation is avoided, and logically consistent segment merging and isolated computation are achieved.
It achieves highly robust time series trend prediction without pre-completion and resistant to incomplete interference, reducing human error and improving the accuracy and interpretability of prediction.
Smart Images
Figure CN122064965A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of data processing technology, and in particular to a method for predicting trends in time-series data based on an attention mechanism. Background Technology
[0002] Time series data trend prediction is a core technology in fields such as industrial operation and maintenance, meteorological monitoring, financial analysis, and medical diagnosis. Transformer-type models based on attention mechanisms, with their ability to capture long-sequence dependencies, have become the mainstream technology for time series prediction. However, existing attention-based time series prediction technologies, when faced with frequently occurring incomplete time series data such as intermittent data collection from industrial sensors, lost frames in remote sensing data, and interrupted medical monitoring signals, strictly follow a fixed process of data pre-completion, feature extraction, attention modeling, and prediction output. This process has an unavoidable core flaw: firstly, the pre-completion operation inevitably introduces biases from human assumptions, resulting in spurious data being filled in. The data is learned and amplified by the attention mechanism, ultimately leading to the accumulation of prediction errors and even the complete deviation of the trend from the actual situation. Secondly, traditional segmentation methods often use sliding windows of fixed time lengths, without distinguishing whether the trend logic within the window is coherent. This forces the input of noisy data with broken logic into the model, directly misleading the weight allocation of the attention mechanism, and has extremely poor adaptability to incomplete data. Finally, existing self-attention mechanisms calculate weights based solely on data correlation, which easily assigns high weights to complete data with broken logic but false correlation with the prediction target, failing to effectively focus on the trend logic, resulting in poor robustness and insufficient interpretability of the prediction results. Summary of the Invention
[0003] In view of this, the present invention proposes a time series data trend prediction method based on attention mechanism, which can achieve time series trend prediction without pre-completion, resisting incomplete interference and with high robustness.
[0004] The technical solution of this invention is implemented as follows:
[0005] A time-series data trend prediction method based on attention mechanism includes the following steps:
[0006] Step S1: Obtain the original time series data and extract the trend slope features, periodic fluctuation features, numerical mutation features, and inter-feature correlation features from the original time series data as a feature set;
[0007] Step S2: Based on preset logical coherence rules, the feature set is segmented to obtain several data segments, including logically coherent segments and logically discontinuous segments.
[0008] Step S3: Using the sparse Bayesian learning algorithm, perform logical matching metric calculation on logical coherence segments and logical break segments to obtain the logical matching degree scores of logical coherence segments and logical break segments.
[0009] Step S4: Construct an attention weight constraint mask based on the logical matching score and embed it into the QKV generation process of the attention mechanism to initially constrain the attention weights of logically coherent segments and logically broken segments.
[0010] Step S5: Divide the attention calculation window into logically coherent segments as the basic unit, and perform iterative calculation and optimization of attention weights;
[0011] Step S6: Based on the optimized attention weights, output the trend prediction results of the original time series data through the attention mechanism.
[0012] Preferably, step S1 includes the following steps:
[0013] Step S11: Obtain raw time-series data containing missing values through sensors, acquisition terminals, or databases deployed in the monitoring scenario;
[0014] Step S12: Do not perform pre-completion operation on the original time series data, and extract the trend slope features, periodic fluctuation features, numerical mutation features and inter-feature correlation features of the original time series data as a feature set;
[0015] Step S13: Normalize the feature set while maintaining the distribution of missing values and the temporal order of the feature set.
[0016] Preferably, step S2 includes the following specific steps:
[0017] Step S21: Construct a logically coherent rule that includes basic time-series continuity rules, trend feature consistency rules, and cross-feature association matching rules;
[0018] Step S22: Identify the trend change boundary of the feature set arranged in time order into a full time series using the mutation point detection algorithm, and divide the feature set into several initial candidate segments aligned with the trend change boundary. Set a sliding window for each initial candidate segment, and calculate the comprehensive coherence score of each sub-window in the initial candidate segment based on the logical coherence rule.
[0019] Step S23: When the difference in the overall coherence score of adjacent sub-windows in the same initial candidate segment exceeds the preset segmentation threshold, the initial candidate segment is split based on the sub-window boundary. If the average difference in the overall coherence score of adjacent initial candidate segments is lower than the preset merging threshold, the adjacent initial candidate segments are merged to obtain several temporary segments.
[0020] Step S24: Calculate the average coherence score of each temporary segment and compare it with the preset coherence threshold. When the average coherence score is greater than the preset coherence threshold, the temporary segment is output as a logically coherent segment. When the average coherence score is less than the preset coherence threshold, the temporary segment is output as a logically broken segment.
[0021] Preferably, the logical coherence rules include:
[0022] Basic temporal continuity rule: The numerical jump amplitude of adjacent sampling points in the time series shall not exceed the preset global data standard deviation threshold;
[0023] Trend feature consistency rule: The fluctuation range of the trend slope within the sub-window does not exceed the preset slope threshold, and the phase matching degree of periodic fluctuation is not lower than the preset matching degree;
[0024] Cross-feature association matching rule: The fluctuation range of the Pearson correlation coefficient between vibration feature and temperature feature does not exceed the preset correlation threshold.
[0025] Preferably, step S3 includes the following specific steps:
[0026] Step S31: Sparsify the feature data of the feature sets corresponding to logically coherent segments and logically discontinuous segments using the sparse Bayesian learning algorithm.
[0027] Step S32: Based on the posterior probability estimation of the sparse Bayesian learning algorithm, calculate the logical matching score between each logically coherent segment and its preceding and following adjacent data segments.
[0028] Step S33: The logical matching score of the logically broken segment is uniformly set to a preset base score, which is lower than the lowest logical matching score of all logically coherent segments.
[0029] Preferably, the specific steps of step S32 are as follows:
[0030] A Bayesian probability distribution model is constructed using the feature data of logically coherent segments as input.
[0031] Extract the core features of adjacent data segments before and after the target logical coherent segment, and input them into the constructed Bayesian probability distribution model;
[0032] The posterior probability estimation using the sparse Bayesian learning algorithm is used to calculate the fit of trend features between the target logical coherent segment and adjacent data segments.
[0033] After normalizing the fit of the trend features, the logical matching score of the target logical coherence segment is obtained.
[0034] Preferably, step S4 includes the following specific steps:
[0035] Step S41: Construct an attention weight constraint mask whose matrix dimension is consistent with the number of features and temporal length of the feature set;
[0036] Step S42: Set the upper limit of attention weight for logically coherent segments based on the logical matching score, and suppress the upper limit of attention weight for logically fragmented segments based on the preset suppression threshold.
[0037] Step S43: In the QKV generation process of the attention mechanism, the attention weight constraint mask is multiplied element-wise with the QKV matrix.
[0038] Preferably, step S5 includes the following specific steps:
[0039] Step S51: Divide the attention calculation window into basic units based on logical coherence segments, and merge adjacent logical coherence segments into one attention calculation window.
[0040] Step S52: During iterative calculation, first calculate the self-attention weights within each attention calculation window, and then calculate the feature association weights between windows through cross-attention calculation.
[0041] Step S53: Use the gradient descent algorithm to optimize the weights with the goal of minimizing the deviation between the logical matching score and the attention weights;
[0042] Step S54: Stop iterating when the number of consecutive iterations is preset, the change in attention weight is lower than a preset change threshold, or the iteration error is lower than a preset error threshold.
[0043] Preferably, the specific steps of step S53 are as follows:
[0044] Obtain the logical matching score, initial attention weight matrix, and attention weight constraint mask for each data segment;
[0045] With minimizing the deviation between the logical matching score and the actual attention weight as the core, an objective loss function for weight optimization is constructed;
[0046] Based on the gradient descent algorithm, the gradient value of the target loss function with respect to the initial attention weight matrix is calculated;
[0047] According to the preset iterative learning rate, the attention weights are updated in a single iteration based on the gradient value;
[0048] Constraint checks are performed on the attention weights after iterative updates based on the upper limit set by the attention weight constraint mask;
[0049] Normalize the attention weights that pass the constraint verification and calculate the actual loss value of the target loss function after this iteration to complete a single weight optimization iteration.
[0050] Preferably, step S6 includes the following specific steps:
[0051] Step S61: The optimized attention weights are weighted and fused with the temporal features of each logical coherent segment. The trend of the temporal interval corresponding to the logical break segment is derived based on the prediction results of the adjacent logical coherent segments.
[0052] Step S62: Based on the weighted fusion results and derivation results, output the time series data trend prediction results, which include the predicted values, trend direction and confidence level for the entire time series interval.
[0053] Compared with the prior art, the beneficial effects of the present invention are:
[0054] The attention-based time-series data trend prediction method of this invention does not perform pre-completion processing after acquiring the original time-series data. Instead, it directly extracts the feature set from the original time-series data. Then, under the preset logical coherence rules, the feature set is divided into several logically coherent segments and logically discontinuous segments. The logical matching degree scores of the logically coherent segments and logically discontinuous segments are quantified by a sparse Bayesian learning algorithm. An attention weight constraint mask is constructed based on the logical matching degree scores, so that the logical matching degree scores are transformed into hard constraints at the bottom layer of the attention mechanism. At the same time, in the iterative calculation and optimization of attention weights, the traditional fixed time window calculation mode is not used. Instead, logically coherent segments are used as basic units to divide the attention calculation window, realizing the merging calculation of logically consistent segments and the isolation calculation of logically inconsistent segments. This allows the attention calculation to be completely centered on the trend logic. Finally, based on the optimized attention weights, the trend prediction result of the original time-series data can be output through the attention mechanism without using any pre-completion spurious data. This achieves time-series trend prediction without pre-completion, resistant to incomplete interference, and highly robust. Attached Figure Description
[0055] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only preferred embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0056] Figure 1 This is a flowchart of the attention mechanism-based time-series data trend prediction method of the present invention;
[0057] Figure 2 This is a flowchart of step S1 of the attention mechanism-based time-series data trend prediction method of the present invention;
[0058] Figure 3 This is a flowchart of step S2 of the attention mechanism-based time-series data trend prediction method of the present invention;
[0059] Figure 4 This is a flowchart of step S3 in the attention mechanism-based time-series data trend prediction method of the present invention;
[0060] Figure 5 This is a flowchart of step S4 of the attention mechanism-based time series data trend prediction method of the present invention;
[0061] Figure 6 This is a flowchart of step S5 of the attention mechanism-based time-series data trend prediction method of the present invention;
[0062] Figure 7 This is a flowchart of step S6 of the attention-based time-series data trend prediction method of the present invention. Detailed Implementation
[0063] To better understand the technical content of this invention, a specific embodiment is provided below, and the invention will be further described in conjunction with the accompanying drawings.
[0064] See Figures 1 to 7 The present invention provides a time-series data trend prediction method based on an attention mechanism, comprising the following steps:
[0065] Step S1: Obtain the original time series data and extract the trend slope features, periodic fluctuation features, numerical mutation features, and inter-feature correlation features from the original time series data as a feature set. Specific steps include:
[0066] Step S11: Obtain raw time-series data containing missing values through sensors, acquisition terminals, or databases deployed in the monitoring scenario;
[0067] Step S12: Do not perform pre-completion operation on the original time series data, and extract the trend slope features, periodic fluctuation features, numerical mutation features and inter-feature correlation features of the original time series data as a feature set;
[0068] Step S13: Normalize the feature set, with a normalization range of [0,1], while maintaining the distribution of missing values and the temporal order in the feature set.
[0069] This invention presents an attention-based time-series data trend prediction method for predicting trends in raw time-series data collected from various monitoring scenarios, such as industrial sensing, satellite remote sensing, and medical monitoring. Traditional time-series data trend prediction methods often suffer from incomplete data due to sensor malfunctions, data loss, and signal interruptions. Conventional methods require pre-completion of this incomplete data for effective trend prediction, which introduces biases from human assumptions, leading to deviations in subsequent trend predictions from reality. Therefore, this invention abandons pre-completion after acquiring the raw time-series data and directly extracts a feature set from it. Simultaneously, when normalizing the feature values, it preserves the true time-series distribution of the original data. The feature set includes trend slope features, periodic fluctuation features, numerical mutation features, and inter-feature correlation features—core features directly and strongly correlated with trend logic. This provides authentic, unaltered basic data for subsequent logical coherence judgments, fundamentally eliminating human error caused by pre-completion.
[0070] Step S2: Based on preset logical coherence rules, the feature set is segmented to obtain several data segments, including logically coherent segments and logically discontinuous segments. Specific steps include:
[0071] Step S21: Construct a logically coherent rule system that includes basic time-series continuity rules, trend feature consistency rules, and cross-feature association matching rules, wherein:
[0072] Basic temporal continuity rule: The numerical jump amplitude of adjacent sampling points in the time series shall not exceed the preset global data standard deviation threshold, which is taken as plus or minus 3 times the global data standard deviation;
[0073] Trend feature consistency rule: The fluctuation range of the trend slope within the sub-window does not exceed the preset slope threshold, and the phase matching degree of periodic fluctuation is not lower than the preset matching degree;
[0074] Cross-feature association matching rule: The fluctuation range of the Pearson correlation coefficient between vibration features and temperature features shall not exceed the preset correlation threshold;
[0075] Step S22: Identify the trend change boundary of the feature set arranged in time order into a full time series using the mutation point detection algorithm, and divide the feature set into several initial candidate segments aligned with the trend change boundary. Set a sliding window for each initial candidate segment, and calculate the comprehensive coherence score of each sub-window in the initial candidate segment based on the logical coherence rule.
[0076] Step S23: When the difference in the overall coherence score of adjacent sub-windows in the same initial candidate segment exceeds the preset segmentation threshold, the initial candidate segment is split based on the sub-window boundary. If the average difference in the overall coherence score of adjacent initial candidate segments is lower than the preset merging threshold, the adjacent initial candidate segments are merged to obtain several temporary segments.
[0077] Step S24: Calculate the average coherence score of each temporary segment and compare it with the preset coherence threshold. When the average coherence score is greater than the preset coherence threshold, the temporary segment is output as a logically coherent segment. When the average coherence score is less than the preset coherence threshold, the temporary segment is output as a logically broken segment.
[0078] After obtaining the feature set, it needs to be segmented according to the preset logical coherence rules, dividing it into several data segments. The data segments include two types: logically coherent segments and logically discontinuous segments. First, global trend coarse segmentation is performed on the normalized feature set. The trend change boundary of the full time series is identified by the improved Petitt mutation point detection algorithm, and the full time series is divided into several initial candidate segments aligned with the trend change boundary. The missing value distribution of the original time series data is not modified throughout the process. Then, for each initial candidate segment, a local fine traversal is performed using a variable step-size sliding window with adaptive residual rate. The window step size is negatively correlated with the data residual rate in the initial candidate segment. During the traversal, the three-level logical coherence comprehensive score corresponding to each sub-window is calculated simultaneously, and the scores of all sub-windows are calculated and summarized.
[0079] Then, based on the overall coherence score of each sub-window obtained by local fine-grained traversal, dynamic splitting and merging are performed on the initial candidate segments. If the difference in the overall coherence score of adjacent sub-windows within the same initial candidate segment exceeds the preset segmentation threshold, it indicates that the logical coherence of the two sub-windows is significantly different. In this case, boundary splitting is performed on the initial candidate segment, and the overall coherence score of each sub-window is retained after splitting. The split data segment is marked as a temporary segment. If the average difference in the overall coherence score of adjacent initial candidate segments is lower than the preset merging threshold, it indicates that the logical coherence of the two initial candidate segments tends to be consistent. In this case, cross-segment merging is performed, and the merged segment is marked as a temporary segment.
[0080] For each temporary segment, calculate its average coherence score and compare it with a preset coherence threshold. If it is greater than the preset coherence threshold, the temporary segment is output as a logically coherent segment; if it is less than the preset coherence threshold, the temporary segment is output as a logically broken segment.
[0081] Step S3: Using a sparse Bayesian learning algorithm, perform a metric calculation of logical matching for logically coherent segments and logically discontinuous segments to obtain logical matching scores. Specific steps include:
[0082] Step S31: Sparsify the feature data of the feature sets corresponding to logically coherent segments and logically discontinuous segments using the sparse Bayesian learning algorithm.
[0083] Step S32: Based on the posterior probability estimation using the sparse Bayesian learning algorithm, calculate the logical matching score between each logically coherent segment and its preceding and following adjacent data segments. The specific steps are as follows:
[0084] A Bayesian probability distribution model is constructed using the feature data of logically coherent segments as input.
[0085] Extract the core features of adjacent data segments before and after the target logical coherent segment, and input them into the constructed Bayesian probability distribution model;
[0086] The posterior probability estimation using the sparse Bayesian learning algorithm is used to calculate the fit of trend features between the target logical coherent segment and adjacent data segments.
[0087] After normalizing the fit of the trend features, the logical matching score of the target logical coherence segment is obtained.
[0088] Step S33: The logical matching score of the logically broken segment is uniformly set to a preset base score, which is lower than the lowest logical matching score of all logically coherent segments.
[0089] The labeled information and feature data of logically coherent segments and logically discontinuous segments are input into a sparse Bayesian learning algorithm to construct a logical matching metric model. The feature data is sparsified using the sparse Bayesian learning algorithm to remove redundant features that do not contribute to the logical matching degree calculation, retaining core features. Then, based on the posterior probability estimation of the sparse Bayesian algorithm, the logical matching degree score of each logically coherent segment and its preceding and following adjacent data segments is calculated, with the score range being [0,1]. The specific calculation process is as follows: after constructing a Bayesian probability distribution model using the feature data of the logically coherent segments as input, the core features of the preceding and following adjacent data segments are simultaneously input into the Bayesian probability distribution model, and the sparse Bayesian learning algorithm calculates the logical matching degree score. The Yeesian learning algorithm calculates the trend feature fit between the target logically coherent segment and its adjacent data segments. Then, it normalizes the trend feature fit and uses it as the logical matching score of the logically coherent segment. For logically discontinuous segments, their logical matching scores are directly and uniformly set to a preset base score, and the base score must be lower than the lowest logical matching score of all logically coherent segments. This clearly distinguishes the predictive value of logically coherent segments from logically discontinuous segments, providing a clear quantitative basis for subsequent attention weight constraints. At the same time, by setting a base score lower than the lowest score of logically coherent segments, the attention weight of logically discontinuous segments can be forcibly suppressed, avoiding their meaningless features from interfering with the overall time series trend prediction.
[0090] Step S4: Construct an attention weight constraint mask based on the logical matching score and embed it into the QKV generation process of the attention mechanism. Initial constraints are applied to the attention weights of logically coherent segments and logically discontinuous segments. Specific steps include:
[0091] Step S41: Construct an attention weight constraint mask whose matrix dimension is consistent with the number of features and temporal length of the feature set;
[0092] Step S42: Set the upper limit of attention weight for logically coherent segments based on the logical matching score, and suppress the upper limit of attention weight for logically fragmented segments based on the preset suppression threshold.
[0093] Step S43: In the QKV generation process of the attention mechanism, the attention weight constraint mask is multiplied element-wise with the QKV matrix.
[0094] The attention weight constraint mask is a two-dimensional matrix, with the matrix dimension consistent with the number of features and the time sequence length of the feature set. Then, an upper limit for the attention weight of logically coherent segments is set, where the higher the logical matching score, the higher the corresponding upper limit of attention weight, and the two have a positive linear correlation. Then, in the QKV matrix generation stage of the attention mechanism, the attention weight constraint mask is multiplied element-wise with the QKV matrix to force the attention score of each data segment to not exceed the corresponding upper limit of weight. For logically broken segments, the corresponding mask elements are aligned to suppress the weight, compressing its upper limit of attention weight to a preset suppression threshold, thereby achieving attention weight suppression of logically broken segments.
[0095] Step S5: Divide the attention calculation window into basic units based on logically coherent segments, and perform iterative calculation and optimization of attention weights. Specific steps include:
[0096] Step S51: Divide the attention calculation window into basic units based on logical coherence segments, and merge adjacent logical coherence segments into one attention calculation window.
[0097] Step S52: During iterative calculation, first calculate the self-attention weights within each attention calculation window, and then calculate the feature association weights between windows through cross-attention calculation.
[0098] Step S53: Employ the gradient descent algorithm to optimize the weights with the goal of minimizing the deviation between the logical matching score and the attention weights. The specific steps are as follows:
[0099] The algorithm acquires the logical matching score, initial attention weight matrix, and attention weight constraint mask for each data segment. With the core objective of minimizing the deviation between the logical matching score and the actual attention weights, it constructs a target loss function for weight optimization. Based on the gradient descent algorithm, it calculates the gradient of the target loss function with respect to the initial attention weight matrix. Following a preset learning rate, it performs a single-iteration update of the attention weights based on the gradient values. It then performs constraint verification on the updated attention weights based on the upper limit set by the attention weight constraint mask, ensuring that all weight values do not exceed the upper limit set by the mask. The attention weights that pass the constraint verification are then normalized to meet the probability distribution constraints of the attention weights. Finally, it calculates the actual loss value of the target loss function after this iteration, completing a single weight optimization iteration.
[0100] Step S54: Stop iterating when the number of consecutive iterations is preset, the change in attention weight is lower than a preset change threshold, or the iteration error is lower than a preset error threshold.
[0101] The attention window is not divided based on a fixed time length, but only on logically coherent segments as the basic unit. Adjacent logically coherent segments can be merged into one attention calculation window. Iterative calculation and optimization of weights are performed based on the divided attention calculation windows. During iterative calculation, self-attention weights and feature association weights are calculated sequentially. During optimization, the Adam gradient descent algorithm is used, with the optimization objective being to minimize the deviation between the logical matching score and the actual attention weights, while ensuring the normalization constraint of the attention weights. Iteration stops when the iteration termination condition is met, and the optimized attention weights are obtained. By optimizing the attention weight allocation, it is ensured that the attention mechanism prioritizes the effective features of logically coherent segments, conforming to the logical matching constraint requirements. At the same time, through iterative optimization and constraint verification, the weights are prevented from exceeding the preset upper limit or deviating from the trend logic, thereby improving the accuracy and stability of attention calculation and laying the foundation for subsequent predictions.
[0102] Step S6: Based on the optimized attention weights, output the trend prediction results of the original time series data through the attention mechanism. Specific steps include:
[0103] Step S61: The optimized attention weights are weighted and fused with the temporal features of each logical coherent segment. The trend of the temporal interval corresponding to the logical break segment is derived based on the prediction results of the adjacent logical coherent segments.
[0104] Step S62: Based on the weighted fusion results and derivation results, output the time series data trend prediction results, which include the predicted values, trend direction and confidence level for the entire time series interval.
[0105] The optimized attention weights are weighted and fused with the temporal features of each logically coherent segment. Logically broken segments do not participate in prediction calculations independently; the trend of their corresponding temporal intervals is derived from the prediction results of adjacent logically coherent segments. Finally, based on the weighted fusion result and the derivation result, the temporal data trend prediction result can be output. Unlike the traditional method of first completing incomplete data and then making temporal predictions, this invention is based on the principle of logic taking precedence over data completeness. It uses a sparse Bayesian learning algorithm for quantification and then deeply embeds the attention mechanism into the entire computational process. This achieves temporal trend prediction without pre-completion, resistant to incomplete interference, and highly robust. It fundamentally solves the problems of human error introduced by pre-completion in traditional methods, the attention mechanism being misled by false associations, and short-term noise interfering with trend judgment.
[0106] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.
Claims
1. A time-series data trend prediction method based on attention mechanism, characterized in that, Includes the following steps: Step S1: Obtain the original time series data and extract the trend slope features, periodic fluctuation features, numerical mutation features, and inter-feature correlation features from the original time series data as a feature set; Step S2: Based on preset logical coherence rules, the feature set is segmented to obtain several data segments, including logically coherent segments and logically discontinuous segments. Step S3: Using the sparse Bayesian learning algorithm, perform logical matching metric calculation on logical coherence segments and logical break segments to obtain the logical matching degree scores of logical coherence segments and logical break segments. Step S4: Construct an attention weight constraint mask based on the logical matching score and embed it into the QKV generation process of the attention mechanism to initially constrain the attention weights of logically coherent segments and logically broken segments. Step S5: Divide the attention calculation window into logically coherent segments as the basic unit, and perform iterative calculation and optimization of attention weights; Step S6: Based on the optimized attention weights, output the trend prediction results of the original time series data through the attention mechanism.
2. The time-series data trend prediction method based on attention mechanism according to claim 1, characterized in that, The specific steps of step S1 include: Step S11: Obtain raw time-series data containing missing values through sensors, acquisition terminals, or databases deployed in the monitoring scenario; Step S12: Do not perform pre-completion operation on the original time series data, and extract the trend slope features, periodic fluctuation features, numerical mutation features and inter-feature correlation features of the original time series data as a feature set; Step S13: Normalize the feature set while maintaining the distribution of missing values and the temporal order of the feature set.
3. The time-series data trend prediction method based on attention mechanism according to claim 1, characterized in that, The specific steps of step S2 include: Step S21: Construct a logically coherent rule that includes basic time-series continuity rules, trend feature consistency rules, and cross-feature association matching rules; Step S22: Identify the trend change boundary of the feature set arranged in time order into a full time series using the mutation point detection algorithm, and divide the feature set into several initial candidate segments aligned with the trend change boundary. Set a sliding window for each initial candidate segment, and calculate the comprehensive coherence score of each sub-window in the initial candidate segment based on the logical coherence rule. Step S23: When the difference in the overall coherence score of adjacent sub-windows in the same initial candidate segment exceeds the preset segmentation threshold, the initial candidate segment is split based on the sub-window boundary. If the average difference in the overall coherence score of adjacent initial candidate segments is lower than the preset merging threshold, the adjacent initial candidate segments are merged to obtain several temporary segments. Step S24: Calculate the average coherence score of each temporary segment and compare it with the preset coherence threshold. When the average coherence score is greater than the preset coherence threshold, the temporary segment is output as a logically coherent segment. When the average coherence score is less than the preset coherence threshold, the temporary segment is output as a logically broken segment.
4. The time-series data trend prediction method based on attention mechanism according to claim 3, characterized in that, The logical coherence rules include: Basic temporal continuity rule: The numerical jump amplitude of adjacent sampling points in the time series shall not exceed the preset global data standard deviation threshold; Trend feature consistency rule: The fluctuation range of the trend slope within the sub-window does not exceed the preset slope threshold, and the phase matching degree of periodic fluctuation is not lower than the preset matching degree; Cross-feature association matching rule: The fluctuation range of the Pearson correlation coefficient between vibration feature and temperature feature does not exceed the preset correlation threshold.
5. The time-series data trend prediction method based on attention mechanism according to claim 1, characterized in that, The specific steps of step S3 include: Step S31: Sparsify the feature data of the feature sets corresponding to logically coherent segments and logically discontinuous segments using the sparse Bayesian learning algorithm. Step S32: Based on the posterior probability estimation of the sparse Bayesian learning algorithm, calculate the logical matching score between each logically coherent segment and its preceding and following adjacent data segments. Step S33: The logical matching score of the logically broken segment is uniformly set to a preset base score, which is lower than the lowest logical matching score of all logically coherent segments.
6. The time-series data trend prediction method based on attention mechanism according to claim 5, characterized in that, The specific steps of step S32 are as follows: A Bayesian probability distribution model is constructed using the feature data of logically coherent segments as input. Extract the core features of adjacent data segments before and after the target logical coherent segment, and input them into the constructed Bayesian probability distribution model; The posterior probability estimation using the sparse Bayesian learning algorithm is used to calculate the fit of trend features between the target logical coherent segment and adjacent data segments. After normalizing the fit of the trend features, the logical matching score of the target logical coherence segment is obtained.
7. The time-series data trend prediction method based on attention mechanism according to claim 1, characterized in that, The specific steps of step S4 include: Step S41: Construct an attention weight constraint mask whose matrix dimension is consistent with the number of features and temporal length of the feature set; Step S42: Set the upper limit of attention weight for logically coherent segments based on the logical matching score, and suppress the upper limit of attention weight for logically fragmented segments based on the preset suppression threshold. Step S43: In the QKV generation process of the attention mechanism, the attention weight constraint mask is multiplied element-wise with the QKV matrix.
8. The time-series data trend prediction method based on attention mechanism according to claim 1, characterized in that, The specific steps of step S5 include: Step S51: Divide the attention calculation window into basic units based on logical coherence segments, and merge adjacent logical coherence segments into one attention calculation window. Step S52: During iterative calculation, first calculate the self-attention weights within each attention calculation window, and then calculate the feature association weights between windows through cross-attention calculation. Step S53: Use the gradient descent algorithm to optimize the weights with the goal of minimizing the deviation between the logical matching score and the attention weights; Step S54: Stop iterating when the number of consecutive iterations is preset, the change in attention weight is lower than a preset change threshold, or the iteration error is lower than a preset error threshold.
9. The time-series data trend prediction method based on attention mechanism according to claim 8, characterized in that, The specific steps of step S53 are as follows: Obtain the logical matching score, initial attention weight matrix, and attention weight constraint mask for each data segment; With minimizing the deviation between the logical matching score and the actual attention weight as the core, an objective loss function for weight optimization is constructed; Based on the gradient descent algorithm, the gradient value of the target loss function with respect to the initial attention weight matrix is calculated; According to the preset iterative learning rate, the attention weights are updated in a single iteration based on the gradient value; Constraint checks are performed on the attention weights after iterative updates based on the upper limit set by the attention weight constraint mask; Normalize the attention weights that pass the constraint verification and calculate the actual loss value of the target loss function after this iteration to complete a single weight optimization iteration.
10. The time-series data trend prediction method based on attention mechanism according to claim 1, characterized in that, The specific steps of step S6 include: Step S61: The optimized attention weights are weighted and fused with the temporal features of each logical coherent segment. The trend of the temporal interval corresponding to the logical break segment is derived based on the prediction results of the adjacent logical coherent segments. Step S62: Based on the weighted fusion results and derivation results, output the time series data trend prediction results, which include the predicted values, trend direction and confidence level for the entire time series interval.