An industrial time series prediction method fusing granger causality and frequency domain features

Abstract

The application discloses an industrial time series prediction method fusing Granger causality and frequency domain characteristics, and belongs to the technical field of industrial process parameter prediction and traditional Chinese medicine intelligent manufacturing. The method carries out Granger causality test on each variable of a traditional Chinese medicine extraction process to obtain a Granger causality matrix, and then divides an input sequence into multiple segments: a frequency domain branch obtains frequency domain characteristics based on original segments; a time domain main part decouples the input sequence into a trend component and a shape component, and performs variable dimension self-attention calculation based on the Granger causality matrix on the last segment of the shape component; the trend component is re-involved in segment dimension attention operation through a value path, and finally, time domain characteristics are obtained; then, the time domain characteristics and the frequency domain characteristics are fused, linear projection and inverse normalization are performed to obtain multi-step prediction results. The application overcomes the insufficiency of existing block prediction models in period component expression and the lack of prior constraints in cross-variable dependence, and improves the prediction accuracy of key parameters of an industrial process.

Description

Technical Field

[0001] This invention relates to an industrial time series prediction method that integrates Granger causality and frequency domain features, belonging to the fields of industrial process parameter prediction and intelligent manufacturing of traditional Chinese medicine. Background Technology

[0002] Extraction of traditional Chinese medicine is the first and core process in the production of prepared Chinese medicines. The temperature of the liquid in the extraction tank directly determines the degree of dissolution of active ingredients and the retention level of heat-sensitive components. Taking honeysuckle extraction as an example, its active ingredients include chlorogenic acid and luteolin, which are quite sensitive to temperature: when the temperature rises, chlorogenic acid and other components will decompose and lose their activity due to heat; when the temperature is too low, the solvent's ability to penetrate the medicinal material decreases, making it difficult to fully extract the active ingredients. Therefore, if the trend of temperature change in the liquid inside the tank can be predicted in advance, operators can be guided to adjust the steam valves in a timely manner to ensure the stable operation of the extraction process.

[0003] The honeysuckle extraction process operates in a cyclical manner, following the sequence of feeding, water injection, heating, heat preservation extraction, and liquid discharge. A single extraction cycle involves multiple stages, with significant differences in the level and fluctuation patterns of the temperature curves at each stage. Multiple extraction tanks typically operate in parallel within the workshop, with a dense network of pipelines and numerous sensors and valves. The tank temperature is influenced by dozens of upstream and downstream variables, including the opening of the steam regulating valve, the steam flow in and out of the upper and lower jackets, solvent feeding, and the operation of the cooling circulating water. These variables include both continuous physical quantities and Boolean valve signals that only take two states (0 and 1). Significant nonlinear coupling and hysteresis effects exist between these variables. Combined with equipment heat transfer inertia, valve adjustment delays, and measurement noise, the tank liquid temperature exhibits strong non-stationarity. Traditional physical sensor monitoring combined with manual adjustment or simple PID control suffers from lag and insufficient accuracy under these conditions, making it difficult to meet the requirements of refined production.

[0004] Early deep learning methods applied to industrial parameter prediction mainly consisted of recurrent neural networks and their variants. For example, Long Short-Term Memory (LSTM) networks, as one of the variants, alleviated the gradient vanishing problem in long sequence training through gating mechanisms, and the bidirectional structure further took into account information from both the preceding and following directions of the sequence. However, such networks rely on recursive information transmission step-by-step, making efficient parallelism difficult and leading to error accumulation on long sequences. Especially when temperatures change rapidly, the information transmission method of the recursive structure causes a significant lag in the prediction curve. The introduction of self-attention mechanisms has changed this situation, and numerous subsequent studies have optimized the efficiency of self-attention long-term modeling from different perspectives: Informer uses ProbSparse sparse attention to reduce the computational cost of long sequences; PatchTST divides the sequence into several segments and calculates attention on a segment-by-segment basis, reducing the size of the attention matrix while preserving local temporal details; iTransformer inverts the input tensor in the variable dimension, treating the entire sequence of each variable as an independent label, thereby modeling coupling relationships in the variable dimension; FreTS transforms the signal into the frequency domain through discrete Fourier transform, mapping the real and imaginary parts of the frequency components respectively; TimeMixer models trend and seasonal components at different time resolutions through multi-scale downsampling.

[0005] To address the interference of amplitude differences between segments on attention modeling (i.e., the attention mechanism preferentially matches segments with similar numerical values, but the overall baseline of each segment is different, thus causing bias), the PMDformer model proposes a block mean decoupling method. This method points out that in non-stationary sequences, the absolute level differences between segments are significant. Directly calculating attention based on level differences can mask the true shape similarity of segments, causing attention to favor segments with similar levels rather than similar shapes. Furthermore, existing segment-level normalization, which subtracts the mean and divides by the standard deviation, compresses the original amplitude of the segment and destroys its inherent shape. Therefore, this method only subtracts the segment mean while retaining the amplitude, separating the segment into a mean component carrying the trend and a residual component carrying the local shape. It also incorporates trend recovery attention to re-inject the mean via values, and proximal variable attention to restrict cross-variable modeling to the segment closest to the prediction time.

[0006] Although the block average decoupling approach has solved the interference problem of amplitude differences between segments to some extent, there are still two shortcomings when applying this approach to the temperature prediction scenario of traditional Chinese medicine extraction process.

[0007] First, such methods model entirely in the time domain. The heating and holding stages of the extraction process are controlled by the on / off state of the steam valve, and the temperature sequence contains periodic components and multi-frequency characteristics determined by the control rhythm. Pure time-domain attention requires indirect fitting of these periodic patterns through multi-layer stacking, which is not direct enough in expressing the periodic components and is prone to error accumulation in multi-step prediction.

[0008] Secondly, the attention weights of such methods in the variable dimension are calculated entirely from data similarity, without introducing prior constraints regarding the actual driving relationships between variables. The extraction process involves complex physical and thermodynamic processes, with directional driving relationships existing between variables such as temperature, pressure, and flow rate. However, most existing self-attention models do not fully consider this causal influence and its sustained effect over time when handling multivariate inputs. In the extracted process data, a significant proportion of the more than fifty variables are valve opening and closing signals. These signals remain constant over long periods, and relying solely on data similarity easily assigns high weights to statistically coincidentally correlated variable pairs that have no physical driving relationship. Summary of the Invention

[0009] To address the problems of existing technologies and further improve the temperature prediction accuracy of traditional Chinese medicine extraction processes, this invention proposes an industrial time series prediction method that integrates Granger causality and frequency domain features. On the one hand, Granger causality tests are used to pre-quantify the temporal driving relationships between variables, injecting them as prior knowledge into cross-variable attention. On the other hand, the frequency domain branch is positioned after segmentation and before mean decoupling, allowing Fourier transform to be applied to the original segments that retain complete amplitude information. The amplitude spectrum can simultaneously reflect the absolute level and fluctuation intensity of the segments. This bypass-style fusion method enables frequency domain information to form an explicit representation in the network's feature paths. Combining these two improvements with mechanisms such as block mean decoupling and trend recovery attention improves the temperature prediction accuracy for traditional Chinese medicine extraction processes, providing support for the stable operation of the extraction process and the consistency of product quality.

[0010] The inventor previously applied for a patent for the CauDformer model (Chinese invention patent publication number CN121010047A) for the honeysuckle three-effect concentration process. This model adopts the variable-dimensional modeling structure of iTransformer, arranging variables according to the physical causal chain order based on the process flow and expert experience. A lower triangular causal mask is applied to the self-attention of the variable dimension, ensuring that each variable only focuses on variables ranked before it. Simultaneously, a frequency-domain auxiliary loss term is constructed using real-number fast Fourier transform, and combined with the time-domain reconstruction loss using adaptive weights. This method achieved good results in predicting concentration parameters, but its causal structure relies on manual sorting based on expert experience, making it difficult to directly construct in extraction processes with dozens of variables and unclear physical chains. Furthermore, its causal constraints use a hard mask, which cannot be adjusted once set; and its frequency-domain information only affects the loss function, failing to form an explicit representation in the network's feature pathways.

[0011] Granger causality tests, as a data-driven measure of causal strength, have been introduced into the field of industrial time series anomaly detection, such as GCAD (Granger Causality Test). Causality based multivariate time The anomaly detection model dynamically constructs a causal graph within the network based on the gradient of the deep predictor and uses a sparsity strategy to identify significant causal relationships. However, this type of method is geared towards anomaly detection tasks and cannot be directly transferred and adapted to industrial time series forecasting tasks. On the one hand, anomaly detection focuses on real-time identification of deviations from operating conditions. Its causal graph is dynamically generated and iteratively updated in real time with the operating conditions, lacking a fixed steady-state structure, making it difficult to transform into reusable and solidifiable static physical prior knowledge for time series forecasting models. On the other hand, this type of method only quantifies the strength of independent binary causal relationships between variables and does not establish a linkage mechanism between causal strength and model feature modeling. It cannot transform the discovered real causal patterns into adjustable soft weights, making it difficult to embed into the cross-variable attention modeling process of multivariate time series forecasting. It cannot fundamentally solve the core problem of existing forecasting models ignoring physical causality and relying on false data association modeling.

[0012] The first objective of this invention is to provide an industrial time series prediction method that integrates Granger causality and frequency domain features. This method is applied to multi-step prediction of the temperature inside the extraction tank during the extraction process of traditional Chinese medicine. The method includes: Step 1: Collect historical parameters of each extraction tank during the extraction process of traditional Chinese medicine to construct an industrial time series dataset; Step 2: Difference processing is performed on the multivariate series composed of historical parameters of each extraction tank. Granger causality test is performed on each pair of the differenced series. The obtained test statistics are used as the causal strength weights of the corresponding variable pairs. Based on this, the Granger causality matrix between variables is constructed. Step 3: Based on the Granger causality matrix obtained in Step 2, construct an industrial time series prediction model that integrates Granger causality and frequency domain features. The model divides the input sequence into several non-overlapping time segments. On one hand, it applies a Fast Fourier Transform along the time dimension within each segment to extract frequency domain features. On the other hand, it decouples the input sequence into trend and shape components. Self-attention modeling is performed on the last segment of the shape component along the variable dimension, with the attention weights modulated by the Granger causality matrix constructed in Step 2 using a multiplicative gating method. The trend component is then re-injected into the segment dimension via a value path for trend recovery attention computation. The computation result and the trend component are then fused again and flattened along the segment dimension to obtain time domain features. Finally, the time domain features and frequency domain features are added and fused using a learnable scalar gating method and then projected as output. Step four: Use the dataset from step one to train the prediction model built in step three, and use the trained model to complete the multi-step prediction of the temperature inside the extraction tank.

[0013] Optionally, the process of constructing the Granger causality matrix in step two includes: Perform first-order differences on multivariate sequences; For any ordered pair of variables (i,j), skip the pair and set it to zero if the standard deviation of the difference sequence of variable i or variable j is zero; otherwise, take variable j as the cause and variable i as the effect, and apply the result to the pair at the maximum lag order. Perform Granger causality tests step by step to obtain the F-statistics for each lag. With the corresponding p value ; The smallest p-value among all lags is taken as the significance criterion. The causal relationship of the ordinal variable pair is retained only when it is less than the significance level of 0.05, and the largest F-statistic among all lags is taken as its causal strength; otherwise, it is set to zero, thus obtaining the Granger causality matrix. The elements:

[0014] Then, the matrix is ​​normalized using the largest element, so that each element takes values ​​in the range [0,1], resulting in the normalized Granger causality matrix. :

[0015] in, Representing the Granger causality matrix The element in the i-th row and j-th column.

[0016] Optionally, the industrial time series prediction model that integrates Granger causality and frequency domain features constructed in step three includes a block and mean decoupling module, a frequency domain feature extraction branch module, a Granger causality-guided proximal variable attention module, a trend recovery attention module, and a gated fusion projection output module. The block segmentation and mean decoupling module is used to segment the input sequence into several non-overlapping time segments, calculate the mean, and decouple them into trend components and shape components. The frequency domain feature extraction branch module is used to apply Fast Fourier Transform along the time dimension of the original segments obtained from the segmentation to extract frequency domain features. The Granger causality-guided proximal variable attention module is used to perform self-attention modeling on the last segment of the shape component based on the constructed Granger causality matrix. The trend recovery attention module is used to apply parameter-shared attention operation to the segment sequence of each variable along the segment dimension, and perform trend recovery attention operation based on the output of the Granger causality-guided proximal variable attention module, and then fuse it with the trend component again to obtain the time domain features. The gated fusion projection output module is used to perform gated addition and fusion of the time domain features and frequency domain features and then linearly project the output.

[0017] Optionally, before the block segmentation and mean decoupling module performs segmentation and decoupling on the input sequence in step three, it first performs reversible instance normalization on the input sequence along the time dimension:

[0018] in Represents the normalized data. Represents the original data. and These represent the mean and variance calculated along the time dimension, respectively. To ensure the stability of extremely small positive numbers; The normalized input sequence is then... The time segment is divided into N non-overlapping time segments based on the segment length S and the step size S. The number of segments is... for:

[0019] L is the length of the input sequence; For the j-th segment of the i-th variable Calculate its mean as the trend component. The shape component is obtained by subtracting it from the mean. :

[0020]

[0021] in, Let S be a completely single vector of length S. This represents the k-th data point in the j-th segment of the i-th variable; The shape components are embedded into the d-dimensional dimension via shared linear projection and superimposed with learnable fragment position codes to obtain a d-dimensional embedding representation. :

[0022] in , These are the shared embedding weights and the linear projection layer bias, respectively. Encode the learnable segment position for the j-th segment.

[0023] Optionally, the frequency domain feature extraction branch module is located after segmentation and before mean decoupling, and its input is the original segment without mean deprocessing. ; For each original fragment Along the time dimension within the fragment Apply a real-valued Fast Fourier Transform and obtain its amplitude spectrum:

[0024] Projected to dimension via linear layer Then, the arithmetic mean is taken along the segment dimension to complete the aggregation, resulting in the frequency domain features:

[0025] in , For frequency domain branching linear layers, weights and bias vectors can be learned.

[0026] Optionally, the Granger causality-guided proximal variable attention module performs self-attention modeling of the last segment of the shape component based on the constructed Granger causality matrix, including: The d-dimensional embedding representation of the last segment of all C variables is obtained. Using the variable dimension C as the sequence dimension of attention, the shape of the attention weight matrix and the Granger causality matrix are both C. C, thereby achieving dimensional alignment between the two; The attention weights are calculated using a multi-head scaling dot product method, and random deactivation is applied to obtain the attention weight matrix A:

[0027] in, For each attention head dimension, T denotes transpose; Constructing a multiplicative gated matrix from Granger causality matrix For the above attention weight matrix After element-wise modulation, the weight matrix is ​​renormalized row by row to obtain the normalized weight matrix. :

[0028]

[0029] in The learnable causal strength coefficient is initialized to 0.1. To prevent division by zero of extremely small positive numbers; when hour This module degenerates into standard self-attention; The output of the Granger causality-guided proximal variable attention module is processed sequentially through residual connections with random deactivation, layer normalization, and a feedforward network. The processed last segment is spliced ​​together with the remaining segments along the segment dimension to restore the complete segment sequence.

[0030] Optionally, the trend recovery attention module applies parameter-shared attention operations to the segment sequence of each variable along the segment dimension, performs trend recovery attention operations based on the output of the Granger causality-guided proximal variable attention module, and then fuses it again with the trend component to obtain temporal features, including: A sequence of fragments for each variable Attention operations with shared parameters are applied independently along the segment dimension. The query and key are obtained only by linear projection of the shape embedding, while the value is superimposed on the corresponding segment mean on the projection result, thereby achieving the re-injection of trends. The output of the trend recovery attention module is also processed sequentially through residual connections with random deactivation, layer normalization, and a feedforward network to obtain the trend-recovered segment sequence. ; The segment mean is then superimposed onto the trend-recovered segment sequence. The temporal features are obtained by flattening along the segment dimension. .

[0031] Optionally, the gated fusion projection output module performs gated addition and fusion of time-domain features and frequency-domain features, followed by linear projection output, including: Time domain features Frequency domain characteristics The learned scalar gates are added and fused, then mapped to the prediction step size through a linear layer to obtain the prediction result vector. :

[0032]

[0033] in, For learnable frequency domain fusion gating coefficients, This refers to the learnable weight matrix of the linear projection layer in the gated fusion projection output module. Let be the learnable bias vector of this linear projection layer.

[0034] Optionally, the model training uses mean squared error as the loss function:

[0035] in , and These represent the true value, the predicted value, and the sample size, respectively.

[0036] The second objective of this invention is to provide a method for controlling the extraction process of traditional Chinese medicine. This method uses the aforementioned industrial time series prediction method to predict the temperature inside the extraction tank in multiple steps during the extraction process of traditional Chinese medicine, and then performs corresponding variable regulation.

[0037] The beneficial effects of this invention are: Considering that the heating and holding stages of the extraction process are controlled by the on / off state of steam valves, the temperature sequence contains periodic components determined by this control rhythm. Traditional pure time-domain attention models can only indirectly fit the periodic characteristics in time-series data by relying on the stacking of multi-layer network structures. This modeling method is highly indirect, has low modeling efficiency, and is prone to error accumulation in multi-step prediction tasks. This invention designs a frequency domain feature extraction branch module and specifically optimizes the embedding position and feature fusion method of this branch. This invention sets the output position of the frequency domain transformation branch after the segment segmentation is completed and before the mean decoupling, so that the Fourier transform directly acts on the original time-series segment that retains complete amplitude information. Based on this structural design, the frequency domain amplitude spectrum can simultaneously characterize the absolute amplitude baseline and local fluctuation characteristics of the time-series segment, effectively avoiding the problems of zero-frequency component disappearance and segment baseline feature loss caused by performing mean removal operation first and then frequency domain transformation. Building upon this foundation, this invention employs a bypass feature fusion architecture. Frequency domain features, after projection and aggregation, do not participate in the attention calculation process of the intermediate layers of the network. Instead, they are adaptively fused with temporal features only at the model output through learnable scalar gating. Compared to existing techniques that only use frequency domain error as an auxiliary loss function to achieve implicit constraints, this invention can achieve explicit modeling of frequency domain features, allowing the periodicity and multi-frequency features of time-series data to directly participate in the generation of prediction results, effectively improving the model's ability to fit the inherent periodicity of the process. Simultaneously, this bypass fusion architecture fully preserves the mean decoupling and trend recovery mechanism of the temporal branch, preventing frequency domain noise from being amplified layer by layer in the multi-layer attention process. Furthermore, this invention uses a small initial value initialization strategy for the frequency domain gating coefficients, adaptively updating parameters through model iterative training. While effectively introducing frequency domain prior features and optimizing modeling effects, this fully ensures the stability and convergence performance of the network training process.

[0038] This invention also proposes to introduce Granger causality into industrial time series forecasting, and designs a Granger causality-guided proximal variable attention module. Considering that the extraction process involves more than fifty process variables, attention weights calculated solely based on data similarity are prone to assigning high weights to statistically coincidentally correlated but physically unrelated variable pairs. This invention uses Granger causality tests to pre-quantify the temporal driving relationships between variables, injecting this as prior knowledge into cross-variable attention. Moreover, in measuring causal strength, this invention creatively uses the F-statistic to achieve refined quantification of variable causal strength, thereby characterizing the significant differences in causal strength among extraction process variables: the extraction tank temperature is mainly driven by the direct physical forces of variables such as the steam regulating valve opening and steam pressure. These variables have a clear and continuous causal effect on temperature changes and should therefore receive high weights in cross-variable attention; while variables such as process step numbers and some valve switching signals are only statistically significantly correlated with temperature due to the synchronous switching of process stages, and their driving effect is relatively indirect or weak. If the traditional p-value is used to measure causal strength, after passing the 0.05 significance test, the p-value itself will be compressed into the same narrow range, making it difficult to reflect the substantial difference in causal strength between direct driving variables such as steam valve opening and indirect related variables such as process step number. This leads to the multiplicative gating matrix modulating the two variables with similar amplitudes and losing its ability to distinguish between them. However, the F-statistic can maintain an order-of-magnitude difference between weak and strong causality, making the gating strength of strong driving variables such as steam valve opening and steam pressure significantly higher than that of weak driving variables such as process step number. This gives the gating matrix the modulation capability to match the actual driving relationship of the extracted process. At the same time, constant channels with a standard deviation of zero in the difference sequence are skipped. This setting in this application is aimed at the characteristic that a large number of valve signals in the extracted data remain constant over a long period of time, improving the robustness of the causal matrix on such industrial data. In terms of the causal prior injection method, this invention uses multiplicative gating after softmax and renormalization by row. Compared to the approach of ranking variables based on expert experience and then rigidly masking some attention, the causal strength of this invention is automatically obtained from the data through statistical tests, eliminating the need for manual analysis of physical chains. This makes it easier to implement in scenarios with a large number of variables and unclear driving relationships. Its modulation effect is a continuously adjustable weighting rather than a binary masking, allowing weak causal relationships to still participate in modeling. The learnable causal strength coefficient enables the model to adaptively determine the degree of prior intervention based on the data. When the causal relationship is insignificant, the coefficient tends to zero, and the module automatically degenerates into standard self-attention, ensuring the applicability of the method under different operating conditions and in different tanks.

[0039] This invention combines the above two improvements with mechanisms such as block mean decoupling and trend recovery attention. The shape of the Granger causality matrix is ​​determined by the number of variables, while the attention of proximal variables uses the variable dimension as its sequence dimension, resulting in a natural alignment between their shapes. This allows causal priors to be directly injected without interpolation or projection. In contrast, the weight matrix shape of attention at the segment dimension is determined by the number of segments, which does not correspond to the causal matrix. This dimensional alignment determines the injection position of the causal prior. Furthermore, the causal matrix is ​​calculated offline once before training and loaded in a fixed buffer, without participating in gradient updates. Compared to schemes that dynamically construct causal graphs within the network, this does not increase training overhead.

[0040] On a real dataset of nine extraction tanks in the honeysuckle extraction process, the method described in this invention outperforms the best results among five comparative models (PatchTST, Informer, Transformer, FreTS, and TimeMixer) in all nine datasets across the three indices: mean squared error, mean absolute error, and coefficient of determination. Specifically, compared to PatchTST, the mean squared error is reduced by an average of 27.99%, and the mean absolute error by an average of 33.30%; compared to Transformer, the reductions are 49.38% and 62.65%, respectively; compared to TimeMixer, the reductions are 29.48% and 39.76%, respectively; and compared to FreTS, the reductions are 28.50% and 36.08%, respectively. It can be seen that the improvement in mean absolute error is more significant than that in mean squared error, indicating more stable control over point-by-point prediction bias at most times. This invention overcomes the shortcomings of existing block-based prediction models in terms of insufficient expression of periodic components and lack of prior constraints on cross-variable dependencies, improving the monitoring and soft measurement capabilities of key parameters in industrial processes, and providing support for the stable operation of the extraction process and the consistency of product quality. Attached Figure Description

[0041] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0042] Figure 1 This is a flowchart of an industrial time series prediction method that integrates Granger causality and frequency domain features, as provided in Embodiment 1 of the present invention. Figure 2 This is a model architecture diagram of an industrial time series prediction method that integrates Granger causality and frequency domain features, provided in Embodiment 1 of the present invention. Figure 3This is a flowchart of the Granger causality matrix construction module provided in Embodiment 1 of the present invention; Figure 4 This is a schematic diagram of the frequency domain feature extraction branch module and gating fusion in Embodiment 1 of the present invention; Figure 5 This is a schematic diagram of the Granger causality-guided proximal variable attention module in Embodiment 1 of the present invention; Figure 6 This is a comparison chart of the predicted values ​​and actual values ​​of the present invention model and five comparative models on dataset A in Embodiment 2 of the present invention; Figure 7 This is a comparison chart of the predicted and actual values ​​of the model and five comparative models in the second embodiment of the present invention on dataset B; Figure 8 This is a comparison chart of the predicted and actual values ​​of the model and five comparative models in Embodiment 2 of the present invention on dataset C; Figure 9 This is a comparison chart of the predicted and actual values ​​of the present invention model and five comparative models on dataset D in Embodiment 2 of the present invention. Figure 10 This is a comparison chart of the predicted and actual values ​​of the model and five comparative models in Embodiment 2 of the present invention on dataset E; Figure 11 This is a comparison chart of the predicted and actual values ​​of the present invention model and five comparative models on dataset F in Embodiment 2 of the present invention; Figure 12 This is a comparison chart of the predicted and actual values ​​of the model and five comparative models on dataset G in Embodiment 2 of the present invention. Figure 13 This is a comparison chart of the predicted and actual values ​​of the present invention model and five comparative models on dataset H in Embodiment 2 of the present invention; Figure 14 This is a comparison chart of the predicted and actual values ​​of the model and five comparative models in Embodiment 2 of the present invention on dataset I; Detailed Implementation To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be described in further detail below with reference to the accompanying drawings.

[0043] Example 1 This embodiment provides an industrial time series forecasting method that integrates Granger causality and frequency domain features. See [link to relevant documentation]. Figure 1The method first collects historical parameters from the extraction process of traditional Chinese medicine to construct an industrial time series dataset, preprocesses it, and divides it into training, validation, and test sets in a 7:1:2 ratio; then, it constructs the Granger causality matrix between variables offline; next, it constructs an industrial time series prediction model that integrates Granger causality and frequency domain features, and trains the prediction model using the processed dataset; finally, it uses the trained prediction model to complete multi-step prediction of key variables in the industrial process. This embodiment takes the multi-step prediction of the temperature inside the extraction tank as an example for illustration.

[0044] Specifically, the method includes: Step 1: Collect historical parameters from the extraction process of traditional Chinese medicine to construct an industrial time series dataset, preprocess it, and divide it into training set, validation set, and test set in a 7:1:2 ratio.

[0045] The process described in this embodiment is the extraction process of the traditional Chinese medicine honeysuckle. The data used includes historical production data from nine extraction tanks (A to I). All variables were sampled at fixed one-minute intervals. Each tank's data contains 54 process variables, forming a long-range multivariate time series with strong non-stationarity and intermittent dynamics. The collected process parameters include continuous and discrete variables. Continuous process variables include the tank temperature, tank pressure, steam pressure, purified water preheating temperature, solvent flow rate and its cumulative amount, etc. Discrete variables include the on / off status of various valves, process step number, etc. The target variable for multi-step prediction is the tank temperature. The 54 process variables are shown in Table 1 below: Table 1. Extraction process parameters of honeysuckle

[0046] The collected data was divided into segments according to chronological order, with the first 70% used as the training set, the middle 10% as the validation set, and the last 20% as the test set, in a ratio of 7:1:2.

[0047] Data preprocessing employs reversible instance normalization. The mean and standard deviation required for normalization are obtained solely from the training set statistics and then applied to the validation and test sets to prevent test information from leaking into the training phase. The calculation formula is as follows:

[0048] in Represents the normalized data. Represents the original data. and These represent the mean and variance calculated along the time dimension, respectively. To ensure the stability of the smallest positive number.

[0049] Step 2: Construct the Granger causality matrix between variables offline.

[0050] For each extraction tank, a unique Granger causality matrix is ​​constructed. The construction process is as follows: Figure 3 As shown.

[0051] First, the multivariate sequences are first-differenced to eliminate trends and satisfy the stationarity requirement of the Granger causality test. Then, all variable pairs are iterated. For any ordered variable pair (i,j), if the standard deviation of the difference sequence of variable i or variable j is zero (i.e., the channel is constant during the sampling period, commonly seen in valve signals that remain open or closed for extended periods), the variable pair is skipped and its corresponding element is set to zero; otherwise, variable j is used as the cause and variable i as the effect, and the result is determined at the maximum lag order. Perform Granger causality tests step by step, and take the smallest p-value among all lags. To determine significance, take the largest F-statistic among all lag orders. The Granger causality matrix was constructed using causality strength. :

[0052] in, Representing the Granger causality matrix The element in the i-th row and j-th column, express l The corresponding p-value for the test under the first lag. express l The F-statistic corresponding to the first lag; Then, the matrix is ​​normalized using the largest element, so that all elements take values ​​in the range [0, 1], resulting in the normalized Granger causality matrix. :

[0053] The maximum lag order in this example We set it to 5, consistent with the prediction step size. The resulting matrix is... The square matrix is ​​stored offline as a file for subsequent model training and inference, and serves as a fixed prior that does not participate in gradient updates.

[0054] Existing methods that assign attention weights solely based on data similarity can easily lead to assigning high weights to statistically correlated but physically unrelated variable pairs. This application, however, avoids this problem by constructing a Granger causality matrix specific to each extraction tank. Furthermore, it distinguishes the temporal leading relationships between variables, rather than merely capturing static covariate correlations. First-order differencing preprocessing removes trend interference, focusing on the transmission logic between variable fluctuations, aligning with the temporal characteristics of disturbance propagation in industrial processes. Simultaneously, an independent causal structure is generated for each extraction tank, fully adapting to individual differences in equipment status, scaling levels, control loops, and operating conditions. This avoids applying a single universal correlation model to all equipment, enabling the identification of tank-specific disturbance propagation paths, closed-loop control behaviors, and stagnant measurement points. Using this causal matrix as the basis for weighting shifts the model weights from "static data similarity" to "statistically verifiable temporal driving effects," improving the physical interpretability of the weights and reducing the interference of spurious correlations on subsequent modeling, fault tracing, and operating condition analysis.

[0055] Furthermore, using the F-statistic to refine the causal strength of variables can also reflect the significant differences in causal strength among extraction process variables, maintain the order-of-magnitude difference between weak and strong causality, achieve differentiated configuration of association weights, suppress the accumulation of iteration errors caused by weak causality, and reduce the drift risk of long-term predictions. At the same time, it retains the transmission information of key process disturbances, making multi-step predictions more consistent with the actual dynamic changes of the extraction tank, and improving the accuracy of medium- and long-term predictions and the physical interpretability of the results.

[0056] Step 3: Construct an industrial time series forecasting model that integrates Granger causality and frequency domain features; The overall architecture of the prediction model constructed in this embodiment is as follows: Figure 2 As shown, it includes a block and mean decoupling module, a frequency domain feature extraction branch module, a Granger causality-guided proximal variable attention module, a trend recovery attention module, and a gated fusion projection output module.

[0057] In this embodiment, the input sequence length L is 10, the segment length S and step size S are both 5, the model hidden dimension d is 512, the number of attention heads is 8, the feedforward network intermediate dimension is 2048, the prediction step size T is 5, and the number of variables C is 54. That is, historical observation data from the past 10 consecutive time steps is used as the input window to predict the future. T =Time series with 5 time steps.

[0058] (1) Block mean decoupling module. The normalized sequence is divided into segments with a length of 5 and a step size of 5 to obtain the number of segments. :

[0059] That is, each variable is divided into Non-overlapping segments For the j-th segment of variable i Calculate its mean as the trend component. And obtain the shape components by subtraction. :

[0060]

[0061] in, Let S be a completely single vector of length S. This represents the k-th data point in the j-th segment of the i-th variable.

[0062] Shape components By embedding the fragment into a 512-dimensional space using shared linear projection and superimposing the learnable fragment position code, a 512-dimensional embedding representation of the fragment is obtained:

[0063] in , These are the shared embedding weights and the linear projection layer bias, respectively. This is for encoding the position of learnable segments.

[0064] Set a shape as Learnable position embedding matrix , Taken from matrix The Okay, that is This matrix is ​​updated along with the other parameters during model training.

[0065] (2) Frequency domain feature extraction branch module. This branch is introduced after segmentation and before mean decoupling. Figure 2 Because the segmentation and mean decoupling are treated as a single module, the location of the frequency domain feature extraction branch module (extracted after segmentation and before mean decoupling) cannot be directly indicated by its position. Its input is the original segment without mean removal, and the processing and fusion process is as follows: Figure 4 As shown. The original segment is processed along the time dimension within the segment. Apply a real-valued Fast Fourier Transform and take the amplitude spectrum, the length of which is... :

[0066] Projected to dimension via linear layer Then, the arithmetic mean is taken along the segment dimension and aggregated to obtain the frequency domain features:

[0067] in , These are the learnable weights and bias vectors of the frequency domain branch linear layer, respectively; Frequency domain characteristics Store temporarily, and participate in the fusion after the time domain backbone completes trend recovery.

[0068] (3) Granger causality-guided proximal variable attention module.

[0069] The embedding d-dimensional representation of the last segment of all C=54 variables Using a variable dimension of 54 as the sequence dimension of attention, the shapes of the attention weight matrix and the Granger causality matrix are both... To achieve dimensional alignment between the two, such as Figure 5 As shown, attention weights are calculated using a multi-head scaling dot product method and random deactivation is applied to obtain the attention weight matrix A:

[0070] in, For each dimension of attention head, , To hide dimensions in the model, For the number of attention heads.

[0071] Construct a multiplicative gating matrix using the normalized Granger causality matrix obtained in step two. Modulate the attention weight matrix A element-wise and then renormalize it row-wise to obtain the normalized attention weight matrix. :

[0072]

[0073] Among them, the learnable causal strength coefficient The initial value is set to 0.1. Matrix Indicates The attention weight matrix before normalization and random deactivation, and causal gating modulation. Representation matrix The OK Column elements, i.e., variables For variables The original attention weights, Representation matrix The OK Column elements are the terms used for summation during row-based normalization. Represents the gating matrix The OK Column elements, i.e., variables For variables Represents the gating matrix The OK Column elements are the terms used for summation during row-based normalization. To prevent extremely small positive numbers from being divided by zero, Represents the normalized attention weight matrix The OK Column elements.

[0074] The attention output is processed sequentially through residual connections with random deactivation, layer normalization, and a feedforward network, with two stacked coding layers:

[0075]

[0076]

[0077]

[0078] in This indicates multi-head attention modulated by causal gating. , The learnable parameters of the feedforward network, and the intermediate dimensions of the feedforward network. The default value is the model dimension. 4 times, The Gaussian error linear unit activation function is used; the above coding layers are stacked into two layers, and the end is normalized once more before output; The processed last segment and index are The remaining segments are spliced ​​along the segment dimension to restore the complete segment sequence.

[0079] (4) Trend recovery attention module.

[0080] Based on the recovered complete fragment sequence, the fragment sequence for each variable. Attention operations with shared parameters are applied independently along the fragment dimension d. The query and key are obtained solely from the linear projection of the shape embedding, while the value is superimposed on the corresponding fragment mean in the projection result. : , ,

[0081] in The projection matrix is ​​learnable. Align by broadcast method The dimension; since the segment mean is only superimposed on the value and not on the query and key, the calculation of attention weight is not affected by the absolute level difference of each segment, so that attention can focus on the shape similarity between segments. At the same time, the decoupled trend information participates in the weighted sum of the output through the value, realizing the re-injection of the trend. The output of the trend recovery attention is also processed sequentially through residual connections with random deactivation, layer normalization, and a feedforward network. Its structure is consistent with the encoding layer of the Granger causality-guided proximal variable attention module: first, residual connections with random deactivation and a first layer normalization are performed; then, a feedforward network consisting of two layers of linear transformation and GELU activation is applied; finally, a second residual connection and layer normalization are performed. These encoding layers are stacked twice, and the output undergoes another layer normalization. The sequence of segments output by these encoding layers is denoted as... ; The segment mean is then superimposed onto the trend-restored segment representation and flattened along the segment dimension to obtain the temporal features. :

[0082] in Indicates the flattening operation. The dimension is The real number space.

[0083] (5) Gated fusion projection output module.

[0084] Time domain features Frequency domain characteristics The learned scalar gates are added and fused, then mapped to the prediction step size through a linear layer to obtain the prediction result vector. :

[0085]

[0086] Among them, the learnable frequency domain fusion gating coefficients The initial value is set to 0.1. This refers to the learnable weight matrix of the linear projection layer in the gated fusion projection output module. Let be the learnable bias vector of this linear projection layer.

[0087] The prediction results are finally restored to their original dimensions by inverse transformation of instance normalization:

[0088] Step 4: Use the dataset from Step 1 to train the prediction model built in Step 3, and use the trained model to complete the multi-step prediction of the temperature inside the extraction tank.

[0089] The model uses mean squared error as the loss function. :

[0090] Training was performed using the Adam optimizer with a learning rate of 0.0001, a batch size of 32, 30 training epochs, and an early stopping patience value of 10. After training, the model weights that performed best on the validation set were loaded, and multi-step predictions of the temperature inside the extraction tank were performed on the test set.

[0091] Example 2 Based on the prediction model given in Example 1, this embodiment conducted a set of comparative experiments on nine real extraction tank datasets (denoted as datasets A to I) of the honeysuckle extraction process to verify the performance of the method of the present invention.

[0092] The prediction step size was fixed at 5 for all experiments, and all comparison methods used the same data partitioning and preprocessing strategies to ensure fair evaluation. In addition to the model of this invention, the models included in the comparison were: PatchTST, Informer, Transformer, FreTS, and TimeMixer time series prediction models.

[0093] Transformer can refer to A. Vaswani et al., “Attention is all you need,” inProc. Adv. Neural Inf. Process. Syst. (NeurIPS), vol. 30, 2017, pp. 5998–6008. For Informer, please refer to H. Zhou, S. Zhang, J. Peng, S. Zhang, J. Li, H. Xiong, and W. Zhang, "Informer: Beyond efficient transformer for long sequence time-series forecasting," in Proc. AAAI Conf. Artif. Intell., vol. 35, no. 12,2021, pp. 11 106–11 115. For FreTS, please refer to K. Yi et al., “Frequency-domain MLPs are more effective learners in time series forecasting,” in Proc. Adv. Neural Inf. Process.Syst. (NeurIPS), vol. 36, 2023, pp. 7625–7645. For TimeMixer, please refer to S. Wang et al., "TimeMixer: Decomposable multiscalemixing for time series forecasting," in Proc. Int. Conf. Learn. Represent.(ICLR), 2024. For PatchTST, please refer to Y. Nie, NH Nguyen, P. Sinthong, and J. Kalagnanam, "Time series forecasting with PatchTST," in Proc. Int. Conf. Learn.Represent. (ICLR), 2023. This embodiment uses mean absolute error (MAE), mean squared error (MSE), and coefficient of determination (CPE). As a standard for evaluating the performance of each model, it is expressed as:

[0094]

[0095]

[0096] in , , Let N represent the true value, predicted value, mean true value, and sample size, respectively. Mean absolute error (MAE) reflects the average deviation between predicted and true values, treating all errors equally and exhibiting strong robustness. Mean squared error (MSE) imposes a stronger penalty on larger errors, is more sensitive to outliers, and highlights the model's performance on extreme samples. The coefficient of determination (COP)... The value of a model is used to measure how well it explains the variation in the data; the closer it is to 1, the better the fit.

[0097] The performance of each dataset (AI) on PatchTST, Informer, Transformer, FreTS, and TimeMixer is shown in Table 2. Table 2 shows the performance of the dataset AI on the five models.

[0098] As shown in Table 2, across the nine extraction tank datasets, the method of this invention ranks first in all three metrics—mean squared error, mean absolute error, and coefficient of determination—consistently outperforming the five comparative models. In terms of average metrics, the method of this invention achieves the lowest mean squared error (4.65645) and mean absolute error (0.50595), and the highest coefficient of determination (0.98907), outperforming Transformer-based strong prediction models such as PatchTST and TimeMixer, as well as the frequency domain-based FreTS. In each dataset, the method of this invention ranks first in all three metrics, with significant improvements, particularly in datasets D, F, and H, where the mean squared error is significantly reduced compared to the best comparative method. These results demonstrate that the proposed frequency domain feature extraction and Granger causality-guided cross-variable modeling effectively suppress large prediction biases and improve the overall fit under diverse operating conditions.

[0099] Regarding mean absolute error, the method of this invention also achieves the best average performance, and its improvement is more significant than that of mean square error. This indicates that the method not only reduces the impact of large errors but also consistently reduces point-by-point prediction bias at most times. It is worth noting that on datasets D and H, some Transformer-based models show significant degradation, while the method of this invention maintains stable accuracy across all three metrics, highlighting its robustness to noise and non-stationarity. Overall, the results show that the proposed method achieves a good balance between accuracy, robustness, and stability, which is crucial for the prediction of key parameters in industrial processes.

[0100] Figures 6 to 14 The comparison curves of the predicted and actual values ​​of the model of this invention and five comparative models on datasets A to I are presented respectively. It can be seen that during the rapid temperature changes in the heating and holding stages of the extraction process, the predicted curve of the model of this invention significantly overlaps with the actual value better than the comparative models, and can more accurately track the temperature peaks and turning points; the comparative models exhibit varying degrees of phase shift and amplitude deviation at points of drastic temperature changes. This indicates that the frequency domain feature extraction branch and Granger causality-guided proximal variable attention proposed in this invention effectively improve the model's ability to model periodic components and cross-variable coupling relationships.

[0101] Some steps in the embodiments of the present invention can be implemented using software, and the corresponding software program can be stored in a readable storage medium, such as an optical disc or a hard disk.

[0102] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for industrial time series forecasting that integrates Granger causality and frequency domain features, characterized in that, The method is applied to the multi-step prediction of temperature inside the extraction tank during the extraction process of traditional Chinese medicine. The method includes: Step 1: Collect historical parameters of each extraction tank during the extraction process of traditional Chinese medicine to construct an industrial time series dataset; Step 2: Difference processing is performed on the multivariate series composed of historical parameters of each extraction tank. Granger causality test is performed on each pair of the differenced series. The obtained test statistics are used as the causal strength weights of the corresponding variable pairs. Based on this, the Granger causality matrix between variables is constructed. Step 3: Based on the Granger causality matrix obtained in Step 2, construct an industrial time series prediction model that integrates Granger causality and frequency domain features. The model divides the input sequence into several non-overlapping time segments. On one hand, it applies a Fast Fourier Transform along the time dimension within each segment to extract frequency domain features. On the other hand, it decouples the input sequence into trend and shape components. Self-attention modeling is performed on the last segment of the shape component along the variable dimension, with the attention weights modulated by the Granger causality matrix constructed in Step 2 using a multiplicative gating method. The trend component is then re-injected into the segment dimension via a value path for trend recovery attention computation. The computation result and the trend component are then fused again and flattened along the segment dimension to obtain time domain features. Finally, the time domain features and frequency domain features are added and fused using a learnable scalar gating method and then projected as output. Step four: Use the dataset from step one to train the prediction model built in step three, and use the trained model to complete the multi-step prediction of the temperature inside the extraction tank.

2. The method according to claim 1, characterized in that, The process of constructing the Granger causality matrix in step two includes: Perform first-order differences on multivariate sequences; For any ordered pair of variables (i,j), skip the pair and set it to zero if the standard deviation of the difference sequence of variable i or variable j is zero; otherwise, take variable j as the cause and variable i as the effect, and apply the result to the pair at the maximum lag order. Perform Granger causality tests step by step to obtain the F-statistics for each lag. With the corresponding p value ; The smallest p-value among all lags is taken as the significance criterion. The causal relationship of the ordinal variable pair is retained only when it is less than the significance level of 0.05, and the largest F-statistic among all lags is taken as its causal strength; otherwise, it is set to zero, thus obtaining the Granger causality matrix. The elements: Then, the matrix is ​​normalized using the largest element, so that each element takes values ​​in the range [0,1], resulting in the normalized Granger causality matrix. : in, Representing the Granger causality matrix The element in the i-th row and j-th column.

3. The method according to claim 2, characterized in that, The industrial time series prediction model constructed in step three, which integrates Granger causality and frequency domain features, includes a block and mean decoupling module, a frequency domain feature extraction branch module, a Granger causality-guided proximal variable attention module, a trend recovery attention module, and a gated fusion projection output module. The block segmentation and mean decoupling module is used to segment the input sequence into several non-overlapping time segments, calculate the mean, and decouple them into trend components and shape components. The frequency domain feature extraction branch module is used to apply Fast Fourier Transform along the time dimension of the original segments obtained from the segmentation to extract frequency domain features. The Granger causality-guided proximal variable attention module is used to perform self-attention modeling on the last segment of the shape component based on the constructed Granger causality matrix. The trend recovery attention module is used to apply parameter-shared attention operation to the segment sequence of each variable along the segment dimension, and perform trend recovery attention operation based on the output of the Granger causality-guided proximal variable attention module, and then fuse it with the trend component again to obtain the time domain features. The gated fusion projection output module is used to perform gated addition and fusion of the time domain features and frequency domain features and then linearly project the output.

4. The method according to claim 3, characterized in that, Before the block segmentation and mean decoupling module in step three performs segmentation and decoupling on the input sequence, it first performs reversible instance normalization on the input sequence along the time dimension: in Represents the normalized data. Represents the original data. and These represent the mean and variance calculated along the time dimension, respectively. To ensure the stability of extremely small positive numbers; The normalized input sequence is then... The time segment is divided into N non-overlapping time segments based on the segment length S and the step size S. The number of segments is... for: L is the length of the input sequence; For the j-th segment of the i-th variable Calculate its mean as the trend component. The shape component is obtained by subtracting it from the mean. : in, Let S be a completely single vector of length S. This represents the k-th data point in the j-th segment of the i-th variable; The shape components are embedded into the d-dimensional dimension via shared linear projection and superimposed with learnable fragment position codes to obtain a d-dimensional embedding representation. : in , These are the shared embedding weights and the linear projection layer bias, respectively. Encode the learnable segment position for the j-th segment.

5. The method according to claim 4, characterized in that, The frequency domain feature extraction branch module is located after segmentation and before mean decoupling, and its input is the original segment without mean deprocessing. ; For each original fragment Along the time dimension within the fragment Apply a real-valued Fast Fourier Transform and obtain its amplitude spectrum: Projected to dimension via linear layer Then, the arithmetic mean is taken along the segment dimension to complete the aggregation, resulting in the frequency domain features: in , For frequency domain branching linear layers, weights and bias vectors can be learned.

6. The method according to claim 5, characterized in that, The Granger causality-guided proximal variable attention module performs self-attention modeling of the last segment of the shape component based on the constructed Granger causality matrix, including: The d-dimensional embedding representation of the last segment of all C variables is obtained. Using the variable dimension C as the sequence dimension of attention, the shape of the attention weight matrix and the Granger causality matrix are both C. C, thereby achieving dimensional alignment between the two; The attention weights are calculated using a multi-head scaling dot product method, and random deactivation is applied to obtain the attention weight matrix A: in, For each attention head dimension, T denotes transpose; Constructing a multiplicative gated matrix from Granger causality matrix For the above attention weight matrix After element-wise modulation, the weight matrix is ​​renormalized row by row to obtain the normalized weight matrix. : in The learnable causal strength coefficient is initialized to 0.

1. To prevent division by zero of extremely small positive numbers; when hour This module degenerates into standard self-attention; The output of the Granger causality-guided proximal variable attention module is processed sequentially through residual connections with random deactivation, layer normalization, and a feedforward network. The processed last segment is spliced ​​together with the remaining segments along the segment dimension to restore the complete segment sequence.

7. The method according to claim 6, characterized in that, The trend recovery attention module applies parameter-shared attention operations to the segment sequence of each variable along the segment dimension. Based on the output of the Granger causality-guided proximal variable attention module, it performs trend recovery attention operations, and then fuses these with the trend component again to obtain temporal features, including: A sequence of fragments for each variable Attention operations with shared parameters are applied independently along the segment dimension. The query and key are obtained only by linear projection of the shape embedding, while the value is superimposed on the corresponding segment mean on the projection result, thereby achieving the re-injection of trends. The output of the trend recovery attention module is also processed sequentially through residual connections with random deactivation, layer normalization, and a feedforward network to obtain the trend-recovered segment sequence. ; The segment mean is then superimposed onto the trend-recovered segment sequence. The temporal features are obtained by flattening along the segment dimension. .

8. The method according to claim 7, characterized in that, The gated fusion projection output module performs gated addition and fusion of time-domain and frequency-domain features, followed by linear projection output, including: Time domain features Frequency domain characteristics The learned scalar gates are added and fused, then mapped to the prediction step size through a linear layer to obtain the prediction result vector. : in, For learnable frequency domain fusion gating coefficients, This refers to the learnable weight matrix of the linear projection layer in the gated fusion projection output module. Let be the learnable bias vector of this linear projection layer.

9. The method according to claim 8, characterized in that, The model training uses mean squared error as the loss function: in , and These represent the true value, the predicted value, and the sample size, respectively.

10. A method for controlling the extraction process of traditional Chinese medicine, characterized in that, The method employs the industrial time series prediction method described in any one of claims 1-9 to achieve multi-step prediction of the temperature inside the extraction tank during the extraction process of traditional Chinese medicine, and then performs corresponding variable regulation.

Images