An industrial control timing prediction method based on multi-scale time-frequency joint perception

Abstract

This invention belongs to the field of network security technology and discloses a method for predicting industrial control system (ICS) time series based on multi-scale time-frequency joint sensing. It proposes an end-to-end learnable adaptive time-frequency feature extractor to replace the low-pass filtering method relying on fixed parameters in existing technologies. A data-driven dynamic multi-scale partitioning mechanism is proposed, which adaptively adjusts the scale partitioning method according to the actual distribution characteristics and fluctuation patterns of ICS time series data. A scale-adaptive residual projector is designed, and through multi-dimensional spatial projection and residual feature compensation, the problem of multi-scale feature dimension mismatch is solved, establishing a feature cascade enhancement relationship between coarse and fine scales, thus strengthening the effective fusion of multi-scale features and the ability to model long-term time series dependencies. A time-frequency cross-modal collaborative calibration mechanism is proposed, breaking through the limitation of traditional channel attention mechanisms that rely only on a single time dimension. It achieves joint learning of dual-domain information by fusing time-domain statistics and frequency-domain context.

Description

Technical Field

[0001] This invention relates to the field of network security technology, and in particular to an industrial control timing prediction method based on multi-scale time-frequency joint sensing. Background Technology

[0002] Industrial control systems (ICS), as the core of modern production facilities, are widely used in key sectors such as power, petrochemicals, manufacturing, and transportation. In these systems, sensor data, controller status signals, and various communication messages together constitute complex time-series data streams. Accurate prediction of this industrial time-series data is a crucial technological support for achieving early warning of industrial equipment failures, optimization of production processes, and detection of network intrusions.

[0003] Industrial control system (ICS) time-series forecasting refers to the process of accurately estimating and inferring the values ​​of key process variables at one or more future points in time by using historical observation data combined with advanced mathematical models and data mining techniques. Its importance lies primarily in the global perception and proactive control of industrial operating conditions. On the one hand, through accurate prediction of equipment operating parameters, the system can issue timely warnings before key indicators deviate from preset thresholds or exhibit abnormal trends, thereby transforming passive "post-fault maintenance" into proactive "predictive maintenance," significantly reducing unplanned downtime and optimizing operating costs. On the other hand, in terms of ICS network security defense, anomaly prediction of time-series data such as network traffic characteristics and communication access frequencies has become a key technical means to identify latent network attacks and unauthorized data transmissions. This has profound strategic significance for building a real-time, reliable industrial network security situational awareness system and ensuring the stable operation of control systems.

[0004] Current research on industrial control system (ICS) timing prediction primarily focuses on deep learning models. Compared to traditional monitoring methods based on physical models or manually set fixed thresholds, deep learning models demonstrate stronger feature representation capabilities when processing ICS data with multidimensional characteristics and complex temporal correlations. These models can automatically identify and extract deep time-dependent features and coupling patterns between variables through nonlinear mapping of large-scale historical sensor data, effectively addressing the non-stationarity and high sampling rate characteristics of ICS operating states. Furthermore, existing advanced deep learning architectures have provided a preliminary technical foundation for proactive fault warning and refined management in assisting in the identification of equipment degradation trends or unknown network attack patterns.

[0005] The paper "Liu S, et al. Pyraformer: Low-complexity pyramidal attention for long-range time series modeling and forecasting[C] / / International Conference on Learning Representations (ICLR), 2022." proposes a prediction model based on a pyramidal attention mechanism. This model constructs a multi-scale hierarchical structure to decompose the input sequence into segments of different resolutions and utilizes the pyramidal attention mechanism to model the temporal dependencies between different levels. However, because the pyramidal structure's scale division is fixed in advance, it is difficult to adapt to the complex and ever-changing operating conditions and data fluctuations in industrial settings. Furthermore, the hierarchical feature aggregation only focuses on the dependencies within segments, ignoring the dynamic cross-scale connections between segments, which can easily lead to a loss of crucial information for predicting sudden anomalies, thus limiting the model's flexibility and prediction accuracy.

[0006] The paper "Zhou T, Ma Z, Wen Q, et al. FEDformer: Frequency enhanced decomposed transformer for long-term series forecasting[C] / / International Conference on Machine Learning (ICML). PMLR, 2022: 27268-27286" proposes a frequency-enhanced deep decomposition Transformer model. It utilizes the Fast Fourier Transform (FFT) to map time series data to the frequency domain, filtering and selecting key frequency components through frequency domain operators to capture the inherent periodic evolution patterns in industrial control data. However, frequency domain analysis mechanisms are static and lack data-driven adaptive frequency range adjustment capabilities when processing non-stationary industrial control signals, making them highly susceptible to introducing high-frequency noise from electromagnetic interference or equipment jitter.

[0007] In summary, existing industrial control time series prediction methods suffer from problems such as insufficient mining of time-frequency features, weak multi-scale feature fusion capabilities, and poor adaptability to non-stationary industrial control data with strong noise. As a result, the prediction accuracy is difficult to meet the actual needs of industrial sites, and the practicality and promotion of these methods in complex industrial control scenarios are low. Summary of the Invention

[0008] The purpose of this invention is to overcome the shortcomings of existing technologies and provide a method for predicting industrial control timing based on multi-scale time-frequency joint sensing. An end-to-end learnable adaptive time-frequency extractor is proposed to replace the low-pass filtering method relying on fixed parameters in existing technologies. A data-driven dynamic multi-scale partitioning mechanism is proposed, which adaptively adjusts the scale partitioning method according to the actual distribution characteristics and fluctuation patterns of industrial control timing data. A time-frequency cross-modal collaborative calibration mechanism is proposed, breaking through the limitation of traditional channel attention mechanisms that rely only on a single time dimension, and achieving joint learning of dual-domain information by fusing time-domain statistics and frequency-domain context.

[0009] The technical solution of this invention is as follows: A method for predicting industrial control time series based on multi-scale time-frequency joint sensing, comprising the following steps: Step 1: The adaptive time-frequency extractor enhances the time-frequency of industrial control time-series data by performing frequency domain real-valued transformation, frequency mask modulation, and extraction of global frequency domain context, combined with inverse Fourier transform, and generates a high signal-to-noise ratio feature sequence. Step 2: After the encoder encodes the feature sequence, the multi-scale fusion module achieves feature decoupling through exponential decay pooling, and completes cross-scale mapping and residual compensation by combining with an adaptive residual projector to extract the fused multi-scale features. Step 3: Based on the fused multi-scale features, cross-modal fusion is performed by spatial compression of the time-domain features and the global frequency-domain context generated in Step 1 to complete the coupled modeling of time-frequency features, thereby generating attention weights for the channel dimension, recalibrating the fused features for the channel dimension, and generating a channel-enhanced time-frequency co-feature sequence. Step 4: The decoder takes the time-frequency co-processed feature sequence generated in Step 3 as input, completes the future value mapping of the industrial control time series data, achieves feature dimension matching through linear projection, and finally generates the prediction result of the future time step.

[0010] Step one specifically involves: Step 1.1: Process industrial control timing data After performing mean and variance standardization, the time-domain signal is transformed into a frequency-domain representation using FFT. Raw industrial control timing data , For batch size, For sequence length, Given the number of feature channels, we obtain the frequency domain complex feature tensor. Frequency domain frequency points A strategy of equivalent mapping from complex to real number domains is adopted to obtain the real-valued representation of the frequency domain complex feature tensor; The real and imaginary parts are concatenated along the last dimension to form a real vector, as shown in the formula: in, Represents the real part of the complex eigenvector in the frequency domain. Represents the imaginary part of the complex eigenvectors in the frequency domain. This indicates that the splicing operation is performed in the last dimension; Step 1.2. Design a frequency mask modulation network to generate a frequency mask that is strongly correlated with the input data. First, flatten the input real vector as shown in the formula: in This represents flattening out the three-dimensional tensor from the frequency point dimension, transforming it into a two-dimensional vector. An initial frequency mask is generated using a fully connected layer. As shown in the formula: in and Represents the learnable weight matrix and bias vector. represent An activation function is used to constrain the mask values ​​to the [0,1] interval, achieving soft selection of frequency components; finally, the mask is reshaped to generate a frequency mask with dimensions matching the input features. As shown in the formula: in This means that the flattened mask is reshaped into a three-dimensional tensor to ensure that it matches the dimension of the input features. Step 1.3. Frequency Mask Frequency mask: Applied directly to real number vectors via element-wise multiplication. As global frequency domain context The direct source is extracted after the modulation operation and before the inverse Fourier transform. As shown in the formula: in, Its amplitude dimension is , The amplitude representing the frequency domain characteristics after gating. The magnitude tensor represents the value from Flattened , and For projection layer parameters, ensure dimension matching during matrix multiplication. The dimension of the frequency domain context vector; Step 1.4. Frequency Domain to Time Domain Conversion and Recovery: The real number vector after frequency mask modulation is divided into real and imaginary parts along the channel dimension and recombined into a complete frequency domain complex representation; then, an inverse Fourier transform is performed to map the frequency domain enhanced features back to the time domain, resulting in a time-frequency enhanced sequence that filters out high-frequency noise and enhances key periodic information; finally, the time-frequency enhanced sequence is denormalized to restore the original physical dimensions of the data, ensuring that the output scale is consistent with the input scale.

[0011] Step two specifically involves: Step 2.1. Scale Decoupling and Feature Parsing: The encoder encodes the feature sequence output by the adaptive time-frequency extractor. An exponentially decaying average pooling operator decomposes the encoded feature sequence into multi-level feature sets. The pooling kernel size is generated using an exponentially decaying strategy as shown in the formula: in Indicates the first The kernel size of each pooling layer It is a scale factor that controls the interval between scales. The number of scale layers; Apply simultaneously to each input channel Average pooling layers of different scales Perform multi-scale decomposition and generate multi-level feature sets for each channel: ; Step 2.2. Adaptive cross-scale mapping: Design a scale-adaptive residual projector to extract fused multi-scale features.

[0012] The scale-adaptive residual projector consists of a multidimensional spatial projection network and a residual feature compensation loop; the multidimensional spatial projection network employs a two-layer fully connected mapping, and the weight matrix is ​​trained... , It automatically captures the nonlinear mapping pattern from coarse-scale features to fine-scale features, thereby establishing a cascaded enhancement relationship between different granularities, as shown in the formula: in This represents the enhanced features after projection. Representing the Scale of input features, , Represents the bias vector; The residual feature compensation loop uses additive coupling nodes to superimpose the projected enhanced features with the original features at the next scale point by point, as shown in the formula: .

[0013] Step three specifically involves: Step 3.1. Temporal feature compression: Features of multi-scale fusion Global average pooling is performed to extract global statistical features for each channel, eliminating spatial information in the sequence dimension and obtaining temporal compressed features that retain only the channel dimension. As shown in the formula: Step 3.2. Time-Frequency Joint Feature Generation: The time-domain compressed features are concatenated with the global frequency domain context generated by the adaptive time-frequency extractor to form a joint feature representation, as shown in the formula: in This represents a concatenation operation along the channel dimension. Step 3.3. Channel Attention Weight Generation: Generate Joint Feature Representations The input is fed into the activation network, where channel-dimensional attention weights are generated through a combination of "two-layer linear transformation and nonlinear gating" logic. The calculation formula is: The first fully connected layer uses weights To achieve dimensionality upscaling of features, The first layer randomly discards some features with a set probability to suppress overfitting of the activation network. The second fully connected layer uses weights... The feature dimensions are restored to the original number of channels to match the channel dimensions of the input features. represent The activation function maps the output value to the [0,1] interval; Step 3.4. Time-Frequency Co-calibration of Features: To achieve attention weights for multi-scale fusion features Channel-by-channel precise control is employed, and a space broadcast multiplication mechanism is used to recalibrate the features, resulting in recalibrated time-frequency co-located features. .

[0014] The spatial broadcast multiplication mechanism specifically involves: first, applying the attention weights of the channel dimension... Broadcast expansion along the sequence dimension to construct a weight tensor consistent with the original feature dimension. As shown in the formula: Using weight tensors The multi-scale fused features are weighted element-wise to obtain the recalibrated time-frequency co-functional features. As shown in the formula: .

[0015] Step four specifically involves: Step 4.1. Decoder Sequence Initialization and Timing Prior Injection: Take the last bit of the original industrial control timing data. Each time step serves as the initial input to the decoder, ensuring the continuity between the prediction and historical conditions; and sinusoidal positional encoding is added to the input sequence to preserve temporal order information, adapting to the attention mechanism's modeling requirements for temporal dependencies; Step 4.2. Decoder Processing: Used to model the inherent temporal dependencies of the predicted target sequence, through the lower triangular mask matrix. To prevent future information leaks, the formula is as follows: in, The lower triangular mask matrix ensures that the decoder can only focus on the current and previous time steps, capture the autocorrelation and periodic evolution characteristics of the sequence to be predicted, and construct the time domain benchmark of the prediction target. Cross-attention layer: based on decoder features The time-frequency co-enhancement features output in step three for and This achieves semantic alignment between the "state of the sequence to be predicted" and the "historical time-frequency enhanced working condition features," as shown in the formula: The cross-attention layer dynamically injects the time-frequency collaborative feature sequence output from step three into the prediction sequence generation process; The feedforward neural network performs nonlinear transformation and high-dimensional mapping on the attention output features, through residual connections and layer normalization; Step 4.3. Prediction Output Generation: The high-dimensional features of the decoder are mapped to the target prediction dimension through a linear projection layer, as shown in the formula: in Represents the decoder output characteristics. Represents the projection layer parameters. This represents the prediction result for the entire sequence; Take the last output of the decoder The final prediction result is calculated using the given time steps, and the formula is as follows: in Dimensions , The number of batch samples representing a single prediction Represents the preset future prediction step size. This represents the number of industrial control parameters to be predicted.

[0016] The beneficial effects of this invention: Addressing the technical shortcomings of existing industrial control system time series prediction methods, such as insufficient time-frequency feature mining, weak multi-scale feature fusion capability, and poor adaptability to non-stationary, high-noise industrial control data, this invention proposes an industrial control system time series prediction method based on multi-scale time-frequency joint sensing, demonstrating the following significant advantages: (1) This invention proposes an end-to-end learnable adaptive time-frequency extractor, abandoning the traditional fixed-parameter filtering method. Through real-valued mapping and frequency mask modulation mechanism, the frequency domain signal generated by Fourier transform is converted into a real-valued tensor that can be parameterized and controlled by a deep neural network, realizing data-driven dynamic spectrum filtering, effectively eliminating the masking of periodic evolution law by industrial control noise in time-series signals. At the same time, the global frequency domain context vector is extracted, providing support for subsequent time-frequency cross-modal fusion, solving the problem that static frequency domain analysis cannot adapt to the dynamic fluctuations of industrial control data.

[0017] (2) This invention proposes a multi-scale fusion mechanism, breaking away from the fixed-scale partitioning paradigm of traditional static pooling. It adopts an exponential decay strategy to generate adaptive multi-scale pooling kernels, achieving hierarchical decoupling of the different periodic evolution patterns of industrial control data. Furthermore, a scale-adaptive residual projector is designed to solve the problem of multi-scale feature dimension mismatch through multi-dimensional spatial projection and residual feature compensation, establishing a feature cascade enhancement relationship between coarse and fine scales, and strengthening the effective fusion of multi-scale features and the ability to model long-term time-series dependencies.

[0018] (3) This invention proposes a time-frequency collaborative calibration mechanism, constructing a cross-modal collaborative calibration path based on frequency domain priors. Time-frequency joint excitation input is generated through global compression in the time domain and deep splicing of the frequency domain context. A two-layer nonlinear gating network is used to dynamically recalibrate and constrain the time-domain channel weights by using the frequency domain context as a priori constraint factor. Furthermore, precise channel-by-channel recalibration of the fused multi-scale features is performed through spatial broadcast multiplication, achieving semantic depth alignment of time-frequency features. This effectively suppresses redundant interference while significantly improving the model's ability to identify abnormal operating states under complex dynamic conditions. Attached Figure Description

[0019] Figure 1 This is a schematic diagram of the overall process of the industrial control timing prediction method based on multi-scale time-frequency joint sensing; Figure 2 This is a schematic diagram of an adaptive time-frequency feature extractor. Figure 3 A schematic diagram of the cascaded mapping of the dynamic multi-scale feature fusion module; Figure 4 This is a schematic diagram of a time-frequency cross-modal collaborative calibration mechanism; Figure 5 This diagram illustrates the performance comparison of different time-series prediction techniques on the ECL dataset; (a) shows the MSE performance; (b) shows the MAE performance. Figure 6 The diagram shows the performance comparison of different time series prediction techniques on the ETTh1 dataset; (a) shows the MSE performance; (b) shows the MAE performance. Detailed Implementation

[0020] The overall process diagram of this invention is as follows: Figure 1 As shown.

[0021] Step 1: The adaptive time-frequency extractor transforms non-stationary industrial control time-series data into high signal-to-noise ratio, context-dependent feature sequences through data-driven dynamic filtering, thereby providing high-quality input for subsequent processing. A flowchart is shown below. Figure 2 As shown.

[0022] 1. Adapting to real-number inputs for deep learning: For the original sequence After performing mean and variance standardization, the time-domain signal is transformed into a frequency-domain representation using FFT. Let the original time-domain sequence be... ,in For batch size, For sequence length, The number of feature channels. The resulting frequency domain complex feature tensor. The number of frequency points in the frequency domain Since standard deep learning network layers only support real-valued tensor inputs and cannot directly process frequency domain features in complex form, this method employs an equivalent mapping strategy from complex to real-valued domains to achieve real-valued representation of frequency domain complex features without loss of feature information. Specifically, this method will... The real and imaginary parts are concatenated along the last dimension to form a real vector, as shown in the formula: in Represents the real part of the complex eigenvector in the frequency domain. Represents the imaginary part of the complex eigenvectors in the frequency domain. This indicates that the splicing operation is performed in the last dimension (channel dimension).

[0023] This strategy enables subsequent linear layers to learn a composite real matrix, thereby accurately simulating the ability to apply linear filters to the spectrum in the complex domain and enhancing the modulation capability of this method for frequency domain information.

[0024] 2. Design a lightweight, trainable frequency mask modulation network to generate a frequency mask that is strongly correlated with the input data. First, flatten the input real vector as shown in the formula: in This means flattening out the three-dimensional tensor starting from the second dimension (frequency point dimension), converting it into a two-dimensional vector to adapt to the input of the fully connected layer.

[0025] Then, an initial frequency mask is generated through a fully connected layer. As shown in the formula: in and Represents the learnable weight matrix and bias vector. represent An activation function is used to constrain the mask values ​​to the [0,1] interval, achieving soft selection of frequency components; finally, the mask is reshaped to generate a frequency mask with dimensions matching the input features. As shown in the formula: in This means that the flattened mask is reshaped into a three-dimensional tensor to ensure that it matches the dimension of the input features.

[0026] 3. Frequency mask Frequency mask: Applied directly to real number vectors via element-wise multiplication. It inherently encodes the spectral structure and modulation intensity of the current input instance, therefore this method treats it as a global frequency context. The direct source is extracted after the modulation operation and before the inverse Fourier transform. As shown in the formula: in, Its amplitude dimension is , The amplitude representing the frequency domain characteristics after gating. The magnitude tensor represents the value from Flattened , and For projection layer parameters, ensure dimension matching during matrix multiplication. is the dimension of the frequency domain context vector.

[0027] 4. Frequency Domain to Time Domain Conversion and Recovery: The real-number vector modulated by the frequency mask is divided into real and imaginary parts along the channel dimension and recombined into a complete complex representation in the frequency domain. Then, an inverse Fourier transform is performed to map the enhanced features back to the time domain, resulting in a time-frequency enhanced sequence that filters out high-frequency noise and enhances key periodic information. Finally, inverse normalization is performed on the time-frequency enhanced sequence to restore the original physical dimensions of the data, ensuring that the output scale is consistent with the input scale.

[0028] Step 2: Couple traditional static pooling with an adaptive residual projection mechanism to achieve effective extraction and fusion of features across multiple time scales. A flowchart is shown below. Figure 3 As shown.

[0029] 1. Scale Decoupling and Feature Parsing: The encoder encodes the feature sequence output by the adaptive time-frequency extractor. Using an average pooling operator with exponential decay, the encoded feature sequence is decomposed into multi-level feature sets. The pooling kernel size is generated using an exponential decay strategy as shown in the formula: in Indicates the first The kernel size of each pooling layer It is a scale factor that controls the interval between scales. The number of scale layers.

[0030] Apply simultaneously to each input channel Average pooling layers of different scales Perform multi-scale decomposition and generate multi-level feature sets for each channel: .

[0031] 2. Adaptive Cross-Scale Mapping: To address the dimensionality mismatch problem in multi-scale feature flows, this method designs a scale-adaptive residual projector, consisting of a multi-dimensional spatial projection network and a residual feature compensation loop. The multi-dimensional spatial projection network employs a two-layer fully connected mapping, trained using a weight matrix... , It automatically captures the nonlinear mapping pattern from coarse-scale features to fine-scale features, thereby establishing a "feature cascade enhancement" relationship between different granularities, as shown in the formula: in This represents the enhanced features after projection. Representing the Scale of input features, , This represents the bias vector.

[0032] The residual feature compensation loop uses additive coupling nodes to superimpose the projected enhanced features with the original features at the next scale point by point, as shown in the formula: Step 3: By spatial compression of time-domain features and cross-modal fusion of frequency-domain context features, coupled modeling of time-frequency features is completed. This generates attention weights for the channel dimension, and the original fused features are recalibrated along the channel dimension. This achieves synergistic enhancement and redundancy suppression of time-frequency cross-modal information. A flowchart is shown below. Figure 4 As shown.

[0033] 1. Temporal feature compression: Features of multi-scale fusion Global average pooling is performed to extract global statistical features for each channel, eliminating spatial information in the sequence dimension and obtaining temporal compressed features that retain only the channel dimension. As shown in the formula: 2. Time-Frequency Joint Feature Generation: The time-domain compressed features are concatenated with the global frequency domain context generated by the adaptive time-frequency extractor to form a joint feature representation, as shown in the formula: in This represents the splicing operation of the channel dimension.

[0034] 3. Channel attention weight generation: Representing joint features The input is fed into the activation network, where channel-dimensional attention weights are generated through a combination of "two-layer linear transformation and nonlinear gating" logic. The calculation formula is: The first fully connected layer uses weights (Without bias) Achieve dimensionality-up transformation of features. The first layer randomly discards some features with a set probability to suppress overfitting of the activation network. The second fully connected layer passes through weights. The feature dimensions are restored to the original number of channels to match the channel dimensions of the input features. represent The activation function maps the output value to the interval [0,1].

[0035] 4. Time-frequency collaborative feature recalibration: To achieve attention weights for multi-scale fusion features For precise channel-by-channel control, this invention employs a spatial broadcast multiplication mechanism to recalibrate features. Firstly, the channel attention weights are... Broadcast expansion along the sequence dimension to construct a weight tensor consistent with the original feature dimension. As shown in the formula: Then, using the weight tensor The multi-scale fused features are weighted element-wise to obtain the recalibrated time-frequency co-functional features. As shown in the formula: Step 4: After completing the collaborative enhancement and fusion of time and frequency features, the future value mapping of industrial control time series data is completed based on the encoder-decoder architecture. Feature dimension matching is achieved through linear projection, and the prediction results of future time steps are finally generated.

[0036] 1. Enhanced Context Mapping of Features: Recalibrated Time-Frequency Co-location Features This feature tensor integrates the original time-domain fluctuation patterns, the frequency-domain global context, and the multi-scale industrial evolution logic, forming a highly representative and high signal-to-noise ratio feature benchmark for subsequent decoding and prediction tasks.

[0037] 2. Decoder sequence initialization and temporal prior injection: Take the last bit of the encoder input. Each time step serves as the initial input to the decoder, ensuring the continuity of predictions with historical conditions. Furthermore, sinusoidal positional encoding is added to the input sequence to preserve temporal order information, adapting to the attention mechanism's modeling requirements for temporal dependencies.

[0038] 3. Decoder Processing: Used to model the inherent temporal dependencies of the predicted target sequence, through a lower triangular mask matrix. To prevent future information leaks, the formula is as follows: in The lower triangular mask matrix ensures that the decoder can only focus on the current and previous time steps, capture the autocorrelation and periodic evolution characteristics of the sequence to be predicted, and construct the time domain benchmark of the prediction target.

[0039] Cross-attention layer: based on decoder features The time-frequency collaborative enhancement features output above for and This achieves semantic alignment between the "state of the sequence to be predicted" and the "historical time-frequency enhanced working condition features," as shown in the formula: By dynamically injecting multi-scale temporal features extracted by the encoder and frequency domain context into the prediction sequence generation process, the traditional cross-attention method, which relies solely on pure temporal features, is able to achieve deep semantic interaction across scales and time frequencies.

[0040] Feedforward neural networks: They perform nonlinear transformations and high-dimensional mappings on the attention output features, and avoid gradient vanishing and information degradation in deep networks through residual connections and layer normalization, ensuring the stable transmission of multi-scale time-frequency features and improving model training efficiency and generalization ability.

[0041] 4. Predicted Output Generation: The high-dimensional features of the decoder are mapped to the target prediction dimension through a linear projection layer, as shown in the formula: in Represents the decoder output characteristics. Represents the projection layer parameters. This represents the prediction result for the entire sequence.

[0042] Take the last output of the decoder The final prediction result is calculated using the given time steps, and the formula is as follows: in Dimensions , The number of batch samples representing a single prediction Represents the preset future prediction step size. This represents the number of industrial control parameters to be predicted (such as temperature, pressure, flow rate, etc.).

[0043] To verify the effectiveness of this invention, this study selected two publicly available time series datasets, ETTh1 and ECL, from typical power industry scenarios, to conduct comparative experiments. The datasets were comprehensively evaluated against mainstream baseline models such as PatchTST, Crossformer, Pyraformer, Informer, and Autoformer in multivariate long-series prediction tasks. Mean squared error (MSE) and mean absolute error (MAE), commonly used in industrial control time series prediction, were adopted as core evaluation indicators. MSE reflects the squared mean of the deviations between predicted and actual values, while MAE represents the average absolute error. Lower values ​​indicate higher prediction accuracy.

[0044] Experimental results are as follows Figure 5 , Figure 6 The results show that, under different prediction lengths, the MSE and MAE of this method are significantly better than all baseline models, especially in long-term time-series prediction tasks with the longest prediction step. This fully demonstrates that the present invention can effectively capture the long-term evolution of industrial time-series data, maintain stable prediction performance under complex industrial conditions with non-stationarity and strong noise, and has good scenario adaptability and generalization ability, which can meet the actual needs of industrial Internet scenarios for high-precision long-term forward-looking prediction.

Claims

1. A method for predicting industrial control timing based on multi-scale time-frequency joint sensing, characterized in that, The steps include the following: Step 1: The adaptive time-frequency extractor enhances the time-frequency of industrial control time-series data by performing frequency domain real-valued transformation, frequency mask modulation, and extracting frequency domain context, combined with inverse Fourier transform, and generates a high signal-to-noise ratio feature sequence. Step 2: After the encoder encodes the feature sequence, the multi-scale fusion module achieves feature decoupling through exponential decay pooling, and completes cross-scale mapping and residual compensation by combining with an adaptive residual projector to extract the fused multi-scale features. Step 3: Based on the fused multi-scale features, cross-modal fusion is performed by spatial compression of the time-domain features and the global frequency-domain context generated in Step 1 to complete the coupled modeling of time-frequency features, thereby generating attention weights for the channel dimension, recalibrating the fused features for the channel dimension, and generating a channel-enhanced time-frequency co-feature sequence. Step 4: The decoder takes the time-frequency co-processed feature sequence generated in Step 3 as input, completes the future value mapping of the industrial control time series data, achieves feature dimension matching through linear projection, and finally generates the prediction result of the future time step.

2. The industrial control timing prediction method based on multi-scale time-frequency joint sensing according to claim 1, characterized in that, Step one specifically involves: Step 1.1: Process industrial control timing data After performing mean and variance standardization, the time-domain signal is transformed into a frequency-domain representation using FFT. Raw industrial control timing data , For batch size, For sequence length, Given the number of feature channels, we obtain the frequency domain complex feature tensor. Frequency domain frequency points A strategy of equivalent mapping from complex to real number domains is adopted to obtain a real-valued representation of the frequency domain complex feature tensor; The real and imaginary parts are concatenated along the last dimension to form a real vector, as shown in the formula: in, Represents the real part of the complex eigenvector in the frequency domain. Represents the imaginary part of the complex eigenvectors in the frequency domain. This indicates that the splicing operation is performed in the last dimension; Step 1.

2. Design a frequency mask modulation network to generate a frequency mask that is strongly correlated with the input data. First, flatten the input real vector as shown in the formula: in This represents flattening out the three-dimensional tensor from the frequency point dimension, transforming it into a two-dimensional vector. An initial frequency mask is generated using a fully connected layer. As shown in the formula: in and Represents the learnable weight matrix and bias vector. represent An activation function is used to constrain the mask values ​​to the [0,1] interval, achieving soft selection of frequency components; finally, the mask is reshaped to generate a frequency mask with dimensions matching the input features. As shown in the formula: in This means that the flattened mask is reshaped into a three-dimensional tensor to ensure that it matches the dimension of the input features. Step 1.

3. Frequency Mask Frequency mask: Applied directly to real number vectors via element-wise multiplication. As frequency domain context The direct source is extracted after the modulation operation and before the inverse Fourier transform. As shown in the formula: in, Its amplitude dimension is , The amplitude representing the frequency domain characteristics after gating. The magnitude tensor represents the value from Flattened , and For projection layer parameters, ensure dimension matching during matrix multiplication. The dimension of the frequency domain context vector; Step 1.

4. Frequency Domain to Time Domain Conversion and Recovery: The real vector modulated by the frequency mask is divided into real and imaginary components along the channel dimension and recombined into a complete frequency domain complex representation; then, an inverse Fourier transform is performed to map the enhanced features back to the time domain, resulting in a time-frequency enhanced sequence that filters out high-frequency noise and enhances key periodic information; finally, the time-frequency enhanced sequence is denormalized to restore the original physical dimensions of the data, ensuring that the output scale is consistent with the input scale.

3. The industrial control timing prediction method based on multi-scale time-frequency joint sensing according to claim 1, characterized in that, Step two specifically involves: Step 2.

1. Scale Decoupling and Feature Parsing: The encoder encodes the feature sequence output by the adaptive time-frequency extractor. An exponentially decaying average pooling operator decomposes the encoded feature sequence into multi-level feature sets. The pooling kernel size is generated using an exponentially decaying strategy as shown in the formula: in Indicates the first The kernel size of each pooling layer It is a scale factor that controls the interval between scales. The number of scale layers; Apply simultaneously to each input channel Average pooling layers of different scales Perform multi-scale decomposition and generate multi-level feature sets for each channel: ; Step 2.

2. Adaptive cross-scale mapping: Design a scale-adaptive residual projector to extract multi-scale fused features.

4. The industrial control timing prediction method based on multi-scale time-frequency joint sensing according to claim 3, characterized in that, The scale-adaptive residual projector consists of a multidimensional spatial projection network and a residual feature compensation loop; the multidimensional spatial projection network employs a two-layer fully connected mapping, and the weight matrix is ​​trained... , It automatically captures the nonlinear mapping pattern from coarse-scale features to fine-scale features, thereby establishing a cascaded enhancement relationship between different granularities, as shown in the formula: in This represents the enhanced features after projection. Representing the Scale of input features, , Represents the bias vector; The residual feature compensation loop uses additive coupling nodes to superimpose the projected enhanced features with the original features at the next scale point by point, as shown in the formula: Finally, all the scale-enhanced features are fused to obtain multi-scale fused features. .

5. The industrial control timing prediction method based on multi-scale time-frequency joint sensing according to claim 1, characterized in that, Step three specifically involves: Step 3.

1. Temporal feature compression: The multi-scale fusion features output from step two above Global average pooling is performed to extract global statistical features for each channel, eliminating spatial information in the sequence dimension and obtaining temporal compressed features that retain only the channel dimension. As shown in the formula: Step 3.

2. Time-Frequency Joint Feature Generation: The time-domain compressed features are concatenated with the frequency-domain context generated by the adaptive time-frequency extractor to form a joint feature representation, as shown in the formula: in This represents a concatenation operation along the channel dimension. Step 3.

3. Channel Attention Weight Generation: Generate Joint Feature Representations The input is fed into the activation network, where channel-dimensional attention weights are generated through a combination of "two-layer linear transformation and nonlinear gating" logic. The calculation formula is: The first fully connected layer uses weights To achieve dimensionality upscaling of features, The first layer randomly discards some features with a set probability to suppress overfitting of the activation network. The second fully connected layer uses weights... The feature dimensions are restored to the original number of channels to match the channel dimensions of the input features. represent The activation function maps the output value to the [0,1] interval; Step 3.

4. Time-Frequency Co-calibration of Features: To achieve attention weights for multi-scale fusion features Channel-by-channel precise control is employed, and a space broadcast multiplication mechanism is used to recalibrate the features, resulting in recalibrated time-frequency co-located features. .

6. The industrial control timing prediction method based on multi-scale time-frequency joint sensing according to claim 5, characterized in that, The spatial broadcast multiplication mechanism specifically involves: first, applying the attention weights of the channel dimension... Broadcast expansion along the sequence dimension to construct a weight tensor consistent with the original feature dimension. As shown in the formula: Using weight tensors The multi-scale fused features are weighted element-wise to obtain the recalibrated time-frequency co-functional features. As shown in the formula: 。 7. The industrial control timing prediction method based on multi-scale time-frequency joint sensing according to claim 6, characterized in that, Step four specifically involves: Step 4.

1. Decoder Sequence Initialization and Timing Prior Injection: Take the last bit of the original industrial control timing data. Each time step serves as the initial input to the decoder, ensuring the continuity between the prediction and historical conditions; and sinusoidal positional encoding is added to the input sequence to preserve temporal order information, adapting to the attention mechanism's modeling requirements for temporal dependencies; Step 4.

2. Decoder Processing: Used to model the inherent temporal dependencies of the predicted target sequence, through the lower triangular mask matrix. To prevent future information leaks, the formula is as follows: in, The lower triangular mask matrix ensures that the decoder can only focus on the current and previous time steps, capture the autocorrelation and periodic evolution characteristics of the sequence to be predicted, and construct the time domain benchmark of the prediction target. Cross-attention layer: based on decoder features The time-frequency co-enhancement features output in step three for and This achieves semantic alignment between the "state of the sequence to be predicted" and the "historical time-frequency enhanced working condition features," as shown in the formula: The cross-attention layer dynamically injects the time-frequency collaborative feature sequence output from step three into the prediction sequence generation process; The feedforward neural network performs nonlinear transformation and high-dimensional mapping on the attention output features, through residual connections and layer normalization; Step 4.

3. Prediction Output Generation: The high-dimensional features of the decoder are mapped to the target prediction dimension through a linear projection layer, as shown in the formula: in Represents the decoder output characteristics. Represents the projection layer parameters. This represents the prediction result for the entire sequence; Take the last output of the decoder The final prediction result is calculated using the given time steps, and the formula is as follows: in Dimensions , The number of batch samples representing a single prediction Represents the preset future prediction step size. This represents the number of industrial control parameters to be predicted.

Images