Industrial multivariable time sequence prediction method fusing multi-view information

Abstract

The invention provides an industrial multivariable time sequence prediction method fusing multi-view information, and the method comprises the steps: carrying out the normalization processing of an industrial multivariable time sequence, and carrying out the modeling of the global space-time correlation of time sequence data through a space-time Transform module, and then capturing a local mode with finer granularity in the sequence through a dual-channel local information extraction module, and finally carrying out adaptive fusion on the extracted global and local features. According to the industrial multivariable time sequence prediction method, a time sequence analysis framework capable of integrating multi-view information is designed, so that the effect of improving prediction performance is achieved.

Description

Technical Field

[0001] This invention relates to the field of industrial time series forecasting, and more specifically, to a method for industrial multivariate time series forecasting that integrates multi-perspective information. Background Technology

[0002] Industrial intelligence is an inevitable choice for leading modern industry towards high-quality development. Industrial multivariate time series forecasting technology is the core cornerstone enabling this transformation. By building high-precision forecasting models, enterprises can achieve predictive maintenance and intelligent management, and anticipate potential problems in the production process. This is of great significance for optimizing production processes and ensuring the stable and efficient operation of production systems.

[0003] Industrial multivariate time series forecasting refers to the joint prediction of multiple related time-series variables over a future period based on historical observation data. Deep learning, with its powerful feature extraction and nonlinear relationship modeling capabilities, provides effective technical support for achieving high-accuracy predictions. Meanwhile, with the rapid development of real-time monitoring and sensing technologies, their application in industrial data acquisition is becoming increasingly widespread. This results in acquired data exhibiting complex characteristics such as high dimensionality and multiple dependencies. However, current research focuses more on mining effective information from a single perspective, lacking a comprehensive framework to fully integrate complex patterns and dynamic dependencies in time series data. Therefore, designing a time series analysis framework that can integrate multi-perspective information to deeply mine data multi-dependency patterns and improve the analytical performance and robustness of the model by combining the advantages of different perspectives has become a critical problem that urgently needs to be solved.

[0004] In view of this, the present invention is hereby proposed. Summary of the Invention

[0005] In view of this, the present invention discloses a prediction method for industrial multivariate time series that integrates multi-perspective information. This method improves prediction performance by designing a time series analysis framework that can integrate multi-perspective information.

[0006] Specifically, the present invention is achieved through the following technical solutions:

[0007] This invention provides a method for industrial multivariate time series forecasting that integrates multi-perspective information, comprising the following steps:

[0008] First, the industrial multivariate time series is normalized. Then, the spatiotemporal Transformer module is used to model the global spatiotemporal correlation of the time series data. Next, a dual-channel local information extraction module is used to capture finer-grained local patterns in the sequence. Finally, the extracted global and local features are adaptively fused.

[0009] This invention also provides a prediction system for the above-mentioned industrial multivariate time series prediction method, comprising:

[0010] Global module: Used to first normalize industrial multivariate time series, and then use the spatiotemporal Transformer module to model the global spatiotemporal correlation of time series data;

[0011] Local module: Used to capture finer-grained local patterns in the sequence through the dual-channel local information extraction module;

[0012] Output adjustment module: Used to adaptively fuse the extracted global and local features.

[0013] This invention provides a method for industrial multivariate time series forecasting that integrates multi-perspective information, aiming to achieve accurate prediction of key operating parameters of industrial equipment, thereby providing reliable data support for equipment operation optimization and forward-looking decision-making. Addressing existing problems in the field of industrial multivariate time series forecasting, this invention refines the complex pattern information extraction process into multiple sub-tasks, each undertaken by an independent module. Different modules focus on capturing specific aspects of data dynamics, effectively revealing various complex dependencies within the data.

[0014] Specifically, the input sequence is first normalized. Then, a spatiotemporal Transformer module is used to model the global spatiotemporal correlation of the time series data. Building upon this, a dual-channel local information extraction module is used to capture finer-grained local patterns in the sequence. Finally, the extracted global and local features are adaptively fused, combining the complementary advantages of features from different perspectives to enhance the model's predictive performance. Attached Figure Description

[0015] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Throughout the drawings, the same reference numerals denote the same parts.

[0016] Figure 1 This is a schematic diagram of the overall model structure of an embodiment of the present invention;

[0017] Figure 2 This is a schematic diagram of the spatiotemporal Transformer module structure according to an embodiment of the present invention;

[0018] Figure 3 This is a schematic diagram of the dual-channel local information extraction module according to an embodiment of the present invention;

[0019] Figure 4This is a flowchart illustrating a computer device provided in an embodiment of the present invention. Detailed Implementation

[0020] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure; rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0021] The terminology used in this disclosure is for the purpose of describing particular embodiments only and is not intended to be limiting of this disclosure. The singular forms “a,” “the,” and “the” as used in this disclosure and the appended claims are also intended to include the plural forms. Unless the context clearly indicates otherwise, it should be understood that the term “and / or” as used herein refers to and includes any and all possible combinations of one or more of the associated listed items.

[0022] It should be understood that although the terms first, second, third, etc. may be used in this disclosure to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this disclosure, first information may also be referred to as second information, and similarly, second information may also be referred to as first information, depending on the context. For example, the word "if" as used herein may be interpreted as "when" or "in response to determination".

[0023] Example

[0024] The following is a specific implementation example to illustrate the technical solution of the present invention in detail. The data used in this embodiment are collected from multiple fields, including six datasets: ETTh1, ETTm1, Weather, Electricity, ILI, and Traffic.

[0025] (1) This invention proposes an industrial multivariate time series forecasting method that integrates multi-perspective information, called MPFN. The MPFN model mainly includes a spatiotemporal Transformer module, a dual-channel local information extraction module, and an output adjustment module. Its overall structure is as follows: Figure 1 As shown.

[0026] (2) The input sequence of the model is defined as ,in, Indicates the length of the input sequence. This represents the number of features. To reduce the impact of differences in data distribution and enhance the training stability of the model, the input sequence is normalized using formula (a).

[0027] Formula (a) is:

[0028]

[0029] In the formula, This represents the normalized data. This represents the mean and variance of the input data. This represents a trainable parameter vector used to adapt normalization to different data distributions. It represents a very small constant scalar, used to avoid the denominator being 0.

[0030] (3) Normalize the data The input is fed into the spacetime Transformer module, and its workflow is as follows: Figure 2 As shown. First, Transformed into a high-dimensional embedding representation through a linear projection layer. This transformation helps subsequent network layers capture complex dependencies in the sequence; then, the learnable positional encoding vector is combined with... Superimposed to introduce sequence order information.

[0031] Formula (b) is:

[0032]

[0033] In the formula, This represents an embedding vector containing location information. This indicates a position encoding operation.

[0034] (4) embedding vectors with location information The input is fed into Encoder-1, where it is transformed by the spatial attention layer to obtain the spatial representation. .

[0035] (5) To more effectively capture the periodic patterns of the sequence, Fourier transform is used to... The signal is transformed to the frequency domain, and key frequency components are automatically selected using a learnable parameter matrix. The filtered frequency domain signal is then compared with its spatial representation. The inputs are fed into the temporal attention layer to obtain a temporal representation. .

[0036] (6) After fusing the temporal and spatial representations, the data is processed sequentially through the Add&Norm layer and the feedforward neural network to obtain the global output representation of Encoder-1. .

[0037] (7) In obtaining Based on this, the learning bias of Encoder-1 is calculated according to formula (c). Subsequent Encoders use the learning bias of the previous Encoder as input, and repeat steps (4)-(6) to progressively extract the deep global representation. The cascading processing of the layer encoders yields... representation vectors The above The vectors are summed to obtain the output of the spatiotemporal Transformer module. .

[0038] Formula (c) is:

[0039]

[0040] In the formula, and They represent the first The input and output of the layer encoder, and the learning bias. for and The difference, which is also the first The input to the layer encoder. Specifically, when... hour, .

[0041] (8) Output the results of the spacetime Transformer module The input is fed into the dual-channel local information extraction module, and its workflow is as follows: Figure 3 As shown. First, using a size of The sliding window, according to the specified sliding step size right Segment the data to generate chunks. .

[0042] (9) The block data The inputs are fed in parallel to the inner branch network and the cross-domain branch network. The inner branch network uses convolutional layers to capture local details within a block, resulting in output features. Cross-domain branch network usage A learnable vector is used as a query token, and inter-block contextual information is aggregated through a multi-head attention mechanism to obtain the output feature. .

[0043] (10) Output features of the inner branch network and the cross-domain branch network output features The data is merged and transformed using a linear layer to obtain the output of the dual-channel local information extraction module. .

[0044] (11) Output the spatiotemporal Transformer module according to formula (d). The output results of the dual-channel local information extraction module Adaptive fusion is performed, and the final prediction result is generated through a fully connected network.

[0045] Formula (d) is:

[0046]

[0047] In the formula, This represents a learnable weight term used to dynamically adjust the fusion ratio of global and local features, enhance the contribution of key features, and thus optimize the model's predictive performance.

[0048] In addition to providing a prediction method, this invention also provides a prediction system, comprising:

[0049] Global module: Used to first normalize industrial multivariate time series, and then use the spatiotemporal Transformer module to model the global spatiotemporal correlation of time series data;

[0050] Local module: Used to capture finer-grained local patterns in the sequence through the dual-channel local information extraction module;

[0051] Output adjustment module: Used to adaptively fuse the extracted global and local features.

[0052] In practice, the above modules can be implemented as independent entities or combined in any way to be implemented as the same or several entities. For the specific implementation of each unit, please refer to the previous method implementation examples, which will not be repeated here.

[0053] Application examples

[0054] The MPFN model of the above embodiments of the present invention was compared with the performance of eight mainstream time series forecasting methods, including: Autoformer, Pyraformer, FEDformer, Crossformer, DLinear, MSGNet, iTransformer, and AMD. MSE and MAE were used as performance evaluation metrics, with lower MSE or MAE indicating more accurate prediction performance.

[0055] Table 1. Prediction performance of the method in this embodiment and other comparative methods on various datasets.

[0056]

[0057] Of the 58 evaluation metrics mentioned above, MPFN achieved first place in 40 and second place in 15. The comparison results are shown in Table 1 above. The MPFN model demonstrates leading prediction accuracy across most datasets and different prediction lengths.

[0058] The beneficial effects of this invention are as follows:

[0059] The MPFN model of this invention mainly includes a spatiotemporal Transformer module, a dual-channel local information extraction module, and an output adjustment module. Specifically, (1) the spatiotemporal Transformer module extracts global temporal and spatial dependency information from the data through a stacked encoder structure. Furthermore, the idea of ​​bias learning is incorporated between each encoder, so that each encoder can focus on the feature variables with large prediction bias, thereby gradually optimizing the prediction results of the model. (2) The dual-channel local information extraction module focuses on the impact of local semantic information on the prediction target. It extracts rich local detail features in the time series through the synergistic effect of internal branch networks and cross-domain branch networks. (3) The output adjustment module is used to adaptively fuse the extracted global and local features, and enhance the model's prediction performance by combining the complementary advantages of features from different perspectives.

[0060] Figure 4 This is a schematic diagram of the structure of a computer device disclosed in this invention. (Reference) Figure 4 As shown, the computer device 400 includes at least a memory 402 and a processor 401; the memory 402 is connected to the processor via a communication bus 403 and is used to store computer instructions executable by the processor 401. The processor 401 is used to read computer instructions from the memory 402 to implement the steps of the method described in any of the above embodiments.

[0061] For the above-described apparatus embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The apparatus embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this disclosure according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0062] Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, such as semiconductor memory devices (e.g., EPROM, EEPROM, and flash memory devices), magnetic disks (e.g., internal disks or removable disks), magneto-optical disks, and CD-ROMs and DVD-ROMs. Processors and memory may be supplemented by or incorporated into dedicated logic circuitry.

[0063] Finally, it should be noted that although this specification contains many specific implementation details, these should not be construed as limiting the scope of any invention or the scope of the claims, but rather are primarily used to describe the features of specific embodiments of a particular invention. Certain features described in the various embodiments of this specification may also be implemented in combination in a single embodiment. On the other hand, various features described in a single embodiment may also be implemented separately in various embodiments or in any suitable sub-combination. Furthermore, while features may function in certain combinations as described above and even initially claimed in this way, one or more features from a claimed combination may be removed from that combination in some cases, and a claimed combination may refer to a sub-combination or a variation of a sub-combination.

[0064] Similarly, although the operations are depicted in a specific order in the accompanying drawings, this should not be construed as requiring these operations to be performed in the specific order shown or sequentially, or requiring all illustrated operations to be performed to achieve the desired result. In some cases, multitasking and parallel processing may be advantageous. Furthermore, the separation of various system modules and components in the above embodiments should not be construed as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

[0065] Thus, specific embodiments of the subject matter have been described. Other embodiments are within the scope of the appended claims. In some cases, the actions recited in the claims may be performed in a different order and still achieve the desired result. Furthermore, the processes depicted in the drawings are not necessarily shown in a specific order or sequence to achieve the desired result. In some implementations, multitasking and parallel processing may be advantageous.

[0066] The above description is merely a preferred embodiment of this disclosure and is not intended to limit this disclosure. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A method for industrial multivariate time series forecasting that integrates multi-perspective information, characterized in that, Includes the following steps: First, the industrial multivariate time series is normalized. Then, the spatiotemporal Transformer module is used to model the global spatiotemporal correlation of the time series data. Next, a dual-channel local information extraction module is used to capture finer-grained local patterns in the sequence. Finally, the extracted global and local features are adaptively fused.

2. The industrial multivariate time series forecasting method according to claim 1, characterized in that, Methods for normalization include: The industrial multivariate time series is defined as ,in, Indicates the length of the input sequence. Indicate the number of features; normalize the input sequence using formula (a): Formula (a) is: In the formula, This represents the normalized data. This represents the mean and variance of the input data. This represents a trainable parameter vector used to adapt normalization to different data distributions. It represents a very small constant scalar, used to avoid the denominator being 0.

3. The industrial multivariate time series forecasting method according to claim 2, characterized in that, The method for modeling the global spatiotemporal correlation of time series data using the Spatiotemporal Transformer module includes the following steps: normalizing the data... The input is fed into the spatiotemporal Transformer module, where it is used to generate embedded representations through a linear projection layer. According to formula (b), the learnable location encoding vector is combined with... Superposition: Formula (b) is: In the formula, This represents an embedding vector containing location information. Indicates position encoding operation; then... The input is fed into Encoder-1, where it is transformed by the spatial attention layer to obtain the spatial representation. Using Fourier transform to The signal is transformed to the frequency domain, and key frequency components are automatically selected using a learnable parameter matrix. The filtered frequency domain signal is then compared with the spatial representation. The inputs are fed into the temporal attention layer to obtain a temporal representation. After fusing the temporal and spatial representations, the data is processed sequentially through an Add&Norm layer and a feedforward neural network to obtain the global output representation. This is to achieve the global spatiotemporal correlation of time series data.

4. The industrial multivariate time series forecasting method according to claim 3, characterized in that, The global output representation Further calculation of the learning bias includes: calculating the learning bias of Encoder-1 according to formula (c), and then using the learning bias of the previous layer's Encoder as input for each subsequent Encoder layer to progressively extract the deep global representation. The cascading processing of the layer encoders yields... representation vectors , will the The vectors are summed to obtain the output of the spatiotemporal Transformer module. Formula (c) is: In the formula, and They represent the first The input and output of the layer encoder, and the learning bias. for and The difference, which is also the first The input to the layer encoder, especially when hour, .

5. The industrial multivariate time series forecasting method according to claim 4, characterized in that, The method for capturing finer-grained local patterns in a sequence using a dual-channel local information extraction module includes: processing the output results... The input is fed into the dual-channel local information extraction module, using a size of The sliding window, according to the specified sliding step size right Segment the data to generate chunks. ; the segmented data Parallel inputs are fed into the inner branch network and the cross-domain branch network; the inner branch network uses convolutional layers to capture local details within blocks to obtain output features. Cross-domain branch network uses A learnable vector is used as a query token, and inter-block contextual information is aggregated through a multi-head attention mechanism to obtain the output feature. .

6. The industrial multivariate time series forecasting method according to claim 5, characterized in that, Methods for adaptively fusing extracted global and local features include fusing intra-block output features and inter-block output features. and The data is merged and transformed using a linear layer to obtain the output of the dual-channel local information extraction module. The spatiotemporal Transformer module outputs the results according to formula (d). The output results of the dual-channel local information extraction module Adaptive fusion is performed, and the final prediction result is generated through a fully connected network: Formula (d) is: In the formula, This represents the learnable weights.

7. The prediction system of the industrial multivariate time series prediction method according to any one of claims 1-6, characterized in that, It includes: a global module, which is used to normalize industrial multivariate time series data and model the global spatiotemporal correlation of time series data through a spatiotemporal Transformer module; a local module, which is used to capture finer-grained local patterns in the sequence through a dual-channel local information extraction module; and an output adjustment module, which is used to adaptively fuse the extracted global and local features.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed, it implements the steps of the prediction method according to any one of claims 1-6.

9. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the prediction method as described in any one of claims 1-6.

Images