Time sequence data prediction method and device based on wavelet time-frequency domain

Abstract

The invention provides a time series data prediction method and device based on a wavelet time-frequency domain. The prediction method comprises the following steps: acquiring multi-dimensional time series industrial data through an industrial gateway deployed in a working area; preprocessing the collected data to obtain an overall multi-dimensional industrial time series data set; selecting a key index influencing a production decision as a task prediction target; detail features and approximate features in the data are separated through a wavelet decomposition module; inputting the separated details and approximate data into different data prediction modules; training a network model through a loss function iterative optimization method and storing optimal parameters; and inputting actually collected industrial time series data into the prediction model to obtain a prediction result of the task target. According to the method, accurate prediction of the industrial multi-dimensional time series data can be provided, so that the production decision is optimized, and the industrial production efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the field of time series data prediction, and in particular to a time series data prediction method and device based on wavelet time-frequency domain. Background Art

[0002] In industrial production, industrial equipment and its sensors generate a large amount of time series data. Industrial gateways collect this data and generate multi-dimensional industrial time series data sets. By analyzing and learning this data, key data can be accurately predicted, thereby achieving overall optimization of production decisions. Therefore, the accuracy of prediction is crucial to improving industrial production efficiency and ensuring production safety. High-accuracy predictions can not only optimize resource allocation and reduce waste, but also detect potential failures and risks in advance, and improve the stability and reliability of the production process.

[0003] In existing technologies, mainstream methods use deep learning networks for data learning and prediction, but these methods are usually based on extracting features in a single time domain or frequency domain, which makes it difficult to fully capture the complex relationship between multi-dimensional data in industrial scenarios. The details and approximate data information in industrial data have different characteristics. Sending them uniformly into the network model will lead to insufficient model parameter training, thereby reducing the accuracy of the prediction results. Some methods extract approximate data by applying a moving average kernel to the sequence, but this method often causes the data to be too smooth, thereby losing the key information of the approximate features.

[0004] Therefore, it is necessary to provide a new method to solve the above problems. Summary of the invention

[0005] Based on the above problems existing in the prior art, the purpose of the embodiments of the present invention is to provide a time series data prediction method and device based on wavelet time and frequency domain, which is used to learn and predict multi-dimensional industrial time series data to improve prediction accuracy and optimize the production process.

[0006] The object of the present invention is achieved through the following technical solution: a time series data prediction method based on wavelet time-frequency domain, comprising:

[0007] Obtain time series data from the production process and select production-related parameters as actual values ​​of the task objectives;

[0008] Inputting the time series data into a neural network model to obtain a regression value of a task prediction target;

[0009] Based on the actual value and the regression value, a trained neural network model is obtained through a loss function iterative optimization method.

[0010] Furthermore, the obtaining of time series data in the production process includes:

[0011] Acquire data from different types of industrial sensors deployed in the work area through industrial gateways;

[0012] The data is preprocessed and information of different types and dimensions is aligned on the time axis to obtain an overall multi-dimensional time series data set.

[0013] Further, the neural network includes a wavelet decomposition module, a time domain linear prediction module, a frequency domain attention interpolation prediction module and a linear data projection module;

[0014] Sending the time series data to a wavelet decomposition module to separate detail features and approximate features in the data;

[0015] The separated detail feature data is input into the time domain linear prediction module to learn the features of the detail data;

[0016] The separated approximate feature data is input into the frequency domain attention interpolation prediction module to learn the features of the approximate data;

[0017] After combining the learned details with the approximate features, they are sent to the linear data projection module to obtain the regression value of the task prediction target.

[0018] Further, based on the actual value and the regression value, a trained neural network is obtained by an iterative optimization method of a loss function, including:

[0019] The mean square error (MSE) indicator is calculated using the regression value of the task prediction target and the actual value of the task target, and this value is used as the loss function;

[0020] The gradient of the loss function is calculated for each batch of samples and smoothed; the smoothed gradient is back-propagated to the parameters of the module networks at all levels, and the network parameters are updated as a training for the entire network; in this way, the entire network model is iteratively trained; after training, the network model and the best parameters after training are saved.

[0021] Furthermore, the actually collected industrial time series data is input into the trained neural network to obtain the prediction results of the task objectives.

[0022] Furthermore, the wavelet decomposition module uses Haar wavelet as wavelet basis function.

[0023] Furthermore, the time domain linear prediction module is composed of a neural network of a linear transformation layer, and its output is the detailed feature information extracted from the data.

[0024] Furthermore, the frequency domain attention interpolation prediction module includes a Fourier transform layer, a self-attention network module including a plurality of parallel self-attention heads, a residual connection normalization layer, a feedforward network layer, a post-residual connection normalization layer, a linear network projection layer, and an inverse Fourier transform layer;

[0025] The formula of the self-attention network module is: For an attention model with d hidden channel variables and h self-attention heads, the query, key, and value on the i-th head corresponds to Q i , K i , V i , Q i ,K i ,V i ∈, W Output Represents the learning parameters, and the process is as follows:

[0026]

[0027] head i =Attention(Q i ,K i ,V i )

[0028] MultiHead(Q,K,V)=W Output *Concat(head1,···,head h )

[0029] The superscript T indicates transposition;

[0030] The residual connection normalization layer and the post-residual connection normalization layer add the frequency domain input data and the output data of the previous layer after dropout processing to form a residual connection; then the result is normalized to stabilize the training process;

[0031] The feedforward network layer consists of the first linear transformation, activation function, and the second linear transformation, which is used to introduce the nonlinear representation ability of the model and enhance the model's ability to process complex data;

[0032] The approximate feature data is converted into frequency domain data through a Fourier transform layer, and then sequentially passes through a self-attention module, a residual connection normalization layer, a feedforward network layer, and a post-residual connection normalization layer to learn high-dimensional features in the frequency domain; the frequency domain interpolation feature scale is standardized through a linear network projection layer, and finally converted back to time domain space through an inverse Fourier transform layer to output the extracted data approximate feature information.

[0033] Furthermore, the linear data projection module is composed of a linear neural network layer, which inputs the detail feature information and approximate feature information extracted by the time domain linear prediction module and the frequency domain attention interpolation prediction module, and outputs the regression value of the task prediction target.

[0034] The present invention also provides a time series data prediction device based on wavelet time-frequency domain, comprising:

[0035] The data acquisition module is used to obtain time series data in the production process and select production-related parameters as the actual values ​​of the task objectives;

[0036] A model training module is used to input the time series data into a neural network model to obtain a regression value of a task prediction target; based on the actual value and the regression value, a trained neural network model is obtained by an iterative optimization method of a loss function;

[0037] The target prediction module is used to input the actually collected industrial time series data into the trained neural network model to obtain the prediction results of the task target.

[0038] The beneficial effects of the present invention are as follows: a time series data prediction method based on wavelet time and frequency domain of the present invention includes: the industrial gateway collects sensor information, and the data is preprocessed to obtain an overall multi-dimensional time series data set; some key parameters related to production decisions are selected as task objectives; the detailed features and approximate features in the data are separated by a wavelet decomposition module; the separated different feature data are input into the data prediction module, and the network parameters are trained and the optimal model parameters are saved by the loss function optimization method; the real-time data of the industrial production environment is input into the prediction model to obtain the prediction results of the relevant data. The present invention uses a wavelet decomposition module to effectively separate the detailed feature data and the approximate feature data, and respectively uses a time domain linear prediction module and a frequency domain attention interpolation prediction module for the details and approximate data, and trains a suitable prediction model according to different data characteristics to improve the accuracy of multi-dimensional industrial time series data prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0040] Figure 1 A schematic flow chart of a time series data prediction method based on wavelet time-frequency domain provided in the first embodiment of the present invention;

[0041] Figure 2A schematic diagram of a module of a time series data prediction device based on wavelet time-frequency domain provided in the first embodiment of the present invention;

[0042] Figure 3 A schematic diagram of a neural network model structure provided in Embodiment 1 of the present invention;

[0043] Figure 4 A schematic diagram of the structure of the industrial gateway power data prediction system provided in Example 2 of the present invention. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0045] First embodiment:

[0046] Glossary:

[0047] Detail feature: Detail feature is the detail coefficient part of wavelet decomposition, which describes the local changes and rapid fluctuations of the time series signal.

[0048] Approximate features: Approximate features are the approximate coefficient part of wavelet decomposition, which describes the global characteristics and overall information of the time series signal.

[0049] In order to improve industrial production efficiency and optimize production planning, it is necessary to predict some industrial data. The first embodiment of the present invention provides a time series data prediction method based on wavelet time-frequency domain, which is described in detail below.

[0050] like Figure 1 As shown, it is a flowchart of a time series data prediction method based on wavelet time-frequency domain provided by an embodiment of the present invention.

[0051] Step S1, obtaining time series data in the production process, and selecting parameters related to production as actual values ​​of the task target;

[0052] Specifically, in the industrial production process, a variety of different types of industrial sensors are usually deployed to collect the required data. The industrial gateway includes a configuration module and a collection module. The configuration module sets the collection module information so that the collection module can correctly obtain the data sent by the industrial sensors.

[0053] The collected sensor data is preprocessed and calculated on the edge data nodes of the network topology through the Lua script to obtain the initial data information. The multi-dimensional initial data information of different sensors is aligned on the time axis dimension to obtain the overall multi-dimensional industrial time series data set. Indicators that affect production decisions (i.e., production-related parameters), such as the load temperature of the power transformer, are selected as the task prediction targets of this data set.

[0054] Step S2, inputting the time series data into a neural network model to obtain a regression value of the task prediction target;

[0055] Specifically, the multidimensional time series data set is used as input data and sent to the wavelet decomposition module to separate the detail features and approximate features in the data. The separated detail feature data is input into the time domain linear prediction module to learn the features of the detail data. The separated approximate feature data is input into the frequency domain attention interpolation prediction module to learn the high-dimensional features of the approximate data. After combining the detail and approximate features, they are sent to the linear data projection module to obtain the regression value of the task prediction target.

[0056] Step S3, training the model to obtain the optimal model network parameters.

[0057] Specifically, the mean square error (MSE) indicator is calculated using the task target regression value and the task target actual value, and this value is used as the loss function. The gradient of the loss function is calculated for each batch of samples and smoothed. The smoothed gradient is back-propagated to the parameters of the module network at each level, and the network parameters are updated as a training of the entire network. In this way, the entire network model is iteratively trained. After the training is completed, the network model and the best parameters after training are saved.

[0058] Network training is an iterative optimization process. When the loss function changes less than the preset threshold or reaches the maximum number of iterations, the network training is stopped, and the optimal network parameters are saved during the training process. The network optimization method selects the Adam optimization algorithm based on gradient descent.

[0059] Step S4, sending the real-time industrial time series data to the trained neural network model to obtain the task target prediction value.

[0060] Specifically, the actually collected industrial time series data is input into the neural network model, and the trained network parameters are used to obtain the prediction results of the task objectives.

[0061] like Figure 2 FIG. 1 is a schematic diagram of a module of a time series data prediction device based on wavelet time-frequency domain provided by an embodiment of the present invention, wherein the device comprises:

[0062] The data acquisition module 201 is used to obtain time series data in the production process and select production-related parameters as actual values ​​of the task objectives;

[0063] The model training module 202 is to input the time series data into the neural network model to obtain the regression value of the task prediction target; based on the actual value and the regression value, a trained neural network model is obtained through a loss function iterative optimization method;

[0064] The target prediction module 203 inputs the actually collected industrial time series data into the trained neural network model to obtain the prediction result of the task target.

[0065] After the model training module 202 completes the network training, there is no need to iterate the training model again in the subsequent target prediction environment. It is only necessary to input the data set collected by the data collection module 201 into the target prediction module 203, which effectively reduces the calculation process of the prediction result.

[0066] It should be noted that the device embodiment shown in this embodiment matches the content of the above method embodiment, and the content of the above method embodiment can be referred to, which will not be repeated here.

[0067] like Figure 3 FIG. 1 is a schematic diagram of a neural network model structure provided by an embodiment of the present invention, including:

[0068] The wavelet decomposition module 301 uses Harr wavelet as wavelet basis function to decompose the input data into detail feature data and approximate feature data.

[0069] The time domain linear prediction module 302 is composed of a neural network with a linear transformation layer. The detailed feature data is input into the module, and the linear model is insensitive to noise to learn the characteristics of this type of data, and the extracted detailed feature information is output.

[0070] The frequency domain attention interpolation prediction module 303 includes a Fourier transform layer, a self-attention network module including several (e.g., 8) parallel self-attention heads, a residual connection normalization, a feedforward network layer, a post-residual connection normalization layer, a linear network projection layer, and an inverse Fourier transform layer. This module learns approximate feature data to improve the prediction accuracy of this type of data.

[0071] The formula of the self-attention network module is: For an attention model with d hidden channel variables and h self-attention heads, the query, key, and value on the i-th head corresponds to Q i , K i , V i , Q i ,K i ,Vi ∈, W Output Represents the learning parameters, and the process is as follows:

[0072]

[0073] head i =Attention(Q i ,K i ,V i )

[0074] MultiHead(Q,K,V)=W Output *Concat(head1,···,head h )

[0075] The residual connection normalization layer and the post-residual connection normalization layer add the frequency domain input data and the previous layer output data after dropout processing to form a residual connection. This result is then normalized to stabilize the training process.

[0076] The feedforward network layer consists of the first linear transformation, the activation function, and the second linear transformation, which is used to introduce the nonlinear representation capability of the model and enhance the model's ability to process complex data.

[0077] The approximate feature data is converted into frequency domain data through the Fourier transform layer, and then passes through the self-attention module, residual connection normalization layer, feedforward network layer and post-residual connection normalization layer in sequence to learn the high-dimensional features in the frequency domain. The frequency domain interpolation feature scale is standardized through the linear network projection layer, and finally converted back to the time domain space through the inverse Fourier transform layer to output the extracted data approximate feature information.

[0078] The linear data projection module 304 is composed of a linear neural network layer. This module combines the feature data output by the time domain linear prediction module 302 and the frequency domain attention interpolation prediction module 303, and finally outputs the regression value of the task prediction target.

[0079] The neural network model shown in the structural diagram needs to undergo iterative optimization training, and the mean square error MSE indicator is calculated using the task target regression value and the task target actual value as the loss function. When the loss function changes less than the preset threshold or reaches the maximum number of iterations, the network training is stopped, and the optimal network parameters are saved during the training process. The network optimization method selects the Adam optimization algorithm based on gradient descent.

[0080] Second implementation method:

[0081] The second embodiment of the present invention provides an industrial gateway power data prediction system. Figure 4As shown, it includes: a power sensor unit 401, a data transmission unit 402, a data preprocessing unit 403, a data storage unit 404, and a data prediction unit 405.

[0082] Specifically, the power sensor unit 401 is used to collect power data of industrial power equipment; the data transmission unit 402 is used to transmit data from the sensor to the industrial gateway through the 5G mobile network; the data preprocessing unit 403 is used to preprocess the original sensor data sent by the transmission module to obtain initial data information; the data storage unit 404 stores the initial power data for subsequent use; the data prediction unit 405 stores a time series data prediction method based on the wavelet time and frequency domain, and predicts future power data based on the stored historical power data, including the future power consumption of different production modules and equipment, the future total power consumption, etc.

[0083] It is not difficult to find that in this embodiment, a system embodiment corresponding to the first embodiment is described. This embodiment and the first embodiment can be applied in combination with each other. The technical details mentioned in the first embodiment are also applicable to this embodiment. In order to avoid duplication, it is not described in detail here. Accordingly, the relevant technical details in this embodiment can also be applied to the first embodiment.

[0084] The above is only an embodiment of the present invention. The common sense such as the known specific structure and characteristics in the scheme is not described in detail here. The ordinary technicians in the relevant field know all the common technical knowledge in the technical field of the invention before the application date or priority date, can obtain all the existing technologies in the field, and have the ability to apply the conventional experimental means before that date. The ordinary technicians in the relevant field can improve and implement this scheme in combination with their own abilities under the enlightenment given by this application. Some typical known structures or known methods should not become obstacles for ordinary technicians in the relevant field to implement this application. It should be pointed out that for those skilled in the art, without departing from the structure of the present invention, several deformations and improvements can be made, which should also be regarded as the protection scope of the present invention, which will not affect the effect of the implementation of the present invention and the practicality of the patent. The protection scope required by this application shall be based on the content of its claims, and the specific implementation methods and other records in the specification can be used to interpret the content of the claims.

[0085] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A time series data prediction method based on wavelet time-frequency domain, characterized in that: include: Obtain time series data from the production process and select production-related parameters as the actual values ​​of the task objectives; Inputting the time series data into a neural network model to obtain a regression value of a task prediction target; Based on the actual value and the regression value, a trained neural network model is obtained through a loss function iterative optimization method.

2. The method according to claim 1, characterized in that The obtaining of time series data in the production process includes: Acquire data from different types of industrial sensors deployed in the work area through industrial gateways; The data is preprocessed and information of different types and dimensions is aligned on the time axis to obtain an overall multi-dimensional time series data set.

3. The method according to claim 1, characterized in that The neural network includes a wavelet decomposition module, a time domain linear prediction module, a frequency domain attention interpolation prediction module and a linear data projection module; Sending the time series data to a wavelet decomposition module to separate detail features and approximate features in the data; The separated detail feature data is input into the time domain linear prediction module to learn the features of the detail data; The separated approximate feature data is input into the frequency domain attention interpolation prediction module to learn the features of the approximate data; After combining the learned details with the approximate features, they are sent to the linear data projection module to obtain the regression value of the task prediction target.

4. The method according to claim 1, characterized in that: Based on the actual value and the regression value, a trained neural network is obtained by an iterative optimization method of a loss function, including: The mean square error (MSE) indicator is calculated using the regression value of the task prediction target and the actual value of the task target, and this value is used as the loss function; The gradient of the loss function is calculated for each batch of samples and smoothed; the smoothed gradient is back-propagated to the parameters of the module networks at all levels, and the network parameters are updated as a training for the entire network; in this way, the entire network model is iteratively trained; after training, the network model and the best parameters after training are saved.

5. The method according to claim 1, characterized in that The actual collected industrial time series data is input into the trained neural network to obtain the prediction results of the task objectives.

6. The method according to claim 3, characterized in that The wavelet decomposition module uses Haar wavelet as wavelet basis function.

7. The method according to claim 3, characterized in that The time domain linear prediction module is composed of a neural network of a linear transformation layer, and its output is the detailed feature information extracted from the data.

8. The method according to claim 3, characterized in that The frequency domain attention interpolation prediction module includes a Fourier transform layer, a self-attention network module including a plurality of parallel self-attention heads, a residual connection normalization layer, a feedforward network layer, a post-residual connection normalization layer, a linear network projection layer, and an inverse Fourier transform layer; The formula of the self-attention network module is: For an attention model with d hidden channel variables and h self-attention heads, the query, key, and value on the i-th head corresponds to Q i , K i , V i , Q i ,K i ,V i ∈, W Output Represents the learning parameters, and the process is as follows: head i =Attention(Q i ,K i ,V i ) MultiHead(Q,K,V)=W Output *Concat(head1,···,head h ) The superscript T indicates transposition; The residual connection normalization layer and the post-residual connection normalization layer add the frequency domain input data and the output data of the previous layer after dropout processing to form a residual connection; then the result is normalized to stabilize the training process; The feedforward network layer consists of the first linear transformation, activation function, and the second linear transformation, which is used to introduce the nonlinear representation ability of the model and enhance the model's ability to process complex data; The approximate feature data is converted into frequency domain data through a Fourier transform layer, and then sequentially passes through a self-attention module, a residual connection normalization layer, a feedforward network layer, and a post-residual connection normalization layer to learn high-dimensional features in the frequency domain; the frequency domain interpolation feature scale is standardized through a linear network projection layer, and finally converted back to time domain space through an inverse Fourier transform layer to output the extracted data approximate feature information.

9. The method according to claim 3, characterized in that: The linear data projection module is composed of a linear neural network layer, which inputs the detail feature information and approximate feature information extracted by the time domain linear prediction module and the frequency domain attention interpolation prediction module, and outputs the regression value of the task prediction target.

10. A time series data prediction device based on wavelet time-frequency domain, characterized in that: include: The data acquisition module is used to obtain time series data in the production process and select production-related parameters as the actual values ​​of the task objectives; A model training module, used to input the time series data into a neural network model to obtain a regression value of a task prediction target; Based on the actual value and the regression value, a trained neural network model is obtained by using a loss function iterative optimization method; The target prediction module is used to input the actually collected industrial time series data into the trained neural network model to obtain the prediction results of the task target.

Images