Prediction method and device of time series data and electronic equipment

Abstract

The invention discloses a time series data prediction method and device and electronic equipment. The method comprises the steps that time series data related to a prediction task in the industrial production process are acquired, and the time series data at least comprise raw material test data, product test data, process variables and operation variables related to the prediction task; a prediction model is adopted to process the time series data to obtain a prediction result corresponding to the prediction task, the prediction model at least comprises a blocking layer and a normalization layer, the blocking layer is used for blocking the time series data according to the cycle length of each variable data in the time series data, and the normalization layer is used for normalizing the time series data according to the cycle length of each variable data in the time series data; and the normalization layer is used for carrying out normalization processing on the time sequence data after block processing. According to the method and the device, the technical problems that the data frequency difference, the distribution drift and the confidence coefficient change along with time existing in the time sequence data of different frequencies in the process industry cannot be effectively handled by the prediction method in the related technology are solved.

Description

Technical Field

[0001] The present application relates to the field of data processing, and in particular to a method, device and electronic device for predicting time series data. Background Art

[0002] In the field of process industry, the widespread application of automated monitoring and control systems has generated a large amount of time series data, covering all aspects from raw material input to product output, such as key parameters such as temperature, pressure, flow, and chemical composition. These data are not only large in volume, but also collected at different frequencies, forming a complex multi-frequency time series data set. For example, sensor data is often collected at a frequency of seconds or minutes, while laboratory data related to quality inspection may only be updated at the hourly level or longer. Therefore, due to the heterogeneity and multi-frequency of multi-frequency time series data in process industry, it poses a huge challenge to time series prediction methods.

[0003] At present, time series prediction mainly relies on three methods: statistical methods, traditional machine learning and deep learning. Statistical methods, such as ARIMA (Autoregressive Integrated Moving Average), make predictions by capturing the autocorrelation and seasonal patterns of data, but their flexibility and adaptability to complex patterns are limited. Traditional machine learning methods, such as linear regression, random forests, and ensemble learning algorithms, although capable of handling time series prediction tasks, usually require manual feature engineering and preprocessing of data to extract useful features, which is not only time-consuming but also difficult to maintain consistency in data of different frequencies. Deep learning, especially the Transformer model based on the self-attention mechanism, has made breakthrough progress in the fields of natural language processing and computer vision in recent years. Its powerful performance in processing sequence data has also attracted attention in the field of time series prediction. However, most of the Transformer-based time series prediction methods in related technologies are designed for same-frequency data. For different-frequency data, simple filling or interpolation methods are usually used to convert them into same-frequency data, which leads to the model's neglect of the confidence changes of low-frequency data and the lack of feature fusion of multi-frequency data.

[0004] In summary, in the prediction of multi-frequency time series data in process industries, the prediction models in related technologies are still unable to effectively handle problems such as data of different frequencies, dynamic changes in the confidence of low-frequency data, and feature fusion.

[0005] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0006] The embodiments of the present application provide a method, device and electronic device for predicting time series data, so as to at least solve the technical problems that the prediction methods in the related art cannot effectively deal with the data frequency differences, distribution drift and confidence changes over time existing in time series data of different frequencies in the process industry.

[0007] According to one aspect of an embodiment of the present application, a method for predicting time series data is provided, comprising: acquiring time series data related to a prediction task in an industrial production process, wherein the time series data at least includes raw material test data, product test data, process variables and operating variables related to the prediction task; using a prediction model to process the time series data to obtain a prediction result corresponding to the prediction task, wherein the prediction model at least includes a block layer and a normalization layer, the block layer is used to perform block processing on the time series data according to the period length of each variable data in the time series data, and the normalization layer is used to normalize the time series data after the block processing.

[0008] Optionally, the sampling frequency of the raw material test data and the product test data is a first frequency, and the sampling frequency of the process variables and the operation variables is a second frequency, and the first frequency is lower than the second frequency.

[0009] Optionally, after acquiring the time series data related to the prediction task in the industrial production process, it also includes data preprocessing of the time series data, wherein the data preprocessing method includes at least: outlier removal, missing value filling, frequency filling processing and normalization processing, wherein the frequency filling processing includes: determining a first sampling point and a second sampling point at a first frequency, and determining a target sampling value at the first sampling point, wherein the first sampling point is before the second sampling point, and the target sampling value is used to represent the raw material test data value and the product test data value at the first sampling point; determining a target sampling point between the first sampling point and the second sampling point according to the second frequency, wherein the target sampling point is used to represent the same sampling point as the second frequency; and filling the target sampling value into the target sampling point.

[0010] Optionally, after filling the target sampling value into the target sampling point, the method further includes: determining a first confidence at the first sampling point; determining a time distance between the first sampling point and the target sampling point; and determining a target confidence at the target sampling point based on the first confidence and the time distance, wherein the target confidence decreases exponentially with the time distance at the target sampling point.

[0011] Optionally, the period length of each variable data in the time series data is determined in the following manner: determine the time domain signal corresponding to each variable data in the time series data, wherein the time domain signal is used to represent the data sequence of each variable data changing with time; convert the time domain signal into a frequency domain signal by Fourier transform, wherein the frequency domain signal is used to represent the data sequence of each variable data changing with frequency; determine the period length of the target frequency in the frequency domain signal, wherein the target frequency is used to represent the frequency with the largest amplitude in the frequency domain signal; determine the period length of the target frequency as the period length of each variable data, wherein the period length of each variable data does not exceed a preset window length, and the preset window length is determined based on the historical time series data.

[0012] Optionally, the prediction model also includes an embedding layer, a flattening layer, an encoding layer and an output layer. The embedding layer is used to embed the normalized time series data, the flattening layer is used to flatten the time series data after the embedded representation, the encoding layer is used to encode the flattened time series data through a multi-head attention mechanism, and the output layer is used to generate the predicted length of each variable data in the encoded time series data.

[0013] Optionally, the method also includes: obtaining the predicted length of each variable data in the time series data; determining the mean and variance of the time series data within a preset time period, wherein the end time of the preset time period is earlier than the start time of the normalization processing; and performing denormalization processing on the predicted length based on the mean and variance to obtain a prediction result.

[0014] According to another aspect of an embodiment of the present application, a prediction device for time series data is also provided, including: an acquisition module, used to acquire time series data related to a prediction task in an industrial production process, wherein the time series data at least includes raw material test data, product test data, process variables and operation variables related to the prediction task; a processing module, used to process the time series data using a prediction model to obtain a prediction result corresponding to the prediction task, wherein the prediction model at least includes a block layer and a normalization layer, the block layer is used to perform block processing on the time series data according to the period length of each variable data in the time series data, and the normalization layer is used to normalize the time series data after block processing.

[0015] According to another aspect of an embodiment of the present application, there is also provided an electronic device, comprising: a memory and a processor, wherein the memory is used to store program instructions; the processor is connected to the memory and is used to execute the above-mentioned time series data prediction method.

[0016] According to another aspect of the embodiments of the present application, a non-volatile storage medium is provided, the non-volatile storage medium includes a stored computer program, wherein the device where the non-volatile storage medium is located executes the above-mentioned time series data prediction method by running the computer program.

[0017] According to another aspect of the embodiments of the present application, a computer program product is also provided, including computer instructions, which implement the above-mentioned time series data prediction method when executed by a processor.

[0018] In an embodiment of the present application, by acquiring time series data related to a prediction task in an industrial production process, wherein the time series data at least includes raw material test data, product test data, process variables and operating variables related to the prediction task; a prediction model is used to process the time series data to obtain a prediction result corresponding to the prediction task, wherein the prediction model at least includes a block layer and a normalization layer, the block layer is used to block the time series data according to the period length of each variable data in the time series data, and the normalization layer is used to normalize the time series data after the block processing, thereby achieving the purpose of effectively integrating and utilizing time series data of different frequencies for prediction, thereby achieving high-precision prediction of multi-frequency time series data in process industries, and improving the accuracy and adaptability of prediction models. The technical effect solves the technical problem that the prediction method in the related art cannot effectively deal with the data frequency differences, distribution drift and confidence changes over time in time series data of different frequencies in process industries. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0020] Figure 1 is a hardware structure diagram of a computer terminal for implementing a method for predicting time series data according to an embodiment of the present application;

[0021] Figure 2 is a flow chart of a method for predicting time series data according to an embodiment of the present application;

[0022] Figure 3 is a schematic diagram of a prediction effect of time series data according to an embodiment of the present application;

[0023] Figure 4 It is a structural diagram of a time series data prediction device according to an embodiment of the present application. DETAILED DESCRIPTION

[0024] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0025] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0026] First, some nouns or terms that appear in the process of explaining the embodiments of the present application are subject to the following explanations:

[0027] Transformer architecture: A deep learning architecture based on attention mechanism, originally proposed in the field of natural language processing for sequence-to-sequence tasks such as machine translation. It solves the efficiency and performance problems of recurrent neural networks and long short-term memory networks in processing long sequences through the combination of self-attention mechanism and feedforward neural network.

[0028] Online learning: A machine learning method in which the model can be updated and learned in real time while receiving new data, without waiting for the entire data set to be processed. It is particularly suitable for data stream or real-time data processing scenarios, and can quickly adapt to new information, improving the real-time and adaptability of the model.

[0029] Distribution drift: refers to the phenomenon that the statistical characteristics of data (such as mean, variance, and distribution shape) change over time in time series data. In industrial scenarios, this may be caused by changes in the production process, equipment aging, or adjustments to operating conditions, posing a challenge to the long-term stability and prediction accuracy of the model.

[0030] FFT (Fast Fourier Transform): An efficient algorithm for performing Fourier transforms, converting time-domain signals into frequency-domain representations. In this application, FFT is used to analyze the frequency characteristics of time series data, identify periodic patterns and characteristic frequencies, and provide information about the periodicity of the data to the model.

[0031] In order to solve the problem of poor time series data prediction efficiency in the related art, the present application embodiment provides a time series data prediction method, which can be run on Figure 1 In the computer terminal shown, the computer terminal is described below.

[0032] The time series data prediction method embodiment provided in the embodiment of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Figure 1 FIG. 1 shows a hardware structure block diagram of a computer terminal for implementing a method for predicting time series data. Figure 1 As shown, the computer terminal 10 may include one or more (102a, 102b, ..., 102n are used to illustrate) processors (the processor may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 104 for storing data, and a transmission module 106 for communication functions connected via a wired and / or wireless network. In addition, it may also include: a display, a keyboard, a cursor control device, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the I / O interface), a network interface, and a BUS bus. Those skilled in the art can understand that Figure 1 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.

[0033] It should be noted that the one or more processors and / or other data processing circuits described above may generally be referred to herein as "data processing circuits". The data processing circuits may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. In addition, the data processing circuit may be a single independent processing module, or may be incorporated in whole or in part into any of the other components in the computer terminal 10. As described in the embodiments of the present application, the data processing circuit acts as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0034] The memory 104 can be used to store software programs and modules of application software, such as program instructions / data storage devices corresponding to the prediction method of time series data in the embodiment of the present application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory 104, that is, realizing the above-mentioned prediction method of time series data. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor, and these remote memories may be connected to the computer terminal 10 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0035] The transmission module 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the computer terminal 10. In one example, the transmission module 106 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission module 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0036] The display may be, for example, a touch screen liquid crystal display (LCD) that enables a user to interact with a user interface of the computer terminal 10 .

[0037] It should be noted that, in some optional embodiments, the above Figure 1 The computer terminal shown may include hardware elements (including circuits), software elements (including computer code stored on a computer-readable medium), or a combination of hardware elements and software elements. It should be noted that Figure 1 This is merely one example of a particular embodiment and is intended to illustrate the types of components that may be present in the computer terminal described above.

[0038] In the above-mentioned operating environment, an embodiment of the present application provides an embodiment of a method for predicting time series data. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0039] Figure 2 is a flowchart of a method for predicting time series data according to an embodiment of the present application. Figure 2 As shown, the method comprises the following steps:

[0040] Step S202, acquiring time series data related to the forecasting task in the industrial production process, wherein the time series data at least includes raw material test data, product test data, process variables and operation variables related to the forecasting task.

[0041] In the above step S202, it is necessary to collect a series of time series data closely related to the prediction task. The time series data is composed of multiple variable data, including but not limited to the test data of raw materials, the test data of products (such as aromatics yield, etc.), process variables in the production process (such as temperature, pressure, flow, etc.), and operating variables (such as valve opening, etc.).

[0042] In the embodiment of the present application, taking the time series data of a continuous reforming unit operation as an example, a total of 267 variable data are included. The time series data covers multiple dimensions and time scales of the production process, providing a rich source of information for the model.

[0043] Step S204, use the prediction model to process the time series data to obtain a prediction result corresponding to the prediction task, wherein the prediction model includes at least a block layer and a normalization layer, the block layer is used to block the time series data according to the period length of each variable data in the time series data, and the normalization layer is used to normalize the time series data after block processing.

[0044] In the above step S204, the prediction model is based on the transformer architecture, including a block layer, a sample normalization layer, an embedding layer, a flattening layer, a coding layer and an output layer. Among them, the block layer is the core component in the prediction model, which performs intelligent block division according to the period length of each variable data in the time series data. Specifically, the main period of different variables is determined by fast Fourier transform (FFT), so that the data is divided into processable blocks, and the length of each block is determined according to its periodic characteristics. This block strategy is more flexible than the traditional method, and can adapt to data of different frequencies, especially the processing of low-frequency data is more refined. By introducing the confidence weight of time variation, the model can better understand and distinguish the timeliness of the data and improve the accuracy of the prediction.

[0045] The normalization layer plays the role of standardizing data in the data preprocessing stage, that is, normalizing each variable data after block division to ensure that the data is at the same level. In view of the characteristics of process industry data, such as distribution drift, the dynamic normalization method is adopted to automatically adjust the normalization parameters according to the data distribution in each time block, which helps the model capture the dynamic changes of data over time and reduce the prediction error caused by changes in data distribution.

[0046] Furthermore, the embedding layer is used to embed the normalized data through numerical encoding and position encoding; the flattening layer is used to flatten all variable data blocks into a one-dimensional sequence; the encoding layer is used to process the flattened sequence data using the transformer encoding mechanism and the multi-head attention mechanism; the output layer is used to map the encoded data into prediction results through a fully connected network, and then perform denormalization.

[0047] In summary, the prediction model achieves efficient processing and high-precision prediction of multi-frequency time series data in process industry through the synergy of block layer, sample normalization layer, embedding layer, flattening layer, encoding layer and output layer. The design of this prediction model fully considers the periodicity, distribution characteristics and time sequence of the data. Through the self-attention mechanism and dynamic normalization technology, it effectively solves the limitations of traditional prediction methods in processing heterogeneous frequency data, and provides a powerful tool for the intelligence and optimization of process industry.

[0048] Through the above steps S202 to S204, the purpose of effectively integrating and using time series data of different frequencies for prediction is achieved, thereby achieving high-precision prediction of multi-frequency time series data in process industries and improving the accuracy and adaptability of prediction models, thereby solving the technical problems that the prediction methods in related technologies cannot effectively deal with the data frequency differences, distribution drift and confidence changes over time in time series data of different frequencies in process industries. The following is a detailed description.

[0049] In the above step S202, the sampling frequency of the raw material test data and the product test data is a first frequency, and the sampling frequency of the process variables and the operation variables is a second frequency, and the first frequency is lower than the second frequency.

[0050] In the present application embodiment, time series data is divided into two kinds of frequencies: the sampling frequency of raw material test data and product test data is the first frequency (such as once every 8 hours), and the sampling frequency of process variables and operating variables is the second frequency (such as once every minute), that is, the first frequency (i.e., the frequency of raw material test data and product test data) is obviously lower than the second frequency (i.e., the frequency of process variables and operating variables). This is due to different types of monitoring and measurement requirements in industrial production. For example, the test data of raw materials and products need to be chemically analyzed, which usually involves complex laboratory processes, so the sampling period is relatively long, such as once every 8 hours. The monitoring of process variables (such as temperature, pressure, flow, etc.) and operating variables (such as valve opening, etc.) is carried out in real time by sensors installed on the production line, so high-frequency (such as minute-level) data collection can be achieved.

[0051] Optionally, after acquiring the time series data related to the prediction task in the industrial production process, it also includes data preprocessing of the time series data, wherein the data preprocessing method includes at least: outlier removal, missing value filling, frequency filling processing and normalization processing, wherein the frequency filling processing includes: determining a first sampling point and a second sampling point at a first frequency, and determining a target sampling value at the first sampling point, wherein the first sampling point is before the second sampling point, and the target sampling value is used to represent the raw material test data value and the product test data value at the first sampling point; determining a target sampling point between the first sampling point and the second sampling point according to the second frequency, wherein the target sampling point is used to represent the same sampling point as the second frequency; and filling the target sampling value into the target sampling point.

[0052] Furthermore, after filling the target sampling value into the target sampling point, it also includes: determining a first confidence at the first sampling point; determining a time distance between the first sampling point and the target sampling point; and determining a target confidence at the target sampling point based on the first confidence and the time distance, wherein the target confidence at the target sampling point decreases exponentially with the time distance.

[0053] In the embodiments of the present application, the time series data collected during the industrial production process often has certain defects, such as outliers, missing values, and sampling at different frequencies, etc. These defects may affect the performance of the prediction model and the accuracy of the prediction results. Therefore, after obtaining the time series data related to the prediction task, it is crucial to implement the data preprocessing step. The specific data preprocessing process can be as follows:

[0054] 1. Outlier elimination: For all variables, a reasonable threshold range is set according to the equipment process characteristics, and data outside this range is marked as outliers and eliminated. This processing method can ensure that the model is not misled by extreme or erroneous data and improve the robustness of the model.

[0055] 2. Missing value filling: Forward filling method is used to handle missing values. If the consecutive missing values ​​do not exceed two time steps, they are directly filled; if the consecutive missing values ​​exceed two time steps, they are kept as they are without filling. This processing can preserve the continuity of the data while avoiding the deviation introduced by large amounts of filling.

[0056] 3. Frequency filling and weight value generation: For non-co-frequency data, that is, the first frequency (low frequency, such as laboratory data, the sampling period may be as long as several hours) and the second frequency (high frequency, such as sensor data, the sampling period may be minutes or shorter), the data frequency is filled. Specifically, the low-frequency data is converted to the same frequency as the high-frequency data through forward filling, that is, the second frequency. At the same time, a confidence weight is added in the frequency filling operation. The weight is 1 at the sampling time and decreases exponentially with the increase of the distance sampling time to reflect the decrease in the confidence of the low-frequency data over time. The specific process can be as follows:

[0057] First, determine the first sampling point and the second sampling point at the first frequency, and determine the target sampling value at the first sampling point, wherein the target sampling value is used to represent the raw material test data value and the product test data value at the first frequency. Secondly, determine the target sampling points between the first sampling point and the second sampling point according to the second frequency (i.e., the sampling period of the high-frequency data), and these target sampling points are used to represent the same sampling points as the second frequency, that is, the sampling of the low-frequency data is complemented by the frequency of the high-frequency data. Finally, fill the target sampling value to the target sampling point. Specifically, for low-frequency data, the value of each sampling point is filled to the sampling point of the high-frequency data until the next sampling point, forming a sequence with the same frequency as the high-frequency data. In this process, each filled value is associated with a confidence weight that changes over time to reflect that the validity of the low-frequency data decreases over time.

[0058] Specifically, at the first sampling point (i.e., the actual sampling point of the low-frequency data), the confidence is initialized to 1, i.e., the first confidence mentioned above, which represents that at the moment when the data is actually sampled, its confidence is the highest, because this is the latest and most accurate state of the data. Subsequently, the time interval between the first sampling point and the target sampling point is calculated. In the industrial production process, this time distance can be understood as the time from data sampling to the time when the data is used for prediction. For low-frequency data, the longer the time distance, the more likely the validity of the data is to be questioned. Based on the first confidence and time distance, the model calculates the target confidence at the target sampling point, which decreases exponentially with the increase of time distance. This step reflects the understanding of the timeliness of data, that is, the closer the data is to the actual sampling time, the higher its confidence. As time goes by, the confidence gradually decreases, reflecting the natural decay of the validity of the data over time.

[0059] 4. Normalization: Normalize each variable to ensure that the value range of each variable is unified to the standard range with a mean of 0 and a variance of 1. This processing method can eliminate the dimensional influence between different variables and make the model more efficient and stable during the training process.

[0060] In the above step S204, the period length of each variable data in the time series data is determined in the following manner: determine the time domain signal corresponding to each variable data in the time series data, wherein the time domain signal is used to represent the data sequence of each variable data changing with time; convert the time domain signal into a frequency domain signal by Fourier transform, wherein the frequency domain signal is used to represent the data sequence of each variable data changing with frequency; determine the period length of the target frequency in the frequency domain signal, wherein the target frequency is used to represent the frequency with the largest amplitude in the frequency domain signal; determine the period length of the target frequency as the period length of each variable data, wherein the period length of each variable data does not exceed a preset window length, and the preset window length is determined based on the historical time series data.

[0061] In the embodiment of the present application, determining the period length of each variable data is achieved by converting the time domain signal into the frequency domain signal, which involves Fourier transform. The specific process can be as follows:

[0062] First, the time domain signal corresponding to each variable is identified from the original time series data collected during the industrial production process. The time domain signal directly reflects the change of the variable data over time and is the basis of time series analysis.

[0063] Secondly, by applying the fast Fourier transform, the time domain signal is converted into a frequency domain signal. The frequency domain signal shows the characteristics of each variable data changing with frequency, especially the periodic patterns in the data. FFT can quickly calculate the spectrum of the signal. For industrial time series data, it can help identify the main periodic components in the data, such as the target frequency mentioned above.

[0064] In the frequency domain signal, find the frequency with the largest amplitude, that is, the target frequency, which usually corresponds to the most significant periodic change in the data. For example, for some industrial processes, there may be obvious daily or weekly changes.

[0065] Finally, the period length corresponding to the target frequency is determined as the period length of each variable data. This step is crucial for capturing and understanding periodic patterns in the data.

[0066] It should be noted that the cycle length of each variable data does not exceed the preset window length. The preset window length is determined based on the analysis of historical time series data, and it represents the maximum time range of the forecast model when looking back at historical data. Ensuring that the cycle length does not exceed the preset window length helps the model focus on the most relevant periodic information when processing data, avoids interference from excessively long cycles, and improves the accuracy of forecasts.

[0067] In industrial production scenarios, this adaptive processing method based on cycle length can better adapt to data changes in different production stages and operating conditions, and provides a solid foundation for the construction of prediction models. For example, the cycle of raw material test data and product test data may be as long as several hours, while the cycle of process variables and operating variables may be as short as a few minutes. By determining the cycle length of each variable through FFT, the model can effectively distinguish and process data of different cycles, especially when dealing with periodic changes in low-frequency data, it can more accurately identify its regularity over time, which is crucial for predicting future changes in such data. The innovation of this cycle determination method is that it can not only automatically identify the periodic characteristics of different variables, but also limit the cycle length to within the preset window length, ensuring that the model can focus on the most relevant time period information when processing and predicting data, thereby improving the efficiency and accuracy of the prediction.

[0068] Optionally, the prediction model also includes an embedding layer, a flattening layer, an encoding layer and an output layer. The embedding layer is used to embed the normalized time series data, the flattening layer is used to flatten the time series data after the embedded representation, the encoding layer is used to encode the flattened time series data through a multi-head attention mechanism, and the output layer is used to generate the predicted length of each variable data in the encoded time series data.

[0069] In the embodiment of the present application, the prediction model realizes efficient processing and high-precision prediction of multi-frequency time series data in process industry through the synergy of the block layer, sample normalization layer, embedding layer, flattening layer, encoding layer and output layer. The above modules are described in detail below.

[0070] Block layer: used to divide the data into different time series according to the period length p of each variable data. (j) Perform non-overlapping segmentation on the time series data X. The specific expression is as follows:

[0071]

[0072] N (j) p =[L / p (j) ]

[0073] Where X represents time series data, p (j) represents the cycle length of each variable, L represents the lookback window length (i.e. the preset window length), and Padding(.) represents the period length when L cannot be p (j) When the number is divisible, the tail is padded with 0 until it is divisible; Slicing (.) means that according to p (j) Divide X into non-overlapping blocks; N (j) pIndicates the time block contained in each variable data after segmentation.

[0074] Sample normalization layer: used to normalize each variable data after block division. The specific expression is as follows:

[0075]

[0076] in, represents the normalized data, It represents the data at the i-th time point in the n-th block of the j-th variable data in the time series data, and α and β represent the model parameters.

[0077] Embedding layer: used to embed the normalized data through numerical encoding and position encoding. Specifically, the embedding layer includes two parts: feature embedding and position embedding. First, each time block is mapped to a vector of fixed size as its numerical encoding through value embedding. Secondly, time information is added through position embedding to ensure that the model can distinguish the time order of the data. Finally, the numerical encoding and position encoding are added to obtain the final embedded representation of each time block.

[0078] Flattening layer: It is used to reorganize all variable data processed by the embedding layer into a one-dimensional sequence to facilitate the subsequent attention calculation of the Transformer encoding layer. This processing method solves the problem of inconsistent lengths of different variables and enables the model to uniformly process diverse time series.

[0079] Coding layer: used to perform in-depth analysis and feature extraction on the flattened sequence data through a multi-head attention mechanism. Specifically, the coding layer is based on the Transformer architecture and uses a multi-head attention mechanism to perform calculations from multiple different attention spaces at the same time, improving the model's ability to process sequence data. Each "head" calculates the attention weight independently, and then merges these calculation results to obtain a more comprehensive and sophisticated data representation. In the embodiment of the present application, an 8-head attention mechanism is selected.

[0080] Output layer: used to convert the encoded data into prediction results through a fully connected network. Specifically, the output layer usually includes a fully connected neural network that maps the output of the encoding layer to a prediction length. The prediction length matches the prediction target of the input data. For example, for high-frequency data, the prediction length may be 720 time points, while for low-frequency data (such as laboratory data), the prediction length may be shorter, a mean prediction of 1 time point.

[0081] Optionally, the above method also includes: obtaining the predicted length of each variable data in the time series data; determining the mean and variance of the time series data within a preset time period, wherein the end time of the preset time period is earlier than the start time of the normalization processing; and performing denormalization processing on the predicted length based on the mean and variance to obtain a prediction result.

[0082] In the embodiment of the present application, the prediction length corresponding to each variable data output by the output layer needs to be denormalized to restore the scale of the original data, and finally output the prediction result corresponding to the prediction task. The details are as follows:

[0083] After the model outputs the normalized prediction length, these prediction values ​​need to be converted back to the original data scale to facilitate comparison and application with actual production data. The denormalization process is based on the dynamic mean and variance before normalization. Specifically, for each prediction value, first restore its original scale by multiplying it by the square root of the variance and adding the mean, and then use these restored prediction values ​​as the final prediction results. Among them, the dynamic mean and variance are determined before the normalization process, that is, before model training and prediction, determine the statistical characteristics of each variable within a preset time period. The selection of the preset time period should be based on historical data to ensure that the typical performance of the data and potential distribution drift are covered.

[0084] Among them, when determining the dynamic mean and variance, the dynamic mean network and dynamic variance network are composed of two linear layers, the first layer uses ReLU as the activation function, and the second layer uses the Identity function. This network structure can learn the nonlinear relationship in the data while ensuring that the output mean and variance are reasonable in value.

[0085] By obtaining the prediction length of the variables, determining the mean and variance within the preset time period, and performing denormalization, it is ensured that the model can not only capture the inherent laws of the data when processing complex multi-frequency time series data in process industries, but also generate predictive information that can be directly used for production management, providing strong technical support for intelligent production.

[0086] In an embodiment of the present application, a prediction model training method is also provided. The prediction model is similar to the above-mentioned prediction method for time series data, and includes the following steps:

[0087] S1. Obtain historical time series data related to the forecasting task of the industrial production process, including historical raw material test data, historical product test data, historical process variables and historical operation variables.

[0088] S2. Preprocess the historical time series data, including outlier removal, missing value filling, data frequency completion, confidence weight generation and normalization.

[0089] S3. Divide the historical time series data after data preprocessing into a training data set and a test data set.

[0090] S4. For the training data set, use the Fast Fourier Transform (FFT) module to analyze the periodicity of the data and determine the block period of each variable. If the period exceeds the lookback window length L, L is taken as the period length.

[0091] S5. Cut the training data set into a sample set with a length of L, a prediction step length T, in a sliding window manner. Specifically, the sample set is constructed as follows:

[0092] First, construct a feature dataset {X i:i+L}, using the form of sliding window, taking a sample every s steps, as follows:

[0093]

[0094] Among them, X i:i+L ∈R L×M , i is the index in the data, 1≤i≤(NLT) / s, is the value of the jth characteristic variable at the i+tth moment, y i+t is the value at the i+tth moment, where 1≤t≤L, N is the total amount of data, L is the length of the lookback window, T is the length of the prediction window, M is the total number of variables, and m is the amount to be predicted. In the embodiment of the present application, L=1440, T=720, M=267, and m=141.

[0095] Secondly, construct the prediction target dataset {Y i+L:i+L+T}, which is also constructed by sliding window, where T is the prediction length. The prediction lengths for data of different frequencies are different. For minute-level data, the prediction window length is T=720, and for low-frequency data such as laboratory data, the prediction length is 1 point, which is the mean value of T=720.

[0096] It should be noted that the feature dataset is used as the input of the model to train the model to identify and learn the potential temporal dependencies and patterns in the data; the target dataset is used as the label of the model to define the future data sequence that the model needs to predict. Together, they form the basis of model learning. By constructing a sample set through a sliding window, the model can effectively process multi-frequency time series data in process industries, learn the intrinsic patterns and temporal dependencies of the data, and generate accurate prediction results.

[0097] S6. Construct an initial prediction model based on the Transformer architecture, including blocking, sample normalization layer, embedding layer, flattening layer, encoding layer and output layer, and use the constructed sample set to train the initial prediction model.

[0098] S7. Use the test data set to verify the initial prediction model, and select the model with the lowest loss in the verification set as the final prediction model.

[0099] Specifically, the initial prediction model is trained using the MAE loss function and the Adam optimizer, with a learning rate of 10 -4 , the number of iterations is 100, and the batch_size is 128. Subsequently, the model performance is evaluated on the validation dataset, and the model with the lowest validation loss is selected as the final model. At the same time, during the prediction model training process, the parameters are dynamically adjusted to ensure that the model can adapt to the characteristics of time series data with different frequencies.

[0100] Figure 3 Schematic diagram of a prediction effect according to an embodiment of the present application. Figure 3 As shown in the figure, for high-frequency data (such as feed temperature), the model's prediction of T202 feed temperature is highly consistent with the actual value. Although the temperature fluctuates greatly, the predicted curve can still accurately follow the changing trend of the actual curve, showing the model's strong predictive ability when processing minute-level high-frequency data. Even if the temperature has obvious up and down fluctuations in a short period of time, the predicted data can still maintain a high accuracy, which shows that the model can effectively capture and predict short-term and long-term dependencies in time series. For low-frequency data (such as xylene content), the model also shows excellent predictive performance. As time goes by, the xylene content gradually decreases, and this long-term trend is accurately predicted by the model. Even if the xylene content changes slowly, the predicted curve still closely follows the actual curve, showing the stability of the model when processing low-frequency data. This shows that the model can adapt to time series data of different frequencies, whether the changes are fast or slow, it can maintain a high prediction accuracy.

[0101] The entire model training process is designed for the multi-frequency time series data prediction task in process industries, taking full account of the complexity and heterogeneity of the data. Through a series of innovative data processing and model training strategies, the accuracy and generalization ability of the prediction are improved.

[0102] In the embodiments of the present application, the limitations of traditional prediction methods in processing data of different frequencies are effectively solved through dynamic blocking, confidence weighting, dynamic normalization and denormalization, and a specially designed model structure. Among them, the FFT layer is used to determine the period of each variable, which ensures that the model can capture the periodic characteristics of the data; the position encoding and multi-head attention mechanism enhances the model's ability to process time series data. The dynamically adjusted mean and variance are used in the normalization and denormalization process to ensure the accuracy of the prediction results and the degree of matching with the actual data. The combination of this series of technical solutions enables the model to provide more accurate predictions when processing diversified data in process industries, significantly improves the prediction effect, and is of great value to industrial process control and optimization.

[0103] According to an embodiment of the present application, a prediction device for time series data is provided. It should be noted that the prediction device for time series data in the embodiment of the present application can be used to execute the prediction method for time series data provided in the embodiment of the present application. The prediction device for time series data provided in the embodiment of the present application is introduced below.

[0104] Figure 4 is a structural diagram of a time series data prediction device provided according to an embodiment of the present application. Figure 4 As shown, the device comprises:

[0105] An acquisition module 40 is used to acquire time series data related to the prediction task in the industrial production process, wherein the time series data at least includes raw material test data, product test data, process variables and operation variables related to the prediction task;

[0106] The processing module 42 is used to process the time series data using a prediction model to obtain a prediction result corresponding to the prediction task, wherein the prediction model includes at least a block layer and a normalization layer, the block layer is used to perform block processing on the time series data according to the period length of each variable data in the time series data, and the normalization layer is used to normalize the time series data after block processing.

[0107] Through the acquisition module 40 and the processing module 42 in the above-mentioned time series data prediction device, the purpose of effectively integrating and utilizing time series data of different frequencies for prediction is achieved, thereby realizing high-precision prediction of multi-frequency time series data in process industries and improving the accuracy and adaptability of prediction models. The technical problem that the prediction methods in related technologies cannot effectively deal with the data frequency differences, distribution drift and confidence changes over time in time series data of different frequencies in process industries is solved.

[0108] The prediction device for time series data provided in the embodiment of the present application also includes a preprocessing module 44, which is used to perform data preprocessing on the time series data, wherein the data preprocessing method at least includes: outlier removal, missing value filling, frequency filling processing and normalization processing, wherein the frequency filling processing includes: determining a first sampling point and a second sampling point at a first frequency, and determining a target sampling value at the first sampling point, wherein the first sampling point is before the second sampling point, and the target sampling value is used to represent the raw material test data value and the product test data value at the first sampling point; determining a target sampling point between the first sampling point and the second sampling point according to the second frequency, wherein the target sampling point is used to represent the same sampling point as the second frequency; and filling the target sampling value into the target sampling point.

[0109] In the prediction device for time series data provided in an embodiment of the present application, the preprocessing module is also used to determine a first confidence at a first sampling point; determine a time distance between the first sampling point and a target sampling point; and determine a target confidence at the target sampling point based on the first confidence and the time distance, wherein the target confidence decreases exponentially with the time distance at the target sampling point.

[0110] The prediction device for time series data provided in the embodiment of the present application also includes a determination module 46, which is used to determine the time domain signal corresponding to each variable data in the time series data, wherein the time domain signal is used to represent the data sequence of each variable data changing with time; convert the time domain signal into a frequency domain signal through Fourier transform, wherein the frequency domain signal is used to represent the data sequence of each variable data changing with frequency; determine the period length of the target frequency in the frequency domain signal, wherein the target frequency is used to represent the frequency with the largest amplitude in the frequency domain signal; determine the period length of the target frequency as the period length of each variable data, wherein the period length of each variable data does not exceed a preset window length, and the preset window length is determined based on the historical time series data.

[0111] In the prediction device for time series data provided in an embodiment of the present application, the processing module is also used to obtain the predicted length of each variable data in the time series data; determine the mean and variance of the time series data within a preset time period, wherein the end time of the preset time period is earlier than the start time of the normalization processing; and perform denormalization processing on the predicted length based on the mean and variance to obtain a prediction result.

[0112] An embodiment of the present application also provides an electronic device, including: a memory and a processor, wherein the memory is used to store program instructions; the processor is connected to the memory and is used to execute the above-mentioned time series data prediction method.

[0113] It should be noted that the above electronic equipment is used to execute Figure 2The prediction method for time series data shown in the figure, therefore the relevant explanations and instructions in the above-mentioned prediction method for time series data are also applicable to the electronic device and will not be repeated here.

[0114] An embodiment of the present application further provides a non-volatile storage medium, which includes a stored computer program, wherein the device where the non-volatile storage medium is located executes the above-mentioned time series data prediction method by running the computer program.

[0115] It should be noted that the above non-volatile storage medium is used to execute Figure 2 The prediction method for time series data shown in the figure, therefore the relevant explanations in the above-mentioned prediction method for time series data are also applicable to the non-volatile storage medium, and will not be repeated here.

[0116] An embodiment of the present application also provides a computer program product, including computer instructions, which implement the above-mentioned time series data prediction method when executed by a processor.

[0117] It should be noted that the above-mentioned computer program product is used to execute Figure 2 The prediction method for time series data shown in the figure, therefore the relevant explanations in the above-mentioned prediction method for time series data are also applicable to the computer program product and will not be repeated here.

[0118] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0119] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0120] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0121] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0122] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0123] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk and other media that can store program codes.

[0124] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A method for predicting time series data, characterized in that: include: Acquire time series data related to the forecasting task in the industrial production process, wherein the time series data at least includes raw material test data, product test data, process variables and operation variables related to the forecasting task; The time series data is processed using a prediction model to obtain a prediction result corresponding to the prediction task, wherein the prediction model includes at least a block layer and a normalization layer, the block layer is used to perform block processing on the time series data according to the period length of each variable data in the time series data, and the normalization layer is used to normalize the time series data after the block processing.

2. The method according to claim 1, characterized in that The sampling frequency of the raw material test data and the product test data is a first frequency, and the sampling frequency of the process variable and the operation variable is a second frequency, and the first frequency is lower than the second frequency.

3. The method according to claim 2, characterized in that After obtaining the time series data related to the forecasting task in the industrial production process, the method further includes performing data preprocessing on the time series data, wherein the data preprocessing method at least includes: outlier removal, missing value filling, frequency filling processing and normalization processing, wherein the frequency filling processing includes: Determine a first sampling point and a second sampling point at the first frequency, and determine a target sampling value at the first sampling point, wherein the first sampling point is before the second sampling point, and the target sampling value is used to represent the raw material test data value and the product test data value at the first sampling point; Determine a target sampling point between the first sampling point and the second sampling point according to the second frequency, wherein the target sampling point is used to represent the same sampling point as the second frequency; Fill the target sampling point with the target sampling value.

4. The method according to claim 3, characterized in that After filling the target sampling value to the target sampling point, the method further includes: determining a first confidence level at the first sampling point; Determining a time distance between the first sampling point and the target sampling point; A target confidence at the target sampling point is determined according to the first confidence and the time distance, wherein the target confidence at the target sampling point decreases exponentially with the time distance.

5. The method according to claim 1, characterized in that The period length of each variable data in the time series data is determined by: Determine a time domain signal corresponding to each variable data in the time series data, wherein the time domain signal is used to represent a data sequence of each variable data changing over time; Converting the time domain signal into a frequency domain signal by Fourier transform, wherein the frequency domain signal is used to represent a data sequence of each variable data varying with frequency; Determining a period length of a target frequency in the frequency domain signal, wherein the target frequency is used to represent a frequency with a maximum amplitude in the frequency domain signal; The cycle length of the target frequency is determined as the cycle length of each variable data, wherein the cycle length of each variable data does not exceed a preset window length, and the preset window length is determined based on historical time series data.

6. The method according to claim 1, characterized in that The prediction model also includes an embedding layer, a flattening layer, an encoding layer and an output layer. The embedding layer is used to embed the normalized time series data, the flattening layer is used to flatten the embedded time series data, the encoding layer is used to encode the flattened time series data through a multi-head attention mechanism, and the output layer is used to generate the predicted length of each variable data in the encoded time series data.

7. The method according to claim 6, characterized in that The method further comprises: Obtaining the predicted length of each variable data in the time series data; Determining the mean and variance of the time series data within a preset time period, wherein the end time of the preset time period is earlier than the start time of the normalization process; The predicted length is denormalized according to the mean and the variance to obtain the prediction result.

8. A prediction device for time series data, characterized in that: include: An acquisition module, used for acquiring time series data related to the forecasting task in the industrial production process, wherein the time series data at least includes raw material test data, product test data, process variables and operation variables related to the forecasting task; A processing module is used to process the time series data using a prediction model to obtain a prediction result corresponding to the prediction task, wherein the prediction model includes at least a block layer and a normalization layer, the block layer is used to perform block processing on the time series data according to the period length of each variable data in the time series data, and the normalization layer is used to normalize the time series data after the block processing.

9. An electronic device, characterized in that: include: A memory and a processor, wherein the memory is used to store program instructions; The processor is connected to the memory and is used to execute the time series data prediction method described in any one of claims 1 to 7.

10. A non-volatile storage medium, characterized in that: The non-volatile storage medium includes a stored computer program, wherein the device where the non-volatile storage medium is located executes the time series data prediction method according to any one of claims 1 to 7 by running the computer program.

11. A computer program product comprising computer instructions, characterized in that: When the computer instructions are executed by a processor, the method for predicting time series data described in any one of claims 1 to 7 is implemented.

Images