Method, device and electronic equipment for time series data prediction

Abstract

This application discloses a method, apparatus, and electronic device for predicting time series data. The method includes: acquiring time series data related to a prediction task in an industrial production process, wherein the time series data includes at least raw material testing data, product testing data, process variables, and operational variables related to the prediction task; processing the time series data using a prediction model to obtain a prediction result corresponding to the prediction task, wherein the prediction model includes at least a block layer and a normalization layer, the block layer being used to block the time series data according to the period length of each variable data in the time series data, and the normalization layer being used to normalize the block-processed time series data. This application solves the technical problems that prediction methods in related technologies cannot effectively address the differences in data frequency, distribution drift, and changes in confidence over time in time series data of different frequencies in process industries.

Description

Technical Field

[0001] This application relates to the field of data processing, and more specifically, to a method, apparatus, and electronic device for predicting time series data. Background Technology

[0002] In the process industries, the widespread application of automated monitoring and control systems generates a massive amount of time-series data, covering all stages from raw material input to product output, including key parameters such as temperature, pressure, flow rate, and chemical composition. This data is not only voluminous but also collected at varying frequencies, forming complex, multi-frequency time-series datasets. For example, sensor data is often collected at the second or minute level, while quality control-related laboratory data may only be updated at the hourly or longer interval. Therefore, the heterogeneity and frequency of multi-frequency time-series data in the process industries pose a significant challenge to time-series forecasting methods.

[0003] Currently, time series forecasting mainly relies on three methods: statistical methods, traditional machine learning, and deep learning. Statistical methods, such as ARIMA (Autoregressive Integrated Moving Average), make predictions by capturing the autocorrelation and seasonal patterns of data, but their flexibility and adaptability to complex patterns are limited. Traditional machine learning methods, such as linear regression, random forests, and ensemble learning algorithms, while capable of handling time series forecasting tasks, typically require manual feature engineering to preprocess the data to extract useful features, which is not only time-consuming but also difficult to maintain consistency across different frequency data. Deep learning, especially the Transformer model based on the self-attention mechanism, has made breakthroughs in natural language processing and computer vision in recent years, and its powerful performance in processing sequential data has attracted attention in the field of time series forecasting. However, most Transformer-based time series forecasting methods are designed for data with the same frequency. For data with different frequencies, simple padding or interpolation methods are usually used to convert it into data with the same frequency. This leads to the model ignoring the confidence changes of low-frequency data and insufficient feature fusion for multi-frequency data.

[0004] In summary, in the prediction of multi-frequency time series data in the process industry, the prediction models in related technologies still cannot effectively handle problems such as different frequency data, dynamic changes in confidence of low-frequency data, and feature fusion.

[0005] There is currently no effective solution to the above problems. Summary of the Invention

[0006] This application provides a method, apparatus, and electronic device for predicting time series data, which at least solves the technical problems that prediction methods in related technologies cannot effectively address the differences in data frequency, distribution drift, and changes in confidence over time in time series data of different frequencies in process industries.

[0007] According to one aspect of the embodiments of this application, a method for predicting time series data is provided, comprising: acquiring time series data related to a prediction task in an industrial production process, wherein the time series data includes at least raw material test data, product test data, process variables, and operational variables related to the prediction task; processing the time series data using a prediction model to obtain a prediction result corresponding to the prediction task, wherein the prediction model includes at least a block layer and a normalization layer, the block layer being used to block the time series data according to the period length of each variable data in the time series data, and the normalization layer being used to normalize the block-processed time series data.

[0008] Optionally, the sampling frequency of raw material test data and product test data is the first frequency, and the sampling frequency of process variables and operational variables is the second frequency, with the first frequency being lower than the second frequency.

[0009] Optionally, after acquiring time-series data related to the forecasting task during industrial production, the method further includes data preprocessing of the time-series data. The data preprocessing methods include at least: outlier removal, missing value imputation, frequency completion processing, and normalization processing. The frequency completion processing includes: determining a first sampling point and a second sampling point at a first frequency, and determining a target sampling value at the first sampling point, wherein the first sampling point is before the second sampling point, and the target sampling value represents the raw material test data value and product test data value at the first sampling point; determining a target sampling point between the first and second sampling points based on a second frequency, wherein the target sampling point represents a sampling point with the same second frequency; and filling the target sampling point with the target sampling value.

[0010] Optionally, after filling the target sampling point with the target sample value, the method further includes: determining a first confidence level at the first sampling point; determining the time distance between the first sampling point and the target sampling point; and determining a target confidence level at the target sampling point based on the first confidence level and the time distance, wherein the target confidence level decreases exponentially with the time distance at the target sampling point.

[0011] Optionally, the period length of each variable data in the time series data is determined as follows: a time-domain signal corresponding to each variable data in the time series data is determined, wherein the time-domain signal represents the data sequence of each variable data changing over time; the time-domain signal is converted into a frequency-domain signal through Fourier transform, wherein the frequency-domain signal represents the data sequence of each variable data changing with frequency; the period length of a target frequency in the frequency-domain signal is determined, wherein the target frequency represents the frequency with the largest amplitude in the frequency-domain signal; the period length of the target frequency is determined as the period length of each variable data, wherein the period length of each variable data does not exceed a preset window length, which is determined based on historical time series data.

[0012] Optionally, the prediction model also includes an embedding layer, a flattening layer, an encoding layer, and an output layer. The embedding layer is used to embed the normalized time series data, the flattening layer is used to flatten the embedded time series data, the encoding layer is used to encode the flattened time series data through a multi-head attention mechanism, and the output layer is used to generate the prediction length of each variable in the encoded time series data.

[0013] Optionally, the method further includes: obtaining the prediction length of each variable in the time series data; determining the mean and variance of the time series data within a preset time period, wherein the end time of the preset time period is earlier than the start time of the normalization process; and performing inverse normalization on the prediction length based on the mean and variance to obtain the prediction result.

[0014] According to another aspect of the embodiments of this application, a time series data prediction device is also provided, comprising: an acquisition module, configured to acquire time series data related to a prediction task in an industrial production process, wherein the time series data includes at least raw material test data, product test data, process variables, and operational variables related to the prediction task; and a processing module, configured to process the time series data using a prediction model to obtain a prediction result corresponding to the prediction task, wherein the prediction model includes at least a block layer and a normalization layer, the block layer being configured to block the time series data according to the period length of each variable data in the time series data, and the normalization layer being configured to normalize the block-processed time series data.

[0015] According to another aspect of the embodiments of this application, an electronic device is also provided, including: a memory and a processor, wherein the memory is used to store program instructions; and the processor is connected to the memory and used to execute the prediction method for the time series data described above.

[0016] According to another aspect of the embodiments of this application, a non-volatile storage medium is also provided, the non-volatile storage medium including a stored computer program, wherein the device where the non-volatile storage medium is located executes the above-mentioned prediction method for time series data by running the computer program.

[0017] According to another aspect of the embodiments of this application, a computer program product is also provided, including computer instructions, which, when executed by a processor, implement the above-described method for predicting time series data.

[0018] In this embodiment, time series data related to the prediction task in the industrial production process is acquired. This time series data includes at least raw material testing data, product testing data, process variables, and operational variables related to the prediction task. A prediction model is used to process the time series data to obtain prediction results corresponding to the prediction task. The prediction model includes at least a block layer and a normalization layer. The block layer is used to block the time series data according to the period length of each variable data in the time series data. The normalization layer is used to normalize the block-processed time series data. This achieves the purpose of effectively integrating and utilizing time series data of different frequencies for prediction, thereby realizing high-precision prediction of multi-frequency time series data in the process industry and improving the accuracy and adaptability of the prediction model. This solves the technical problem that prediction methods in related technologies cannot effectively cope with the differences in data frequency, distribution drift, and changes in confidence over time in time series data of different frequencies in the process industry. Attached Figure Description

[0019] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments of this application and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0020] Figure 1 This is a hardware structure diagram of a computer terminal for implementing a prediction method for time series data according to an embodiment of this application;

[0021] Figure 2 This is a flowchart of a time series data prediction method according to an embodiment of this application;

[0022] Figure 3 This is a schematic diagram illustrating the prediction effect of time series data according to an embodiment of this application;

[0023] Figure 4 This is a structural diagram of a time series data prediction device according to an embodiment of this application. Detailed Implementation

[0024] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0025] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0026] First, some nouns or terms that appear in the explanation of the embodiments of this application shall be interpreted as follows:

[0027] Transformer architecture: A deep learning architecture based on attention mechanisms, originally proposed in the field of natural language processing for sequence-to-sequence tasks such as machine translation. It addresses the efficiency and performance issues of recurrent neural networks and long short-term memory networks when processing long sequences by combining self-attention mechanisms and feedforward neural networks.

[0028] Online learning: A machine learning method where the model can update and learn in real time while receiving new data, without waiting for the entire dataset to be processed. It is particularly suitable for data stream or real-time data processing scenarios, enabling rapid adaptation to new information and improving the model's real-time performance and adaptability.

[0029] Distribution drift refers to the phenomenon where the statistical properties (such as mean, variance, and distribution shape) of time-series data change over time. In industrial scenarios, this can be caused by changes in production processes, equipment aging, or adjustments to operating conditions, posing a challenge to the long-term stability and predictive accuracy of models.

[0030] FFT (Fast Fourier Transform): An efficient algorithm for performing Fourier transforms, converting time-domain signals into frequency-domain representations. In this application, FFT is used to analyze the frequency characteristics of time-series data, identify periodic patterns and characteristic frequencies, and provide the model with information about the periodicity of the data.

[0031] To address the problem of poor prediction efficiency for time series data in related technologies, this application provides a method for predicting time series data, which can be implemented in... Figure 1 The computer terminal shown is described below.

[0032] The time series data prediction method provided in this application can be executed on a mobile terminal, computer terminal, or similar computing device. Figure 1 A hardware block diagram of a computer terminal for implementing a prediction method for time series data is shown. Figure 1 As shown, the computer terminal 10 may include one or more processors (shown as 102a, 102b, ..., 102n in the figure) (the processor may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.), a memory 104 for storing data, and a transmission module 106 for communication functions connected via wired and / or wireless networks. In addition, it may also include: a display, a keyboard, a cursor control device, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the I / O interface), a network interface, and a BUS bus. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the aforementioned electronic device. For example, computer terminal 10 may also include... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0033] It should be noted that the aforementioned one or more processors and / or other data processing circuits are generally referred to herein as "data processing circuits". These data processing circuits may be implemented wholly or partially as software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuits may be a single, independent processing module, or may be wholly or partially integrated into any other element in the computer terminal 10. As involved in the embodiments of this application, the data processing circuits serve as processor control (e.g., selection of a variable resistor termination path connected to an interface).

[0034] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the time series data prediction method in this embodiment. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory 104, thereby realizing the aforementioned time series data prediction method. The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor, and these remote memories can be connected to the computer terminal 10 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0035] The transmission module 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of the computer terminal 10. In one example, the transmission module 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission module 106 may be a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0036] The display may be, for example, a touchscreen liquid crystal display (LCD) that allows the user to interact with the user interface of the computer terminal 10.

[0037] It should be noted here that, in some optional embodiments, the above... Figure 1 The computer terminal shown may include hardware elements (including circuitry), software elements (including computer code stored on a computer-readable medium), or a combination of both hardware and software elements. It should be noted that... Figure 1 This is only one instance of a specific particular instance, and is intended to illustrate the types of components that may exist in the aforementioned computer terminal.

[0038] In the above operating environment, this application provides an embodiment of a method for predicting time series data. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0039] Figure 2 This is a flowchart of a time series data prediction method according to an embodiment of this application, such as... Figure 2 As shown, the method includes the following steps:

[0040] Step S202: Obtain time series data related to the prediction task in the industrial production process, wherein the time series data includes at least raw material test data, product test data, process variables and operation variables related to the prediction task.

[0041] In step S202 above, it is necessary to collect a series of time series data closely related to the prediction task. This time series data consists of multiple variable data, including but not limited to raw material test data, product test data (such as aromatic yield), process variables in the production process (such as temperature, pressure, flow rate, etc.), and operational variables (such as valve opening degree, etc.).

[0042] In this embodiment of the application, taking the time series data of a certain continuous reforming unit as an example, a total of 267 variable data are included. This time series data covers multiple dimensions and time scales of the production process, providing a rich source of information for the model.

[0043] Step S204: The time series data is processed using a prediction model to obtain the prediction result corresponding to the prediction task. The prediction model includes at least a block layer and a normalization layer. The block layer is used to divide the time series data into blocks according to the period length of each variable data in the time series data. The normalization layer is used to normalize the time series data after block processing.

[0044] In step S204 above, the prediction model is based on a transformer architecture, including a block layer, a sample normalization layer, an embedding layer, a flattening layer, an encoding layer, and an output layer. The block layer is the core component of the prediction model; it intelligently blocks the data based on the period length of each variable in the time series data. Specifically, the main period of different variables is determined by the Fast Fourier Transform (FFT), thereby dividing the data into processable blocks, the length of which depends on its periodic characteristics. This block strategy is more flexible than traditional methods and can adapt to data of different frequencies, especially providing more refined processing for low-frequency data. By introducing confidence weights based on time variations, the model can better understand and distinguish the timeliness of the data, improving prediction accuracy.

[0045] The normalization layer plays a crucial role in the data preprocessing stage by standardizing the data. It normalizes the data for each variable after data is divided into blocks, ensuring the data is on the same order of magnitude. Given the characteristics of process industry data, such as distribution drift, a dynamic normalization method is employed. This method automatically adjusts the normalization parameters based on the data distribution within each time block, helping the model capture the dynamic changes in data over time and reducing prediction errors caused by variations in data distribution.

[0046] Furthermore, the embedding layer is used to embed the normalized data into a representation using numerical encoding and positional encoding; the flattening layer is used to flatten all variable data blocks into a one-dimensional sequence; the encoding layer is used to process the flattened sequence data using the transformer's encoding mechanism and a multi-head attention mechanism; and the output layer is used to map the encoded data into prediction results through a fully connected network, followed by inverse normalization processing.

[0047] In summary, the prediction model achieves efficient processing and high-precision prediction of multi-frequency time series data in process industries through the synergistic effect of the block layer, sample normalization layer, embedding layer, flattening layer, encoding layer, and output layer. The model's design fully considers the periodicity, distribution characteristics, and temporal order of the data. Through self-attention mechanisms and dynamic normalization techniques, it effectively overcomes the limitations of traditional prediction methods when processing heterogeneous frequency data, providing a powerful tool for the intelligentization and optimization of process industries.

[0048] Through steps S202 to S204 above, the goal of effectively fusing and utilizing time series data of different frequencies for prediction is achieved. This enables high-precision prediction of multi-frequency time series data in the process industry, improves the accuracy and adaptability of the prediction model, and solves the technical problems that prediction methods in related technologies cannot effectively cope with the differences in data frequency, distribution drift, and changes in confidence over time in time series data of different frequencies in the process industry. A detailed explanation follows.

[0049] In step S202 above, the sampling frequency of raw material test data and product test data is the first frequency, and the sampling frequency of process variables and operation variables is the second frequency. The first frequency is lower than the second frequency.

[0050] In this embodiment, time series data is divided into two frequencies: the sampling frequency for raw material and product testing data is the first frequency (e.g., once every 8 hours), while the sampling frequency for process variables and operational variables is the second frequency (e.g., once per minute). The first frequency (i.e., the frequency of raw material and product testing data) is significantly lower than the second frequency (i.e., the frequency of process variables and operational variables). This is due to the different types of monitoring and measurement needs in industrial production. For example, raw material and product testing data require chemical analysis, which typically involves complex laboratory procedures, thus requiring a longer sampling cycle, such as once every 8 hours. In contrast, the monitoring of process variables (e.g., temperature, pressure, flow rate, etc.) and operational variables (e.g., valve opening, etc.) is performed in real-time using sensors installed on the production line, thus enabling high-frequency (e.g., minute-level) data acquisition.

[0051] Optionally, after acquiring time-series data related to the forecasting task during industrial production, the method further includes data preprocessing of the time-series data. The data preprocessing methods include at least: outlier removal, missing value imputation, frequency completion processing, and normalization processing. The frequency completion processing includes: determining a first sampling point and a second sampling point at a first frequency, and determining a target sampling value at the first sampling point, wherein the first sampling point is before the second sampling point, and the target sampling value represents the raw material test data value and product test data value at the first sampling point; determining a target sampling point between the first and second sampling points based on a second frequency, wherein the target sampling point represents a sampling point with the same second frequency; and filling the target sampling point with the target sampling value.

[0052] Furthermore, after filling the target sampling point with the target sample value, the method further includes: determining the first confidence level at the first sampling point; determining the time distance between the first sampling point and the target sampling point; and determining the target confidence level at the target sampling point based on the first confidence level and the time distance, wherein the target confidence level decreases exponentially with the time distance at the target sampling point.

[0053] In this embodiment, time-series data collected during industrial production often contains certain defects, such as outliers, missing values, and sampling at different frequencies. These defects may affect the performance of the prediction model and the accuracy of the prediction results. Therefore, after obtaining the time-series data related to the prediction task, implementing a data preprocessing step is crucial. The specific data preprocessing process can be as follows:

[0054] 1. Outlier Removal: For all variables, a reasonable threshold range is set based on the characteristics of the equipment and processes. Data exceeding this range is marked as outliers and removed. This approach ensures that the model is not misled by extreme or erroneous data, improving its robustness.

[0055] 2. Missing Value Imputation: Forward imputation is used to handle missing values. If consecutive missing values ​​are within two time steps, imputation is performed directly; if consecutive missing values ​​are within two time steps, the data is left as is without imputation. This approach preserves data continuity while avoiding bias introduced by excessive imputation.

[0056] 3. Frequency Completion Processing and Weight Generation: For non-co-frequency data, i.e., the first frequency (low frequency, such as laboratory data, with sampling periods potentially lasting several hours) and the second frequency (high frequency, such as sensor data, with sampling periods potentially lasting minutes or less), frequency completion is performed. Specifically, low-frequency data is forward-padded to the same frequency as the high-frequency data, i.e., the second frequency. Simultaneously, a confidence weight is added during the frequency completion operation. This weight is 1 at the sampling time and decreases exponentially with increasing sampling time to reflect the decrease in confidence of low-frequency data over time. The specific process can be as follows:

[0057] First, a first sampling point and a second sampling point are determined at a first frequency, and a target sampling value is determined at the first sampling point. This target sampling value represents the raw material test data value and the product test data value at the first frequency. Second, target sampling points are determined between the first and second sampling points based on a second frequency (i.e., the sampling period for high-frequency data). These target sampling points represent sampling points with the same frequency as the second frequency, i.e., the sampling of low-frequency data is supplemented by the frequency of high-frequency data. Finally, the target sampling values ​​are filled into the target sampling points. Specifically, for low-frequency data, the value of each sampling point is filled into the sampling points of high-frequency data until the next sampling point, forming a sequence with the same frequency as the high-frequency data. In this process, each filled value is associated with a confidence weight that changes over time to reflect the decreasing validity of low-frequency data over time.

[0058] Specifically, at the first sampling point (i.e., the actual sampling point of the low-frequency data), the confidence level is initialized to 1, which is the aforementioned first confidence level. This represents the highest confidence level at the moment the data is actually sampled, as this is the most up-to-date and accurate state of the data. Subsequently, the time interval between the first sampling point and the target sampling point is calculated. In industrial production processes, this time distance can be understood as the period from data sampling to the time the data is used for prediction. For low-frequency data, the longer the time distance, the more likely the data's validity is to be questioned. Based on the first confidence level and the time distance, the model calculates the target confidence level at the target sampling point. This target confidence level decreases exponentially with increasing time distance. This step reflects the understanding of data timeliness: the closer the data is to the actual sampling time, the higher its confidence level; as time progresses, the confidence level gradually decreases, reflecting the natural decay of data validity over time.

[0059] 4. Normalization: Normalize each variable to ensure that its numerical range is uniformly within a standard range of mean 0 and variance 1. This process eliminates the influence of different variables' dimensions, making the model more efficient and stable during training.

[0060] In step S204 above, the period length of each variable data in the time series data is determined in the following way: determining the time domain signal corresponding to each variable data in the time series data, wherein the time domain signal is used to represent the data sequence of each variable data changing over time; converting the time domain signal into a frequency domain signal through Fourier transform, wherein the frequency domain signal is used to represent the data sequence of each variable data changing with frequency; determining the period length of the target frequency in the frequency domain signal, wherein the target frequency is used to represent the frequency with the largest amplitude in the frequency domain signal; determining the period length of the target frequency as the period length of each variable data, wherein the period length of each variable data does not exceed a preset window length, the preset window length being determined based on historical time series data.

[0061] In this embodiment, the period length of each variable data is determined by converting the time-domain signal into a frequency-domain signal, which involves Fourier transform. The specific process is as follows:

[0062] First, the time-domain signal corresponding to each variable is identified from the raw time-series data collected during the industrial production process. This time-domain signal directly reflects the changes in the variable data over time and is the foundation of time series analysis.

[0063] Secondly, by applying the Fast Fourier Transform (FFT), the time-domain signal is converted into a frequency-domain signal. The frequency-domain signal reveals the characteristics of each variable's data as a function of frequency, and in particular, can reveal periodic patterns in the data. The FFT can quickly calculate the signal's spectrum, and for industrial time-series data, it can help identify the main periodic components in the data, such as the target frequency mentioned above.

[0064] In the frequency domain, the frequency with the largest amplitude, i.e., the target frequency, is usually found, as it typically corresponds to the most significant periodic change in the data. For example, some industrial processes may exhibit obvious daily or weekly cycles.

[0065] Finally, the period length corresponding to the target frequency is determined as the period length of each variable data. This step is crucial for capturing and understanding periodic patterns in the data.

[0066] It should be noted that the period length of each variable's data does not exceed the preset window length. This preset window length is determined based on the analysis of historical time series data, and it represents the maximum time range for the predictive model when backtracking historical data. Ensuring that the period length does not exceed the preset window length helps the model focus on the most relevant periodic information when processing data, avoids interference from excessively long periods, and improves the accuracy of predictions.

[0067] In industrial production scenarios, this adaptive processing method based on cycle length can better adapt to data changes under different production stages and operating conditions, providing a solid foundation for predictive model construction. For example, the cycle length of raw material and product testing data may be as long as several hours, while the cycle length of process and operational variables may be as short as a few minutes. By determining the cycle length of each variable through FFT, the model can effectively distinguish and process these data with different cycles. Especially when dealing with the periodic changes of low-frequency data, it can more accurately identify their regularity over time, which is crucial for predicting the future trends of such data. The innovation of this cycle determination method lies in its ability not only to automatically identify the periodic characteristics of different variables, but also to ensure that the model focuses on the most relevant time cycle information when processing and predicting data by limiting the cycle length within a preset window length, thereby improving the efficiency and accuracy of prediction.

[0068] Optionally, the prediction model also includes an embedding layer, a flattening layer, an encoding layer, and an output layer. The embedding layer is used to embed the normalized time series data, the flattening layer is used to flatten the embedded time series data, the encoding layer is used to encode the flattened time series data through a multi-head attention mechanism, and the output layer is used to generate the prediction length of each variable in the encoded time series data.

[0069] In this embodiment, the prediction model achieves efficient processing and high-precision prediction of multi-frequency time series data in the process industry through the synergistic effect of the block layer, sample normalization layer, embedding layer, flattening layer, encoding layer, and output layer. The following provides a detailed description of these modules.

[0070] Block layer: used to divide the time series data according to the period length p of each variable data. (j) Perform non-overlapping partitioning on the time series data X. The specific expression is as follows:

[0071]

[0072] N (j) p =[L / p (j) ]

[0073] Where X represents time series data, p (j) This represents the period length of each variable, L represents the backtracking window length (i.e., the preset window length), and Padding(.) indicates when L cannot be p... (j) When divisible, pad the end with 0s until it is divisible; Slicing(.) means according to p (j) Divide X into non-overlapping blocks; N (j) pThis represents the time block contained in each variable's data after the segmentation.

[0074] Sample normalization layer: Used to normalize the data of each variable after being divided into blocks. The specific expression is as follows:

[0075]

[0076] in, This represents the normalized data. This represents the data at the i-th time point in the n-th block of the j-th variable data in the time series data, where α and β represent the model parameters.

[0077] Embedding layer: This layer is used to embed the normalized data into a representation using numerical and positional encoding. Specifically, the embedding layer consists of two parts: feature embedding and positional embedding. First, value embedding maps each time block to a fixed-size vector as its numerical encoding. Second, positional embedding adds temporal information to ensure the model can distinguish the temporal order of the data. Finally, the numerical and positional encodings are added together to obtain the final embedding representation of each time block.

[0078] Flattening layer: This layer reorganizes all the variable data processed by the embedding layer into a one-dimensional sequence, facilitating subsequent attention calculations by the Transformer encoding layer. This approach addresses the issue of inconsistent variable lengths, enabling the model to uniformly handle diverse time series data.

[0079] Encoding layer: Used for deep analysis and feature extraction of the flattened sequence data through a multi-head attention mechanism. Specifically, the encoding layer is based on the Transformer architecture and utilizes a multi-head attention mechanism, enabling simultaneous computation from multiple different attention spaces, thus improving the model's ability to process sequence data. Each "head" independently calculates attention weights, and then these calculation results are merged to obtain a more comprehensive and refined data representation. In this embodiment, an 8-head attention mechanism is selected.

[0080] Output layer: Used to transform the encoded data into prediction results through a fully connected network. Specifically, the output layer typically includes a fully connected neural network that maps the output of the encoding layer to the prediction length. The prediction length matches the prediction target of the input data. For example, for high-frequency data, the prediction length might be 720 time points, while for low-frequency data (such as laboratory data), the prediction length might be shorter, such as the mean prediction for 1 time point.

[0081] Optionally, the above method further includes: obtaining the prediction length of each variable in the time series data; determining the mean and variance of the time series data within a preset time period, wherein the end time of the preset time period is earlier than the start time of the normalization process; and performing inverse normalization processing on the prediction length based on the mean and variance to obtain the prediction result.

[0082] In this embodiment, the prediction length output by the output layer corresponding to each variable data needs to be denormalized to restore the scale of the original data, and finally the prediction result corresponding to the prediction task is output. Specifically:

[0083] After the model outputs the normalized prediction length, these predictions need to be converted back to the original data scale for comparison and application with actual production data. The denormalization process is based on the dynamic mean and variance before normalization. Specifically, for each prediction value, its original scale is restored by multiplying it by the square root of the variance and adding the mean. These restored prediction values ​​are then used as the final prediction results. The dynamic mean and variance are determined before normalization; that is, before model training and prediction, the statistical characteristics of each variable within a preset time period are determined. The selection of this preset time period should be based on historical data to ensure that it covers typical data performance and potential distribution drift.

[0084] In determining the dynamic mean and variance, the dynamic mean network and dynamic variance network consist of two linear layers. The first layer uses ReLU as the activation function, and the second layer uses the Identity function. This network structure can learn the non-linear relationships in the data while ensuring that the output mean and variance are numerically reasonable.

[0085] By obtaining the prediction length of variables, determining the mean and variance within a preset time period, and performing inverse normalization, the model ensures that when processing complex, multi-frequency time series data in the process industry, it can not only capture the inherent patterns of the data, but also generate predictive information that can be directly used for production management, providing strong technical support for intelligent manufacturing.

[0086] In this application embodiment, a method for training a prediction model is also provided. This prediction model is similar to the prediction method for time series data described above, and includes the following steps:

[0087] S1. Obtain historical time series data related to the prediction task of industrial production process, including historical raw material test data, historical product test data, historical process variables, and historical operation variables.

[0088] S2. Preprocess the historical time series data, including outlier removal, missing value filling, data frequency completion, generation of confidence weights, and normalization.

[0089] S3. Divide the preprocessed historical time series data into training datasets and test datasets.

[0090] S4. For the training dataset, use the Fast Fourier Transform (FFT) module to analyze the periodicity of the data and determine the block period of each variable. If the period exceeds the backtracking window length L, then take L as the period length.

[0091] S5. Divide the training dataset into sample sets of length L (backtracking window length) and prediction step size T using a sliding window method. Specifically, the sample set is constructed as follows:

[0092] First, construct a feature dataset {X} with a backtracking length of L. i:i+L The method uses a sliding window to take a sample every s steps, as follows:

[0093]

[0094] Among them, X i:i+L ∈R L×M Let i be the index in the data, 1 ≤ i ≤ (NLT) / s. Let y be the value of the j-th feature variable at time i+t. i+t Let be the value at time i+t, where 1≤t≤L, N is the total amount of data, L is the backtracking window length, T is the prediction window length, M is the total number of variables, and m is the quantity to be predicted. In the embodiment of this application, L=1440, T=720, M=267, m=141.

[0095] Secondly, construct the prediction target dataset {Y} i+L:i+L+T The prediction window is also constructed using a sliding window method, where T is the prediction length. The prediction length varies for data of different frequencies. For minute-level data, the prediction window length is T = 720, while for low-frequency data such as laboratory data, the prediction length is 1 point, which is the mean of T = 720.

[0096] It should be noted that the feature dataset serves as the input to the model, used to train the model to identify and learn potential temporal dependencies and patterns in the data; while the target dataset serves as the model's label, defining the future data sequences that the model needs to predict. Together, they form the basis of model learning. By constructing a sample set through a sliding window, the model can effectively process multi-frequency time series data in the process industry, learn the inherent patterns and temporal dependencies of the data, and thus generate accurate prediction results.

[0097] S6. Construct an initial prediction model based on the Transformer architecture, including a block, sample normalization layer, embedding layer, flattening layer, encoding layer and output layer, and train the initial prediction model using the constructed sample set.

[0098] S7. Validate the initial prediction model on the test dataset, and select the model with the lowest loss on the validation set as the final prediction model.

[0099] Specifically, the initial prediction model is trained using the MAE loss function and the Adam optimizer, with a learning rate of 10. -4 The model underwent 100 iterations with a batch size of 128. Subsequently, its performance was evaluated on a validation dataset, and the model with the lowest validation loss was selected as the final model. Simultaneously, during the training of the prediction model, parameters were dynamically adjusted to ensure the model could adapt to the characteristics of time series data at different frequencies.

[0100] Figure 3 This is a schematic diagram illustrating a prediction effect according to an embodiment of this application. For example... Figure 3 As shown, for high-frequency data (such as feed temperature), the model's prediction of the T202 feed temperature is highly consistent with the actual value. Although the temperature fluctuates significantly, the predicted curve still accurately follows the trend of the actual curve, demonstrating the model's strong predictive ability when processing minute-level high-frequency data. Even with significant temperature fluctuations in a short period, the predicted data maintains high accuracy, indicating that the model can effectively capture and predict short-term and long-term dependencies in time series. For low-frequency data (such as xylene content), the model also demonstrates excellent predictive performance. As time progresses, the xylene content gradually decreases, and this long-term trend is accurately predicted by the model. Even though the change in xylene content is relatively slow, the predicted curve still closely follows the actual curve, demonstrating the model's stability when processing low-frequency data. This shows that the model can adapt to time series data of different frequencies, maintaining high predictive accuracy regardless of whether the changes are rapid or slow.

[0101] The entire model training process was designed for the prediction of multi-frequency time series data in the process industry. It fully considered the complexity and heterogeneity of the data and improved the accuracy and generalization ability of the prediction through a series of innovative data processing and model training strategies.

[0102] In this embodiment, the limitations of traditional prediction methods in handling data of different frequencies are effectively addressed through dynamic block partitioning, confidence weighting, dynamic normalization and denormalization, and a specially designed model structure. Specifically, the FFT layer is used to determine the periodicity of each variable, ensuring the model can capture the periodic characteristics of the data; positional encoding and multi-head attention mechanisms enhance the model's ability to process time series data. Dynamically adjusted mean and variance are used in the normalization and denormalization processes, guaranteeing the accuracy of the prediction results and their match with actual data. This combination of technologies enables the model to provide more accurate predictions when processing diverse data in process industries, significantly improving prediction performance and possessing significant value for industrial process control and optimization.

[0103] According to embodiments of this application, a time series data prediction apparatus is provided. It should be noted that the time series data prediction apparatus of this application embodiment can be used to execute the time series data prediction method provided in this application embodiment. The time series data prediction apparatus provided in embodiments of this application is described below.

[0104] Figure 4 This is a structural diagram of a time series data prediction device provided according to an embodiment of this application. Figure 4 As shown, the device includes:

[0105] The acquisition module 40 is used to acquire time series data related to the prediction task in the industrial production process, wherein the time series data includes at least raw material test data, product test data, process variables and operation variables related to the prediction task;

[0106] The processing module 42 is used to process time series data using a prediction model to obtain prediction results corresponding to the prediction task. The prediction model includes at least a block layer and a normalization layer. The block layer is used to divide the time series data into blocks according to the period length of each variable data in the time series data, and the normalization layer is used to normalize the time series data after block processing.

[0107] Through the acquisition module 40 and processing module 42 in the aforementioned time series data prediction device, the purpose of effectively integrating and utilizing time series data of different frequencies for prediction is achieved. This enables high-precision prediction of multi-frequency time series data in the process industry and improves the accuracy and adaptability of the prediction model. In turn, it solves the technical problems that the prediction methods in related technologies cannot effectively cope with the differences in data frequency, distribution drift, and changes in confidence over time in time series data of different frequencies in the process industry.

[0108] The time series data prediction device provided in this application embodiment further includes a preprocessing module 44, which is used to preprocess the time series data. The data preprocessing method includes at least: outlier removal, missing value filling, frequency completion processing, and normalization processing. The frequency completion processing includes: determining a first sampling point and a second sampling point at a first frequency, and determining a target sampling value at the first sampling point, wherein the first sampling point is before the second sampling point, and the target sampling value is used to represent the raw material test data value and the product test data value at the first sampling point; determining a target sampling point between the first sampling point and the second sampling point according to a second frequency, wherein the target sampling point is used to represent a sampling point with the same second frequency; and filling the target sampling point with the target sampling value.

[0109] In the time series data prediction apparatus provided in this application embodiment, the preprocessing module is further configured to determine a first confidence level at a first sampling point; determine the time distance between the first sampling point and the target sampling point; and determine a target confidence level at the target sampling point based on the first confidence level and the time distance, wherein the target confidence level decreases exponentially with the time distance at the target sampling point.

[0110] The time series data prediction device provided in this application embodiment further includes a determination module 46, which is used to determine the time domain signal corresponding to each variable data in the time series data, wherein the time domain signal is used to represent the data sequence of each variable data changing over time; convert the time domain signal into a frequency domain signal through Fourier transform, wherein the frequency domain signal is used to represent the data sequence of each variable data changing with frequency; determine the period length of the target frequency in the frequency domain signal, wherein the target frequency is used to represent the frequency with the largest amplitude in the frequency domain signal; and determine the period length of the target frequency to the period length of each variable data, wherein the period length of each variable data does not exceed a preset window length, the preset window length being determined based on historical time series data.

[0111] In the time series data prediction device provided in this application embodiment, the processing module is further configured to obtain the prediction length of each variable data in the time series data; determine the mean and variance of the time series data within a preset time period, wherein the end time of the preset time period is earlier than the start time of the normalization process; and perform inverse normalization processing on the prediction length based on the mean and variance to obtain the prediction result.

[0112] This application also provides an electronic device, including: a memory and a processor, wherein the memory is used to store program instructions; and the processor is connected to the memory and used to execute the prediction method for the time series data described above.

[0113] It should be noted that the aforementioned electronic equipment is used to perform Figure 2The time series data prediction method shown above is also applicable to this electronic device, and will not be repeated here.

[0114] This application also provides a non-volatile storage medium including a stored computer program, wherein the device containing the non-volatile storage medium executes the above-mentioned time series data prediction method by running the computer program.

[0115] It should be noted that the aforementioned non-volatile storage media is used for execution. Figure 2 The time series data prediction method shown above is also applicable to this non-volatile storage medium, and will not be repeated here.

[0116] This application also provides a computer program product, including computer instructions that, when executed by a processor, implement the above-described method for predicting time series data.

[0117] It should be noted that the above-mentioned computer program product is used to execute Figure 2 The time series data prediction method shown above is also applicable to this computer program product, and will not be repeated here.

[0118] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0119] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0120] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0121] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0122] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0123] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0124] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A method for predicting time series data, characterized in that, include: Acquire time series data related to the prediction task in the industrial production process, wherein the time series data includes at least raw material test data, product test data, process variables and operation variables related to the prediction task; The time series data is processed using a prediction model to obtain prediction results corresponding to the prediction task. The prediction model includes at least a block layer and a normalization layer. The block layer is used to divide the time series data into blocks according to the period length of each variable data in the time series data. The normalization layer is used to normalize the time series data after block processing. The method further includes: performing data preprocessing on the time series data, wherein the data preprocessing method includes at least: outlier removal, missing value imputation, frequency completion processing, and normalization processing, wherein the frequency completion processing includes: determining a first sampling point and a second sampling point at a first frequency, and determining a target sampling value at the first sampling point, wherein the first frequency is the sampling frequency of the raw material test data and the product test data, the first sampling point is before the second sampling point, and the target sampling value is used to represent the raw material test data value and the product test data value at the first sampling point; determining a target sampling point between the first sampling point and the second sampling point based on a second frequency, wherein the second frequency is the sampling frequency of the process variable and the operation variable, and the target sampling point is used to represent a sampling point with the same second frequency; and filling the target sampling value into the target sampling point.

2. The method according to claim 1, characterized in that, After filling the target sampling point with the target sample value, the method further includes: Determine the first confidence level at the first sampling point; Determine the time distance between the first sampling point and the target sampling point; A target confidence level is determined at the target sampling point based on the first confidence level and the time distance, wherein the target confidence level decreases exponentially with the time distance at the target sampling point.

3. The method according to claim 1, characterized in that, The period length of each variable in the time series data is determined in the following way: Determine the time-domain signal corresponding to each variable data in the time series data, wherein the time-domain signal is used to represent the data sequence of each variable data changing over time; The time-domain signal is converted into a frequency-domain signal by Fourier transform, wherein the frequency-domain signal is used to represent the data sequence of each variable data as a function of frequency; Determine the period length of the target frequency in the frequency domain signal, wherein the target frequency is used to represent the frequency with the largest amplitude in the frequency domain signal; The period length of the target frequency is determined as the period length of each variable data, wherein the period length of each variable data does not exceed a preset window length, which is determined based on historical time series data.

4. The method according to claim 1, characterized in that, The prediction model further includes an embedding layer, a flattening layer, an encoding layer, and an output layer. The embedding layer is used to embed the normalized time series data. The flattening layer is used to flatten the embedded time series data. The encoding layer is used to encode the flattened time series data using a multi-head attention mechanism. The output layer is used to generate the prediction length of each variable in the encoded time series data.

5. The method according to claim 4, characterized in that, The method further includes: Obtain the prediction length of each variable in the time series data; Determine the mean and variance of the time series data within a preset time period, wherein the end time of the preset time period is earlier than the start time of the normalization process; The prediction length is inversely normalized based on the mean and variance to obtain the prediction result.

6. A device for predicting time series data, characterized in that, include: The acquisition module is used to acquire time series data related to the prediction task in the industrial production process, wherein the time series data includes at least raw material test data, product test data, process variables and operation variables related to the prediction task; The processing module is used to process the time series data using a prediction model to obtain a prediction result corresponding to the prediction task. The prediction model includes at least a block layer and a normalization layer. The block layer is used to divide the time series data into blocks according to the period length of each variable data in the time series data. The normalization layer is used to normalize the time series data after block processing. The device further includes a preprocessing module for preprocessing the time series data. The preprocessing methods include at least outlier removal, missing value imputation, frequency completion processing, and normalization. The frequency completion processing includes: determining a first sampling point and a second sampling point at a first frequency, and determining a target sampling value at the first sampling point, wherein the first frequency is the sampling frequency of the raw material test data and the product test data, the first sampling point precedes the second sampling point, and the target sampling value represents the raw material test data value and the product test data value at the first sampling point; determining a target sampling point between the first sampling point and the second sampling point based on a second frequency, wherein the second frequency is the sampling frequency of the process variable and the operational variable, and the target sampling point represents a sampling point with the same second frequency; and filling the target sampling point with the target sampling value.

7. An electronic device, characterized in that, include: A memory and a processor, wherein the memory is used to store program instructions; The processor, connected to the memory, is used to execute the prediction method for time series data according to any one of claims 1 to 5.

8. A non-volatile storage medium, characterized in that, The non-volatile storage medium includes a stored computer program, wherein the device containing the non-volatile storage medium executes the prediction method for time series data according to any one of claims 1 to 5 by running the computer program.

9. A computer program product comprising computer instructions, characterized in that, When the computer instructions are executed by the processor, they implement the time series data prediction method according to any one of claims 1 to 5.

Images