Multi-time-slot industrial time series data prediction system based on optimized LSTM (Long Short Term Memory)

Abstract

The invention provides a multi-time-slot industrial time series data prediction system based on an optimized LSTM, and relates to the technical field of time series data prediction. The system comprises a mixed data window at an input end, an improved LSTM model unit and a data cleaning module adapted to the time sequence data of the industrial instrument, wherein the mixed data window is of a mixed window structure formed by a plurality of historical true values and a plurality of predicted values; the number of network nodes of the improved LSTM model unit is 128, and a network activation function of an output gate in the original model is replaced by linear; the data cleaning module is used for obtaining data with null values removed and a determined sampling period. According to the optimization method, the data cleaning module, the mixed data window and the improved LSTM model unit are combined, and the feasibility and the stability during model training and the accuracy and the reliability during multi-time-slot data prediction task execution are remarkably improved for an industrial instrument and meter time sequence data scene.

Description

Technical Field

[0001] The present invention relates to the technical field of time series data prediction, and in particular to a multi-time slot industrial time series data prediction system based on optimized LSTM. Background Art

[0002] In actual industrial production, there is often a need to predict instrument trend data. Taking a sewage treatment plant as an example, during the treatment process, the amount of resources invested is positively correlated with the amount of sewage to be treated. Therefore, if the inflow flow rate in the future can be predicted, the sewage treatment plant can reasonably allocate various resources such as electricity and reagents based on the prediction results; thereby achieving the goal of reducing costs and increasing efficiency while ensuring that the treatment results meet quality standards.

[0003] At present, there are a lot of research related to time series data prediction both at home and abroad. The proven effective methods include RNN, LSTM, GRU, linear regression and other research contents. However, if this topic is subdivided into some specific fields, especially in the aspects related to industrial instrumentation, such as vortex meters, oil meters, etc., there are fewer effective and stable methods, and most of them are still based on LSTM. In addition, in addition to the above data-driven methods, there are also mechanism modeling-driven methods. However, since the output of this modeling is usually not reused and can only be used in specific scenarios, it is less versatile and is not a research hotspot.

[0004] In general, although LSTM is a more suitable method in the field of industrial instrumentation time series data prediction; however, the drawback of being unable to accurately predict the results of data after multiple time slots has seriously affected the actual application effect of LSTM. For example, assuming that the sampling frequency of the original data is once every 20 seconds, then LSTM can only predict the value after 20 seconds, but cannot predict the value after 40 seconds, 60 seconds, etc. Of course, simply inputting the predicted data repeatedly into the model can also predict the data after multiple time slots; however, its error will be gradually amplified with the running time, the accuracy will only be maintained for a very short time, and eventually the prediction ability will be almost completely lost. Summary of the invention

[0005] The purpose of the present invention is to provide a multi-slot industrial time series data prediction system based on optimized LSTM to solve the problem of multi-slot industrial time series data prediction in view of the deficiencies of the above-mentioned prior art.

[0006] To achieve the above purpose, the technical solution adopted by the present invention is as follows:

[0007] The present invention provides a multi-time slot industrial time series data prediction system based on optimized LSTM, the system comprising: a mixed data window at the input end, an improved LSTM model unit, and a data cleaning module adapted to industrial instrument time series data,

[0008] The hybrid data window is a hybrid window structure consisting of multiple historical real values ​​plus multiple predicted values;

[0009] The number of network nodes of the improved LSTM model unit is 128, and the network activation function of the output gate in the original LSTM model is replaced from tanh to linear;

[0010] The data cleaning module adapted to the time series data of industrial instruments is used to first fill the empty values ​​of the original data of the time series data of industrial instruments, and then average the data in each time slot in turn to obtain data with the empty values ​​removed and the sampling period determined.

[0011] Optionally, the specific use process of the mixed data window is as follows: i It represents the historical real value, i=tk,t-k+1,…,t,P j It represents the predicted value, j = t+1, t+2, ..., t+m, t represents the time, k and m are positive integers, at time t, the historical real data will only end at X t , and there is no true value after time t; therefore, first t+1 , P t+2 ,…,P t+m-1 Instead of adding a window, these m-1 values ​​are predicted by the LSTM model unit in the previous cycle, and then X t-k To P t+m-1 These k+m values ​​are input into the LSTM model unit to predict P t+m , when it reaches the next moment t+1, we have the true value X at the current moment t+1 and the predicted value P obtained at the previous moment t+m At this time, based on the mixed data window at time t, first t-k Delete and then change P t+1 Change to X t+1 , and add P at the end of the window t+m , we get the mixed data window at time t+1, and then input the data in this window into the LSTM model unit to get the predicted value P t+m+1 .

[0012] Optionally, k=6, and m=4.

[0013] The beneficial effects of the present invention include:

[0014] The multi-slot industrial time series data prediction system based on optimized LSTM provided by the present invention includes: a mixed data window at the input end, an improved LSTM model unit, and a data cleaning module adapted to industrial instrument time series data, wherein the mixed data window is a mixed window structure composed of multiple historical real values ​​plus multiple predicted values; the number of network nodes of the improved LSTM model unit is 128, and the network activation function of the output gate in the original LSTM model is replaced from tanh to linear; the data cleaning module adapted to industrial instrument time series data is used to: first fill the original data null values ​​of the industrial instrument time series data, and then average the data in each time slot in turn to obtain data with null values ​​removed and a determined sampling period. The present invention proposes an optimized LSTM method combining a data cleaning module + a mixed data window + an improved LSTM model unit, which, under the premise of facing the industrial instrument time series data scenario, is significantly improved in terms of feasibility, stability, etc. during model training, as well as accuracy and reliability when performing multi-slot data prediction tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0016] Figure 1 The schematic diagram of the structure of the original LSTM model unit is shown;

[0017] Figure 2 A schematic diagram of the structure of a hybrid input window provided by an embodiment of the present invention is shown;

[0018] Figure 3 A schematic diagram of the structure of an improved LSTM model unit provided in an embodiment of the present invention is shown;

[0019] Figure 4 A schematic flow chart of a data cleaning method for adapting industrial instrument time series data provided by an embodiment of the present invention is shown;

[0020] Figure 5 A schematic diagram of the workflow of a multi-slot industrial time series data prediction system based on optimized LSTM provided in an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0021] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0022] Long Short-term Memory Networks (LSTM) is a special type of Recurrent Neural Network (RNN). By introducing the gating mechanism of selective update and selective forgetting, the gradient vanishing and gradient exploding problems encountered by RNN during back propagation can be effectively avoided. An LSTM is composed of several stacked LSTM units. The structure of the original LSTM unit is as follows: Figure 1 shown. Figure 1 In, C t It is the cell state that determines what information will be retained and what information will be forgotten; x t It represents the input of the current unit; h t-1 The dotted box 1 represents the forget gate, which is responsible for deciding how much of the cell state at the previous moment is retained to the cell state at the current moment; the dotted box 2 represents the input gate, which is responsible for deciding how much of the current moment's input is retained to the cell state at the current moment; the dotted box 3 represents the output gate, which is responsible for deciding how much output the cell state at the current moment has. In addition, tanh 20 , o · 20 They represent hidden deep networks with activation functions of tanh and sigmoid and 20 nodes. The LSTM unit is based on h t-1 and xt, after a series of calculations, h will be output t , which is the predicted value of the target.

[0023] From the above, we can see that under the structure of the original LSTM unit, if we want to get h t , then you must have x t ; If only x t-1 、x t-2 If we wait for input from an earlier time, we cannot obtain the predicted value h t. In other words, LSTM alone can only make single-time-slot predictions, and cannot obtain numerical results after multiple time slots. Furthermore, the original LSTM unit structure is not reasonable enough. The number of hidden layer network nodes is too small, resulting in a lack of sufficient feature extraction capabilities, and the output gate activation function tanh will further limit the corrected output range. In addition, LSTM has high requirements for input data, but is not equipped with a corresponding data cleaning module, which makes it difficult to train and predict using industrial instrumentation time series data.

[0024] Although LSTM is a relatively suitable method in the field of industrial instrumentation time series data prediction, its inability to accurately predict the results of data after multiple time slots has seriously affected the actual application effect of LSTM. For example, assuming that the sampling frequency of the original data is once every 20 seconds, then LSTM can only predict the value after 20 seconds, but cannot predict the value after 40 seconds, 60 seconds, etc. Of course, simply inputting the predicted data repeatedly into the model can also predict the data after multiple time slots; however, its error will be gradually amplified with the running time, the accuracy will only be maintained for a very short time, and eventually the prediction ability will be almost completely lost.

[0025] In view of the above situation, this paper proposes a multi-time slot industrial time series data prediction system based on optimized LSTM. First, a mixed data window is added to the input end; second, the LSTM unit structure is improved, and the model's ability to fit numerical values ​​is improved by increasing the number of nodes and optimizing the activation function of the output gate; third, a novel data cleaning algorithm is proposed, thereby completing the overall optimization of the LSTM method, enabling it to more accurately complete the task of traffic data prediction after multiple time slots, and playing a role in helping related fields.

[0026] The embodiment of the present invention provides a multi-time-slot industrial time series data prediction system based on optimized LSTM, the system comprising: a mixed data window at an input end, an improved LSTM model unit, and a data cleaning module adapted to industrial instrument time series data, the mixed data window is a mixed window structure composed of multiple historical real values ​​plus multiple predicted values; the number of network nodes of the improved LSTM model unit is 128, and the network activation function of the output gate in the original LSTM model is replaced from tanh to linear; the data cleaning module adapted to industrial instrument time series data is used for: first filling the original data null values ​​of the industrial instrument time series data, and then averaging the data in each time slot in turn to obtain data with null values ​​removed and a determined sampling period.

[0027] Optionally, the specific use process of the mixed data window is as follows: i It represents the historical real value, i=tk,t-k+1,…,t,P jIt represents the predicted value, j = t+1, t+2, ..., t+m, t represents the time, k and m are positive integers, at time t, the historical real data will only end at X t , and there is no true value after time t; therefore, first t+1 , P t+2 ,…,P t+m-1 Instead of adding a window, these m-1 values ​​are predicted by the LSTM model unit in the previous cycle, and then X t-k To P t+m-1 These k+m values ​​are input into the LSTM model unit to predict P t+m , when it reaches the next moment t+1, we have the true value X at the current moment t+1 and the predicted value P obtained at the previous moment t+m At this time, based on the mixed data window at time t, first t-k Delete and then change P t+1 Change to X t+1 , and add P at the end of the window t+m , we get the mixed data window at time t+1, and then input the data in this window into the LSTM model unit to get the predicted value P t+m+1 .

[0028] As mentioned above, the original LSTM can only make single time slot predictions, but not multi-time slot predictions. One feasible approach is to use the predicted values ​​for iteration, but this is only effective in the short term, but not in the long term. Therefore, the present invention refers to the idea of ​​sliding windows and predicted value iteration to construct a window structure that integrates past historical values ​​+ partial predicted values ​​to cope with multi-time slot time series data prediction tasks. Here, a mixed data window with a length of 10 and a prediction time slot of 4 is taken as an example (at this time, k=6, and m=4), and the structure of the mixed data window is described, as shown below. Figure 2 As shown. Figure 2 In the above example, X represents the historical real value and P represents the predicted value. At time t, the historical data will only be up to X. t , and there will be no true value after time t; therefore, first t+1 , P t+2 and P t+3 Instead of adding a window, these three values ​​are predicted by the LSTM in the previous cycles. Then, X t-6 To P t+3 These ten values ​​are input into the LSTM network to get P t+4 When the next moment t+1 is reached, the real value X at the current moment is obtained. t+1 and the predicted value P obtained at the previous moment t+4At this time, based on the mixed input window at time t, first convert the X t-6 Delete and then change P t+1 Change to X t+1 , and add P at the end of the window t+4 , we get the mixed data window at time t+1. Then input the data in this window into LSTM to get the predicted value P t+5 Subsequently, by repeating the above process, the model can continuously perform multi-slot time series data prediction tasks; at each moment, the ratio of historical values ​​to predicted values ​​is still 7:3, and the ratio of predicted values ​​has not increased, so the error will not be amplified over time, solving the accuracy defect of iterative prediction.

[0029] The original LSTM unit activation function and the number of hidden nodes are set as Figure 1 When it is directly used for trend prediction of industrial time series data, it is found that the results are usually not as expected. The reasons are as follows: 1. The number of hidden layer nodes is set too small, and the basic information collected is insufficient, which leads to insufficient deep features extracted by the transposed matrix; 2. The activation function tanh limits the output range, which reduces the influence of the cell state on the predicted value, thereby weakening the correction ability of the network.

[0030] Therefore, the present invention makes corresponding structural improvements to the LSTM neurons, and the specific structure is as follows: Figure 3 As shown. Figure 1 , the number of original network nodes is increased from 20 to 128, which increases the amount of basic information obtained, so that more invisible high-order features can be mined; in addition, the network activation function above the output gate is also changed from tanh to linear, and the output range is expanded from (-1, +1) to (-∞, +∞), which strengthens the influence of cell state on the predicted value. After the above two improvement measures work together, the fitting ability of the LSTM model unit will be strengthened, and the quality of the final prediction results will become more accurate.

[0031] Common industrial instrumentation time series data mainly has two major problems: uneven sampling period and null values ​​at a small number of moments. Among them, inputting the former into model training will make the model have no practical application value, because LSTM will regard all data as the same time interval, which contradicts the underlying principle; and inputting the latter into the model will make the model unable to complete training, because LSTM cannot read null values, which contradicts the input structure. Therefore, if the above two points are not handled, even if the LSTM model used can be implemented, its prediction results will not be very accurate.

[0032] Therefore, the present invention designs a data cleaning algorithm for common problems of industrial instrumentation time series data. The specific process is as follows: Figure 4 As shown in the figure. First, fill in the empty values ​​of the original data, and then average the data in each time slot T in turn, so as to obtain data with empty values ​​removed and a certain sampling period, thereby solving the above-mentioned two types of problems and solving the data foundation for applying LSTM to multi-time slot industrial time series data prediction tasks.

[0033] Figure 5 The following is a schematic diagram showing the workflow of a multi-slot industrial time series data prediction system based on optimized LSTM provided by an embodiment of the present invention. Figure 5 The specific logic of the sliding window is as shown in Figure 2 As mentioned above, the function is the code that implements the logic; the unit structure of the LSTM model is as shown in Figure 3 As mentioned above, it needs to be trained before executing the prediction task to determine all node parameters; the detailed process of the data cleaning module is as shown in Figure 4 As mentioned above, the original data set can only be read by the model to complete the training after passing through this module. Figure 5 As shown in the figure, after each round of new real-time data arrives, the sliding window function will first update the data of the mixed data window; then, the current mixed data window is input into the model, so that the model can predict the data after multiple moments. By combining the above two according to this process, the prediction task after multiple moments can be completed continuously and accurately until the command to end the prediction is input. In actual applications, the historical data of the target industrial instrumentation point is obtained and referenced. Figure 4 The data cleaning algorithm of the embodiment of the present invention described above obtains training set data that can be used for model training; uses the improved LSTM model unit to build a suitable LSTM model; and inputs the training set data into the model to complete the model training and determine all its node parameters; according to the reference Figure 2 The logic shown in the figure completes the coding of the hybrid data window and its update method and places it at the front end of the model. Finally, the real-time data of the target industrial instrumentation points is connected to start the continuous multi-slot data prediction task until the system issues an end command.

[0034] In summary, this paper proposes a multi-time-slot industrial time series data prediction system based on the LSTM method for optimization. First, a data window is added to the input end of the model, and the historical value + predicted value is mixed to maintain it at a fixed ratio, so as to solve the problem of the continuous decline in accuracy during the iteration of the pure predicted value, so that the model can stably perform the multi-time-slot data prediction task; secondly, the LSTM unit is structurally improved, the number of hidden layer nodes is increased, and the activation function of the final output gate is optimized to solve the problem of insufficient fitting ability of the original LSTM unit, so that the accuracy of the model prediction result is improved; furthermore, a data cleaning algorithm for industrial instrumentation time series data is designed to solve common problems such as null values ​​and uneven time slots, so that the model can be trained and used normally. Compared with the prior art, the present invention proposes an optimized LSTM method combining data cleaning module + mixed data window + improved LSTM model unit. Under the premise of facing the industrial instrumentation time series data scenario, both the feasibility and stability of the model training and the accuracy and reliability of the multi-time-slot data prediction task are significantly improved.

[0035] The above embodiments are only for illustrating the technical concept and features of the present invention, and their purpose is to enable ordinary technicians in the field to understand the content of the present invention and implement it. They cannot be used to limit the protection scope of the present invention. Any equivalent changes or modifications made according to the spirit of the present invention should be included in the protection scope of the present invention.

Claims

1. A multi-slot industrial time series data prediction system based on optimized LSTM, characterized in that: The system includes: a mixed data window at the input end, an improved LSTM model unit, and a data cleaning module adapted to the time series data of industrial instruments. The hybrid data window is a hybrid window structure consisting of multiple historical real values ​​plus multiple predicted values; The number of network nodes of the improved LSTM model unit is 128, and the network activation function of the output gate in the original LSTM model is replaced from tanh to linear; The data cleaning module adapted to the time series data of industrial instruments is used to first fill in the original data null values ​​of the time series data of industrial instruments, and then average the data in each time slot in turn to obtain data with null values ​​removed and a determined sampling period.

2. The multi-slot industrial time series data prediction system based on optimized LSTM according to claim 1 is characterized in that: The specific use process of the mixed data window is as follows: i It represents the historical real value, i=tk,t-k+1,…,t,P j It represents the predicted value, j = t+1, t+2, ..., t+m, t represents the time, k and m are positive integers, at time t, the historical real data will only end at X t , and there is no true value after time t; therefore, first t+1 , P t+2 ,…,P t+m-1 Instead of adding a window, these m-1 values ​​are predicted by the LSTM model unit in the previous cycle, and then X t-k To P t+m-1 These k+m values ​​are input into the LSTM model unit to predict P t+m , when it reaches the next moment t+1, we have the true value X at the current moment t+1 and the predicted value P obtained at the previous moment t+m At this time, based on the mixed data window at time t, first t-k Delete and then change P t+1 Change to X t+1 , and add P at the end of the window t+m , we get the mixed data window at time t+1, and then input the data in this window into the LSTM model unit to get the predicted value P t+m+1 .

3. The multi-slot industrial time series data prediction system based on optimized LSTM according to claim 2 is characterized in that: k=6, and m=4.

Images