Industrial process missing value filling method and system based on time sequence

Abstract

The invention relates to the technical field of data processing and industrial automation, in particular to an industrial process missing value filling method and system based on a time sequence. The method comprises the steps of obtaining original sequence data collected by various sensors in the industrial process and performing missing processing, constructing a generator by stacking LSTM layers of which the number of multiple layers of units is sequentially increased, establishing a time sequence characteristic model of a generative adversarial network, training the time sequence characteristic model by using the original sequence data subjected to missing processing, and constructing a time sequence characteristic model of the generative adversarial network. Calculating a loss function of the discriminator, setting a dynamic consistency loss function to constrain filling data generated by the generator in a sliding time window, obtaining a trained time sequence characteristic model through back propagation, inputting industrial sequence data of a to-be-filled missing value into the trained time sequence characteristic model, and obtaining a to-be-filled missing value; and evaluating the filling sequence data obtained through output. According to the method, data changes under different time scales can be adaptively processed, and the data processing efficiency and the generalization ability of the model are effectively improved.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing and industrial automation, and in particular to a method and system for filling missing values ​​in an industrial process based on time series. Background Art

[0002] In modern industrial production systems, sensors serve as core data collection devices, providing real-time monitoring of key parameters such as physical and chemical quantities involved in the production process. This data, like the nervous system of industrial production, provides crucial support for key processes such as process optimization, equipment monitoring, intelligent control, and fault diagnosis. However, due to the complexity of real-world production environments, sensor data may be lost due to factors such as equipment failure, network communication impairments, and data storage anomalies. This data loss not only impacts system stability but also weakens data-driven optimization and prediction capabilities, thereby reducing production efficiency and safety.

[0003] While Generative Adversarial Networks (GANs) have achieved some success in imputing missing values ​​and have improved accuracy compared to traditional simple interpolation methods, they still face numerous challenges. In particular, when industrial process data exhibits significant temporal characteristics, existing GAN methods fail to fully exploit the dynamic nature of the data, ignoring the temporal dependencies and trends. For example, in industries such as chemical production and semiconductor manufacturing, key process parameters such as temperature, pressure, and concentration often exhibit significant continuity and trends over time. Traditional GAN ​​methods fail to effectively capture these temporal characteristics when filling missing values, potentially causing the interpolated data to deviate from the actual trend. This deviation not only compromises data integrity and consistency but can also lead to significant errors in predictions based on the interpolated data, compromising the scientific nature and reliability of production decisions. Summary of the Invention

[0004] The technical problem to be solved by the embodiments of the present invention is to provide a method and system for filling missing values ​​in industrial processes based on time series, so as to solve the problem in the prior art that the interpolated data deviates from the actual change trend because the GAN method cannot effectively capture the temporal characteristics of industrial process data.

[0005] The present invention discloses a method for filling missing values ​​in an industrial process based on time series, comprising: Obtaining raw sequence data collected by various sensors in the industrial process and performing missing processing on the raw sequence data; The generator is constructed by stacking multiple layers of LSTM layers with increasing numbers of units, and the discriminator is constructed by fully connected layers, thus establishing a temporal characteristic model of the generative adversarial network. The time series characteristic model is trained using the original sequence data after missing data processing, the loss function of the discriminator is calculated, and a dynamic consistency loss function is set to constrain the padded data generated by the generator within the sliding time window. The function expression of the dynamic consistency loss function is:

[0006] Where, represents the dynamic consistency loss function of the generator, represents the padding value at time step t+1, represents the padding value at time step t, represents the true value at time step t+1, represents the true value at time step t; Alternatingly updating the weights of the generator and the discriminator through back propagation until the loss function of the discriminator and the dynamic consistency loss function are minimized, thereby obtaining the trained temporal characteristic model; The industrial sequence data to be filled with missing values ​​is input into the trained time series characteristic model, and the filled sequence data is output, and the filling evaluation is performed by comparing the deviation between the filled sequence data and the real sequence data.

[0007] Optionally, performing deletion processing on the original sequence data includes: Cleaning the obtained raw sequence data to remove invalid or erroneous data points in the raw sequence data; Mapping the cleaned raw sequence data to a preset interval through normalization processing to obtain a standardized time series data set; A binary mask matrix of the same dimension as the time series dataset is established, the missing positions in the time series dataset are traversed, and missing marks are performed at the corresponding positions in the binary mask matrix to obtain missing sequence data containing the binary mask matrix.

[0008] Optionally, the generator is constructed by stacking LSTM layers with increasing numbers of multi-layer units, including: Add the first fully connected layer and use the activation function ReLU to construct the input layer of the generator; Adding a first Reshape layer between the first fully connected layer and the first LSTM layer to convert the output of the first fully connected layer into a three-dimensional tensor form matching the LSTM layer; Arranging a stack of multiple LSTM layers on the output side of the first fully connected layer, and setting parameters of each LSTM layer to output a hidden state of a time step; A second fully connected layer is added and the activation function Sigmoid is used to construct the output layer of the generator. A second Reshape layer is added on the output side of the second fully connected layer to convert the output of the second fully connected layer into a three-dimensional tensor form.

[0009] Optionally, the training process of the temporal characteristic model includes training the generator, including: generating a random noise vector having the same number of features and sequence length as the missing sequence data according to the number of features and sequence length of the missing sequence data, and inputting the missing sequence data and the random noise vector into the generator; Determining the number of neurons in the first fully connected layer according to the length of the random noise vector, and performing feature transformation on the random noise vector through the first fully connected layer; Using the first Reshape layer, the one-dimensional vector output by the first fully connected layer is converted into three-dimensional tensor data with time steps and features; Dynamically adjust the number of units in the first LSTM layer according to the number of features and sequence length of the missing sequence data, and use multiple layers of the LSTM layer to sequentially capture dependencies in the time dimension from the sequence output by the first Reshape layer; Determining the number of neurons in the second fully connected layer according to the size of the target time window and the number of features of the missing sequence data, and performing dimension conversion on the sequence output by the last LSTM layer through the second fully connected layer; The second Reshape layer is used to convert the one-dimensional vector output by the second fully connected layer and output three-dimensional tensor data with time steps and features, and the padded sequence samples are input into the discriminator to train the generator.

[0010] Optionally, inputting the padded sequence samples into the discriminator to train the generator includes: According to the periodic characteristics of industrial process data, the length and sliding step of the sliding time window are determined, and a dynamic window sliding strategy is established to associate the sliding time window with the generator training. The function expression of the dynamic window sliding strategy is:

[0011] Where, represents the length of the sliding time window, represents the padding value at time step t, represents the true value at time step t, represents the convergence function; Fixing the weight parameters of the discriminator, gradually moving the sliding time window along the time axis of the original sequence data after the missing processing, and after each movement, filling the sequence data in the sliding time window by the generator to obtain a filled sequence sample; Inputting the generated padded sequence samples into the discriminator to obtain a discrimination result, and calculating a dynamic consistency loss function according to the discrimination result of the discriminator; The weight of the generator is updated through back propagation until the dynamic consistency loss function is minimized to obtain the trained generator.

[0012] Optionally, constructing a discriminator by a fully connected layer includes: Add a third fully connected layer and use the ReLU activation function to construct the discriminator, and add a Flatten layer on the input side of the third fully connected layer to flatten the three-dimensional tensor data output by the generator into a one-dimensional vector; A fourth fully-connected layer is added on the output side of the third fully-connected layer and an activation function ReLU is used, and the number of neurons in the fourth fully-connected layer is lower than that in the third fully-connected layer. A fifth fully-connected layer is also added on the output side of the fourth fully-connected layer and an activation function Sigmoid is used. The fifth fully-connected layer is used to output an interval probability value representing the confidence of the true data.

[0013] Optionally, the training process of the temporal characteristic model includes training the discriminator, including: The weight parameters of the generator are fixed, the padded sequence samples and the real sequence data generated by the generator are input into the discriminator, and the loss function of the discriminator is calculated. The functional expression of the discriminator loss function is:

[0014] Where, represents the loss function of the discriminator, Indicates the expected value, represents the real sequence data, represents the distribution of real sequence data, Output of real sequence data, represents the padded sequence sample, represents the distribution of the filled sequence samples, Represents the output of the discriminator for the padded sequence sample; The weights of the discriminator are updated by back propagation until the loss function of the discriminator is minimized, thereby obtaining the trained discriminator.

[0015] Optionally, evaluating the imputed sequence data by comparing the deviation between the imputed sequence data and the true sequence data comprises: According to the padded sequence data output by the time series characteristic model, the root mean square error, mean absolute error and mean square error of the padded sequence data and the real sequence data are calculated respectively. The function expressions of the root mean square error, mean absolute error and mean square error are respectively:

[0016]

[0017]

[0018] Where, represents the root mean square error, represents the mean absolute error, represents the mean square error, Indicates the number of padded sequence data, represents the i-th true value in the real sequence data, Indicates the i-th filling value in the filling sequence data; The root mean square error, mean absolute error, and mean square error of the padded sequence data are compared with the errors corresponding to the padded data of the GAN model, and an evaluation curve is drawn.

[0019] The present invention also discloses a filling system, which adopts the above-mentioned time series-based industrial process missing value filling method, including: The raw data processing module is used to obtain the raw sequence data collected by various sensors in the industrial process and perform missing processing on the raw sequence data; The temporal characteristic model building module is used to build a generator by stacking multiple layers of LSTM layers with increasing numbers of units, and a discriminator by fully connected layers, thereby establishing a temporal characteristic model of the generative adversarial network; a data filling constraint module, configured to train the time series characteristic model using the original sequence data after missing data processing, calculate the loss function of the discriminator, and set a dynamic consistency loss function to constrain the filled data generated by the generator within a sliding time window; A model training module, configured to alternately update the weights of the generator and the discriminator through back propagation until the loss function of the discriminator and the dynamic consistency loss function are minimized, thereby obtaining the trained temporal characteristic model; The filling data evaluation module is used to input the industrial sequence data with missing values ​​to be filled into the trained time series characteristic model, output the filled sequence data, and evaluate the filled sequence data by comparing the deviation between the filled sequence data and the real sequence data.

[0020] The present invention also discloses a computer-readable storage medium having a computer program stored thereon, characterized in that when the computer program is executed by a processor, the computer program implements the above-mentioned method for filling missing values ​​in an industrial process based on time series.

[0021] Compared with the prior art, the method and system for filling missing values ​​in industrial processes based on time series provided by the embodiments of the present invention have the following beneficial effects: By constructing a multi-layer LSTM generator structure with incremental units, the time series characteristic model is equipped with the core capability to capture the nonlinear dynamic characteristics of industrial data. By strictly aligning the change in the padded value between adjacent time steps with the change trend of the true value, the generated data is ensured to maintain consistency in the dynamic trend of the original data during the padded process, avoiding data feature distortion caused by padded data and ensuring the authenticity and validity of the data. Furthermore, the introduction of a dynamic sliding window training strategy automatically optimizes according to different operating conditions and adaptively handles data changes at different time scales, effectively improving data processing efficiency and the model's generalization capabilities. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments, in which: Figure 1 A schematic block diagram of the overall steps of the method for filling missing values ​​in an industrial process provided by an embodiment of the present invention; Figure 2 A schematic block diagram of a generator according to an embodiment of the present invention confronting a discriminator under the constraints of a sliding time window and dynamic consistency loss; Figure 3 A schematic block diagram of the execution flow of the method for filling missing values ​​in an industrial process provided by an embodiment of the present invention; Figure 4 This is a diagram showing the filling effect of the method of the present invention in the first application provided by an embodiment of the present invention when the missing rate is 40%; Figure 5 A schematic diagram comparing the RMSE, MSE, and MAE of the method of the present invention and the traditional GAN ​​model in Application 1 provided in an embodiment of the present invention; Figure 6 This is a diagram showing the filling effect of the method of the present invention in the second application provided by the embodiment of the present invention when the missing rate is 40%; Figure 7A schematic diagram comparing the RMSE, MSE, and MAE of the method of the present invention and the traditional GAN ​​model in Application 2 provided in an embodiment of the present invention; Figure 8 This is a diagram showing the filling effect of the method of the present invention in the third application provided by the embodiment of the present invention when the missing rate is 40%; Figure 9 This is a diagram of the filling effect of the method of the present invention in application three provided in an embodiment of the present invention when the missing rate is 40%. DETAILED DESCRIPTION

[0023] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in this application can be combined with each other. Now, in conjunction with the accompanying drawings, the preferred embodiments of the present invention will be described in detail.

[0024] The present invention discloses a method for filling missing values ​​in industrial processes based on time series. Figure 1 and Figure 2 Shown, including: S1. Obtain the original sequence data collected by various sensors in the industrial process and perform missing processing on the original sequence data; S2: The generator is constructed by stacking multiple layers of LSTM layers with increasing numbers of units, and the discriminator is constructed by fully connected layers, thus establishing a temporal characteristic model of the generative adversarial network. S3. Use the original sequence data after missing data processing to train the time series feature model, calculate the loss function of the discriminator, and set the dynamic consistency loss function to constrain the filled data generated by the generator within the sliding time window. The function expression of the dynamic consistency loss function is:

[0025] Where, represents the dynamic consistency loss function of the generator, represents the padding value at time step t+1, represents the padding value at time step t, represents the true value at time step t+1, represents the true value at time step t; S4. Alternately update the weights of the generator and discriminator through backpropagation until the discriminator loss function and the dynamic consistency loss function are minimized to obtain the trained temporal feature model; S5. Input the industrial sequence data with missing values ​​to be filled into the trained time series characteristic model, output the filled sequence data, and evaluate the filling by comparing the deviation between the filled sequence data and the real sequence data.

[0026] By implementing the aforementioned missing value imputation method in industrial processes, the GAN is integrated with a long short-term memory (LSTM) network to construct a multi-layer LSTM generator structure with incrementally increasing units. This in turn establishes a temporal characteristic model of a generative adversarial network. This model leverages the strengths of both GAN and LSTM, accurately modeling long-term data dependencies and deeply exploring temporal correlations. GAN, through its adversarial training strategy, continuously optimizes the quality of imputed data, making the imputed results more realistic, reasonable, and random, while avoiding the pattern fixation issues that can occur with traditional interpolation methods. The LSTM's unique gating mechanism enables it to maintain a strong memory capacity while learning the dynamic patterns of industrial process data, ensuring that the imputed data not only conforms to historical trends but also remains consistent and stable over long time spans. This gives the temporal characteristic model the core capability to capture the nonlinear dynamic characteristics of industrial data. Furthermore, by strictly aligning the imputed value changes between adjacent time steps with the true value trends, the generated data maintains consistency with the original data in terms of dynamic trends during the imputation process, avoiding data feature distortion caused by imputation and guaranteeing data authenticity and validity. The introduction of a dynamic sliding window training strategy automatically optimizes according to different operating conditions and adaptively handles data changes at different time scales, effectively improving data processing efficiency and model generalization capabilities. Compared to traditional methods for imputing missing values ​​in industrial processes, the method of this embodiment not only overcomes the limitations of a single model when processing complex industrial data, but also better adapts to the dynamic changes and uncertainties of industrial process data through dynamic consistency constraints and sliding time window optimization.

[0027] Furthermore, the original sequence data is processed for missing information, including: Clean the acquired raw sequence data to remove invalid or erroneous data points; The cleaned raw sequence data is mapped to the preset interval through normalization processing to obtain a standardized time series data set; A binary mask matrix with the same dimension as the time series dataset is established, the missing positions in the time series dataset are traversed, and missing marks are made at the corresponding positions in the binary mask matrix to obtain the missing sequence data containing the binary mask matrix.

[0028] By implementing the above-mentioned method for filling missing values ​​in industrial processes, we first collect time series data collected by various sensors in the industrial process, and then preliminarily organize the data to remove data points with obvious errors or anomalies. The organized data is then normalized and mapped to the interval [0, 1] or [-1, 1]. Using the common normalization formula Take the data mapping to [0, 1] as an example: is the original data value, and are the maximum and minimum values ​​in the dataset, respectively. This approach can accelerate model training and improve training stability. Furthermore, by marking missing locations in the data and generating a binary mask matrix (0 indicates missing, 1 indicates present), this mask matrix serves as one of the inputs for training the time series feature model and guides the improved generator to fill in the missing areas.

[0029] Furthermore, the generator is constructed by stacking multiple LSTM layers with increasing numbers of units, including: Add the first fully connected layer and use the ReLU activation function to construct the input layer of the generator; Add the first Reshape layer between the first fully connected layer and the first LSTM layer to convert the output of the first fully connected layer into a three-dimensional tensor form that matches the LSTM layer; Set up a stack of multi-layer LSTM layers on the output side of the first fully connected layer, and set the parameters of each LSTM layer to output the hidden state of the time step; Add a second fully connected layer and use the activation function Sigmoid to construct the output layer of the generator. Add a second Reshape layer on the output side of the second fully connected layer to convert the output of the second fully connected layer into a three-dimensional tensor form.

[0030] Furthermore, the training process of the time series feature model includes the training of the generator, including: According to the number of features and sequence length of the missing sequence data, a random noise vector with the same number of features and sequence length is generated, and the missing sequence data and the random noise vector are input into the generator; Determine the number of neurons in the first fully connected layer according to the length of the random noise vector, and perform feature transformation on the random noise vector through the first fully connected layer; Use the first Reshape layer to convert the one-dimensional vector output by the first fully connected layer into a three-dimensional tensor data with time steps and features; Dynamically adjust the number of units in the first LSTM layer according to the number of features and sequence length of the missing sequence data, and use multiple LSTM layers to capture the dependencies in the time dimension from the sequence output by the first Reshape layer. The number of neurons in the second fully connected layer is determined based on the size of the target time window and the number of features of the original sequence data. The sequence output by the last LSTM layer is transformed in dimension through the second fully connected layer. The second Reshape layer is used to transform the one-dimensional vector output by the second fully connected layer and output three-dimensional tensor data with time steps and features, and the padded sequence samples are input into the discriminator to train the generator.

[0031] Furthermore, the padded sequence samples are input into the discriminator to train the generator, including: According to the periodic characteristics of industrial process data, the length and sliding step of the sliding time window are determined, and a dynamic window sliding strategy is established to associate the sliding time window with generator training. The function expression of the dynamic window sliding strategy is:

[0032] Where, represents the length of the sliding time window, represents the padding value at time step t, represents the true value at time step t, represents the convergence function; The weight parameters of the discriminator are fixed, and the sliding time window is gradually moved along the time axis of the original sequence data after missing processing. After each movement, the sequence data in the sliding time window is filled by the generator to obtain the filled sequence samples; The generated padded sequence samples are input into the discriminator to obtain the discrimination result, and the dynamic consistency loss function is calculated based on the discrimination result of the discriminator; The weights of the generator are updated through backpropagation until the dynamic consistency loss function is minimized to obtain the trained generator.

[0033] Through the implementation of the above-mentioned industrial process missing value filling method embodiment, the main task of the generator is to generate filling data similar to the real data distribution based on the input random noise vector. In the present invention, the improved generator uses LSTM as the core component of the generator in order to capture the long-term dependency of time series data. In the input stage, the length of the random noise vector will be dynamically adjusted according to the complexity of the missing sequence data and the size of the feature space. That is, the length of the random noise vector can be determined by calculating the comprehensive index of the number of features of the missing sequence data and the sequence length. Afterwards, the random noise vector will be converted through the first fully connected layer. The number of neurons in the first fully connected layer is determined according to a certain proportion of the length of the noise vector to perform a preliminary feature transformation on the input noise vector. The first Reshape layer converts the output of the first fully connected layer into a three-dimensional tensor form suitable for the input of the LSTM layer. The shape of this three-dimensional tensor needs to be determined according to the sequence length of the missing sequence data and the processing requirements of the subsequent LSTM layer to ensure that the data can enter the LSTM layer in a suitable format for processing. Next, the data passes through multiple LSTM layers in sequence. For example, in the first LSTM layer, the number of units is dynamically adjusted based on the time series length and number of features in the input data. Return_sequences=True is also set to output the hidden state at each time step, enabling subsequent LSTM layers to learn richer time series information. The second LSTM layer has a further increase in the number of units, typically set to twice that of the first LSTM layer, to extract deeper time series features. At the output stage, the second fully connected layer transforms the output of the final LSTM layer into a dimension corresponding to the target time window size and number of features. The target time window size can be adjusted based on the periodicity and frequency of change of the data. Finally, a second Reshape layer transforms the output into the appropriate shape to output the required padded data.

[0034] Furthermore, the sliding time window is optimized to determine its length and sliding step size based on the cyclical characteristics of industrial process data. In time series processing, different phases may exhibit different patterns of change, so a sliding time window strategy is employed to balance short-term fluctuations with long-term trends. During training, the sliding time window is incrementally moved along the time series axis, with only the data within the sliding window being trained and constrained locally at each time, enhancing the model's ability to capture local dynamic features.

[0035] Furthermore, the discriminator is constructed by the fully connected layer, including: Add a third fully connected layer and use the ReLU activation function to build the discriminator. Add a Flatten layer on the input side of the third fully connected layer to flatten the three-dimensional tensor data output by the generator into a one-dimensional vector. A fourth fully connected layer is added on the output side of the third fully connected layer and the activation function ReLU is used. The number of neurons in the fourth fully connected layer is lower than that in the third fully connected layer. A fifth fully connected layer is also added on the output side of the fourth fully connected layer and the activation function Sigmoid is used. The fifth fully connected layer is used to output an interval probability value that represents the confidence of the true data.

[0036] Furthermore, the training process of the time series feature model includes the training of the discriminator, including: Fix the weight parameters of the generator, input the padded sequence samples and real sequence data generated by the generator into the discriminator, and calculate the loss function of the discriminator. The function expression of the discriminator loss function is:

[0037] Where, represents the loss function of the discriminator, Indicates the expected value, represents the real sequence data, represents the distribution of real sequence data, Output of real sequence data, represents the padded sequence sample, represents the distribution of the filled sequence samples, Represents the output of the discriminator for the padded sequence sample; The weights of the discriminator are updated through back propagation until the loss function of the discriminator is minimized to obtain the trained discriminator.

[0038] Through the implementation of the above-mentioned method for imputing missing values ​​in an industrial process, the discriminator primarily processes data in the time dimension and determines the authenticity of the input data. To achieve this, the present invention first adds a Flatten layer to flatten the input three-dimensional tensor data into a one-dimensional vector, facilitating processing by the subsequent fully connected layer. The data then passes through a third fully connected layer for feature extraction. The number of neurons in this third fully connected layer is proportional to the input data dimension (i.e., the length of the flattened one-dimensional vector), thereby extracting preliminary features of the input data. Next, a fourth fully connected layer, with half the number of neurons in the third fully connected layer, further mines features. Finally, a fifth fully connected layer outputs a probability value between 0 and 1, which is used to determine whether the input data is real or fake data generated by the generator. In this way, the discriminator can effectively assess the authenticity of the input data.

[0039] As mentioned above, before training the time series characteristic model, the model training parameters are determined first. For example, the learning rate is generally set between 0.0001 and 0.001, and the Adam optimizer is used. The batch size is usually set to 16, 32, 64, etc. The number of training rounds can be adjusted according to actual conditions, generally between 100 and 500 rounds, and can be dynamically adjusted according to the convergence situation.

[0040] Furthermore, the imputed sequence data is evaluated by comparing the deviations between the imputed sequence data and the true sequence data, including: According to the padded sequence data output by the time series characteristic model, the root mean square error, mean absolute error and mean square error of the padded sequence data and the real sequence data are calculated respectively. The function expressions of the root mean square error, mean absolute error and mean square error are:

[0041]

[0042]

[0043] Where, represents the root mean square error, represents the mean absolute error, represents the mean square error, Indicates the number of padded sequence data, represents the i-th true value in the real sequence data, Indicates the i-th filling value in the filling sequence data; The root mean square error, mean absolute error, and mean square error of the padded sequence data are compared with the errors corresponding to the padded data of the GAN model, and an evaluation curve is drawn.

[0044] By implementing the above-described embodiment of the method for imputing missing values ​​in an industrial process, after model training is completed, the industrial process time series data requiring imputation is input into the trained generator. The generator then outputs the imputed time series data, which is then denormalized and restored to its original dimensions, completing the missing value imputation task. The imputed time series data is then compared with the data imputed using a traditional GAN ​​model using comprehensive metrics such as root mean square error (RMSE), mean absolute error (MAE), and mean square error (MSE). An evaluation curve is then plotted to comprehensively evaluate the imputed sequence data generated by the present method, thereby assessing its accuracy.

[0045] Comprehensive industrial process missing value filling method of the present invention, combined with Figure 3As shown, the specific execution process is as follows: Read the raw sequence data collected by various sensors in the industrial process; Perform preprocessing such as cleaning, normalization, and deletion on the raw sequence data; Check whether there are missing positions in the preprocessed original sequence data. If there are missing positions, generate missing sequence data containing a binary mask matrix; According to the present invention, a temporal characteristic model of a generative adversarial network is established. The batch-generated missing sequence data is divided into a training set and a test set. The training set is input into the temporal characteristic model and constrained by the dynamic window sliding strategy of the present invention. The generator captures the temporal dependency relationship based on the LSTM layer depth with increasing multi-layer units and preliminarily fills the missing positions of the missing sequence data in the training set. The discriminator then mines the features in the filled data based on temporal processing and judges the authenticity of the data filled by the generator. Based on adversarial training between the generator and the discriminator, the discriminator minimizes the loss function, continuously improving its ability to identify subtle time series features. This in turn forces the generator to produce more realistic sequence waveforms. Under the constraint of the dynamic consistency loss function, the logical correlation between adjacent time steps is forced to be corrected, so that the fill-in values ​​conform to both the overall data rules and the local dynamic continuity, thus obtaining a trained time series characteristic model. Input the industrial sequence data with missing values ​​to be filled in the test set into the trained time series characteristic model, and calculate the root mean square error, mean absolute error, and mean square error of the output filled sequence data. Record the evaluation indicators of the time series characteristic model based on the comparison between the filled sequence data and the real sequence data; Compare the root mean square error, mean absolute error, and mean square error of the padded sequence data with the errors corresponding to the GAN model padded data, and record the evaluation indicators of the time series characteristic model again based on the comparison between the padded sequence data and the GAN model padded data; Traverse the preprocessed original sequence data to see if there are missing positions. If there are no missing positions, draw an evaluation curve; if there are missing positions, continue to generate missing sequence data containing a binary mask matrix and repeat the above process.

[0046] The present invention further illustrates the above-mentioned method for filling missing values ​​in industrial processes with specific application examples: Application 1: Daily exchange rates for eight countries from 1990 to 2016 (country numbers range from 0-6-OT, 0: Australia exchange rate, 1: UK exchange rate, 2: Canada exchange rate, 3: Switzerland exchange rate, 4: China exchange rate, 5: Japan exchange rate, 6: New Zealand exchange rate, OT: Singapore exchange rate): S1. First, the data is preliminarily sorted and normalized, and the data is missing. The initial data of each missing rate is generated, and the missing values ​​of the data are marked to generate a mask matrix.

[0047] S2. Model Building: First, the input random noise vector of length 100 is transformed through a first fully connected layer with 128 neurons and a ReLU activation function. The first Reshape layer then converts the output into a three-dimensional tensor suitable for LSTM layer input. It then passes through two LSTM layers. The first LSTM layer contains 256 units and sets return_sequences=True to output the hidden state of each time step, allowing the subsequent LSTM layers to learn richer time series information. The second LSTM layer contains 512 units to further extract time series features. The output is then converted to a dimension related to the window size and number of features through a second fully connected layer, using a Sigmoid activation function. Finally, a second Reshape layer converts the output to the appropriate shape to output the required padded data.

[0048] To process data in the temporal dimension, the discriminator S3 adds a Flatten layer to flatten the input three-dimensional tensor data into a one-dimensional vector. Feature extraction is then performed through a third fully connected layer with 512 neurons and a ReLU activation function. Further feature mining is performed through a fourth fully connected layer with 256 neurons. Finally, a fifth fully connected layer with a Sigmoid activation function outputs a probability value between 0 and 1, which is used to determine the authenticity of the input data.

[0049] S4. Determine the model training parameters and constraint strategy. Set the Adam learning rate to 0.0005, the batch size to 64, the number of training rounds to 300, and the initial length of the time window to 30.

[0050] S5. Train the model: Alternately update the generator and discriminator parameters through backpropagation to minimize the dynamic consistency loss and the discriminator loss, training the generator and discriminator. Repeat this training process as the sliding window moves across the time series data until the model converges.

[0051] S6. Fill in missing values ​​in the data: Input the industrial process time series data that need to fill in missing values ​​into the model, and denormalize the obtained complete data to restore it to the original dimension to complete the missing value filling task. Take data with a missing rate of 40% and select 300 data to show the effect. Figure 4The following is a graph showing the effect of model filling, and a comparison graph showing the original complete data and the filled data. RMSE, MAE, and MSE indicators are calculated and compared with the data filled using the traditional GAN ​​model under various missing rates. Figure 5 .like Figure 4 and Figure 5 As shown in the figure, the data filled by the industrial process missing value filling method of the present invention is closer to the true value than the traditional GAN ​​model filling method.

[0052] Application 2: The sensor data for a chemical plant's continuous production process includes 40 parameters such as temperature, pressure, flow, and pH value, with an interval of 10 minutes: S1. First, the data is preliminarily sorted and normalized, and the data is missing. The initial data of each missing rate is generated, and the missing values ​​of the data are marked to generate a mask matrix.

[0053] S2. Constructing the Generator and Discriminator: A random noise vector of length 150 is used and transformed through the first fully connected layer, which has 200 neurons and uses the LeakyReLU activation function. Subsequently, the first Reshape layer transforms the output of the fully connected layer into a three-dimensional tensor suitable for input to the LSTM layer. The data then passes through three LSTM layers in sequence. The first LSTM layer has 128 units and is set with return_sequences=True. This setting enables it to output the hidden state for each time step, allowing subsequent LSTM layers to learn richer time series information. The second LSTM layer has 256 units and further extracts time series features. The third LSTM layer has 512 units and can more deeply explore long-term dependencies in the data. The output of the LSTM layer is transformed into a dimension related to the window size and number of features by the second fully connected layer. The activation function of this fully connected layer, Tanh, has a range of [-1, 1], making it well suited for generating values ​​that resemble the real data distribution. The second Reshape layer is used to transform the output into a shape of (20, 5) to output the padded data that meets the requirements.

[0054] S3. Determine the training parameters and constraint strategy of the model: set the Adam learning rate to 0.0001, the batch size to 128, the number of training rounds to 400, and the initial length of the time window to 50.

[0055] S4. Train the model: Alternately update the generator and discriminator parameters through backpropagation to minimize the dynamic consistency loss and the discriminator loss, training the generator and discriminator. Repeat this training process as the sliding window moves across the time series data until the model converges.

[0056] S5. Fill in missing values ​​in the data: Input the industrial process time series data that need to fill in missing values ​​into the model, and denormalize the obtained complete data to restore it to the original dimension to complete the missing value filling task. 300 data are selected from the data with a missing rate of 30% to demonstrate the effect. Figure 6 The following is a graph showing the effect of model filling, and a comparison graph showing the original complete data and the filled data. RMSE, MAE, and MSE indicators are calculated, and compared with the data filled using the traditional GAN ​​model under various missing rates. Figure 7 .like Figure 6 and Figure 7 As shown in the figure, the data filled by the industrial process missing value filling method of the present invention is closer to the true value than the traditional GAN ​​model filling method.

[0057] Application 3: Parameter data collected during the crystal pulling process at the National and Local Joint Engineering Research Center for Crystal Growth Equipment and System Integration at Xi'an University of Technology includes five characteristics: average growth rate, maximum V / G (crystal growth rate V and solid-liquid interface temperature gradient G), maximum stress, interface deformation, and oxygen content. S1. First, conduct preliminary collation of industrial process time series data. To ensure better stability and convergence during model training, normalize the data using a normalization method, mapping the data's numerical range to an appropriate interval. Next, perform data missingness processing, setting a series of different missingness rates and generating initial data with varying degrees of missingness based on these missingness rates. At the same time, mark the missing values ​​in the data and generate a corresponding mask matrix. This matrix clearly indicates where missing values ​​exist in the data, providing key information for subsequent model training.

[0058] S2. Building the generator and discriminator is the core of the model. The generator uses a random noise vector of length 200 as input. This random noise vector first passes through the first fully connected layer, which has 250 neurons and uses the LeakyReLU activation function, which effectively mitigates the vanishing gradient problem and enhances the model's learning ability. The first Reshape layer then converts the output of the fully connected layer into a three-dimensional tensor suitable for input to the LSTM layer. The data then passes through three LSTM layers in sequence. The first LSTM layer contains 150 units and has return_sequences=True set. This allows it to output the hidden state at each time step, helping the subsequent LSTM layers learn richer and more detailed time series information. The second LSTM layer has 300 units and further extracts time series features. The third LSTM layer has 600 units and can further explore long-term dependencies in the data. After processing by the LSTM layer, the output is converted to a dimension related to the target time window size and number of features by the second fully connected layer. The activation function of the second fully connected layer uses Tanh, whose value range is between [-1, 1], making it ideal for generating values ​​that approximate the distribution of real data. Finally, the second Reshape layer transforms the output to a shape of (25, 6), thereby obtaining padded data that meets the requirements. The discriminator receives the input data, processes it, and determines the probability of whether the data is real or generated.

[0059] S3. Training the model: Determine the model's training parameters and constraint strategy. Set the Adam learning rate to 0.00005, the batch size to 256, the number of training epochs to 400, and the initial time window length to 60.

[0060] S4. Train the model: Alternately update the generator and discriminator parameters through backpropagation to minimize the dynamic consistency loss and the discriminator loss, training the generator and discriminator. Repeat this training process as the sliding window moves across the time series data until the model converges.

[0061] S5. Fill in missing values ​​in the data: Input the industrial process time series data that need to fill in missing values ​​into the model, and denormalize the obtained complete data to restore it to the original dimension to complete the missing value filling task. 300 data are selected from the data with a missing rate of 30% to demonstrate the effect. Figure 8 The following is a graph showing the effect of model filling, and a comparison graph showing the original complete data and the filled data. RMSE, MAE, and MSE indicators are calculated, and compared with the data filled using the traditional GAN ​​model under various missing rates. Figure 9 .like Figure 8 and Figure 9 As shown in the figure, the data filled by the industrial process missing value filling method of the present invention is closer to the true value than the traditional GAN ​​model filling method.

[0062] The present invention also discloses a filling system, which adopts the above-mentioned time series-based industrial process missing value filling method, including: The raw data processing module is used to obtain the raw sequence data collected by various sensors in the industrial process and perform missing processing on the raw sequence data; The temporal characteristic model building module is used to build a generator by stacking multiple layers of LSTM layers with increasing numbers of units, and a discriminator by fully connected layers, thereby establishing a temporal characteristic model of the generative adversarial network; The data filling constraint module is used to train the time series feature model using the original sequence data after missing processing, calculate the loss function of the discriminator, and set the dynamic consistency loss function to constrain the filled data generated by the generator within the sliding time window; The model training module is used to alternately update the weights of the generator and discriminator through backpropagation until the discriminator loss function and the dynamic consistency loss function are minimized to obtain the trained time series feature model; The imputed data evaluation module is used to input the industrial sequence data with missing values ​​to be filled into the trained time series characteristic model, output the imputed sequence data, and evaluate the imputed sequence data by comparing the deviation between the imputed sequence data and the real sequence data.

[0063] The present invention also discloses a computer-readable storage medium having a computer program stored thereon, characterized in that when the computer program is executed by a processor, the steps of the above-mentioned method for filling missing values ​​in an industrial process based on time series are implemented.

[0064] The present invention also discloses a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the above-mentioned method for filling missing values ​​in an industrial process based on time series are implemented.

[0065] The present invention is described based on flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to specific embodiments. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0066] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0067] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0068] It should be understood that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Those skilled in the art may modify the technical solutions described in the above embodiments, or replace some of the technical features therein with equivalents; and all these modifications and replacements should fall within the scope of protection of the present invention.

Claims

1. A method for filling missing values ​​in industrial processes based on time series, characterized in that: include: Obtaining raw sequence data collected by various sensors in the industrial process and performing missing processing on the raw sequence data; The generator is constructed by stacking multiple layers of LSTM layers with increasing numbers of units, and the discriminator is constructed by fully connected layers, thus establishing a temporal characteristic model of the generative adversarial network. The time series characteristic model is trained using the original sequence data after missing data processing, the loss function of the discriminator is calculated, and a dynamic consistency loss function is set to constrain the padded data generated by the generator within the sliding time window. The function expression of the dynamic consistency loss function is: Where, represents the dynamic consistency loss function of the generator, represents the padding value at time step t+1, represents the padding value at time step t, represents the true value at time step t+1, represents the true value at time step t; Alternatingly updating the weights of the generator and the discriminator through back propagation until the loss function of the discriminator and the dynamic consistency loss function are minimized, thereby obtaining the trained temporal characteristic model; The industrial sequence data to be filled with missing values ​​is input into the trained time series characteristic model, and the filled sequence data is output, and the filling evaluation is performed by comparing the deviation between the filled sequence data and the real sequence data.

2. The method for filling missing values ​​in industrial processes based on time series according to claim 1, characterized in that: The performing deletion processing on the original sequence data includes: Cleaning the obtained raw sequence data to remove invalid or erroneous data points in the raw sequence data; Mapping the cleaned raw sequence data to a preset interval through normalization processing to obtain a standardized time series data set; A binary mask matrix of the same dimension as the time series dataset is established, the missing positions in the time series dataset are traversed, and missing marks are performed at the corresponding positions in the binary mask matrix to obtain missing sequence data containing the binary mask matrix.

3. The method for filling missing values ​​in industrial processes based on time series according to claim 2, characterized in that: The generator is constructed by stacking LSTM layers with increasing numbers of multi-layer units, including: Add the first fully connected layer and use the activation function ReLU to construct the input layer of the generator; Adding a first Reshape layer between the first fully connected layer and the first LSTM layer to convert the output of the first fully connected layer into a three-dimensional tensor form matching the LSTM layer; Arranging a stack of multiple LSTM layers on the output side of the first fully connected layer, and setting parameters of each LSTM layer to output a hidden state of a time step; A second fully connected layer is added and the activation function Sigmoid is used to construct the output layer of the generator. A second Reshape layer is added on the output side of the second fully connected layer to convert the output of the second fully connected layer into a three-dimensional tensor form.

4. The method for filling missing values ​​in industrial processes based on time series according to claim 3 is characterized in that: The training process of the temporal characteristic model includes training of the generator, including: generating a random noise vector having the same number of features and sequence length as the missing sequence data according to the number of features and sequence length of the missing sequence data, and inputting the missing sequence data and the random noise vector into the generator; Determining the number of neurons in the first fully connected layer according to the length of the random noise vector, and performing feature transformation on the random noise vector through the first fully connected layer; Using the first Reshape layer, the one-dimensional vector output by the first fully connected layer is converted into three-dimensional tensor data with time steps and features; Dynamically adjust the number of units in the first LSTM layer according to the number of features and sequence length of the missing sequence data, and use multiple layers of the LSTM layer to sequentially capture dependencies in the time dimension from the sequence output by the first Reshape layer; Determining the number of neurons in the second fully connected layer according to the size of the target time window and the number of features of the missing sequence data, and performing dimension conversion on the sequence output by the last LSTM layer through the second fully connected layer; The second Reshape layer is used to convert the one-dimensional vector output by the second fully connected layer and output three-dimensional tensor data with time steps and features, and the padded sequence samples are input into the discriminator to train the generator.

5. The method for filling missing values ​​in industrial processes based on time series according to claim 4 is characterized in that: Inputting the padded sequence samples into the discriminator to train the generator includes: According to the periodic characteristics of industrial process data, the length and sliding step of the sliding time window are determined, and a dynamic window sliding strategy is established to associate the sliding time window with the generator training. The function expression of the dynamic window sliding strategy is: Where, represents the length of the sliding time window, represents the padding value at time step t, represents the true value at time step t, represents the convergence function; Fixing the weight parameters of the discriminator, gradually moving the sliding time window along the time axis of the original sequence data after the missing processing, and after each movement, filling the sequence data in the sliding time window by the generator to obtain a filled sequence sample; Inputting the generated padded sequence samples into the discriminator to obtain a discrimination result, and calculating a dynamic consistency loss function according to the discrimination result of the discriminator; The weight of the generator is updated through back propagation until the dynamic consistency loss function is minimized to obtain the trained generator.

6. The method for filling missing values ​​in industrial processes based on time series according to claim 5, characterized in that: The discriminator is constructed by the fully connected layer, including: Adding a third fully connected layer and using the activation function ReLU to construct the discriminator, adding a Flatten layer on the input side of the third fully connected layer to flatten the three-dimensional tensor data output by the generator into a one-dimensional vector; A fourth fully-connected layer is added on the output side of the third fully-connected layer and an activation function ReLU is used. The number of neurons in the fourth fully-connected layer is lower than that in the third fully-connected layer. A fifth fully-connected layer is also added on the output side of the fourth fully-connected layer and an activation function Sigmoid is used to output an interval probability value representing the confidence of the true data.

7. The method for filling missing values ​​in industrial processes based on time series according to claim 6, characterized in that: The training process of the temporal characteristic model includes the training of the discriminator, including: The weight parameters of the generator are fixed, the padded sequence samples and the real sequence data generated by the generator are input into the discriminator, and the loss function of the discriminator is calculated. The functional expression of the discriminator loss function is: Where, represents the loss function of the discriminator, Indicates the expected value, represents the real sequence data, represents the distribution of real sequence data, represents the output of the discriminator for the real sequence data, represents the padded sequence sample, represents the distribution of the filled sequence samples, Represents the output of the discriminator for the padded sequence sample; The weights of the discriminator are updated by back propagation until the loss function of the discriminator is minimized, thereby obtaining the trained discriminator.

8. The method for filling missing values ​​in industrial processes based on time series according to claim 1, characterized in that: The filling evaluation by comparing the deviation between the filled sequence data and the true sequence data comprises: According to the padded sequence data output by the time series characteristic model, the root mean square error, mean absolute error and mean square error of the padded sequence data and the real sequence data are calculated respectively. The function expressions of the root mean square error, mean absolute error and mean square error are respectively: Where, represents the root mean square error, represents the mean absolute error, represents the mean square error, Indicates the number of padded sequence data, represents the i-th true value in the real sequence data, Indicates the i-th filling value in the filling sequence data; The root mean square error, mean absolute error, and mean square error of the padded sequence data are compared with the errors corresponding to the padded data of the GAN model, and an evaluation curve is drawn.

9. A filling system, using the time series-based industrial process missing value filling method according to any one of claims 1 to 8, characterized in that: The filling system includes: The raw data processing module is used to obtain the raw sequence data collected by various sensors in the industrial process and perform missing processing on the raw sequence data; The temporal characteristic model building module is used to build a generator by stacking multiple layers of LSTM layers with increasing numbers of units, and a discriminator by fully connected layers, thereby establishing a temporal characteristic model of the generative adversarial network; a data filling constraint module, configured to train the time series characteristic model using the original sequence data after missing data processing, calculate the loss function of the discriminator, and set a dynamic consistency loss function to constrain the filled data generated by the generator within a sliding time window; A model training module, configured to alternately update the weights of the generator and the discriminator through back propagation until the loss function of the discriminator and the dynamic consistency loss function are minimized, thereby obtaining the trained temporal characteristic model; The filling data evaluation module is used to input the industrial sequence data with missing values ​​to be filled into the trained time series characteristic model, output the filled sequence data, and evaluate the filled sequence data by comparing the deviation between the filled sequence data and the real sequence data.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for filling missing values ​​in an industrial process based on time series according to any one of claims 1 to 8 is implemented.

Images