A method and system for enhancing and predicting small sample industrial time series data

Abstract

This invention provides a method and system for small-sample industrial time-series data augmentation and quality prediction. The method includes the following steps: acquiring a historical dataset as a training dataset; inputting the historical dataset into a variational autoencoder (VAE) model incorporated into the time-series decoding process to generate virtual samples; processing the virtual samples to obtain supplementary process samples; inputting the supplementary process samples into a pre-trained soft sensor model to obtain virtual quality samples; using an extended training dataset composed of the above data to update and train the soft sensor model; and inputting online input variables into the updated soft sensor model to predict online quality variables. The system includes a sample generation module, a hybrid training module, and a prediction module electrically connected in sequence. Compared to traditional VAE-based methods, the addition of a time-series decoder ensures that the generated virtual samples more closely approximate the distribution of real samples, exhibiting better diversity.

Description

Technical Field

[0001] The present invention relates to the field of data processing, and in particular to a method and system for enhancing and predicting quality of small sample industrial time series data. Background Art

[0002] In order to ensure the safety and efficiency of industrial processes, strict control of various process variables is an important task. However, for some key variables, especially quality variables, it is usually difficult to achieve real-time measurement. Soft sensing technology can predict difficult-to-measure quality variables through easy-to-measure process variables. Soft sensing technology can generally be divided into two categories, namely model-based methods and data-based methods. Model-based methods rely on domain knowledge and theoretical models to describe and explain process phenomena, and make predictions and decisions by building mathematical models. These methods have excellent interpretability, but rely heavily on theoretical understanding of systems and processes, and require expertise and experience in model analysis. In contrast, data-driven methods use statistical techniques to extract data features, automatically learn from large amounts of data with greater flexibility, and are particularly good at identifying features such as nonlinearity between high-dimensional and complex data structures. With the growth of process complexity and the development of data transmission technology, data-based soft sensing technology has become mainstream. However, problems such as missing values, low sampling rates, and high sampling repetition rates can lead to insufficient training data for data-based soft sensing neural network models. In this case, the soft sensing neural network model cannot fully capture the dynamics of the process and may not accurately predict quality variables. This phenomenon is called the small sample problem.

[0003] To address the problem of small samples, data augmentation can be performed through virtual sample generation (VSG) technology. VSG technology aims to create new samples based on existing data sets to expand and enrich the training data set, thereby improving the performance of soft sensor neural network models. Sampling-based VSG techniques, such as classic methods such as interpolation and Bootstrap, play an important role in soft sensor applications. Although additional samples are generated, they do not essentially generate new data, but use existing data to generate additional samples. With the rapid development of deep learning technology, feature extraction-based VSG techniques, such as variational autoencoders (VAE) and generative adversarial networks (GAN), have achieved good results in generating high-dimensional data and increasing sample diversity. For example, VAE can be used to extract features from noisy process data, measure the similarity between current samples and historical samples, learn the distribution of high-dimensional complex data, and generate similar samples. Although VAE and GAN-based methods have shown strong capabilities in generating virtual samples, they encounter limitations when applied to time series data, without considering the temporal correlation and dynamic characteristics of process data. Since it cannot fully capture the data distribution of time series data, VAE reduces the diversity of the model to minimize the reconstruction error loss, which leads to data homogeneity. For GAN, although it can produce more diverse samples, it is easy to generate virtual samples that deviate significantly from the actual data. Therefore, it is an urgent problem to provide a small sample industrial time series data enhancement and quality prediction method and system that can take into account both the diversity and authenticity of virtual samples. Summary of the invention

[0004] The technical problem to be solved by the present invention is to provide a small sample industrial time series data enhancement and quality prediction method and system.

[0005] The technical solution of the present invention is: a method for enhancing and predicting the quality of small sample industrial time series data, the method comprising the steps of:

[0006] S1. Obtain historical data sets As a training dataset, the historical dataset D is input into the variational autoencoder model of the time series decoding process to generate virtual samples. , for the virtual sample Samples are processed to obtain additional process samples ;

[0007] S2. Input the pre-trained soft-sensing neural network model to obtain virtual quality samples , will be composed of historical data sets , Supplementary process samples And virtual quality samples The extended training dataset consists of , by expanding the training dataset Update and train the soft-sensing neural network model;

[0008] S3, input variables online Input the updated soft sensor neural network model to predict online quality variables ;

[0009] Wherein, step S1 comprises the steps of:

[0010] S11. Preprocess industrial time series data to obtain historical data sets As a training dataset;

[0011] S12. Generate virtual samples by inputting the historical data set D into the variational autoencoder model of the time series decoding process , for the virtual sample Perform screening and expansion to obtain virtual process samples ;

[0012] Wherein, X represents a general process variable of N×T×M, Y represents a key quality variable of N×T×m, where N represents the variable dimension, T is the time step, and M and m represent the dimensions of the process variable and the quality variable respectively; and step S12 comprises:

[0013] S121, taking the historical data set D as the model input and inputting it into the encoder for encoding, wherein the encoding layer includes several convolutional layers, and after the expansion operation, the latent variable z is calculated and obtained;

[0014] S122, after the latent variable z is combined with the conditional variable c, they enter the decoder as input variables. During the decoding process, the input variables pass through multiple time series modules to extract the time series features under each model;

[0015] S123, combine all the time series features to obtain a new virtual sample ;

[0016] S124. Virtual Sample Conduct screening expansion to obtain supplementary process samples .

[0017] Preferably, in step S122, the multiple temporal modules include a trend module, a plurality of seasonal modules and a residual module.

[0018] Preferably, in step S12, the loss function is calculated as:

[0019]

[0020] In the above formula, is the likelihood function, is the posterior probability distribution of z under the conditions of x and c, where x is the reconstructed time series data and c is the conditional variable. It follows a Gaussian distribution. yes and The KL divergence between . for Abbreviation of for Abbreviation of .

[0021] Preferably, in step S121, Obtain the corresponding latent variable z. After the latent variable z passes through the decoder, a virtual sample can be obtained by reconstruction. ;in is the value randomly sampled from the normal distribution of N(0,1), μ represents the mean of the latent variable, is the standard deviation of the latent variable.

[0022] Preferably, step S2 includes the following steps:

[0023] S21. Using the pre-trained soft-sensing neural network model to measure the virtual process samples Processing to obtain virtual quality samples ;

[0024] S22. Obtaining an extended training data set , expand the training data set Re-input the training model to obtain the updated soft sensor neural network model.

[0025] Preferably, in step S3, the virtual quality sample The acquisition process is:

[0026]

[0027] Among them, the function Refers to the pre-trained soft-sensing neural network model, represents the weight parameter and b represents the bias.

[0028] The present invention also provides a small sample industrial time series data enhancement and quality prediction system, the system comprising:

[0029] Sample generation module to obtain historical data sets As a training dataset, the historical dataset D is input into the variational autoencoder model of the time series decoding process to generate virtual samples. , for the virtual sample Samples are processed to obtain virtual process samples ;

[0030] A hybrid training module is electrically connected to the sample generation module to Input the pre-trained soft-sensing neural network model to obtain virtual quality samples ; The historical data set D and the virtual process sample And virtual quality samples The extended training dataset consists of , by expanding the training dataset Update and train the soft-sensing neural network model;

[0031] The prediction module is electrically connected to the hybrid training module and inputs the online variables Input the updated soft sensor neural network model to predict online quality variables ;

[0032] Wherein, the sample generation module includes:

[0033] Preprocessing unit, used to preprocess industrial time series data and obtain historical data sets As a training dataset;

[0034] A virtual sample generation unit is electrically connected to the preprocessing unit and is used to generate virtual samples by inputting the historical data set D into the variational autoencoder model with the time series decoding process added. , and for the virtual sample Perform screening expansion to obtain virtual process samples ;

[0035] Wherein, the virtual sample generating unit comprises:

[0036] A latent variable generation subunit is electrically connected to the preprocessing unit, and is used to input the historical data set D as a model input and input it into an encoder for encoding, wherein the encoding layer includes a plurality of convolutional layers, and after an expansion operation, calculates and obtains the latent variable z;

[0037] A time series feature extraction subunit is electrically connected to the latent variable generation subunit, and is used to combine the latent variable z with the conditional variable c, and then use them as input variables to enter the decoder. During the decoding process, the input variables are respectively passed through a trend module, several seasonal modules, and a residual module to extract the time series features under each model;

[0038] The virtual sample acquisition subunit is electrically connected to the time series feature extraction subunit to combine and process all the time series features to obtain a new virtual sample. ;

[0039] A virtual process sample acquisition subunit is electrically connected to the virtual sample acquisition subunit to acquire virtual samples. Perform screening and expansion to obtain virtual process samples .

[0040] Preferably, the hybrid training module includes:

[0041] A virtual quality sample generation unit is electrically connected to the virtual sample generation unit and is used to use a pre-trained soft measurement neural network model to generate virtual process samples. Processing to obtain virtual quality samples ;

[0042] An extended training data set generation unit is electrically connected to the virtual quality sample generation unit to obtain an extended training data set , and expand the training dataset Re-input the training model to obtain the updated soft sensor neural network model.

[0043] Preferably, the virtual sample generation unit further includes a loss function calculation subunit electrically connected to the virtual sample acquisition subunit.

[0044] The above technical solution has the following advantages or beneficial effects: In the above-mentioned small sample industrial time series data enhancement and quality prediction method and system, it consists of three parts: sample generation stage, mixed training stage and prediction stage. Compared with the traditional VAE virtual sample generation method, the above method adds a time series decoder part, that is, through the combination of trend module, seasonal module and residual module, a new decoder method is formed, thereby improving the effect of feature extraction of time series samples and virtual sample generation. Since the time series feature extraction is more accurate and conditional variables are added for control, the sample diversity and reliability are improved. In addition, conditional variables are added in the process of generating virtual samples, so that the generated virtual samples are more sensitive to important output variables such as quality variables, thereby improving the value of virtual samples. Finally, this method introduces the conditional variable c in the loss function, and needs to consider complex situations such as the joint probability distribution of x and c, which makes the derivation and calculation of the loss function more difficult. The loss function finally improved and designed can train the key parameters of the model well and achieve good results. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The embodiments of the present invention will be described more fully with reference to the attached drawings, which are provided for illustration and description only and are not intended to limit the scope of the present invention.

[0046] Figure 1 The process diagram of a small sample industrial time series data enhancement and quality prediction method of the present invention is shown in FIG. Figure 1 ;

[0047] Figure 2 The process diagram of a small sample industrial time series data enhancement and quality prediction method of the present invention is shown in FIG. Figure 2 ;

[0048] Figure 3 The process diagram of a small sample industrial time series data enhancement and quality prediction method of the present invention is shown in FIG. Figure 3 ;

[0049] Figure 4 The process diagram of a small sample industrial time series data enhancement and quality prediction method of the present invention is shown in FIG. Figure 4 ;

[0050] Figure 5 The structure of a small sample industrial time series data enhancement and quality prediction system of the present invention is shown in FIG. Figure 1 ;

[0051] Figure 6 The structure of a small sample industrial time series data enhancement and quality prediction system of the present invention is shown in FIG. Figure 2 ;

[0052] Figure 7 The structure of a small sample industrial time series data enhancement and quality prediction system of the present invention is shown in FIG. Figure 3 ;

[0053] Figure 8 The structure of a small sample industrial time series data enhancement and quality prediction system of the present invention is shown in FIG. Figure 4 ;

[0054] In the accompanying drawings: 1. Sample generation module; 11. Preprocessing unit; 12. Virtual sample generation unit; 121. Latent variable generation subunit; 122. Time series feature extraction subunit; 123. Virtual sample acquisition subunit; 124. Virtual process sample acquisition subunit; 125. Loss function calculation subunit; 2. Hybrid training module; 21. Virtual quality sample generation unit; 22. Extended training data set generation unit; 3. Prediction module. DETAILED DESCRIPTION

[0055] The following is a detailed description of a small sample industrial time series data enhancement and quality prediction method and system of the present invention in conjunction with the accompanying drawings and specific embodiments.

[0056] In the description of the present invention, it should be understood that the orientations or positional relationships indicated by the terms "upper", "lower", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc. are based on the orientations or positional relationships shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must and has a specific orientation, is constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention. At the same time, the terms "first", "second", etc. are only used to distinguish the names of the components, and there is no primary and secondary relationship, and therefore cannot be understood as a limitation on the present invention.

[0057] Embodiment 1

[0058] like Figure 1 As shown, a small sample industrial time series data enhancement and quality prediction method includes the following steps:

[0059] S1. Obtain historical data sets As a training dataset, the historical dataset D is input into the variational autoencoder model of the time series decoding process to generate virtual samples. , for the virtual sample Samples are processed to obtain virtual process samples ;

[0060] S2. Input the pre-trained soft-sensing neural network model to obtain virtual quality samples ; The historical data set D and the virtual process sample And virtual quality samples The extended training dataset consists of , by expanding the training dataset Update and train the soft-sensing neural network model;

[0061] S3, input variables online Input the updated soft sensor neural network model to predict online quality variables .

[0062] Specifically, the above-mentioned small sample industrial time series data enhancement and quality prediction method consists of a sample generation phase, a hybrid training phase, and a prediction phase. First, the historical data set D is input into the variational autoencoder model added to the time series decoding process to generate a virtual sample , and then input the historical dataset D into the variational autoencoder model to generate virtual samples Then we enter the hybrid training phase, in which the virtual process samples Input soft-sensing neural network model to obtain virtual quality samples , and then the expanded training data set The soft-sensing neural network model is updated and trained to obtain an updated soft-sensing neural network model. In this process, the regression relationship between process variables and key quality variables is established using the soft-sensing neural network model. Finally, in the prediction stage, the online input variables are Input the updated soft sensor neural network model to predict online quality variables In this method, a new data augmentation method based on time series conditional autoencoder is proposed. Compared with the traditional VAE-based method, a time series decoder part is added, that is, a new decoder structure is formed through the combination of multiple time series modules such as trend module, seasonal module and residual module, thereby improving the feature extraction of time series samples and the effect of virtual sample generation.

[0063] like Figure 2 As shown, further speaking, step S1 includes the steps of:

[0064] S11. Preprocess industrial time series data to obtain historical data sets As a training dataset;

[0065] S12. Generate virtual samples by inputting the historical data set D into the variational autoencoder model of the time series decoding process , for the virtual sample Perform screening and expansion to obtain virtual process samples .

[0066] After preprocessing the industrial time series data, we can obtain the historical data set , where X represents the general process variables of N×T×M, Y represents the key quality variables of N×T×m, where N represents the variable dimension, T is the time step, and M and m represent the dimensions of process variables and quality variables, respectively. After preprocessing the industrial time series data to obtain the historical data set D, since the historical data set D is a small sample data set, it is usually difficult to achieve accurate industrial process quality prediction by directly building a soft measurement neural network model. Therefore, in order to expand the scale and quality of the original data set, sample generation should be performed first, and then hybrid training and online prediction should be performed.

[0067] like Figure 3 As shown, further speaking, step S2 includes the steps of:

[0068] S21. Using the pre-trained soft-sensing neural network model to measure the virtual process samples Processing to obtain virtual quality samples ;

[0069] S22. Obtaining an extended training data set , expand the training data set Re-input the training model to obtain the updated soft sensor neural network model.

[0070] Specifically, in the hybrid training phase, the pre-trained soft sensor neural network model is used to train the virtual process samples. Processing to obtain virtual quality samples . Summarize all the above samples to obtain the extended training data set , for the expanded training data set Re-input the training model to obtain the updated soft sensor neural network model.

[0071] In the final online prediction, the online input variables Input the updated soft sensor neural network model to predict online quality variables .

[0072] like Figure 3 As shown, further, in step S12, the historical data set D is input into the variational autoencoder model with the time series decoding process added to generate virtual samples The process is:

[0073] S121, taking the historical data set D as the model input and inputting it into the encoder for encoding, wherein the encoding layer includes several convolutional layers, and after the expansion operation, the latent variable z is calculated and obtained;

[0074] S122, after the latent variable z is combined with the conditional variable c, they are jointly used as input variables to enter the decoder. During the decoding process, the input variables are respectively passed through a trend module, several seasonal modules and a residual module to extract the time series features under each model;

[0075] S123, combine all the time series features to obtain a new virtual sample ;

[0076] S124. Virtual Sample Perform screening and expansion to obtain virtual process samples .

[0077] Specifically, when expanding the historical data set D, a variational autoencoder model is used. The model consists of an encoder and a decoder, where the decoder is composed of multiple time series modules, such as a trend module, several seasonal modules, and a residual module. During the expansion process, the historical data set is first input into the encoder for encoding, and several convolutional layers are obtained. The convolutional layers are expanded to calculate the latent variable z. After that, the time series features under each model are extracted through the trend module, several seasonal modules, and the residual module. Finally, all the time series features are combined and processed to obtain a new virtual sample. In this process, compared with the traditional VAE virtual sample generation model, the above method adds a time series decoder part, that is, through the combination of trend module, seasonal module and residual module, a new decoder structure is formed, thereby improving the feature extraction of time series samples and the effect of virtual sample generation. In addition, conditional variables are added in the process of generating virtual samples, making the generated virtual samples more sensitive to important output variables such as quality variables, improving the virtual sample value.

[0078] Furthermore, in step S122, the calculation formula 1-3 for extracting the time series feature using the trend module is:

[0079] Formula (1)

[0080] Formula (2)

[0081] Formula (3)

[0082] Where f is the calculation function of the weight matrix elements in the trend module, z represents the latent variable, c represents the conditional variable, is the bias of the pth weight; R=\left [ {1,r,...,{r}^{p}} \right ], r=\left [ {0,1,...,T-1} \right ] / T is the trend prediction result of step length T, represents the weight coefficient determined by the latent variables and conditional variables, represents the trend feature weight matrix, is the feature extracted by the trend module. It is worth pointing out that refers to the dimension, where Is a random number.

[0083] The calculation formula 4-6 for extracting time series features using the seasonal module is:

[0084] = Formula (4)

[0085] Where f is the calculation function of the weight matrix elements in the seasonal module, represents the weight coefficient determined by the latent variables and conditional variables in the seasonal module, is the bias of the sth weight, refers to the dimension, where is a random number. Then,

[0086] Formula (5)

[0087] Formula (6)

[0088] In the formula represents the seasonal feature weight matrix, represents the output of the seasonal module, that is, the time series features extracted under the seasonal module, refers to the dimension, where Is a random number.

[0089] The calculation formula 7-8 for extracting time series features using the residual module is:

[0090] Formula (7)

[0091] Formula (8)

[0092] Formula (9)

[0093] Where H and h are weight coefficients determined by latent variables and conditional variables, represents the output of the residual module, f represents the activation function, represents the weight matrix, and b represents the deviation coefficient. denotes the weight vector and bias vector respectively, in and k denote the number of input channels and the size of the convolution kernel respectively. h[i] denotes the result of the i-th convolution calculation during the encoding process. i is automatically determined by the size of the convolution kernel k and the data size, starting from 1 until the convolution calculation is completed. j denotes the number of offset calculations in a single convolution calculation, starting from 1 until the convolution offset calculation is completed, and is also automatically determined by the size of the convolution kernel k and the data size. Represents the weight parameter of the jth offset calculation in the i-th convolution calculation, represents the deviation of the i-th convolution calculation. μ represents the mean of the latent variable, represents the variance of the latent variable, and h represents the set of h[i].

[0094] After random sampling from a normal distribution, we can Get the corresponding latent variable z, where is a value randomly sampled from a normal distribution of N(0,1), is the standard deviation of the latent variable.

[0095] Furthermore, the virtual sample , that is, virtual sample The input variable is formed by combining the latent variable z with the conditional variable c, and consists of all the time series features obtained after the trend module, seasonal module and residual module.

[0096] Compared with the traditional VAE virtual sample generation model, the above method adds conditional variables in the process of generating virtual samples, making the generated virtual samples more sensitive to important output variables such as quality variables, thereby improving the value of virtual samples.

[0097] Further, in step S121, after the historical data set D is input into the variational autoencoder model of the time series decoding process, a virtual sample is generated. In the process of , the conditional information and time series data are jointly encoded in the encoder network to obtain the mean μ and variance σ of the latent variable 2 It is worth pointing out that the time series data is the general process variable X after data enhancement, and the conditional information is the key quality variable Y after data enhancement. After that, the latent variable z is integrated with the conditional variable c and transmitted as input to the decoder network to generate the reconstructed time series data x. In this process, the loss function of the model is shown in formula (1):

[0098] Formula (11)

[0099] in, is the likelihood function, is the posterior probability distribution of z under the conditions of x and c, where x is the reconstructed time series data and c is the conditional variable. It follows a Gaussian distribution. yes and The KL divergence between . for Abbreviation of for The model maximizes the lower bound of the log-likelihood function, where:

[0100]

[0101]

[0102] Formula (12)

[0103] Compared with the traditional loss function, the conditional variable c is considered here, that is, a new conditional variable c is added to the loss function to adapt to the training process in this method. Among them, the first term can be regarded as The reconstruction error between the generated data and the original data, and the second term represents the posterior probability and The error between . is the posterior probability distribution of x under the conditions of z and c, represents the prior probability of c, represents the posterior probability of z.

[0104] The calculation method of KL divergence is shown in formula (3):

[0105] KL\left ( {P||Q} \right )=\frac {1} {2}\left [ {log\left ( {\frac {{\sigma}_{Q}} {{\sigma}_{P}}} \right )^{2}-1+\frac {{{\sigma}_{P}}^{2}+\left ( {{\mu}_{P}-{\mu}_{Q}} \right )^{2}} {{{\sigma}_{Q}}^{2}}} \right ] Formula (13)

[0106] in and They represent the standard deviation of the two functions, and They represent the means of the two functions respectively.

[0107] The time series conditional variational autoencoder model can more accurately capture and represent the characteristics of time series data, maintain the temporal nature of the data, and achieve comprehensive modeling of seasonal, trend, and non-periodic residuals. In order to make the training process of this model meet the needs of conditional modeling, the loss function is improved compared to VAE. The loss function of VAE only focuses on the distribution of the general variable x; while this model introduces the conditional variable c into the loss function, and needs to consider complex situations such as the joint probability distribution of x and c, which makes the derivation and calculation of the loss function more difficult. The loss function that was finally improved and designed can well train the key parameters of the model and achieve good results.

[0108] Furthermore, in step S3, the virtual quality sample The acquisition process is:

[0109] Formula (14)

[0110] Among them, the function Refers to the pre-trained soft-sensing neural network model, represents the weight parameter and b represents the bias.

[0111] In the above-mentioned small sample industrial time series data enhancement and quality prediction method, it consists of three parts: sample generation stage, mixed training stage and prediction stage. Compared with the traditional VAE virtual sample generation method, the above method adds a time series decoder part, that is, through the combination of trend module, seasonal module and residual module, a new decoder method is formed, thereby improving the feature extraction of time series samples and the effect of virtual sample generation. In addition, conditional variables are added in the process of generating virtual samples, so that the generated virtual samples are more sensitive to important output variables such as quality variables, thereby improving the value of virtual samples. Since the time series feature extraction is more accurate and conditional variables are added for control, the sample diversity and reliability are improved. Finally, this method introduces the conditional variable c in the loss function, and needs to consider complex situations such as the joint probability distribution of x and c, which makes the derivation and calculation of the loss function more difficult. The loss function finally improved and designed can well train the key parameters of the model and achieve good results.

[0112] Embodiment 2

[0113] A small sample industrial time series data enhancement and quality prediction system, comprising:

[0114] like Figure 5 As shown, sample generation module 1 is used to obtain historical data sets As a training dataset, the historical dataset D is input into the variational autoencoder model of the time series decoding process to generate virtual samples. , for the virtual sample Samples are processed to obtain virtual process samples ;

[0115] The mixed training module 2 is electrically connected to the sample generation module 1 to Input the pre-trained soft-sensing neural network model to obtain virtual quality samples ; The historical data set D and the virtual process sample And virtual quality samples The extended training dataset consists of , by expanding the training dataset Update and train the soft-sensing neural network model;

[0116] The prediction module 3 is electrically connected to the hybrid training module 2 and inputs the online variables Input the updated soft sensor neural network model to predict online quality variables .

[0117] Specifically, the above-mentioned small sample industrial time series data enhancement and quality prediction system consists of a sample generation module 1, a hybrid training module 2 and a prediction module 3. First, in the sample generation module 1, the historical data set D is input into the variational autoencoder model added to the time series decoding process to generate a virtual sample , and then input the historical dataset D into the time series conditional variational autoencoder model to generate virtual samples Then, after entering the hybrid training module 2, the virtual process sample Soft-sensing neural network model to obtain virtual quality samples , and then the expanded training data set The soft measurement neural network model is updated and trained to obtain the updated soft measurement neural network model. In this module, the regression relationship between process variables and key quality variables is established using the soft measurement neural network model. Finally, in the prediction module 3, the online input variables are Input the updated soft sensor neural network model to predict online quality variables In this system, a new data enhancement method based on time series conditional autoencoder is proposed. Compared with the traditional prediction system based on data enhancement, the variational autoencoder module in the above system adds a time series decoding process, that is, through the combination of trend module, seasonal module and residual module, a new decoder structure is formed, thereby improving the feature extraction of time series samples and the effect of virtual sample generation.

[0118] like Figure 6 As shown, further speaking, the sample generation module 1 includes:

[0119] The preprocessing unit 11 is used to preprocess the industrial time series data and obtain the historical data set. As a training dataset;

[0120] The virtual sample generation unit 12 is electrically connected to the preprocessing unit 11, and is used to generate virtual samples by inputting the historical data set D into the variational autoencoder model with the time series decoding process. , and for the virtual sample Perform screening expansion to obtain virtual process samples .

[0121] After preprocessing the industrial time series data, we can obtain the historical data set , where X represents the general process variables of N×T×M, Y represents the key quality variables of N×T×m, where N represents the variable dimension, T is the time step, and M and m represent the dimensions of process variables and quality variables, respectively. After preprocessing the industrial time series data to obtain the historical data set D, since the historical data set D is a small sample data set, it is usually difficult to achieve accurate industrial process quality prediction by directly building a soft measurement neural network model. Therefore, in order to expand the scale and quality of the original data set, sample generation should be performed first, and then hybrid training and online prediction should be performed.

[0122] Further, if Figure 7 As shown, the hybrid training module 2 includes:

[0123] The virtual quality sample generation unit 21 is electrically connected to the virtual sample generation unit 12 and is used to use the pre-trained soft measurement neural network model to generate virtual process samples. Processing to obtain virtual quality samples ;

[0124] The extended training data set generation unit 22 is electrically connected to the virtual quality sample generation unit 21 to obtain the extended training data set. , and expand the training dataset Re-input the training model to obtain the updated soft sensor neural network model.

[0125] Specifically, in the virtual quality sample generation unit 21, the virtual process sample is generated by using the pre-trained soft measurement neural network model. Processing to obtain virtual quality samples The extended training data set generation unit 22 aggregates all the above samples to obtain the extended training data set. , for the expanded training data set Re-input the training model to obtain the updated soft sensor neural network model.

[0126] Further, if Figure 8 As shown, the virtual sample generating unit 12 includes:

[0127] The latent variable generating subunit 121 is electrically connected to the preprocessing unit 11, and is used to input the historical data set D as a model input and input it into the encoder for encoding, wherein the encoding layer includes several convolutional layers, and after the expansion operation, the latent variable z is calculated and obtained;

[0128] The time series feature extraction subunit 122 is electrically connected to the latent variable generation subunit 121, and is used to combine the latent variable z with the conditional variable c, and then use them as input variables to enter the decoder. In the decoding process, the input variables are respectively passed through a trend module, several seasonal modules and a residual module, and the time series features under each model are extracted;

[0129] The virtual sample acquisition subunit 123 is electrically connected to the time series feature extraction subunit 122 to combine and process all the time series features to obtain a new virtual sample. ;

[0130] The virtual process sample acquisition subunit 124 is electrically connected to the virtual sample acquisition subunit 123 to acquire the virtual sample. Perform screening and expansion to obtain virtual process samples .

[0131] Specifically, when expanding the historical data set D, the latent variable generation subunit 121 is first used to encode the historical data set D to obtain the latent variable z. Then, the time series feature extraction subunit 122 combines the latent variable z with the conditional variable c, and enters the decoder as an input variable. During the decoding process, the input variables are respectively passed through a trend module, several seasonal modules, and a residual module to extract the time series features under each model. Finally, the virtual sample acquisition subunit 123 combines all the time series features to obtain a new virtual sample. , and then the virtual process sample acquisition subunit 124 performs virtual sample Perform screening and expansion to obtain virtual process samples The above device adds a time series decoder part, that is, through the combination of trend module, seasonal module and residual module, a new decoder structure is formed, thereby improving the effect of feature extraction of time series samples and virtual sample generation. In addition, conditional variables are added in the process of generating virtual samples, making the generated virtual samples more sensitive to important output variables such as quality variables, improving the virtual sample value.

[0132] Furthermore, the virtual sample generation unit 12 also includes a loss function calculation subunit 125, which is electrically connected to the time series feature extraction subunit 122. The loss function calculation subunit 125 adds a conditional variable c, that is, a conditional variable c is added to the loss function to adapt to the training process in this system. Compared with the traditional VAE loss function model that only focuses on the distribution of the general variable x, the virtual sample generation unit 12 introduces the conditional variable c in the loss function, and needs to consider complex situations such as the joint probability distribution of x and c, which makes the derivation and calculation of the loss function more difficult. The loss function that was finally improved and designed can train the key parameters of the model well and achieve good results.

[0133] In the above-mentioned small sample industrial time series data enhancement and quality prediction system, it is composed of three parts: sample generation module 1, hybrid training module 2 and prediction module 3. Compared with the traditional VAE virtual sample generation system, the above-mentioned system adds a time series feature extraction subunit 122, that is, through the combination of trend module, seasonal module and residual module, a new decoder method is formed, thereby improving the feature extraction of time series samples and the effect of virtual sample generation. In addition, conditional variables are added in the process of generating virtual samples, so that the generated virtual samples are more sensitive to important output variables such as quality variables, thereby improving the value of virtual samples. Since the time series feature extraction is more accurate and conditional variables are added for control, the sample diversity and reliability are improved. Finally, the system introduces the conditional variable c in the loss function calculation subunit 125, which needs to consider complex situations such as the joint probability distribution of x and c, and the derivation and calculation of the loss function are more difficult. The loss function finally improved and designed can train the key parameters of the model well and achieve good results.

[0134] For those skilled in the art, various changes and modifications will undoubtedly be obvious after reading the above description. Therefore, the attached claims should be regarded as covering all changes and modifications of the true intent and scope of the present invention. Any and all equivalent ranges and contents within the scope of the claims should be considered to still be within the intent and scope of the present invention.

Claims

1. A small sample industrial time series data enhancement and quality prediction method, characterized in that: The method comprises the steps of: S1. Obtain a historical data set D = {(X, Y)} as a training data set, and input the historical data set D into the variational autoencoder model added to the time series decoding process to generate a virtual sample X vir , for the virtual sample X vir After the sample is processed, a virtual process sample X is obtained new ; S2, the virtual process sample X new Input the pre-trained soft-sensing neural network model to obtain the virtual quality sample Y new , which will be composed of historical data set D, virtual process sample X new and the virtual mass sample Y new The extended training dataset D new , by expanding the training data set D new Update and train the soft-sensing neural network model; S3, input variable x online t Input the updated soft sensor neural network model to predict the online quality variable y t ; Wherein, step S1 comprises the steps of: S11, preprocessing the industrial time series data to obtain a historical data set D = {(X, Y)} as a training data set; S12, the historical data set D is input into the variational autoencoder model added to the time series decoding process to generate a virtual sample X vir , for the virtual sample X vir Perform screening and expansion to obtain virtual process sample X new ; Wherein, X represents a general process variable of N×T×M, Y represents a key quality variable of N×T×m, where N represents the variable dimension, T is the time step, and M and m represent the dimensions of the process variable and the quality variable respectively; and step S12 comprises: S121, taking the historical data set D as the model input and inputting it into the encoder for encoding, wherein the encoding layer includes several convolutional layers, and after the expansion operation, the latent variable z is calculated and obtained; S122, after the latent variable z is combined with the conditional variable c, they enter the decoder as input variables. During the decoding process, the input variables pass through multiple time series modules to extract the time series features under each model; S123, combine all the time series features to obtain a new virtual sample X vir ; S124, for virtual sample X vir Perform screening and expansion to obtain virtual process sample X new ; Wherein, in step S122, the multiple time series modules include a trend module, several seasonal modules and a residual module.

2. The small sample industrial time series data enhancement and quality prediction method according to claim 1 is characterized in that: In step S12, the loss function is calculated as: In the above formula, is the likelihood function, p θ (z|x, c) is the posterior probability distribution of z under the conditions of x and c, where x is the reconstructed time series data and c is the conditional variable. Obeying a Gaussian distribution, yes and p θ The KL divergence between (z|x, c), for Abbreviation, p θ For p θ Abbreviation for (z|x, c).

3. The small sample industrial time series data enhancement and quality prediction method according to claim 1 is characterized in that: In step S121, the corresponding latent variable z is obtained by z = μ + σ·ε. After the latent variable z passes through the decoder, the virtual sample X can be obtained by reconstruction. vir ; where ε is the value randomly sampled from the normal distribution of N(0, 1), μ represents the mean of the latent variable, and σ is the standard deviation of the latent variable.

4. The small sample industrial time series data enhancement and quality prediction method according to claim 1 is characterized in that: Step S2 includes the following steps: S21. Use the pre-trained soft-sensing neural network model to measure the virtual process sample X new Processing is performed to obtain a virtual mass sample Y new ; S22. Obtaining the extended training data set D new ={(X, Y), (X new , Y new )}, expand the training data set D new Re-input the training model to obtain the updated soft sensor neural network model.

5. The small sample industrial time series data enhancement and quality prediction method according to claim 1 is characterized in that: In step S3, the virtual quality sample Y new The acquisition process is: Y new =f(ωX new +b) Here, function f refers to the pre-trained soft-sensing neural network model, ω represents the weight parameter, and b represents the bias.

6. A small sample industrial time series data enhancement and quality prediction system, used to execute the method according to any one of claims 1 to 5, characterized in that: The system comprises: The sample generation module (1) is used to obtain a historical data set D = {(X, Y)} as a training data set, and input the historical data set D into the variational autoencoder model added to the time series decoding process to generate a virtual sample X vir , for the virtual sample X vir After the sample is processed, a virtual process sample X is obtained new ; A mixed training module (2) is electrically connected to the sample generation module (1) and is used to convert X new Input the pre-trained soft-sensing neural network model to obtain the virtual quality sample Y new ; The historical data set D, the virtual process sample X new and the virtual mass sample Y new The extended training dataset D new , by expanding the training data set D new Update and train the soft-sensing neural network model; The prediction module (3) is electrically connected to the hybrid training module (2) and inputs the online variable x t Input the updated soft sensor neural network model to predict the online quality variable y t ; Wherein, the sample generation module (1) comprises: A preprocessing unit (11) is used to preprocess the industrial time series data to obtain a historical data set D = {(X, Y)} as a training data set; A virtual sample generating unit (12) is electrically connected to the preprocessing unit (11) and is used to generate virtual samples X by inputting a historical data set D into a variational autoencoder model with a time series decoding process. vir , and for the virtual sample X vir Perform screening expansion to obtain virtual process sample X new ; Wherein, the virtual sample generating unit (12) comprises: A latent variable generating subunit (121) is electrically connected to the preprocessing unit (11) and is used to input the historical data set D as a model input into an encoder for encoding, wherein the encoding layer includes a plurality of convolutional layers, and after an expansion operation, a latent variable z is calculated and obtained; A time series feature extraction subunit (122) is electrically connected to the latent variable generation subunit (121) and is used to combine the latent variable z with the conditional variable c and use them as input variables to enter the decoder. During the decoding process, the input variables are respectively passed through a trend module, several seasonal modules and a residual module to extract the time series features under each model. The virtual sample acquisition subunit (123) is electrically connected to the time series feature extraction subunit (122) and is used to combine and process all the time series features to obtain a new virtual sample X. vir ; The virtual process sample acquisition subunit (124) is electrically connected to the virtual sample acquisition subunit (123) and is used to obtain the virtual sample X vir Perform screening and expansion to obtain virtual process sample X new .

7. The small sample industrial time series data enhancement and quality prediction system according to claim 6 is characterized in that: The hybrid training module (2) comprises: A virtual quality sample generation unit (21) is electrically connected to the virtual sample generation unit (12) and is used to use a pre-trained soft measurement neural network model to generate a virtual process sample X. new Processing is performed to obtain a virtual mass sample Y new ; An extended training data set generation unit (22) is electrically connected to the virtual quality sample generation unit (21) to obtain an extended training data set D new ={(X, Y), (X new , Y new )}, and expand the training data set D new Re-input the training model to obtain the updated soft sensor neural network model.

8. The small sample industrial time series data enhancement and quality prediction system according to claim 6, characterized in that: The virtual sample generation unit (12) also includes a loss function calculation subunit (125) electrically connected to the time series feature extraction subunit (122).

Images