Process data missing soft-sensor prediction method

Abstract

The application relates to a process data missing soft measurement prediction method, wherein the process data missing soft measurement prediction method comprises the following steps: obtaining target data to be detected in an industrial production process; reading a trained process data missing soft measurement model; constructing a filling reconstruction representation based on the target data to be detected; the filling reconstruction representation is a sequence representation after the target data to be detected is mapped; taking the filling reconstruction representation as input of a filling task embedding, inputting the filling reconstruction representation into a pre-training model, outputting a filling reconstruction result, constructing a soft measurement prediction representation based on the filling reconstruction result, taking the soft measurement prediction representation as input of a soft measurement prediction embedding, inputting the soft measurement prediction representation into the pre-training model, and outputting a process data missing soft measurement prediction result of the target data to be detected; and the application improves the accuracy of process data missing soft measurement model prediction.

Description

Technical Field

[0001] This application relates to the field of soft measurement modeling, and in particular to a method for predicting soft measurement data missing from processes. Background Technology

[0002] Soft sensing technology establishes a mapping relationship between easily accessible process variables and target variables, enabling indirect estimation of indicators that are difficult to detect directly online. This has significant application value in process industry production monitoring, quality control, and safety assurance. As modern industrial systems become increasingly complex, production processes exhibit characteristics such as multivariate complexity, high dimensionality, and dynamic evolution. Compared to methods that rely on physical or chemical reaction mechanisms to construct analytical models, data-driven modeling can directly utilize historical data to uncover the intrinsic relationships between variables, thus gaining widespread attention and application in the field of soft sensing.

[0003] However, in actual industrial settings, production process data often suffers from varying degrees of incompleteness due to factors such as sensor malfunctions, data acquisition failures, communication link interruptions, and complex operating environments. This poses significant challenges to subsequent soft measurement modeling. For multivariate process data, directly deleting samples containing missing values ​​can significantly reduce the amount of usable data. Furthermore, identifying and removing outliers may further exacerbate the data incompleteness problem. At the same time, the forms of data incompleteness in industrial processes are often multifaceted; in addition to random missing values, there may be instances of missing groups of variables, making data modeling for complex scenarios even more difficult.

[0004] There is currently no effective solution to the problem that soft measurement models built on the basis of missing process data have low prediction accuracy in related technologies. Summary of the Invention

[0005] This embodiment provides a soft measurement prediction method for missing process data to solve the problem of low prediction accuracy of soft measurement models based on missing process data in related technologies.

[0006] Firstly, this embodiment provides a soft measurement prediction method for missing process data, applicable to industrial production scenarios. The method includes:

[0007] Acquire target data to be detected during industrial production; the target data to be detected is production result data that is missing during the industrial production process.

[0008] Read the trained process data missing soft measurement model, wherein the soft measurement model includes task filling embedding, soft measurement prediction embedding, pre-trained model and output layer, wherein the pre-trained model is a model that has been trained in advance based on sample data in various industrial production scenarios;

[0009] A filling and reconstruction representation is constructed based on the target data to be detected, wherein the filling and reconstruction representation is a sequence representation of the target data to be detected after mapping;

[0010] The filling and reconstruction representation is used as the input to the filling task embedding, and is fed into the pre-trained model to output the filling and reconstruction result;

[0011] A soft measurement prediction representation is constructed based on the filling and reconstruction results; the soft measurement prediction representation is a sequence representation of the filling and reconstruction results after mapping.

[0012] The soft measurement prediction representation is used as the input to the soft measurement prediction embedding and is fed into the pre-trained model to output the process data missing soft measurement prediction result of the target data to be detected.

[0013] In some embodiments, the training process for the process data-missing soft measurement model includes:

[0014] The original training sample data is collected according to the preset first fixed window, and the original training sample data is masked to obtain the masked original training sample data.

[0015] The original training sample data after the mask is normalized to obtain the target training sample data;

[0016] A fill-in reconstruction representation is constructed based on the target training sample data, wherein the fill-in reconstruction representation is a sequence representation of the target training sample data after mapping.

[0017] The filling and reconstruction representation is used as the input to the filling task embedding, and is fed into the pre-trained model to output the filling and reconstruction result;

[0018] A soft measurement prediction representation is constructed based on the filling and reconstruction results; the soft measurement prediction representation is a sequence representation of the filling and reconstruction results after mapping.

[0019] The soft measurement prediction representation is used as the input to the soft measurement prediction embedding and input into the pre-trained model to output the process data missing soft measurement prediction result of the target training sample data.

[0020] In some embodiments, constructing the imputation reconstruction representation based on the target training sample data includes:

[0021] The target training sample data is processed by spatial vector mapping to obtain the filling reconstruction representation, wherein the filling reconstruction representation includes word embedding representation and position embedding representation.

[0022] In some embodiments, the step of using the infilling reconstruction as input to the infilling task embedding, inputting it into a preset training model, and outputting the infilling reconstruction result includes:

[0023] The word embedding representation and position embedding representation are converted into contextual features for filling and reconstruction using the preset training model. Based on the contextual features of the filling and reconstruction, the filling and reconstruction result is obtained.

[0024] In some of these embodiments, the pre-trained model is updated and optimized using low-rank adaptation.

[0025] In some embodiments, constructing a soft measurement prediction representation based on the infill reconstruction results includes:

[0026] The filling and reconstruction results are patched to obtain the patched filling and reconstruction results;

[0027] The patched reconstruction results are mapped to obtain the soft measurement prediction representation.

[0028] In some embodiments, after mapping the patched reconstruction results to obtain the soft measurement prediction representation, the method further includes:

[0029] The soft measurement prediction representation is segmented according to a preset second fixed window to obtain a segmented soft measurement prediction representation; wherein the preset second fixed window is smaller than the preset first fixed window;

[0030] The segmented soft measurement prediction representation is used as the input for the soft measurement prediction embedding.

[0031] In some embodiments, the process of using the soft measurement prediction representation as input to the soft measurement prediction embedding, inputting it into the pre-trained model, and outputting the target training sample data as a data missing soft measurement prediction result includes:

[0032] The pre-trained model is used to convert the segmented soft measurement prediction representation into contextual features of the soft measurement prediction.

[0033] The context features are upsampled to obtain the upsampled context features for soft measurement prediction;

[0034] Based on the upsampling context features of the soft measurement prediction, the process data missing soft measurement prediction result of the target training sample data is predicted.

[0035] In some of these embodiments, the process data missing soft measurement model is optimized based on the imputation reconstruction loss function and the soft measurement prediction loss function.

[0036] In some of these embodiments, the root mean square error and mean absolute error are used to evaluate the process data missing soft measurement prediction results of the target training sample data.

[0037] Compared with related technologies, the process data missing soft measurement prediction method provided in this embodiment improves the accuracy of process data missing soft measurement model prediction by acquiring target data to be detected in the industrial production process; the target data to be detected is the production result data with missing information in the industrial production process; reading a trained process data missing soft measurement model, wherein the soft measurement model includes a completion task embedding, a soft measurement prediction embedding, a pre-trained model, and an output layer, the pre-trained model being a model pre-trained based on sample data in various industrial production scenarios; constructing a completion reconstruction representation based on the target data to be detected, the completion reconstruction representation being a sequence representation of the target data to be detected after mapping; using the completion reconstruction representation as input to the completion task embedding, inputting it into the pre-trained model, and outputting the completion reconstruction result; constructing a soft measurement prediction representation based on the completion reconstruction result; the soft measurement prediction representation being a sequence representation of the completion reconstruction result after mapping; using the soft measurement prediction representation as input to the soft measurement prediction embedding, inputting it into the pre-trained model, and outputting the process data missing soft measurement prediction result of the target data to be detected, thus improving the accuracy of process data missing soft measurement model prediction.

[0038] Details of one or more embodiments of this application are set forth in the following drawings and description to make other features, objects and advantages of this application more readily apparent. Attached Figure Description

[0039] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0040] Figure 1 This is a flowchart illustrating the training process of the process data missing soft measurement model in the process data missing soft measurement prediction method of this embodiment.

[0041] Figure 2 This is a structural diagram of the process data missing soft measurement model in this embodiment.

[0042] Figure 3 This is the LoRA efficient fine-tuning strategy invoked by the pre-trained model in this embodiment.

[0043] Figure 4 These are the soft measurement prediction result curves for data loss in each process under the random missing scenario in this embodiment.

[0044] Figure 5These are the soft measurement prediction result curves for missing process data in the structural component missing scenario of this embodiment.

[0045] Figure 6 This is a flowchart of the process data missing soft measurement prediction method in this embodiment. Detailed Implementation

[0046] To better understand the purpose, technical solution, and advantages of this application, the application is described and illustrated below in conjunction with the accompanying drawings and embodiments.

[0047] Unless otherwise defined, the technical or scientific terms used in this application shall have the general meaning understood by one of ordinary skill in the art to which this application pertains. Words such as “a,” “an,” “an,” “the,” “the,” and “these” used in this application do not indicate quantitative limitation and may be singular or plural. The terms “comprising,” “including,” “having,” and any variations thereof used in this application are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that comprises a series of steps or modules (units) is not limited to the listed steps or modules (units) but may include steps or modules (units) not listed, or may include other steps or modules (units) inherent to these processes, methods, products, or devices. Words such as “connected,” “linked,” and “coupled” used in this application are not limited to physical or mechanical connections but may include electrical connections, whether direct or indirect. “Multiple” used in this application refers to two or more. “And / or” describes the relationship between related objects, indicating that three relationships may exist; for example, “A and / or B” can represent: A alone, A and B simultaneously, and B alone. Normally, the character " / " indicates that the objects before and after it are in an "or" relationship. The terms "first," "second," "third," etc., used in this application are merely to distinguish similar objects and do not represent a specific order of objects.

[0048] Currently, soft sensor models built on the basis of missing process data in industrial processes typically suffer from the following two problems:

[0049] (1) Filling modeling is usually done independently, ignoring the supervision information contained in the soft measurement dynamic label, which limits the targeted filling for downstream tasks.

[0050] (2) The process is complex, nonlinear, highly dynamic and has diverse missing modes. Ignoring these characteristics will make it difficult to achieve unified modeling of different missing scenarios and will affect the adaptability and generalization ability of the model.

[0051] Therefore, this embodiment provides a method for training a soft measurement model with missing process data to overcome the above problems, and performs soft measurement prediction in industrial processes based on the trained soft measurement model with missing process data. Figure 1 This is a flowchart illustrating the training process of the process data missing soft measurement model in the process data missing soft measurement prediction method of this embodiment. Figure 1 As shown, the training process for the data-missing soft measurement model includes:

[0052] Step S101: Collect the original training sample data according to the preset first fixed window, and perform masking processing on the original training sample data to obtain the masked original training sample data; perform normalization processing on the masked original training sample data to obtain the target training sample data.

[0053] Specifically, the process data-deficient soft measurement model in this embodiment is applicable to any field with soft measurement needs, such as predicting polymerization melt index, reaction conversion rate, and reactor material concentration in chemical reactions; predicting free calcium oxide in clinker, calcination heat consumption, and kiln crusting state in cement and lime kilns; and predicting gasoline octane number, diesel pour point, product sulfur content, and distillation column component purity in oil refining units. The original training sample data consists of production result data from industrial production processes in as many industries as possible. Figure 2 This is a structural diagram of the process data missing soft measurement model in this embodiment, as shown below. Figure 2 As shown, the process data missing soft measurement model in this embodiment includes an imputation module component, a prediction module component, and a pre-trained model component. The layers include an input embedding layer, a pre-trained model, an output layer, and a loss optimization layer. The input embedding layer includes imputation task embedding and soft measurement prediction embedding; the output layer includes imputation task output and soft measurement prediction output; and the loss optimization layer includes reconstruction error and prediction error.

[0054] In this embodiment, the example is free calcium oxide (f-CaO) that has not fully reacted during the calcination of cement clinker.

[0055] Free calcium oxide (f-CaO) is the residue of incompletely reacted cement clinker during calcination, and its content directly reflects clinker quality and process stability. To address the type of missing process data, two sets of different process data were selected as the original training samples: one set showed random missing data, and the other set showed structured grouping missing data. The two sets of data came from different sources, as detailed below:

[0056] Randomly Missing Data Sample Set: Up to 50 types of variable data are available in the cement clinker production chain. In this embodiment, the following 14 process variables are used to construct the soft measurement model: primary outlet gas analysis (carbon dioxide), primary outlet gas analysis (nitrogen dioxide), high-temperature fan frequency conversion, high-temperature fan current, decomposer outlet temperature, flue gas temperature, kiln current, kiln speed, tertiary air temperature, kiln head hood negative pressure, secondary air temperature, kiln head dust collector inlet temperature, kiln tail coal feed rate, and kiln head coal feed rate. In this data set, the sampling period of the process variables in the cement clinker production case is different from that of the target variable f-CaO. The sampling period of the process variables is 3 minutes, while the target variable f-CaO is collected and recorded every 1 hour through offline chemical analysis. To reconcile this difference, in this embodiment, the process variables are divided into non-overlapping time windows with a span of 1 hour. The dataset used spans nearly three months, and the number of samples for the target variable f-CaO is 2088. It is divided into training, validation, and test sets in an approximately 3:1:1 ratio. The missing rate in the process data is 37%, and the missing type is random.

[0057] The data sample set with structurally missing groups: The sampling frequency of the process variable and the target variable f-CaO is consistent, both at 1 hour. A point-by-point prediction strategy is adopted in the soft measurement phase. The target variable f-CaO has 3440 samples, which are divided into training, validation, and test sets in an approximately 3:1:1 ratio. Meanwhile, due to changes in the measurement point recording range, the process variables available here are adjusted compared to the randomly missing dataset, with a process variable missing rate of approximately 17%. Based on the data acquisition and recording range of the clinker calcination process, a total of 14 grouped missing process variables are identified, which can be summarized into three groups in terms of information attributes:

[0058] Group 1: Four variables related to the chemical composition of raw materials entering the kiln, mainly including the contents of silicon dioxide (SiO2), aluminum oxide (Al2O3), ferric oxide (Fe2O3), and calcium oxide (CaO) in the raw materials entering the kiln;

[0059] The second group consists of four variables related to the chemical composition of clinker, mainly including the contents of silicon dioxide (SiO2), aluminum oxide (Al2O3), ferric oxide (Fe2O3), and calcium oxide (CaO) in the clinker.

[0060] The third group consists of six process operation variables that reflect the operating load and thermal state of the calcination system, including feed rate, decomposition furnace outlet temperature, secondary air temperature, pressure of the first and second chambers of the cooler, coal feed rate at the kiln tail, and coal feed rate at the kiln head.

[0061] Since the above three types of variables usually correspond to different collection and recording stages, missing segments can be observed in actual data, that is, the same type of variable is simultaneously missing in certain time intervals.

[0062] After obtaining the original training samples, which are long sequences arranged chronologically and composed of multi-dimensional features, the data is first segmented using a preset first sliding window: a uniform first fixed window size is set, and segments are truncated along the time axis with a specified step size. Each segment is a window sample X. A single segmented observation sample is denoted as X. obs :

[0063] ;

[0064] in, Represents the time step for a single window; This refers to the total number of feature variables statistically represented in the original training sample data.

[0065] Each X obs There exists a binary mask matrix The mask value selection rule is as follows: At time t, it means that the value of the f-th feature at time t is collected normally and belongs to the valid observation data; If a value is missing, it indicates that the data at that location is missing and the original reading is unreliable, thus representing the validity of the observation at that location. The observation matrix and the mask are multiplied element-wise. ( (representing the element-wise product of Hadamard) to obtain the cleaned context truth matrix. Valid observations retain their original measured values, while missing values ​​are uniformly set to 0. This standardized format can be directly used in subsequent missing value imputation modules as a prerequisite context input for model inference and numerical completion.

[0066] To programmatically mark the coordinates of all valid observations, a set of observation indices is defined in advance. This set of observation indexes This is used to represent the correspondence between all observed time periods and variables, and is used during model training to ensure that the reconstruction results remain consistent with the known observation data.

[0067] ;

[0068] t is the window index, f is the feature dimension index, and each set of binary coordinates within the set is... The unique matching window contains one successfully observed (time step, feature) point. Represents the time step of a single window. This refers to the total number of feature variables statistically analyzed in the original training sample data. This is a binary mask matrix. For random missing values, the missing values ​​are distributed dispersedly within the time window; for grouped missing values, the variable is divided into several predefined groups, and each group is missing in the same manner as the whole.

[0069] Then, the original training sample data after masking is normalized: for each X obs In Subtract the mean of each variable Then divide by the standard deviation of the training samples. This allows for the standardization of samples, resulting in the target training sample data X.

[0070] Step S102: Construct a fill-in reconstruction representation based on the target training sample data. The fill-in reconstruction representation is the sequence representation of the target training sample data after mapping.

[0071] The process of constructing a fill-in reconstruction representation based on the target training sample data includes:

[0072] Spatial vector mapping is performed on the target training sample data to obtain the filling reconstruction representation, which includes word embedding representation and position embedding representation.

[0073] Specifically, firstly, the feature dimension F in the target training sample data is mapped to a D-dimensional spatial vector through two sets of one-dimensional convolutions, obtaining the word embedding and position embedding representations respectively, thus mapping the target training sample data into a filling and reconstruction representation Z. (0) This is expressed as:

[0074] ;

[0075] In the formula, Indicates the input mapping layer. This represents position embedding, specifically, generating a fixed set of sine and cosine position encoding vectors for each time step in the sequence to generate the position information of the sequence.

[0076] Step S103: The filling reconstruction representation is used as the input to the filling task embedding and fed into the pre-trained model to output the filling reconstruction result.

[0077] The process involves using the filling reconstruction representation as the input to the filling task embedding, inputting it into a pre-trained model, and outputting the filling reconstruction result. This includes: using the pre-trained model to convert word embedding representations and position embedding representations into contextual features for filling reconstruction, and obtaining the filling reconstruction result based on the contextual features of the filling reconstruction.

[0078] Specifically, the pre-trained model is a model pre-trained based on sample data from industrial production scenarios in various fields. Through pre-training with massive samples from multiple industries, the model extracts general industrial rules that are not limited by specific scenarios, thereby supporting its adaptation to new industries and expanding its application boundaries. In this embodiment, the second-generation generative pre-trained Transformer model (GPT-2) is used as an example, and the soft sensing model based on GPT-2 in this embodiment is named the GPTIS model (GPT-2-based Process-variable Imputation and Soft-sensing framework). The pre-trained network can also be other network models; this embodiment does not specifically limit this, and the choice can be made according to the actual situation.

[0079] The incomplete reconstruction representation is used as the input to the incomplete task embedding and fed into GPT-2. The general decoder of GPT-2 converts the incomplete reconstruction representation into contextual features H.

[0080] ;

[0081] The pre-trained GPT-2 backbone network reads the imputed reconstructed representation and generates hidden sequence features. The superscript LoRA indicates that this backbone network is optimized through low-rank updates in the subsequent Transformer module's Low-Rank Adaptation (LoRA).

[0082] The efficient fine-tuning strategy of LoRA includes keeping the two early attention layers fixed and inserting the LoRA module into subsequent attention layers. This design keeps most pre-trained parameters unchanged while still allowing room for adaptive adjustments for specific scenarios. For the projection matrix within the attention layer, LoRA uses low-rank correction instead of a direct full-matrix update.

[0083] ;

[0084] Where W is the original projection weight matrix of the pre-trained attention layer; To adapt to specific downstream scenarios, the effective projected weight matrix of the attention layer is obtained by superimposing the low-rank correction on the original weights, without needing to directly modify W itself. Let be the matrix updated using LoRA; A and B are two low-rank trainable matrices introduced by LoRA.

[0085] Figure 3 This embodiment utilizes the LoRA efficient fine-tuning strategy called by the pre-trained model when process data is missing, such as... Figure 3 As shown:

[0086] The first two attention layers are determined to be non-target layers (frozen throughout, without LoRA attached), and the remaining attention projection layers are marked as fine-tuning target layers that need to be connected to the LoRA structure, thus completing the delineation of the scope of the layers to be modified.

[0087] Iterate through the attention layers selected in the previous step in the order of network hierarchy, configure the LoRA bypass structure for each layer, and enter the dedicated processing branch for each layer.

[0088] The LoRA layer contains the original weight matrix that has been pre-trained and solidified. and low-rank matrix of bypass branches For the original weight matrix and low-rank matrix of bypass branches Forward propagation is performed, and the equivalent weight of a single layer is obtained by superimposing the original pre-trained weights with the low-rank product increment. The original pre-trained weights carry general industrial prior knowledge accumulated across industries, while the low-rank increment learns the specific process and missing distribution characteristics of this scenario.

[0089] In single-layer, single-number training constraints, freeze the original weight matrix. Only optimize the low-rank matrix of the bypass branch. The gradient of the main intervention training weights is masked and not updated throughout the process, preserving the general temporal patterns and cross-industry prior knowledge learned in the pre-training stage, thus avoiding damage to the original mature representation capabilities of the underlying model. Only a small number of parameters participate in gradient backpropagation updates, significantly compressing the overall number of trainable parameters. Under the condition of limited sample resources in industrial scenarios, the extremely low number of parameters can strongly suppress the risk of overfitting and improve the generalization performance of the fine-tuned model.

[0090] Next, it is determined whether the target layer has been processed. If there are still attention layers that have not been connected to LoRA and the single-layer parameter configuration is completed, the process returns to the target layer traversal stage to continue processing the next target attention layer. If all defined target layers have completed LoRA mounting and parameter freezing configuration, the process moves down to the overall fine-tuning stage.

[0091] During the overall fine-tuning phase, low-rank matrices within all target layers are used. As the only updatable parameter, it imports industrial scene window samples and relies on the observation index set defined above. Construct a loss function to conduct iterative training: low-rank increments continuously fit the unique features of local working conditions and missing patterns, while keeping the original backbone weights unchanged, taking into account both general prior retention and adaptation to downstream industrial tasks.

[0092] The fine-tuning process ends after iterative convergence reaches the accuracy target; during the inference phase, bypass increments can be implemented. and original weights By performing weight fusion, the inference latency is completely on par with the original pre-trained model, without introducing additional computational overhead.

[0093] This operation preserves the knowledge gained from pre-training while fine-tuning for the current industrial task. Because the correction is a low-rank quantity, the number of trainable parameters remains relatively small, thus the backbone network is less prone to overfitting on limited data.

[0094] In the filling and reconstruction phase, such as Figure 2 As shown, after the word embedding representation and position embedding representation are fed into the decoder, temporal features are first extracted through layers N and M in the pre-trained model. Then, the extracted temporal features are converted into contextual features for incomplete reconstruction. Based on the contextual features of the incomplete reconstruction, the incomplete reconstruction result is obtained. Among them, layer N is the natively frozen pre-trained Transformer layer, and layer M is the adapted Transformer layer after being connected to LoRA and fine-tuned.

[0095] First, the attention weights trained in the GPT-2 pre-training stage are reused through N layers of multi-head attention, and the weights are locked and not updated throughout the process. Through a unidirectional self-attention mechanism, the temporal dependencies between different time steps and different features within an industrial time window are modeled to uncover general industrial prior rules such as periodic fluctuations and feature linkages.

[0096] Then, the features of the attention output are normalized and scaled through the first layer of normalization to adapt to the input distribution of the downstream layer; the normalization parameters of this layer are updated with open gradients to adapt to the feature distribution shift of the downstream subdivided industrial scenarios.

[0097] Next, through a feedforward network, the pre-trained weights are completely frozen, and a non-linear feature transformation is performed on the representation of attention output to solidify the universal feature mapping paradigm learned across industries.

[0098] Finally, the output of the feedforward network is normalized and corrected again through a second layer of normalization. The parameters can be fine-tuned to adapt to the changes in the representation distribution caused by local operating conditions.

[0099] Layer M is a later, deeper attention layer where a low-rank LoRA bypass branch is inserted. In layer M, multi-head attention is decomposed into query... ,key ,value All three projection matrices were modified to bypass LoRA:

[0100] Original trunk projection weights , , Perform freeze processing; add a low-rank LoRA matrix as an incremental correction path in the bypass, and the equivalent projection result satisfies... Only bypass low-rank matrix Open training. The two results are summed element-wise and fused to calculate the query after scenario adaptation. ,key ,value The time-series autocorrelation score is calculated by inputting multi-head attention.

[0101] This method avoids directly updating the massive original projection matrix, and achieves scene adaptation with a very small number of low-rank increments, thereby reducing the number of trainable parameters and suppressing overfitting of small industrial datasets.

[0102] Next, in the first normalization layer, the deep representation of the attention output is normalized, and the parameters participate in gradient updates to correct the deep feature distribution shift caused by the industrial subdivisions. The weights of the deep feedforward backbone of the feedforward network use the pre-trained parameters and are locked, preserving the general high-dimensional feature transformation logic. The second normalization layer normalizes the output representation of the feedforward network, which can learn to scale the offset parameters to adapt to the representation input requirements of downstream tasks such as imputation and prediction, and extract temporal features.

[0103] Layer M uses LoRA for lightweight incremental fine-tuning: the main weights are kept frozen, and the industry-standard priors accumulated from pre-training are fully retained; only small-scale LoRA bypass parameters and two layers of normalized parameters are optimized, with very low computing power and data overhead, so that the deep representation can adapt to the current plant's temporal missing distribution and process fluctuation characteristics, and output the adapted high-level hidden state for filling the missing parts of the right branch and the working condition prediction task.

[0104] By prioritizing the feeding of embedded features from time-series samples into N-layer shallow stacked units, the underlying general temporal patterns are extracted by the frozen backbone module, and a small number of trainable normalized parameters are used for basic distribution adaptation. The shallow hidden representations output from the N-layer flow downwards into M-layer LoRA-modified deep stacked units, where LoRA incremental correction is used to fine-tune the high-level representations according to the specific scenario, outputting refined hidden states adapted to downstream tasks. By locking in general features at the shallow layer and using low-rank bypasses for customized adaptation at the deep layer, both the retention of pre-trained general knowledge and the transfer effect to downstream industrial tasks are considered. The overall number of trainable parameters is strictly constrained, significantly reducing the risk of overfitting under the condition of limited labeled samples in industrial scenarios.

[0105] The temporal features are encoded into contextual features H using the universal decoder of GPT-2. The incomplete module reads the contextual features H and outputs the missing process values ​​in the same variable space as the original sequence, thus obtaining the incomplete reconstruction result X'.

[0106] Among them, g imp To fill in the module.

[0107] In practical applications, incomplete process data often results in the loss of some state information required for soft sensing. If the recovered sequence does not match the actual process evolution, downstream quality assessment results will also become unreliable. By using a fill-in module, a signal consistent with the process can be recovered for use in predictive branches, thereby improving the reliability of downstream quality assessment results.

[0108] Step S104: Construct a soft measurement prediction representation based on the filling and reconstruction results; the soft measurement prediction representation is a sequence representation of the filling and reconstruction results after mapping.

[0109] The soft measurement prediction representation based on the filling and reconstruction results includes: patching the filling and reconstruction results to obtain patched filling and reconstruction results; and mapping the patched filling and reconstruction results to obtain the soft measurement prediction representation.

[0110] Specifically, to preserve as much original information as possible in soft measurement tasks, a patching operation is performed on the imputation and reconstruction result X' to retain the original valid information of the target training samples, and the corresponding imputation and reconstruction result is only used where the data is missing:

[0111] ;

[0112] Will As input to the soft measurement subtask, namely the completed process data after filling in the missing parts, for By mapping, we obtain the soft measurement prediction representation. :

[0113] ;in, Indicates the input mapping layer. This indicates positional embedding.

[0114] The soft measurement branch uses the same contextual feature method as the above-described filling and reconstruction process, but reads the process data in a more compressed time-series format.

[0115] For example, the soft measurement prediction representation is segmented according to a preset second fixed window to obtain a segmented soft measurement prediction representation; wherein the preset second fixed window is smaller than the preset first fixed window; the segmented soft measurement prediction representation is used as the input for soft measurement prediction embedding.

[0116] First, the process sequence is divided into several overlapping segments according to a preset second fixed window, with each segment having a length of [length missing]. and step size To obtain each fragment U:

[0117] Patch is used to divide modules.

[0118] Each segment summarizes a local short period in the process trajectory, which helps the model focus on the nearby temporal evolution process when making the final quality prediction.

[0119] Before inputting the embedded sequence into the backbone network, dividing it into multiple short time windows can shorten the sequence length, reduce the impact of isolated noise samples, and help the model focus on short-term process trends that are conducive to prediction.

[0120] Step S105: The soft measurement prediction representation is used as the input to the soft measurement prediction embedding and fed into the pre-trained model. The output is the process data missing soft measurement prediction result of the target training sample data.

[0121] The process of using soft measurement prediction representation as input to soft measurement prediction embedding and inputting it into a pre-trained model to output process data missing soft measurement prediction results includes: converting the segmented soft measurement prediction representation into soft measurement prediction context features through the pre-trained model; performing upsampling processing on the context features to obtain upsampled context features of soft measurement prediction; and predicting process data missing soft measurement prediction results of the target training sample data based on the upsampled context features of soft measurement prediction.

[0122] Specifically, the patch window information is encoded into the same pre-trained backbone weights and restored to the temporal resolution required for the prediction target:

[0123] ;

[0124] Part One Generate block-level contextual features from the stitched view of the prediction branches; Part Two These features are resampled to the original temporal resolution. This method preserves the backbone network architecture shared by the infill reconstruction task and the soft measurement prediction task, while enabling the soft measurement branch to read block-based sequence views. The final quality predictor still needs to output a numerical vector for the entire process window.

[0125] Upsampled contextual features based on soft measurement prediction: Soft measurement generates predicted quality variables.

[0126] g pred To predict branch networks.

[0127] The process data missing soft measurement model transforms the upsampled features into the final quality estimate. The process data missing soft measurement prediction result and the imputation output imputation reconstruction result are independent of each other, but both depend on the same backbone features.

[0128] Step S106: Optimize the process data missing soft measurement model based on the imputation reconstruction loss function and the soft measurement prediction loss function.

[0129] Specifically, for cases where process data is randomly missing, a loss function L for imputation and reconstruction is constructed. res The reconstruction loss function is the mean square error at the observation location:

[0130] ;in, Let t be the set of observation indices, t be the window index, and f be the feature dimension index. For sample data, To fill in the gaps in the reconstruction results.

[0131] This equation acts as a consistency term. The observed data points serve as a reliable baseline within the window, therefore the model needs to make the reconstructed sequence as close as possible to the known actual measurements. This reduces the likelihood that the imputation process will deviate from the true evolutionary process when filling in missing data points.

[0132] For cases where structural groupings of process data are missing, the same method is used to process each group individually:

[0133] ;

[0134] Here Let G be the number of missing structural groups within a group. Since some sensor groups may still have dense data while others may have severely missing data, additional averaging of the groups can prevent a group with good observations from dominating the loss function.

[0135] Constructing the soft measurement prediction loss function L pred Soft measurement prediction loss refers to the mean square error between the true quality vector and the predicted quality vector.

[0136] ;

[0137] In the formula, y represents the number of soft measurement prediction points; q Indicates a soft measurement label, y' q Indicates soft measurement label y q The corresponding predicted value. This equation measures the degree of deviation between the predicted quality variable and the measured value. The smaller the value, the better the soft measurement performance.

[0138] The overall training objective for constructing the total training loss L and the soft sensor model with missing process data is:

[0139] ;

[0140] This loss function indicates that the process data missing soft measurement model must both recover the missing process data and ensure that the recovered data representation has practical value for soft measurement. These two objectives are interrelated: if the process sequence generated by filling in the reconstructed branch cannot reflect the true process trend, the prediction branch will also be affected. The resulting process data missing soft measurement model can adapt to limited industrial data conditions and meets the needs of both random and structured missing process data scenarios.

[0141] Step S107: The root mean square error and mean absolute error are used to evaluate the process data missing soft measurement prediction results of the target training sample data.

[0142] Specifically, the root mean square error (RMSE) and mean absolute error (MAE) are used as evaluation indicators for soft measurement predictions:

[0143] ;

[0144] In the formula For soft measurement tags, For soft measurement prediction results, This indicates the number of soft measurement prediction points.

[0145] When training the process data missing soft sensor model, the hyperparameter settings are as follows. Table 1 shows the parameter settings for training the process data missing soft sensor model in this embodiment. A fixed time window is used to divide the samples into non-overlapping samples. Each sample contains T time steps and F feature variables. The specific parameter settings are shown in Table 1. The learning rate is uniformly set to 0.001 and is automatically adjusted using Adam.

[0146] Table 1

[0147]

[0148] To further verify the performance of the process data missing soft measurement model in this embodiment, this embodiment also selected the following five imputation methods for comparison, including Principal Component Analysis (PCA), Bidirectional Recurrent Imputation for Time Series (BRITS), Gaussian Process Variational Autoencoder (GP-VAE), Process Feature Industrial Data Imputation Model (PFIDM), and Transformer Sequence-to-Sequence (Trans-S2S). Based on the characteristics of the methods, the above methods can be classified into three categories: (1) statistical imputation, such as PCA; (2) machine learning-based independent imputation modeling, such as BRITS and GP-VAE, which do not consider the impact of subsequent tasks; (3) downstream task-oriented imputation modeling methods PFIDM and Trans-S2S, where PFIDM is a diffusion model that introduces downstream soft measurement to achieve task-aware imputation optimization, and Trans-S2S is based on sequence-to-sequence prediction structure, which enhances the consistency between imputation and downstream prediction tasks through joint modeling.

[0149] Table 2 compares the RMSE and MAE of process data missing soft measurement methods under different data missing scenarios in cement clinker production cases:

[0150] Table 2

[0151]

[0152] As shown in Table 2, the PCA method exhibits significant errors in both missing scenarios; the BRITS and GP-VAE methods show improvement over PCA; and PFIDM and Trans-S2S further reduce prediction errors in both missing scenarios. The GPTIS method in this embodiment achieves superior and stable prediction results in both missing scenarios. In the random missing scenario, the root mean square error (RMSE) and mean absolute error (MAE) of the prediction results obtained by the GPTIS method in this embodiment are lower than most of the comparative methods; in the structural component missing scenario, it still maintains a low error level, demonstrating good adaptability to different missing patterns. Figure 4These are the soft measurement prediction result curves for data loss in each process under the random missing scenario in this embodiment. The Real value is the actual result. Figure 5 These are the soft measurement prediction result curves for missing process data in the structural component missing scenario of this embodiment, based on... Figure 4 and Figure 5 As shown, compared with the PCA, BRITS, GP-VAE, PFIDM, and Trans-S2S methods used in the prior art, the comparison results show that the soft measurement prediction method of the soft measurement model in this embodiment can track the trend of the change of the true value well under both missing conditions, and has good stability and prediction accuracy.

[0153] Through steps S101 to S107, GPT-2 is used as a shared encoder, and its attention mechanism is utilized to model global dependencies in multivariate time-series data. Based on these global dependencies, a missing data incompleteness and reconstruction task is performed, and soft measurement prediction is made based on the incompleteness and reconstruction results, thereby improving the representation learning and reconstruction capabilities of missing data. Subsequently, supervision information for the soft measurement task is introduced, and the missing data incompleteness and soft measurement prediction tasks are optimized collaboratively through the shared encoder and joint loss function, making the incompleteness more in line with the prediction requirements and improving the accuracy of the soft measurement model prediction. At the same time, an efficient parameter fine-tuning strategy is adopted to adapt the pre-trained model to industrial process data and complex missing data scenarios.

[0154] This embodiment provides a soft measurement prediction method for missing process data, applicable to industrial production scenarios. Figure 6 This is a flowchart of the process data missing soft measurement prediction method in this embodiment, as shown below. Figure 6 As shown, the process includes the following steps:

[0155] Step S601: Obtain the target data to be detected in the industrial production process; the target data to be detected is the production result data that is missing in the industrial production process.

[0156] Step S602: Read the trained process data missing soft measurement model. The soft measurement model includes task filling embedding, soft measurement prediction embedding, pre-trained model and output layer. The pre-trained model is a model that has been trained in advance based on sample data in various industrial production scenarios.

[0157] Step S603: Construct a filling and reconstruction representation based on the target data to be detected. The filling and reconstruction representation is the sequence representation of the target data to be detected after mapping.

[0158] Step S604: The filling and reconstruction representation is used as the input to the filling task embedding and fed into the pre-trained model to output the filling and reconstruction result;

[0159] Step S605: Construct a soft measurement prediction representation based on the filling and reconstruction results; the soft measurement prediction representation is a sequence representation of the filling and reconstruction results after mapping.

[0160] Step S606: The soft measurement prediction representation is used as the input to the soft measurement prediction embedding and input into the pre-trained model to output the process data missing soft measurement prediction result of the target data to be detected.

[0161] Specifically, the target data to be detected during industrial production is input into the process data missing soft measurement model trained according to the above-mentioned process data missing soft measurement model training process. Spatial vector mapping processing is performed on the target data to be detected to obtain the filling reconstruction representation, which includes word embedding representation and position embedding representation. Through the GPT-2 model, the word embedding representation and position embedding representation are converted into context features of the filling reconstruction. Based on the context features of the filling reconstruction, the filling reconstruction is obtained. The soft measurement prediction representation is constructed with the filling reconstruction result. The soft measurement prediction representation is used as the input of the soft measurement prediction embedding. Through the GPT-2 model, the soft measurement prediction representation is converted into the context features of the soft measurement prediction. The context features are upsampled to obtain the upsampled context features of the soft measurement prediction. Based on the upsampled context features of the soft measurement prediction, the process data missing soft measurement prediction result is predicted.

[0162] Through steps S601 to S606 above, firstly, a shared encoder is constructed using the GPT-2 pre-trained structure, and the global dependencies in the time dimension and feature dimension are captured through the attention mechanism to achieve effective reconstruction of missing process data; then, a task mapping module for missing data reconstruction task and soft measurement prediction task is constructed on the basis of the shared representation, and soft measurement prediction is achieved through the mapping of the two tasks, thereby improving the missing data reconstruction effect and soft measurement prediction accuracy.

[0163] It should be understood that the specific embodiments described herein are merely illustrative of the application and not intended to limit it. All other embodiments derived by those skilled in the art based on the embodiments provided in this application without inventive effort are within the scope of protection of this application.

[0164] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0165] Obviously, the accompanying drawings are merely some examples or embodiments of this application. Those skilled in the art can apply this application to other similar situations based on these drawings without any creative effort. Furthermore, it is understood that although the work done in this development process may be complex and lengthy, for those skilled in the art, certain design, manufacturing, or production modifications made based on the technical content disclosed in this application are merely conventional technical means and should not be considered as insufficient disclosure of this application.

[0166] The term "embodiment" in this application refers to a specific feature, structure, or characteristic described in connection with an embodiment that may be included in at least one embodiment of this application. The appearance of this phrase in various places in the specification does not necessarily imply the same embodiment, nor does it imply that it is mutually exclusive with or independent of other embodiments. It will be clearly or implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0167] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of patent protection. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the appended claims.

Claims

1. A soft measurement prediction method for missing process data, characterized in that, For industrial production scenarios, the method includes: Acquire target data to be detected during industrial production; the target data to be detected is production result data that is missing during the industrial production process. Read the trained process data missing soft measurement model, wherein the soft measurement model includes task filling embedding, soft measurement prediction embedding, pre-trained model and output layer, wherein the pre-trained model is a model that has been trained in advance based on sample data in various industrial production scenarios; A filling and reconstruction representation is constructed based on the target data to be detected, wherein the filling and reconstruction representation is a sequence representation of the target data to be detected after mapping; The filling and reconstruction representation is used as the input to the filling task embedding, and is fed into the pre-trained model to output the filling and reconstruction result; A soft measurement prediction representation is constructed based on the filling and reconstruction results; the soft measurement prediction representation is a sequence representation of the filling and reconstruction results after mapping. The soft measurement prediction representation is used as the input to the soft measurement prediction embedding and is fed into the pre-trained model to output the process data missing soft measurement prediction result of the target data to be detected.

2. The process data missing soft measurement prediction method according to claim 1, characterized in that, The process of training the soft measurement model with missing process data includes: The original training sample data is collected according to the preset first fixed window, and the original training sample data is masked to obtain the masked original training sample data. The original training sample data after the mask is normalized to obtain the target training sample data; A fill-in reconstruction representation is constructed based on the target training sample data, wherein the fill-in reconstruction representation is a sequence representation of the target training sample data after mapping. The filling and reconstruction representation is used as the input to the filling task embedding, and is fed into the pre-trained model to output the filling and reconstruction result; A soft measurement prediction representation is constructed based on the filling and reconstruction results; the soft measurement prediction representation is a sequence representation of the filling and reconstruction results after mapping. The soft measurement prediction representation is used as the input to the soft measurement prediction embedding and input into the pre-trained model to output the process data missing soft measurement prediction result of the target training sample data.

3. The process data missing soft measurement prediction method according to claim 2, characterized in that, The construction of the imputation reconstruction representation based on the target training sample data includes: The target training sample data is processed by spatial vector mapping to obtain the filling reconstruction representation, wherein the filling reconstruction representation includes word embedding representation and position embedding representation.

4. The process data missing soft measurement prediction method according to claim 3, characterized in that, The input of the filling and reconstruction representation as the embedding of the filling task is fed into a preset training model, and the output filling and reconstruction result includes: The word embedding representation and position embedding representation are converted into contextual features for filling and reconstruction using the preset training model. Based on the contextual features of the filling and reconstruction, the filling and reconstruction result is obtained.

5. The process data missing soft measurement prediction method according to claim 2, characterized in that, The pre-trained model is updated and optimized using low-rank adaptation.

6. The process data missing soft measurement prediction method according to claim 2, characterized in that, The construction of a soft measurement prediction representation based on the filling and reconstruction results includes: The filling and reconstruction results are patched to obtain the patched filling and reconstruction results; The patched reconstruction results are mapped to obtain the soft measurement prediction representation.

7. The process data missing soft measurement prediction method according to claim 6, characterized in that, After mapping the patched reconstruction results to obtain the soft measurement prediction representation, the method further includes: The soft measurement prediction representation is segmented according to a preset second fixed window to obtain a segmented soft measurement prediction representation; wherein the preset second fixed window is smaller than the preset first fixed window; The segmented soft measurement prediction representation is used as the input for the soft measurement prediction embedding.

8. The process data missing soft measurement prediction method according to claim 7, characterized in that, The process of using the soft measurement prediction representation as input to the soft measurement prediction embedding, inputting it into the pre-trained model, and outputting the target training sample data to output the data missing soft measurement prediction result includes: The pre-trained model is used to convert the segmented soft measurement prediction representation into contextual features of the soft measurement prediction. The context features are upsampled to obtain the upsampled context features for soft measurement prediction; Based on the upsampling context features of the soft measurement prediction, the process data missing soft measurement prediction result of the target training sample data is predicted.

9. The process data missing soft measurement prediction method according to claim 2, characterized in that, The method further includes optimizing the process data missing soft measurement model based on the imputation reconstruction loss function and the soft measurement prediction loss function.

10. The process data missing soft measurement prediction method according to claim 2, characterized in that, The method further includes: The root mean square error and mean absolute error are used to evaluate the process data missing soft measurement prediction results of the target training sample data.

Images