Industrial modeling system and method based on federated lifelong learning in cloud-edge collaboration scenario

Abstract

The present invention discloses an industrial modeling system and method based on federated lifelong learning in a cloud-edge collaboration scenario. In order to achieve lifelong learning and modeling capabilities of industrial processes in variable working condition scenarios, the present invention introduces a generation-based replay strategy, designs a dedicated diffusion model for each working condition, and realizes the capture, filling and generation of data. The pseudo data generated by the diffusion model is trained collaboratively with the incremental data to consolidate the local historical working condition knowledge. Taking into account the characteristics of high missing rate and poor quality of industrial data, the present method proposes a self-masking technique for training the diffusion model, so that it has both filling and generation capabilities. Taking into account the differences and complementarities of different client working conditions, the present invention designs a federal weighted aggregation strategy to achieve the integration of historical knowledge of other factories. While protecting data privacy, the present invention effectively utilizes the knowledge of each client, and the downstream task model does not forget the historical data distribution during incremental updates, and has stronger generalization capabilities for locally unseen working conditions.

Description

Technical Field

[0001] The present invention belongs to the field of soft measurement of key quality variables in industrial processes, and in particular relates to soft measurement systems and methods for cloud-edge collaboration, federated learning, and lifelong learning. Background Art

[0002] Accurate measurement of key quality variables is crucial to ensuring the safe and economical operation of large-scale manufacturing equipment. However, due to factors such as hardware cost, measurement error, and analysis latency, some key qualities cannot be measured in a timely manner, seriously impacting subsequent process monitoring and advanced control. Data-driven soft sensor models have been proposed in recent years. These regression models use easily measurable process variables as input to achieve real-time prediction of key quality variables. Deep learning-based soft sensor models exhibit excellent performance due to the powerful ability of deep networks to extract nonlinear and temporal relationships. Cloud-edge collaboration has been proposed as a new intelligent manufacturing paradigm, breaking down data silos within the industrial sector, aggregating and organizing massive amounts of information resources, and providing new solutions for data fusion and knowledge aggregation. Federated learning, a prominent paradigm in cloud-edge collaboration, has attracted widespread attention. It enables distributed training of deep learning models without compromising data privacy. Training soft sensor models based on federated learning, also known as federated soft sensor models, can further improve the model's predictive ability for quality variables by integrating complementary knowledge across different factories.

[0003] Although federated soft-sensing algorithms have received widespread attention, most algorithms assume that the data distribution of each client is static, that is, the data distribution of each client will not change over time. However, this assumption is difficult to hold true in actual processes. Due to factors such as changes in set values, switching of operating conditions, and mechanical wear, the working conditions of the actual process are constantly changing, which causes the data collected at different times to have a large distribution offset. This problem is also known as temporal non-independent and identically distributed (temporal non-IID). Affected by this problem, when the local data distribution switches, if the model is not updated in time, the performance of the traditional federated learning model will drop significantly. However, the updated model will forget the working conditions of historical learning, resulting in a catastrophic forgetting problem.

[0004] To address the catastrophic forgetting problem associated with incremental model updates, the lifelong learning paradigm has garnered widespread attention. In particular, the replay-based lifelong learning paradigm has made significant progress in combating forgetting. This paradigm primarily encompasses prototype-based and generation-based replay algorithms. The former retains the data closest to the center of each data distribution, enabling the replay of historical knowledge, but at the expense of sample diversity. The latter, based on adversarial generative networks, mimics each training data distribution, enabling the generation of historical data and the replay of knowledge, but the distribution of the generated data can be biased. Furthermore, these approaches are unable to address the widespread missing data found in industrial data. Summary of the Invention

[0005] The purpose of the present invention is to address the insufficiency of the existing federated soft measurement technology of industrial equipment in handling the ever-changing working conditions, and to provide a federated lifelong modeling system and method based on a diffusion model, which utilizes a fine-grained replay strategy and cloud-based knowledge aggregation. For the fine-grained replay strategy, the present invention designs a dedicated diffusion model for each working condition to achieve data capture, filling and generation. The diffusion model is trained through the proposed self-masking technique, and has both filling and generation capabilities. The pseudo data generated by the diffusion model is trained collaboratively with the incremental data to achieve the consolidation of historical knowledge. For the cloud-based knowledge aggregation part, the present invention designs a federal weighted aggregation strategy to achieve the integration of historical knowledge from other factories. For the federated lifelong soft measurement online application framework, the present invention designs different soft measurement strategies according to the set values ​​of the current control variables to achieve effective prediction of key quality variables.

[0006] The purpose of the present invention is achieved through the following technical solutions:

[0007] A federated lifelong industrial modeling system based on a diffusion model in a cloud-edge collaboration scenario, including:

[0008] C clients, each client includes a data acquisition module, a data memory, a diffusion model library, a local soft measurement model and local soft measurement models corresponding to C-1 other clients; the data acquisition module is used to acquire industrial process variable data in real time; the data memory library updates and stores the data of newly appeared working conditions in real time based on the data acquisition module to construct a training data set for newly appeared working conditions; each sample of the training data set contains industrial process variable data and corresponding soft measurement labels; the diffusion model library includes multiple diffusion models, and the multiple diffusion models correspond to working conditions one by one, and based on the training data set of the corresponding working conditions, the missing part and the non-missing part in the sample are separated to construct sub-samples of the same size as the original training sample, and the sub-samples constructed with the missing part are constructed Diffusion for t steps is performed for the target value to obtain After the diffusion There is no subsample constructed with missing parts and the number of diffusion steps t are input into the diffusion model, which predicts the noise added in the forward diffusion process, and the true value of the noise ε t The constrained diffusion model is trained and obtained; the diffusion model is at least used to fill in the missing industrial process variable data acquired in real time by the data acquisition module under the corresponding working conditions; the local soft measurement model of each client is trained and updated based on the data stored in the corresponding data memory and transmitted to the cloud and other clients respectively;

[0009] A cloud, which is used to receive the local soft measurement model updated by each client and aggregate it to obtain the global model.

[0010] Furthermore, the loss function of the diffusion model training is:

[0011]

[0012] Where, ε θ (*) indicates the noise predicted by the diffusion model, α t =1-β t , β t is a positive number less than 1 that controls the noise amplitude of the diffusion process in step t; ||*| |2 is the L2 norm, Represents t~[1,T], and ε t Calculate the expected value of all possible values ​​of .

[0013] Furthermore, the method for the diffusion model to fill in the missing industrial process variable data acquired in real time by the data acquisition module under the corresponding working conditions is specifically as follows:

[0014] The subsample constructed by using Gaussian noise as the missing part obtained by diffusion in the Tth step The non-missing part of the missing industrial process variable data obtained in real time by the data acquisition module under the corresponding working conditions is constructed as a subsample t and Input into the trained diffusion model to obtain the mean function and variance function estimated by the model, thus obtaining Then and t are input into the diffusion model, thus obtaining Repeat this process until you get Expressed as:

[0015]

[0016]

[0017] Where T is the total number of diffusion steps, ε θ (*) indicates the noise predicted by the diffusion model, is the variance function, α t =1-β t , β t is a positive number less than 1 that controls the noise amplitude of the diffusion process in the t-th step, and ε is Gaussian noise.

[0018] Furthermore, the data memory is based on the data acquisition module, which updates and stores the data of the newly appeared working conditions in real time to construct a training data set for the newly appeared working conditions, specifically:

[0019] Collecting and storing industrial process variable data of newly emerged working conditions acquired by the data acquisition module;

[0020] The non-missing industrial process variable data stored in the collected data are used to construct subsamples and train the corresponding diffusion model;

[0021] Using the trained diffusion model to fill in the missing industrial process variable data stored in the collected data to obtain filled data;

[0022] The offline analysis system is used to label the missing industrial process variable data and fill in the data to obtain a training data set for the newly emerged working conditions.

[0023] Furthermore, the diffusion model also includes a method for generating replay data, specifically:

[0024] The subsample constructed by using Gaussian noise as the missing part obtained by diffusion in the Tth step Let the subsample constructed without missing parts Will t and Input into the trained diffusion model to obtain the mean function and variance function estimated by the model, thus obtaining Then and t are input into the diffusion model, thus obtaining Repeat this process until you get Expressed as:

[0025]

[0026]

[0027] Where T is the total number of diffusion steps, ε θ (*) indicates the noise predicted by the diffusion model, is the variance function, α t =1-βt , β t is a positive number less than 1 that controls the noise amplitude of the diffusion process in the t-th step, and ε is Gaussian noise.

[0028] Furthermore, the data memory also stores a replay data set for each working condition contained in the corresponding client; the replay data set is constructed and obtained by the following method:

[0029] Replay data is obtained using the diffusion model trained under the corresponding working conditions;

[0030] The replay data is predicted based on the local soft measurement model of the corresponding client to obtain the soft measurement results as pseudo labels, and a replay dataset is constructed.

[0031] Furthermore, when a client's data memory is constructed to obtain a training data set for a new working condition, the local soft sensor model and the global model of each client are trained and updated once, specifically:

[0032] For clients with training data sets for new working conditions, the training data sets of the newly appeared working conditions and the replay data sets of other working conditions stored in the corresponding data memory are used as training data for training. Other clients use the replay data sets of all corresponding working conditions stored in the corresponding data memory as training data for training. During the training of the local soft measurement model, each client uploads the local soft measurement model to the cloud at fixed intervals. The cloud performs weighted averaging on the local soft measurement models of each client, obtains the global model, and distributes it to each client. After the client receives the global model, it uses the parameters of the global model as the starting point for subsequent training of the local soft measurement model. This process is repeated until the preset rounds are reached, and the update of each client's local soft measurement model and the global model is completed. The cloud transmits the global model and the local soft measurement models of all clients to each client.

[0033] Furthermore, the structures of the local soft-sensing model and the global model are LSTM, RNN, CNN, ConvLSTM, Transformer, etc.

[0034] A method for federated lifelong industrial modeling based on a diffusion model in a cloud-edge collaborative scenario is implemented based on a federated lifelong industrial modeling system based on a diffusion model in the cloud-edge collaborative scenario, comprising the following steps:

[0035] (1) The data acquisition module of each client obtains industrial process variable data in real time. After the online data is acquired, it determines whether the working conditions of the data have changed;

[0036] (2) According to the judgment result of the working condition, the corresponding operation is performed: if the current working condition is a working condition that has been modeled locally, the local soft measurement model is directly used in the soft measurement task to predict the soft measurement results of the industrial process variable data; if the current working condition is a working condition that has been trained by other clients, the local soft measurement model of the corresponding client is used for the current soft measurement task to predict the soft measurement results of the industrial process variable data; if the current working condition is a working condition that has not been seen by all clients, the global soft measurement model is used for the current soft measurement task to predict the soft measurement results of the industrial process variable data, and at the same time, the industrial process variable data of the current new working condition is collected and stored in the corresponding data memory;

[0037] (3) After collecting industrial process variable data of a certain current new working condition, the corresponding diffusion model is trained based on the collected industrial process variable data of the new working condition and stored in the diffusion model library; the trained diffusion model is used to fill the collected industrial process variable data of the new working condition, and the data is labeled through the offline analysis system to construct a training data set for the new working condition;

[0038] (4) When a client's data memory is constructed to obtain a training data set for a new working condition, the local soft measurement model and the global model of each client are trained and updated once.

[0039] Furthermore, in step (2), if the current working condition is a local modeling working condition, the diffusion model of the local corresponding working condition is used to fill the missing values, otherwise the mean is used to fill the missing values, and then the soft measurement task is performed.

[0040] Traditional federated soft-sensing algorithms are limited by the assumption that client data distribution is static, and are unable to handle situations where client data distribution changes with operating conditions. Blindly updating the model will cause historical soft-sensing knowledge to be forgotten. Compared with existing technologies, the present invention has at least the following beneficial effects:

[0041] 1. A fine-grained replay strategy was designed. This invention employs a dedicated diffusion model for each operating condition to capture, infill, and generate data. The diffusion model is trained using the proposed self-masking technique, enabling both infill and generation capabilities. Pseudo-data generated by the diffusion model is trained collaboratively with incremental data to consolidate historical knowledge.

[0042] 2. A cloud-based knowledge aggregation method is designed. The present invention designs a federal weighted aggregation strategy to achieve the integration of historical knowledge of other factories.

[0043] 3. A federated lifelong soft measurement online application method is proposed. According to the set value of the current control variable, the present invention designs different soft measurement strategies to achieve effective prediction of key quality variables. The predicted quality variables can indicate the status of the current process operation and can be used for subsequent anomaly detection tasks and closed-loop control tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 This is a flow chart of diffusion model training and self-masking training in the present invention;

[0045] Figure 2 It is a diagram of the update process of DMB and EMB of the present invention;

[0046] Figure 3 It is the overall flow chart of the federal lifetime modeling framework based on the diffusion model. DETAILED DESCRIPTION

[0047] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0048] The present invention provides a federated lifelong industrial modeling system based on a diffusion model in a cloud-edge collaboration scenario, including:

[0049] A cloud and C clients; each client contains a data acquisition module, a data memory library, a diffusion model library, a local soft measurement model and local soft measurement models corresponding to C-1 other clients; where:

[0050] The data acquisition module is used to acquire industrial process variable data in real time. The data memory is updated and stored in real time based on the data acquisition module to build a training data set for the new working condition. Each sample in the training data set contains industrial process variable data and the corresponding soft measurement label. The training data set for a certain working condition of a certain client can be expressed as Among them, c represents the cth client, w represents the wth working condition, is the sample size of client c working condition w, and They represent the i-th sample and soft measurement label of client c working condition w data, It is a sliding window of a multivariate time series. The local soft measurement model of each client is trained and updated based on the data stored in the corresponding data memory and transmitted to the cloud and other clients respectively; the cloud is used to receive the local soft measurement model trained and updated by each client and aggregate it to obtain a global model. Based on the industrial process variable data, the trained soft measurement model can be used to predict the corresponding quality variables, thereby realizing quality variable monitoring to ensure the safe and economic operation of large-scale manufacturing equipment. For each client, a training data set of several commonly occurring working conditions can be collected in advance to train the corresponding diffusion model and construct an initial local soft measurement model and a global model. In actual use, the data memory, diffusion model library, local soft measurement model and global model are updated based on the newly occurring working conditions. This embodiment takes the quality monitoring of a three-phase flow industrial process as an example. The selected three-phase flow process contains a total of 24 variables, of which the air delivery pressure, the riser bottom pressure, the riser top pressure and the separator top pressure are used as the quality variables (soft measurement results) of the soft measurement model. The data sampling frequency is 1 Hz and the length of the sliding window is 30. The industrial process variables acquired in real time in this process include air flow rate and water flow rate. Changes in their values ​​affect the changes in data distribution and belong to different working conditions.

[0051] Affected by factors such as signal transmission errors or hardware failures, industrial process data collected in real time contain a large number of missing values. In order to fill in missing values ​​and generate historical data, the present invention system constructs a diffusion model library, which includes multiple diffusion models. The diffusion model is at least used to fill in the missing industrial process variable data acquired in real time by the data acquisition module under the corresponding working conditions; the diffusion model includes a forward diffusion process and a reverse diffusion process. The forward diffusion process is a deterministic Markov process, which converts the original data into a standard Gaussian distribution by gradually adding white noise, specifically:

[0052]

[0053]

[0054] Among them, t represents the t-th step of the diffusion process, and the diffusion process has a total of T steps. represents the original sample that has not undergone the diffusion process, is the noise sample after t steps of diffusion, β t is a positive number less than 1 to control the noise amplitude, I represents the unit covariance matrix, Indicates from is the mean, β t I is the Gaussian distribution sampled from the covariance The probability of Indicates from arrive The conditional probability distribution of .

[0055] By re-parameterization techniques, any time step In a given The conditional probability in the case can be directly given by the following formula

[0056]

[0057] in, α t =1-β t .

[0058] The inverse diffusion process is to reverse the above diffusion process step by step to generate original data from Gaussian noise. This process is also a Markov process and obeys the Gaussian distribution. Specifically,

[0059]

[0060]

[0061] in, and denote the mean function and variance function respectively, Indicates that the mean The variance is The Gaussian distribution of The probability, p θ represents the probability density function of the inverse diffusion process, Indicates known In this case, the independent variable takes the value The probability of Indicates known In this case, sampling is performed when t is from 0 to T The probability of express The prior distribution of .

[0062] By using Bayes' rule, we can deduce that the mean function and variance function have the following forms:

[0063]

[0064]

[0065] Among them, ε θ is the denoising function that needs to be learned by the diffusion model, ε θ (*) represents the noise predicted by the diffusion model, and * is the input of the diffusion model.

[0066] The diffusion model is trained using the proposed self-masking training technique. Multiple diffusion models are mapped one-to-one to the working conditions and are trained using the self-masking training technique based on the training dataset under the corresponding working conditions. The training process is as follows:

[0067] Constructing the training sub-dataset of the diffusion model: For the industrial engineering data samples under the corresponding working conditions, random missing positions are constructed with a fixed probability, and the missing part and the non-missing part of each training sample are separated to construct sub-samples of the same size as the original training sample. The sub-sample constructed from the non-missing part (the non-missing part is the original eigenvalue, and the eigenvalue of the missing part is 0) is expressed as As the original observation sample of the diffusion model, the subsample constructed by the missing part is recorded as as target value;

[0068] Utilize x obs and x tgt , train the diffusion model in a self-supervised manner, such as Figure 1 As shown, specifically: for the target value Perform t-step diffusion to obtain t is uniformly sampled from 1 to T, and the diffused The original observation sample and the number of diffusion steps t are input into the diffusion model, which predicts the noise added in the forward diffusion process, and the true value of the noise ε t Training of the constrained diffusion model. For example, the loss function can be:

[0069]

[0070] ||*| |2 is the L2 norm, Represents t~[1,T], and ε t Calculate the expected value of all possible values ​​of .

[0071] Using the industrial process data of each working condition, a diffusion model (DM) exclusive to that working condition can be established. The trained diffusion model Store in Diffusion Model Base (DMB) c , using the trained diffusion model The industrial engineering data corresponding to the working conditions can be filled and traversed Each sample Get the corresponding Then get the filled data Constructing a filled dataset The specific process of filling the training data using the diffusion model is as follows:

[0072] The subsample constructed by using Gaussian noise as the missing part obtained by diffusion in the Tth step Construct a subsample without missing parts t and Input into the trained diffusion model to obtain the mean function and variance function estimated by the model, thus obtaining Then and t are input into the diffusion model, thus obtaining Repeat this process until you get The process can be written

[0073]

[0074] Among them, ε is the noise sampled from the standard Gaussian distribution, that is, Gaussian noise.

[0075] Furthermore, using the trained diffusion model You can also generate replay data for the corresponding working conditions When a new working condition appears and sufficient training data is accumulated, the local soft measurement model can be trained and updated in combination with the replay data to resist forgetting and maintain the accuracy of the updated local soft measurement model in soft measurement of the original working condition.

[0076] The process of generating replay data using the diffusion model is as follows:

[0077] The subsample constructed by using Gaussian noise as the missing part obtained by diffusion in the Tth step Let the subsample constructed without missing parts (all set to 0), t and Input into the trained diffusion model to obtain the mean function and variance function estimated by the model, thus obtaining Then and t are input into the diffusion model, thus obtaining Repeat this process until you get The number of replay data is set in advance, and the corresponding number of noise input diffusion models are sampled from the standard Gaussian distribution to obtain the generated replay data. The replay data generated Store data in External Memory Bank (EMB) c .

[0078] Furthermore, in step (2) data will be generated The process of storing data in the memory is as follows Figure 2 As shown, specifically:

[0079] Constructing a replay dataset Among them, Placeholder represents the placeholder of the replay data label, and the replay data set is merged into the data memory library. For the replay data label of the working condition that has already appeared, the replay data is directly predicted and pseudo-labeled by the corresponding local soft measurement model, and then the Placeholder is replaced. For the replay data label of the newly appeared working condition, the local soft measurement model based on the padded data set corresponding to the new working condition and the replay data set of other working conditions corresponding to the current client is used as training data to predict and pseudo-label it, and then the Placeholder is replaced. That is: pseudo-labeling is performed after the local soft measurement model is updated, wherein, when there is a client's data memory library to construct a training data set for a new working condition, the local soft measurement model and the global model of each client are trained and updated once; specifically:

[0080] On each client, the data stored in the data memory is used for training. For clients with training data sets for new working conditions, the training data sets (filling data sets) of the new working conditions stored in the corresponding data memory are used. The replay data sets of other working conditions are used as training data for training. Other clients use the replay data sets of all working conditions stored in the corresponding data memory as training data for training;

[0081] The soft measurement model can adopt conventional neural network models, such as LSTM, RNN, CNN, ConvLSTM, Transformer, etc. In this embodiment, a cascade structure of LSTM and fully connected layers is used. The former is used to extract time series features, and the latter is used to obtain the final prediction results. The model structure of each client is the same. For client c, its soft measurement model parameters are The overall model is Input the industrial process data in the training data set into the LSTM network to obtain features Then feature Z c Input the fully connected layer to get the prediction result The parameters of the soft sensor model are optimized by minimizing the bi-norm error between the predicted results and the true values. The loss function of this process can be written as

[0082]

[0083] Where X c represents the industrial process data of the training data used by the c-th client in the current update, Y cFor the corresponding label, for the client with a training dataset of a new working condition, it includes the industrial process data of the filling dataset and the industrial process data of the replay dataset of other working conditions.

[0084] During the training of its local soft measurement model, each client uploads it to the cloud at fixed intervals. The cloud then performs a weighted average of the soft measurement models from each client to generate a global model, which is then distributed to each client. After receiving the global model, the client uses the parameters of the global model as the starting point for subsequent training of its local soft measurement model. This process repeats until the preset number of rounds is reached. Once each client's local soft measurement model and the global model are updated, the cloud transmits the global model and all clients' local soft measurement models to each client.

[0085] For clients with training data sets for new working conditions, after training is completed, the trained soft measurement model is used to replay the data. Anti-counterfeiting label Pseudo-label replacement Specifically: replay data Bring it into the soft measurement model to obtain the pseudo label of the replay data The process can be written as:

[0086]

[0087] The system of the present invention can be used for industrial soft measurement online, such as Figure 3 As shown, the following steps are included:

[0088] Step (1): The data acquisition module of each client obtains industrial process variable data in real time. After the online data is acquired, it is determined whether the working conditions of the data have changed. The judgment basis can be whether the value of the control variable has changed. In this embodiment, it specifically refers to whether the air flow rate and water flow rate have changed.

[0089] Step (2): Execute the corresponding operation according to the judgment result of the working condition. If the current working condition is a locally modeled working condition, the local soft measurement model is directly used in the soft measurement task to predict the soft measurement results of the industrial process variable data; if the current working condition is a working condition trained by other clients, the local soft measurement model of the corresponding client is used in the current soft measurement task to predict the soft measurement results of the industrial process variable data; if the current working condition is a working condition that has not been seen by all clients, the global model is used in the current soft measurement task to predict the soft measurement results of the industrial process variable data, and at the same time, the industrial process variable data of the current new working condition is collected and stored in the corresponding data memory; the predicted quality variable can indicate the status of the current process operation and can be used for subsequent abnormality detection tasks and closed-loop control tasks.

[0090] Step (3): After collecting industrial process variable data of a certain current new working condition, the corresponding diffusion model is trained based on the collected industrial process variable data of the new working condition and stored in the diffusion model library; the trained diffusion model is used to fill the collected industrial process variable data of the new working condition, and the data is labeled through the offline analysis system to construct a training data set for the new working condition for subsequent lifelong incremental updates;

[0091] Step (4): When a client's data memory is constructed to obtain a training data set for a new working condition, the local soft measurement model and the global model of each client are trained and updated once.

[0092] Furthermore, in step (2), if the current working condition is a local modeling working condition, the diffusion model of the local corresponding working condition is used to fill the missing values, otherwise the mean is used to fill the missing values, and then the soft measurement task is performed.

[0093] In this embodiment, there are 4 values ​​for air flow rate and 5 values ​​for water flow rate. Set up 4 different clients (denoted as A, B, C and D), and their air flow rates are different. Each client has 5 working condition data distributions, and there are 20 different working condition data distributions in total. The 5 local data distributions appear in sequence over time, and the soft measurement model needs to be trained and updated when the data distribution changes. The ERR index and BWT index are used to evaluate the performance of the system of the present invention, and Exact Replay (ER), No Replay (NR), prototype-based replay algorithm iCaRL and generation-based replay algorithm DGR are selected as comparison algorithms. The calculation method of ERR is

[0094]

[0095] in, Is the test error matrix, each element R i,j Represents the soft sensor model in the data set After training on the dataset Test error, measured by RMSE and MAE. This metric evaluates the overall predictive ability of the model across all data distributions. The smaller the value, the better the model.

[0096] BWT is calculated as

[0097]

[0098] BWT reflects the impact of incremental model updates on historical learning tasks. The smaller the value, the better the model.

[0099] Table 1 shows the results of the soft-sensing model's anti-forgetting capabilities under different client scenarios. The NR strategy achieves the highest ERR and BWT, indicating that deep learning-based soft-sensing models suffer from catastrophic forgetting. The ER strategy achieves the lowest ERR and BWT, but this strategy significantly increases model training costs, making it unusable in practice. Regardless of the ER strategy, the present invention achieves the lowest ERR and BWT under different client scenarios, demonstrating its superior anti-forgetting capabilities.

[0100] Table 1. Comparison of anti-forgetting results of the present invention and other replay algorithms

[0101]

[0102] The results of the generalization performance of the soft measurement model for seen and unseen distributions under different clients are shown in Table 2. As can be seen from the table, when the cloud-based aggregation is performed, the soft measurement model of the present invention has the smallest error in both RMSE and MAE indicators for seen and unseen distributions, reflecting that the present invention has better generalization performance for unseen distributions.

[0103] Table 2. Generalization performance comparison between the proposed method and the non-federated aggregation method

[0104]

[0105] The present invention is not limited to the field of three-phase flow process soft measurement in the above example. Anyone familiar with the art can make equivalent modifications or substitutions without violating the present invention. These equivalent modifications or substitutions are all included in the scope defined by the claims of this application.

Claims

1. A federated lifelong industrial modeling system based on a diffusion model in a cloud-edge collaboration scenario, characterized by: include: C clients, each client comprising a data acquisition module, a data memory, a diffusion model library, a local soft measurement model, and local soft measurement models corresponding to C-1 other clients; the data acquisition module is used to acquire industrial process variable data in real time; the data memory library is updated and stored in real time based on the data acquisition module to construct a training data set for the newly emerged working conditions; each sample in the training data set comprises industrial process variable data and a corresponding soft measurement label; The diffusion model library includes multiple diffusion models, which correspond to working conditions one by one. Based on the training data set of the corresponding working conditions, the missing parts and the non-missing parts in the sample are separated to construct sub-samples of the same size as the original training sample. Diffusion for t steps is performed for the target value to obtain After the diffusion There is no subsample constructed with missing parts and the number of diffusion steps t are input into the diffusion model, which predicts the noise added in the forward diffusion process, and the true value of the noise ε t The constrained diffusion model is trained and obtained; the diffusion model is at least used to fill in the missing industrial process variable data acquired in real time by the data acquisition module under the corresponding working conditions; the local soft sensor model of each client is trained and updated based on the data stored in the corresponding data memory and transmitted to the cloud and other clients respectively; A cloud, which receives the local soft-sensor model trained and updated by each client and aggregates it to obtain a global model; The diffusion model also includes a method for generating replay data, specifically: The subsample constructed by using Gaussian noise as the missing part obtained by diffusion in the Tth step Let the subsample constructed without missing parts Will t and Input into the trained diffusion model to obtain the mean function and variance function estimated by the model, thus obtaining Then and t are input into the diffusion model, thus obtaining Repeat this process until you get Expressed as: Where T is the total number of diffusion steps, ε θ (*) indicates the noise predicted by the diffusion model, is the variance function, α t =1-β t , β t is a positive number less than 1 that controls the noise amplitude of the diffusion process in the t-th step, and ε is Gaussian noise.

2. The system according to claim 1, wherein: The loss function of the diffusion model training is: Where, ε θ (*) indicates the noise predicted by the diffusion model, α t =1-β t , β t is a positive number less than 1 that controls the noise amplitude of the diffusion process in step t; ||*|| 2 is the L2 norm, Represents t~[1,T], and ε t Calculate the expected value of all possible values ​​of .

3. The system according to claim 1, wherein: The method for the diffusion model to fill in the missing industrial process variable data acquired in real time by the data acquisition module under the corresponding working conditions is specifically as follows: The subsample constructed by using Gaussian noise as the missing part obtained by diffusion in the Tth step The non-missing part of the missing industrial process variable data obtained in real time by the data acquisition module under the corresponding working conditions is constructed as a subsample t and Input into the trained diffusion model to obtain the mean function and variance function estimated by the model, thus obtaining Then and t are input into the diffusion model, thus obtaining Repeat this process until you get Expressed as: Where T is the total number of diffusion steps, ε θ (*) indicates the noise predicted by the diffusion model, is the variance function, α t =1-β t , β t is a positive number less than 1 that controls the noise amplitude of the diffusion process in the t-th step, and ε is Gaussian noise.

4. The system according to claim 1, wherein: The data memory is based on the data acquisition module to update and store the data of the newly appeared working conditions in real time to construct a training data set for the newly appeared working conditions, specifically: Collecting and storing industrial process variable data of newly emerged working conditions acquired by the data acquisition module; The non-missing industrial process variable data stored in the collected data are used to construct subsamples and train the corresponding diffusion model; Using the trained diffusion model to fill in the missing industrial process variable data stored in the collected data to obtain filled data; The offline analysis system is used to label the missing industrial process variable data and fill in the data to obtain a training data set for the newly emerged working conditions.

5. The system according to claim 1, wherein: The data memory also stores a replay data set for each working condition contained in the corresponding client; the replay data set is constructed and obtained by the following method: Replay data is obtained using the diffusion model trained under the corresponding working conditions; The replay data is predicted based on the local soft measurement model of the corresponding client to obtain the soft measurement results as pseudo labels, and a replay dataset is constructed.

6. The system according to claim 5, characterized in that When a client's data memory is constructed to obtain a training data set for a new working condition, the local soft sensor model and the global model of each client are trained and updated once, specifically: For clients with training data sets for new working conditions, the training data sets of the new working conditions and the replay data sets of other working conditions are used for training. Other clients use the replay data sets of all corresponding working conditions as training data for training. During the training of the local soft measurement model, each client uploads the local soft measurement model to the cloud at fixed intervals. The cloud then performs a weighted average of the local soft measurement models of each client, obtains a global model, and distributes it to each client. After receiving the global model, the client uses the parameters of the global model as the starting point for subsequent training of the local soft sensor model; This process is repeated until the preset rounds are reached, and the update of each client's local soft measurement model and the global model is completed. The cloud transmits the global model and the local soft measurement models of all clients to each client.

7. The system according to claim 5, characterized in that The structures of the local soft-sensing model and the global model are LSTM, RNN, CNN, ConvLSTM or Transformer.

8. A federated lifecycle industrial modeling method based on diffusion model in cloud-edge collaboration scenario, characterized by: The implementation of a federated lifelong industrial modeling system based on a diffusion model in a cloud-edge collaboration scenario according to any one of claims 1 to 7 includes the following steps: (1) The data acquisition module of each client obtains industrial process variable data in real time. After the online data is acquired, it determines whether the working conditions of the data have changed; (2) According to the judgment result of the working condition, the corresponding operation is performed: if the current working condition is a working condition that has been modeled locally, the local soft measurement model is directly used in the soft measurement task to predict the soft measurement results of the industrial process variable data; if the current working condition is a working condition that has been trained by other clients, the local soft measurement model of the corresponding client is used for the current soft measurement task to predict the soft measurement results of the industrial process variable data; if the current working condition is a working condition that has not been seen by all clients, the global soft measurement model is used for the current soft measurement task to predict the soft measurement results of the industrial process variable data, and at the same time, the industrial process variable data of the current new working condition is collected and stored in the corresponding data memory; (3) After collecting industrial process variable data of a certain current new working condition, the corresponding diffusion model is trained based on the collected industrial process variable data of the new working condition and stored in the diffusion model library; the trained diffusion model is used to fill the collected industrial process variable data of the new working condition, and the data is labeled through the offline analysis system to construct a training data set for the new working condition; (4) When a client's data memory is constructed to obtain a training data set for a new working condition, the local soft measurement model and the global model of each client are trained and updated once.

9. The method according to claim 8, characterized in that In step (2), if the current working condition is a local modeling working condition, the diffusion model of the local corresponding working condition is used to fill the missing values. Otherwise, the mean is used to fill the missing values ​​and then the soft measurement task is performed.

Images