A product quality prediction system based on federated learning

Abstract

A product quality prediction system based on federated learning includes a server and clients set up on multiple production lines. The server sends an initial product quality prediction model and fused model parameters to the clients. The fused model parameters are obtained by fusing the model parameters sent by each client. The clients include: multiple sensors for collecting product manufacturing process data; a training set construction module for acquiring sensor data and performing feature extraction to construct a local training set; a model training module for training the acquired product quality prediction model using a forward propagation federated training method based on the local training set; encrypting and sending the model parameters during training to the server and updating the local product quality prediction model according to the fused parameters; stopping training when training requirements are met to obtain a trained product quality prediction model; and a product quality prediction module for performing quality prediction based on the trained product quality prediction model.

Description

Technical Field

[0001] This invention relates to the field of product quality prediction technology, and in particular to a product quality prediction system based on federated learning. Background Technology

[0002] With the development and advancement of the manufacturing industry, the production processes of manufacturing enterprises have become increasingly complex, exceeding general statistical rules. Therefore, traditional statistical methods have encountered difficulties in addressing actual production realities. Developing models to predict product quality is a key aspect of product quality control.

[0003] In actual production, data from multiple sources is typically integrated for better quality prediction and analysis. However, with the implementation of privacy regulations in various countries worldwide, organizations in the field of smart manufacturing—which spans multiple units, production lines, and stages—are seeking to protect their data assets, leading to a serious data silo problem. These data silos and the implementation of various privacy regulations make it more difficult to integrate data from multiple sources for research. Current technologies cannot accurately predict product quality while ensuring data privacy. Summary of the Invention

[0004] Based on the above analysis, the embodiments of the present invention aim to provide a product quality prediction system based on federated learning, in order to solve the problem that existing product quality prediction joint training systems cannot protect data privacy.

[0005] This invention provides a product quality prediction system based on federated learning, comprising:

[0006] This includes servers and clients set up on multiple production lines; among which,

[0007] The server is used to send an initial product quality prediction model and fused model parameters to each production line client; the fused model parameters are obtained by fusing the model parameters sent by each production line client.

[0008] The production line client includes:

[0009] Multiple sensors are used to collect product manufacturing process data for this production line;

[0010] The training set construction module is used to acquire sensor data from this production line, extract features from the sensor data, construct a local input feature matrix, and construct a local training set based on the local input feature matrix and the product quality label of this production line.

[0011] The model training module is used to obtain an initial product quality prediction model, train the product quality prediction model using a forward propagation federated training method based on the local training set, encrypt and send the model parameters during the training process to the server, and update the local product quality prediction model according to the fused parameters; stop training when the training requirements are met to obtain a trained product quality prediction model.

[0012] The product quality prediction module is used to predict the quality of the product to be predicted based on the trained product quality prediction model.

[0013] The beneficial effects of the above technical solution are as follows: Each production line client trains the model locally, transmitting model parameters only through encryption. This ensures that the training samples for each production line do not leave the local machine, solving the problem of data silos, improving data privacy protection, and enabling accurate model training to enhance the accuracy of product quality prediction. The system structure is simple and easy to implement.

[0014] Based on further improvements to the above technical solution, the sensor includes an internal sensor and an external sensor. The internal sensor is used to collect information about the internal equipment of the manufacturing production line, and the external sensor is used to collect information about the external environment of the manufacturing production line.

[0015] The beneficial effects of the above technical solution are as follows: By setting up multiple internal and external sensors, data from the production line during the product manufacturing process can be comprehensively collected, thereby providing a data foundation for subsequent training of product quality prediction models.

[0016] Based on further improvements to the above technical solution, the product quality prediction model includes an input layer, a hidden layer, and an output layer connected in sequence; the hidden layer includes an LSTM network layer.

[0017] The beneficial effects of the above technical solution are as follows: By adding an LSTM network layer, the product prediction model can learn the temporal influence relationship of the production line, thereby further improving the accuracy of product quality prediction.

[0018] Based on further improvements to the above technical solution, the server fuses the model parameters according to the following formula.

[0019]

[0020] Where, N i Let θ be the training sample size for the i-th production line. i ′ represents the updated parameters of the i-th production line, θ represents the merged parameters, and n represents the number of production lines.

[0021] The beneficial effects of the above technical solution are as follows: The server uses a weighted average method to fuse the update parameters, and assigns different update parameter weights to different update parameters according to the training sample size of each production line, so that the update of model parameters is more accurate.

[0022] Based on further improvements to the above technical solution, the model parameters include the hidden states, cell states, and update parameters of the LSTM network layers, and the model training module includes:

[0023] The historical hidden state acquisition module is used for each production line client to obtain the hidden state of all its upstream production line clients at the previous moment from the server; and to aggregate the hidden state of the upstream production line clients at the previous moment with the hidden state of the current production line client at the previous moment to obtain historical hidden state information.

[0024] The historical cell state acquisition module is used for each production line client to obtain the cell state of its closest upstream production line client from the server at the previous moment; and to aggregate the cell state of the closest upstream production line client at the previous moment with the cell state of the current production line client at the previous moment to obtain historical cell state information.

[0025] The model parameter acquisition module is used to calculate the hidden state and cell state of the LSTM network layer of the current production line client, as well as the output result of the product quality prediction model, based on the historical hidden state information, the historical cell state information, and the local training set; and to calculate the update parameters of the product quality prediction model based on the output result and quality label of the model quality prediction model.

[0026] The model parameter sending module is used to send the hidden state, cell state, and update parameters of the LSTM network layer of the current production line client to the server.

[0027] The beneficial effects of the above technical solution are as follows: by acquiring the hidden state and cell state of the upstream production line through the historical hidden state acquisition module and the historical cell state module, each production line not only considers the historical state information of its own production line, but also the historical state information of the upstream production line when training the model. Thus, it not only learns the sequential influence relationship between the upstream and downstream production lines, but also learns the influence of the upstream production line's processing and manufacturing process on its own production line. This makes the trained product quality prediction model more accurate and enables more precise prediction of product quality.

[0028] Based on further improvements to the above technical solution, the training set construction module includes an initial feature extraction module, a Gini coefficient calculation module, and a feature matrix construction module; the server includes a common feature extraction module and a final feature determination module.

[0029] The initial feature extraction module is used by each production line client to extract features from the sensor data and send the extracted feature information to the server in an encrypted manner.

[0030] The common feature extraction module is used to extract the common features of each production line client based on the feature information sent by each production line client, and send them to each production line client in an encrypted manner.

[0031] The Gini coefficient calculation module is used to calculate the Gini coefficient of the common feature and send the Gini coefficient of the common feature to the server.

[0032] The final feature determination module is used to select the M common features with the smallest Gini coefficient as the final features and send them to each production line client.

[0033] The feature matrix construction module is used to construct a local input feature matrix based on the final features.

[0034] The beneficial effects of the above technical solution are as follows: by using the Gini coefficient of the calculated features, the feature with the smallest Gini coefficient is selected as the final feature. The smaller the Gini coefficient, the better the feature. This allows the most important features to be selected, making the established model more accurate, improving the accuracy of product quality prediction, and increasing the efficiency of training the product quality prediction model.

[0035] Based on the further improvement of the above technical solution, the model parameter acquisition module calculates the cell state of the LSTM network layer of the client of this production line at the current moment according to the following formula:

[0036]

[0037] in, Let be the output of the forget gate of the LSTM network layer at time t. This provides the historical cell state information for the i-th production line LSTM network layer at time t-1. and All values ​​represent the outputs of the input gates of the LSTM network layer for the i-th production line at time t. Let t represent the cell state of the i-th production line LSTM network layer.

[0038] Based on the further improvement of the above technical solution, the LSTM network layer includes an attention mechanism module, which is used to aggregate the hidden states of the upstream production line at the previous moment using the following formula:

[0039]

[0040]

[0041]

[0042] The LSTM network layer calculates the historical hidden state information according to the following formula:

[0043]

[0044] Where k represents the number of upstream production lines of the i-th production line. This represents the hidden state of the j-th production line at time t-1. Let w and u represent the weight coefficients and biases of the attention mechanism module in the j-th production line, respectively. q is the query vector, s is the attention scoring function, ω represents the learning parameters, and tanh is the activation function. Let v represent the historical hidden state information of the i-th production line at time t-1, v represent the aggregated hidden state of the upstream production line, and ω represent the hidden state of the upstream production line. T This represents the transpose of ω.

[0045] The beneficial effects of the above technical solution are as follows: By introducing an attention mechanism, the attention weights of the hidden states of the upstream production lines of each production line are calculated, thereby determining the most relevant upstream production lines, ignoring noise and redundancy, making the established model more accurate, improving the accuracy of product quality prediction, and improving the training efficiency of the product quality prediction model.

[0046] Based on the further improvement of the above technical solution, the model parameter acquisition module calculates the hidden state of the LSTM network layer of the current production line at the current moment according to the following formula:

[0047]

[0048] in, Let be the hidden state of the LSTM network layer of the i-th production line at time t. This provides the historical hidden state information of the i-th production line at time t-1. W is the input data for the model of the i-th production line at time t. o and b o These are the weights and biases of the output gates of the LSTM network layer for the i-th production line, respectively, σ o Here, sigmoid is the activation function, and tanh is the activation function. Let t represent the cell state of the i-th production line LSTM network layer.

[0049] Based on further improvements to the above technical solution, the model parameter acquisition module calculates the updated parameters of the product quality prediction model using the following formula:

[0050]

[0051]

[0052] θ′ i =θ i -Δθ i

[0053] in, This represents the quality label for the i-th production line during this training session. Let θ represent the output value of the model trained on the i-th production line in this iteration. i The loss represents the model parameters of the current model for the i-th production line. i Let denot θ represent the loss function of the i-th production line, m represent the number of training samples for the i-th production line in this training iteration, and Δθ represent the loss function of the i-th production line. i This indicates updating the gradient, θ i ' indicates updating parameters, Indicates the relationship with θ i Find the derivative.

[0054] In this invention, the above-described technical solutions can be combined with each other to achieve more preferred combinations. Other features and advantages of this invention will be set forth in the following description, and some advantages may become apparent from the description or be learned by practicing the invention. The objects and other advantages of this invention can be realized and obtained from what is particularly pointed out in the description and drawings. Attached Figure Description

[0055] The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Throughout the drawings, the same reference numerals denote the same parts.

[0056] Figure 1 This is a structural block diagram of a product quality prediction system for a manufacturing production line based on federated learning, according to an embodiment of the present invention. Detailed Implementation

[0057] Preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings, which form part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not intended to limit the scope of the present invention.

[0058] In actual production, data from multiple sources is typically integrated for better quality prediction and analysis. However, with the implementation of privacy regulations in various countries worldwide, organizations in the field of smart manufacturing—which spans multiple units, production lines, and stages—are seeking to protect their data assets, leading to a serious data silo problem. These data silos and the implementation of various privacy regulations make it more difficult to integrate data from multiple sources for research. Current technologies cannot accurately predict product quality while ensuring data privacy.

[0059] To address this, this application proposes a product quality prediction system based on federated learning, which enables multiple production lines to jointly train a product quality prediction model without the raw data leaving the local machine. This improves privacy while enhancing the accuracy of product quality prediction results.

[0060] One specific embodiment of this application discloses a production line product quality prediction system based on federated learning, such as... Figure 1 As shown, the system includes a server and clients located on multiple production lines. The server sends an initial product quality prediction model and fused model parameters to each production line client; the fused model parameters are obtained by fusing the model parameters sent by each production line client.

[0061] Production line clients include:

[0062] Multiple sensors are used to collect product manufacturing process data for this production line;

[0063] The training set construction module is used to acquire sensor data from this production line, extract features from the sensor data, construct a local input feature matrix, and construct a local training set based on the local input feature matrix and the product quality label of this production line.

[0064] The model training module is used to obtain an initial product quality prediction model, train the product quality prediction model using a forward propagation federated training method based on the local training set, encrypt and send the model parameters during the training process to the server, and update the local product quality prediction model according to the fused parameters; stop training when the training requirements are met to obtain a trained product quality prediction model.

[0065] The product quality prediction module is used to predict the quality of the product to be predicted based on the trained product quality prediction model.

[0066] Specifically, the sensors include internal sensors and external sensors. The internal sensors are used to collect information about the internal equipment of the manufacturing production line, and the external sensors are used to collect information about the external environment of the manufacturing production line.

[0067] For example, sensors are categorized into internal and external sensors based on the object being measured. Internal sensors primarily detect information such as position, speed, torque, temperature, and abnormal changes within the manufacturing production line; for instance, a speed sensor can be mounted on the drive shaft. External sensors primarily detect the external environmental conditions of the manufacturing production line, and they include corresponding contact sensors (tactile sensors, glide sensors, pressure sensors) and non-contact sensors (vision sensors, ultrasonic ranging, laser ranging). The types of sensors used in a manufacturing plant vary depending on the products being produced. They are generally used to measure the plant's production resources and production volume, and also to measure the performance of production machines, such as operating rate and failure rate. Their placement varies from factory to factory, and they are placed at various stages of the production chain as needed.

[0068] For example, during the manufacturing process of parts, the rotation of the electric motor, the movement of moving parts, and cutting all generate heat, and the temperature distribution is uneven, causing temperature differences. This can cause thermal deformation of the machine tool and affect the machining accuracy of the parts. Therefore, temperature sensors can be installed in certain parts of the machine tool to collect temperature data.

[0069] For example, a position sensor can be installed and configured on the inside of a robotic hand to detect the contact position between the robotic hand and an object. Another example is installing a speed sensor on a rotating component to measure its rotational speed.

[0070] To improve the effectiveness of the data, each client needs to perform data preprocessing before extracting features from the sensor data after acquiring it. Therefore, the production line client also includes a preprocessing module for data deduplication, sparse data processing, data balancing, and data dimensionality reduction.

[0071] For example, data deduplication can be achieved by removing duplicate sample data.

[0072] For example, sparse data processing involves removing overly sparse sample data, such as deleting feature dimensions with less than 10% of the total sample size, to ensure that the data is not too sparse.

[0073] During model training, data balancing is necessary to prevent the impact of data skew. For example, data from each production line can be balanced by extracting positive and negative samples at a certain ratio, such as a 2:1 ratio. This ensures that while balancing the data, a larger proportion of positive samples is retained from the original dataset.

[0074] The Spearman correlation coefficient was used to calculate the correlation of different dimensions. The Spearman correlation coefficient is a non-parametric index that characterizes the correlation. This method uses a monotonic equation to evaluate the correlation between two statistical variables. It does not require the distribution of the original variables and has a wide range of applications. Dimensions with high correlation contain relatively repetitive information, so one of them is retained for training.

[0075] Specifically, to maintain feature alignment across clients, the training set construction module includes an initial feature extraction module, a Gini coefficient calculation module, and a feature matrix construction module; the server includes a shared feature extraction module and a final feature determination module.

[0076] The initial feature extraction module of the client is used by each production line client to extract features from sensor data and send the extracted feature information to the server in an encrypted manner. For example, each production line client can encode the extracted features in a specific way, such as using a hash function to ensure that the feature code is identifiable while maintaining the anonymity of the feature. Each client sends the encoded features to the server in an encrypted manner.

[0077] The server's common feature extraction module is used to extract common features from the feature information sent by each production line client, and then send these features to each production line client in an encrypted manner. Specifically, after receiving the feature information sent by all production line clients, the server's common feature extraction module decodes the information, filters out the common features, and sends the filtered common features to each client in an encrypted manner.

[0078] The client-side Gini coefficient calculation module is used to calculate the Gini coefficient of the common features and send the Gini coefficient of the common features to the server. Specifically, according to the CART method of decision trees, the optimal feature is the feature with the smallest Gini coefficient. Therefore, each production line calculates the Gini coefficient for each common feature and sends the calculated Gini coefficient to the server.

[0079] The server's final feature determination module selects the M common features with the smallest Gini coefficients as the final features and sends them to each production line client. The server filters out the maximum value of the calculation results from each client as the final Gini coefficient of that common feature, and selects the M common features with the smallest final Gini coefficients as the final features to send to each production line. Here, M can be determined based on the actual training situation, for example, based on the training speed.

[0080] The client-side feature matrix construction module is used to construct a local input feature matrix based on the final features. Specifically, the local input feature matrix is ​​obtained by arranging the final features corresponding to each production line according to their Gini coefficients.

[0081] Since preceding processing steps may influence subsequent processing steps during product manufacturing, a product quality prediction model in one embodiment of the present invention includes an input layer, a hidden layer, and an output layer connected in sequence; the hidden layer includes an LSTM network layer. The LSTM network layer can realize state transfer, allowing the model to autonomously select the states to be memorized and learn long-term dependencies in the manufacturing process. The output layer includes a Linear layer, used to convert the output of the LSTM layer into a one-dimensional vector.

[0082] In implementation, the local training set can be divided into a training set and a test set as needed, used for model training and testing respectively. The initial model for each client is the same, a preliminary product quality prediction model constructed by the server based on the final features. During the first training, each production line client trains the model based on the training set and tests the model based on the test set. After the first training, each client sends the hidden states and cell states of the LSTM network layers to the server in an encrypted manner. Each production line client calculates the loss and update parameters of the product quality prediction model according to the loss function formula and update parameter calculation formula, and sends the calculated update parameters to the server in an encrypted manner. The server fuses all received update parameters, encrypts the fused parameters, and sends them to each client. Each client updates its local product quality prediction model based on the fused parameters. Specifically, the loss function and update parameter calculation formulas are as follows:

[0083]

[0084]

[0085] θ′ i =θ i -Δθ i (3)

[0086] in, This represents the quality label for the i-th production line during this training session. Let θ represent the output value of the model trained on the i-th production line in this iteration. i The loss represents the model parameters of the current model for the i-th production line. i Let denot θ represent the loss function of the i-th production line, m represent the number of training samples for the i-th production line in this training iteration, and Δθ represent the loss function of the i-th production line. i This indicates updating the gradient, θ′ i This indicates that the parameters are being updated. Indicates the relationship with θ i Find the derivative.

[0087] Since the upstream production line's processing and manufacturing process may affect the downstream production line's processing and manufacturing process, the model training modules of each client not only need to learn the long-term dependencies in the processing process of each production line itself, but also need to consider the interdependencies in the processing and manufacturing processes of the upstream and downstream production lines during the model training process. Therefore, the model training modules of each client include: historical hidden state acquisition module, historical cell state acquisition module, model parameter acquisition module, and model parameter sending module.

[0088] The historical hidden state acquisition module is used by each production line client to obtain the hidden states of all its upstream production lines at the previous moment from the server. The hidden states of the upstream production lines at the previous moment are then aggregated with the hidden states of the current production line to obtain historical hidden state information. The historical hidden state information obtained by each client includes not only the hidden state information of the current production line's previous training (i.e., the previous moment), but also the hidden state information of all its upstream production lines' previous training. By aggregating these, the historical hidden state information is obtained, thus learning not only the dependencies between the current production line's processing and manufacturing processes, but also the impact of the upstream production line's processing and manufacturing processes on the current production line. This comprehensive consideration of the interdependencies between the upstream and downstream production line processing and manufacturing processes makes the trained product quality prediction model more accurate and enables more precise product quality prediction.

[0089] Specifically, the LSTM network layer introduces an attention mechanism to learn and weightedly average the attention probability distribution values ​​of the hidden states of the upstream production line at the previous time step, thereby selectively filtering out a small amount of important information and focusing on this important information. Specifically, the LSTM network layer includes an attention mechanism module, which aggregates the hidden states of the upstream production line client at the previous time step using the following formula:

[0090]

[0091]

[0092]

[0093] The LSTM network layer is also used to aggregate the hidden state of the upstream production line client at the previous moment with the hidden state of the current production line client at the previous moment according to the following formula to calculate the historical hidden state information:

[0094]

[0095] Where k represents the number of upstream production line clients of the i-th production line client. This represents the hidden state of the j-th production line client at time t-1. Let w and u represent the weight coefficients and biases of the attention mechanism module in the j-th production line, respectively. q is the query vector, s is the attention scoring function, ω represents the learning parameters, and tanh is the activation function. Let v represent the historical hidden state information of the i-th production line client at time t-1, v represent the aggregated hidden state of the upstream production line client, and ω represent the historical hidden state information of the upstream production line client. T This represents the transpose of ω.

[0096] The historical cell state acquisition module is used by each production line client to obtain the cell state of its closest upstream production line client from the server at the previous moment. This cell state is then aggregated with the cell state of the current production line client at the previous moment to obtain historical cell state information. The historical cell state information obtained by each production line client includes not only the cell state information from its previous training (i.e., the previous moment) but also the cell state information from the previous training of its closest upstream production line. The cell state contains the model's long-term memory. By aggregating the cell state of the current production line and the cell state of the closest production line at the previous moment, the learning information from upstream manufacturers can be effectively utilized, combined with time-series production information, to improve the accuracy of product quality prediction.

[0097] Specifically, the historical cell state acquisition module obtains data according to the formula... The cell state of the LSTM network layer in the product quality prediction model of production line i at the previous time step. The cell state at the previous moment relative to the nearest upstream production line j Additive polymerization is performed to obtain the historical cell state.

[0098] The model parameter acquisition module is used to calculate the hidden state, cell state, and output result of the product quality prediction model of the LSTM network layer at the current time, based on the historical hidden state information, the historical cell state information, and the local training set. The module then calculates the update parameters of the product quality prediction model based on the output result and quality labels.

[0099] Specifically, the aggregated historical hidden state information Training data with production line i at the current time (time t) The input is fed into the forget gate of the LSTM network layer. The forget gate consists of a sigmoid neural network layer and a bitwise multiplication operation.

[0100]

[0101] Where σ f W is the activation function for the forget gate. f and bf , represent the weight coefficients and biases of the forget gate in the i-th production line LSTM network layer, respectively. This indicates the output of the forget gate.

[0102] The input gates of an LSTM network layer include sigmoid layers and tanh neural network layers, which are used to determine the information of the new input. and Which information will be retained?

[0103]

[0104]

[0105] in and The outputs of the sigmoid layer and the tanh neural network layer are respectively, σ I W represents the activation function. I ,b I Let W represent the weights and biases of the sigmoid layer at the input gate of the LSTM network layer for the i-th production line, respectively, where tanh represents the activation function. c ,b c These represent the weight coefficients and biases of the tanh neural network layer at the input gate of the LSTM network layer in the i-th production line, respectively.

[0106] The cell state of the LSTM network layer at the current time step is updated using the following formula:

[0107]

[0108] in, Let t represent the cell state of the i-th production line LSTM network layer.

[0109] The output gate of the LSTM network layer, together with the tanh function and bitwise multiplication operation, transmits the cell state and input signal to the output. Based on the output of the gate and the cell state, a new hidden state is obtained.

[0110]

[0111]

[0112] Right now

[0113] in, Let be the hidden state of the LSTM network layer of the i-th production line at time t. This provides the historical hidden state information of the i-th production line at time t-1. W is the input data for the model of the i-th production line at time t.o and b o These represent the weights and biases of the output gates of the LSTM network layer, respectively. For the output of the output gate, σ o Here, sigmoid is the activation function, and tanh is the activation function. Let t represent the cell state of the i-th production line LSTM network layer.

[0114] Specifically, the model parameter acquisition module of each production line client calculates the updated parameters of the local product quality prediction model according to formulas (1)-(3).

[0115] The model parameter sending module is used to send the hidden state, cell state, and update parameters of the LSTM network layer at the current time to the server. Specifically, it sends the hidden state at time t. Cell state and the updated parameters θ′ obtained after training i The encrypted message is sent to the server.

[0116] The server receives and stores the hidden states and cell states sent by clients from each production line, and then fuses all received updated parameters. Specifically, the server fuses the model parameters using the following formula:

[0117]

[0118] Where, N i Let θ′ be the number of training samples in the local training set of the i-th production line client. i Let θ represent the updated parameters for the i-th production line client, where θ represents the merged parameters and n represents the number of production line clients.

[0119] The server encrypts and sends the merged parameters to each production line client. The model training module of each production line client updates its local product quality prediction model according to the merged model parameters. In other words, the model training module of each production line client updates the parameters of its local product quality prediction model to the merged parameters.

[0120] Training ends when the training requirements are met. Optionally, training is met when the losses of all production lines no longer change significantly. After one training session, each production line model training module calculates the loss according to formula (1). If the losses of all production lines no longer change significantly, training ends and a well-trained product quality prediction model is obtained; otherwise, the next training session continues.

[0121] Optionally, the number of iterations can also be set. Once the set number of iterations is reached, the training requirements are met, and training stops to obtain a well-trained product quality prediction model.

[0122] The product quality prediction module for each production line client is used to predict the quality of the product to be predicted based on the trained product quality prediction model.

[0123] Specifically, each production line obtains data collected by sensors during the manufacturing process of the product to be predicted. Based on the final features used during training, the data undergoes feature extraction and dimensionality reduction, retaining only the final features used during training. Missing values ​​are handled and normalization preprocessing is performed to obtain a feature input matrix. The product quality prediction model trained with the feature input matrix of the product to be predicted is used to obtain the prediction result of the product quality of this production line.

[0124] Compared with existing technologies, the product quality prediction system based on federated learning provided by this invention has the following beneficial effects:

[0125] 1. Each production line client trains the model locally, transmitting model parameters only via encryption. This ensures that training samples for each production line do not leave the local machine, solving the problem of data silos, improving data privacy protection, and enabling accurate model training to enhance the accuracy of product quality prediction. The system structure is simple and easy to implement.

[0126] 2. By setting up multiple internal and external sensors, comprehensive data can be collected from the production line during the product manufacturing process, thus providing a data foundation for subsequent training of product quality prediction models.

[0127] 3. By using the Gini coefficient of the calculated features, the feature with the smallest Gini coefficient is selected as the final feature. The smaller the Gini coefficient, the better the feature. This allows the most important features to be selected, making the established model more accurate, improving the accuracy of product quality prediction, and increasing the efficiency of product quality prediction model training.

[0128] 4. By adding an LSTM network layer, the product prediction model can learn the temporal influence relationship of the production line, thereby further improving the accuracy of product quality prediction.

[0129] 5. By acquiring the hidden state and cell state of the upstream production line through the historical hidden state acquisition module and the historical cell state module, each production line not only considers the historical state information of its own production line, but also the historical state information of the upstream production line when training the model. In this way, it not only learns the sequential influence relationship between the upstream and downstream production lines, but also learns the influence of the upstream production line's processing and manufacturing process on its own production line. This makes the trained product quality prediction model more accurate and can more accurately predict product quality.

[0130] 6. By introducing an attention mechanism, the attention weights of the hidden states of the upstream production lines of each production line are calculated, thereby identifying the most relevant upstream production lines, ignoring noise and redundancy, making the established model more accurate, improving the accuracy of product quality prediction, and increasing the efficiency of product quality prediction model training.

[0131] 7. The server uses a weighted average method to fuse the updated parameters, and assigns different weights to different updated parameters according to the training sample size of each production line, so as to make the model parameter updates more accurate.

[0132] Those skilled in the art will understand that all or part of the processes of the methods described in the above embodiments can be implemented by a computer program instructing related hardware, and the program can be stored in a computer-readable storage medium. The computer-readable storage medium may be a disk, optical disk, read-only memory, or random access memory, etc.

[0133] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A product quality prediction system based on federated learning, characterized in that, This includes servers and clients set up on multiple production lines; among which, The server is used to send an initial product quality prediction model and fused model parameters to each production line client; the fused model parameters are obtained by fusing the model parameters sent by each production line client. The production line client includes: Multiple sensors are used to collect product manufacturing process data for this production line; The training set construction module is used to acquire sensor data from the production line, extract features from the sensor data, construct a local input feature matrix, and construct a local training set based on the local input feature matrix and the product quality label of the production line. The model training module is used to obtain an initial product quality prediction model, train the product quality prediction model using a forward propagation federated training method based on the local training set, encrypt and send the model parameters during the training process to the server, and update the local product quality prediction model according to the fused parameters; stop training when the training requirements are met to obtain a trained product quality prediction model. The product quality prediction module is used to predict the quality of the product to be predicted based on the trained product quality prediction model. The training set construction module includes an initial feature extraction module, a Gini coefficient calculation module, and a feature matrix construction module; the server includes a common feature extraction module and a final feature determination module. The initial feature extraction module is used by each production line client to extract features from the sensor data and send the extracted feature information to the server in an encrypted manner. The common feature extraction module is used to extract the common features of each production line client based on the feature information sent by each production line client, and send them to each production line client in an encrypted manner. The Gini coefficient calculation module is used to calculate the Gini coefficient of the common feature and send the Gini coefficient of the common feature to the server. Each production line calculates the Gini coefficient for each common feature. The final feature determination module is used to take the maximum value of the Gini coefficient calculated by each client as the final Gini coefficient of the common feature, and select the M common features with the smallest final Gini coefficient as the final features and send them to each production line client. The feature matrix construction module is used to construct a local input feature matrix based on the final features.

2. The product quality prediction system based on federated learning according to claim 1, characterized in that, The sensors include internal sensors and external sensors. The internal sensors are used to collect information about the internal equipment of the manufacturing production line, and the external sensors are used to collect information about the external environment of the manufacturing production line.

3. The product quality prediction system based on federated learning according to claim 1, characterized in that, The product quality prediction model includes an input layer, a hidden layer, and an output layer connected in sequence; the hidden layer includes an LSTM network layer.

4. The product quality prediction system based on federated learning according to claim 1, wherein the server fuses the model parameters according to the following formula: ； in, Let be the training sample size for the i-th production line client. For the updated parameters of the i-th production line client, This represents the parameters after merging, and n represents the number of production line clients.

5. The product quality prediction system based on federated learning according to claim 3, characterized in that, The model parameters include the hidden states, cell states, and update parameters of the LSTM network layers, and the model training module includes: The historical hidden state acquisition module is used for each production line client to obtain the hidden state of all its upstream production line clients at the previous moment from the server; and to aggregate the hidden state of the upstream production line clients at the previous moment with the hidden state of the current production line client at the previous moment to obtain historical hidden state information. The historical cell state acquisition module is used for each production line client to obtain the cell state of its closest upstream production line client from the server at the previous moment; and to aggregate the cell state of the closest upstream production line client at the previous moment with the cell state of the current production line client at the previous moment to obtain historical cell state information. The model parameter acquisition module is used to calculate the hidden state and cell state of the LSTM network layer of the current production line client, as well as the output result of the product quality prediction model, based on the historical hidden state information, the historical cell state information, and the local training set; and to calculate the update parameters of the product quality prediction model based on the output result of the product quality prediction model and the quality label. The model parameter sending module is used to send the hidden state, cell state, and update parameters of the LSTM network layer of the current production line client to the server.

6. The product quality prediction system based on federated learning according to claim 5, characterized in that, The model parameter acquisition module calculates the cell state of the LSTM network layer of the client in this production line at the current moment according to the following formula: ； in, Let t be the output of the forget gate of the LSTM network layer for the i-th production line client. This provides the historical cell state information of the LSTM network layer for the i-th production line client at time t-1. and All values ​​represent the outputs of the input gates of the LSTM network layer for the i-th production line client at time t. Let t represent the cell state of the LSTM network layer of the i-th production line client.

7. The product quality prediction system based on federated learning according to claim 5, characterized in that, The LSTM network layer includes an attention mechanism module, which aggregates the hidden state of the upstream production line client at the previous time step using the following formula: ； ； ； The LSTM network layer is also used to aggregate the hidden state of the upstream production line client at the previous moment with the hidden state of the current production line client at the previous moment according to the following formula to calculate the historical hidden state information: ； Where k represents the number of upstream production lines for the i-th production line client. This represents the hidden state of the j-th production line client at time t-1. For attention distribution, and Let represent the weight coefficients and biases of the attention mechanism module for the j-th production line client, respectively. Let be the query vector, and s be the attention scoring function. Represents the learning parameters, For activation function, This represents the historical hidden state information of the i-th production line client at time t-1. This indicates the aggregate hidden state of the upstream production line client. express The transpose of .

8. The product quality prediction system based on federated learning according to claim 5, characterized in that, The model parameter acquisition module calculates the hidden state of the LSTM network layer of the current production line at the current moment according to the following formula: ； in, Let be the hidden state of the LSTM network layer of the i-th production line client at time t. This provides the historical hidden state information of the i-th production line client at time t-1. Let be the input data for the model of the i-th production line client at time t. and These represent the weight coefficients and biases of the output gates of the LSTM network layer for the i-th production line client. It is the sigmoid activation function. for Activation function Let t represent the cell state of the LSTM network layer of the i-th production line client.

9. The product quality prediction system based on federated learning according to claim 5, characterized in that, The model parameter acquisition module calculates the updated parameters of the product quality prediction model using the following formula: ； ； ； in, This represents the quality label of the i-th production line client during this training. This represents the model output value trained by the i-th production line client in this iteration. This represents the model parameters of the current model for the i-th production line client. Let represent the loss function for the i-th production line client, and m represent the number of training samples for the i-th production line client in this training iteration. This indicates that the gradient is being updated. This indicates that the parameters are being updated. Indicates to Find the derivative.

Images