A method for product quality prediction in manufacturing production lines based on federated learning

Abstract

A method for predicting product quality in a manufacturing production line based on federated learning comprises: obtaining sensor data from the product manufacturing process of each production line, performing feature extraction on the sensor data, and constructing a local input feature matrix for each production line; each production line constructing a local training set based on the local input feature matrix and the product quality label of the production line; a server sending an initial product quality prediction model to each production line, each production line training the product quality prediction model based on the local training set using a federated training method of forward propagation, and encrypting the model parameters during the training process and sending them to the server; the server fusing all model parameters; each production line updating the local product quality prediction model based on the fused model parameters; stopping training to obtain a trained product quality prediction model when training requirements are met; and performing quality prediction on the product to be predicted based on the trained product quality prediction model.

Description

Technical Field

[0001] The present invention relates to the technical field of product quality prediction for manufacturing production lines, and in particular to a method for predicting product quality for manufacturing production lines based on federated learning. Background Art

[0002] With the development and progress of the manufacturing industry, the production processes of manufacturing companies have become increasingly complex, exceeding the general statistical rules. Therefore, traditional statistical methods have run into difficulties in the face of production reality. Predicting product quality by building models is the key content of product quality control.

[0003] In production, data from multiple sources is often integrated to better predict quality analysis. With the implementation of privacy regulations in countries around the world, organizations across multiple units, production lines, and processes in smart manufacturing are eager to protect their data assets, leading to a serious data silo problem. Data silos and the implementation of various privacy regulations make it more difficult to integrate data from multiple sources for research. Existing technologies cannot accurately predict product quality while protecting data privacy. Summary of the Invention

[0004] In view of the above analysis, an embodiment of the present invention aims to provide a method for predicting product quality of a manufacturing production line based on federated learning, so as to solve the problem that the existing joint training product quality prediction model cannot protect the privacy of data.

[0005] An embodiment of the present invention provides a method for predicting product quality of a manufacturing production line based on federated learning, comprising the following steps:

[0006] Acquire sensor data from the product manufacturing process on each production line, perform feature extraction on the sensor data, and construct a local input feature matrix for each production line. Each production line then constructs a local training set based on the local input feature matrix and the product quality labels of that production line.

[0007] The server sends an initial product quality prediction model to each production line. Each production line trains the product quality prediction model using a federated training method based on a local training set, and encrypts the model parameters during training and sends them to the server. The server fuses all model parameters. Each production line updates its local product quality prediction model based on the fused model parameters. When the training requirements are met, training stops, resulting in a trained product quality prediction model.

[0008] Based on the trained product quality prediction model, quality prediction is performed on the product to be predicted.

[0009] The beneficial effects of the above technical solution are as follows: by adopting the federated training method of forward propagation, each production line trains the model locally and only transmits the model parameters in an encrypted manner, so that the training samples of this production line do not leave the local area, solving the problem of data silos, improving data privacy protection, and accurately training the model, thereby improving the accuracy of product quality prediction.

[0010] Based on a further improvement of the above technical solution, the feature extraction of the sensor data and the construction of a local input feature matrix include:

[0011] Each production line extracts features from the sensor data and sends the extracted feature information to the server in an encrypted manner; the server obtains the common features of each production line and sends them to each production line in an encrypted manner;

[0012] Each production line calculates the Gini coefficient of the common features and sends the Gini coefficients of the common features to the server. The server selects M common features with the smallest Gini coefficients as final features and sends them to each production line.

[0013] Each production line constructs a local input feature matrix according to the final features.

[0014] The beneficial effects of the above technical solution are as follows: by calculating the Gini coefficient of the features, the feature with the smallest Gini coefficient is selected as the final feature. The smaller the Gini coefficient, the better the feature, thereby screening out the most important features, making the established model more accurate, improving the accuracy of product quality prediction, and improving the efficiency of product quality prediction model training.

[0015] Based on the further improvement of the above technical solution, the product quality prediction model includes an LSTM network layer; the model parameters include the hidden state, cell state and update parameters of the LSTM network layer.

[0016] Based on the further improvement of the above technical solution, each production line trains the product quality prediction model using a federated training method of forward propagation based on a local training set, and sends the trained model parameters to the server, including:

[0017] Each production line obtains the hidden states of all its upstream production lines at the previous moment from the server; the hidden states of the upstream production lines at the previous moment are aggregated with the hidden state of the current production line at the previous moment to obtain historical hidden state information;

[0018] Each production line obtains the cell state of its most recent upstream production line from the server; aggregates the cell state of the most recent upstream production line with the cell state of the current production line to obtain historical cell state information;

[0019] Based on the historical hidden state information, the historical cell state information and the local training set, the hidden state and cell state of the LSTM network layer of the current production line and the output result of the product quality prediction model are calculated;

[0020] Calculate the updated parameters of the product quality prediction model based on the output results and quality labels of the model quality prediction model; send the hidden state, cell state and updated parameters of the LSTM network layer of the current production line to the server.

[0021] The beneficial effects of the above technical solution are as follows: by adopting the forward propagation federated training method to train the product quality prediction model, each production line not only considers the historical status information of the production line when training the model, but also considers the historical status information of the upstream production line, thereby not only learning the before and after time sequence influence relationship of the production line, but also learning the influence of the processing and manufacturing process of the upstream production line on the production line, thereby making the trained product quality prediction model more accurate and able to predict product quality more accurately.

[0022] Based on the further improvement of the above technical solution, the LSTM network layer includes an attention mechanism module, which is used to aggregate the hidden state of the upstream production line at a moment through the following formula:

[0023]

[0024]

[0025] The LSTM network layer calculates the historical hidden state information according to the following formula:

[0026]

[0027] Where k represents the number of upstream production lines of the i-th production line, represents the hidden state of the j-th production line at time t-1, is the attention distribution, w and u represent the weight coefficient and bias of the attention mechanism module of the j-th production line, q is the query vector, s is the attention scoring function, ω represents the learning parameter, tanh is the activation function, represents the historical hidden state information of the i-th production line at time t-1, v represents the aggregated hidden state of the upstream production line, ω T represents the transpose of ω.

[0028] The beneficial effects of the above technical solution are as follows: by introducing the attention mechanism, the attention weight of the hidden state of the upstream production line of each production line is calculated, and then the most relevant upstream production line can be determined, ignoring the noise and redundancy therein, making the established model more accurate, improving the accuracy of product quality prediction and improving the efficiency of product quality prediction model training.

[0029] Based on the further improvement of the above technical solution, the cell state at the last moment of the closest upstream production line is aggregated with the cell state at the last moment of the current production line according to the following formula to obtain the historical cell state information in, represents the cell state of production line i at the previous moment, Indicates the cell state of the most upstream production line j of production line i at the previous moment.

[0030] The beneficial effects of the above technical solution are as follows: by aggregating the cell status of the current production line at a certain moment and the cell status of the closest production line at a certain moment, the learning information of upstream manufacturers can be effectively utilized, combined with the time-series production information, to improve the accuracy of product quality prediction.

[0031] Based on the further improvement of the above technical solution, the cell state of the LSTM network layer of this production line at the current moment is calculated according to the following formula:

[0032]

[0033] in, is the output of the forget gate of the LSTM network layer of the i-th production line at time t, is the historical cell state information of the LSTM network layer of the i-th production line at time t-1, and are the outputs of the input gate of the LSTM network layer of the i-th production line at time t, is the cell state of the LSTM network layer of the i-th production line at time t.

[0034] Based on the further improvement of the above technical solution, the hidden state of the LSTM network layer of this production line at the current moment is calculated according to the following formula:

[0035]

[0036] in, is the hidden state of the LSTM network layer of the i-th production line at time t, is the historical hidden state information of the i-th production line at time t-1, is the input data of the model of the i-th production line at time t, W o and b oare the weight coefficient and bias of the output gate of the LSTM network layer of the i-th production line, σ o is the sigmoid activation function, tanh is the tanh activation function, is the cell state of the LSTM network layer of the i-th production line at time t.

[0037] Based on the further improvement of the above technical solution, the updated parameters of the product quality prediction model are calculated by the following formula:

[0038]

[0039]

[0040] θ′ i =θ i -Δθ i

[0041] in, represents the quality label of the i-th production line in this training, represents the model output value of the i-th production line during this training, θ i Represents the model parameters of the current model of the i-th production line, loss i represents the loss function of the i-th production line, m represents the number of training samples for this training of the i-th production line, Δθ i represents the updated gradient, θ′ i Indicates the update parameters, Represents theta i Find the derivative.

[0042] Based on the further improvement of the above technical solution, the server uses the following formula to fuse the model parameters

[0043]

[0044] Among them, N i is the training sample size of the i-th production line, θ′ i is the updated parameter of the i-th production line, θ represents the fused parameter, and n represents the number of production lines.

[0045] The beneficial effects of the above technical solution are as follows: the update parameters are integrated by weighted averaging, and different update parameter weights are assigned to different update parameters according to the training sample size of each production line, so that the update of model parameters is more accurate.

[0046] In the present invention, the above-mentioned technical solutions can be combined with each other to achieve more preferred combinations. Other features and advantages of the present invention will be described in the following description, and some advantages will become apparent from the description or be learned through practice of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the contents particularly pointed out in the description and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] The accompanying drawings are only for the purpose of illustrating particular embodiments and are not to be considered limiting of the present invention. Like reference symbols denote like parts throughout the drawings.

[0048] Figure 1 This is a flowchart of a method for predicting product quality of a manufacturing production line based on federated learning according to an embodiment of the present invention. DETAILED DESCRIPTION

[0049] The preferred embodiments of the present invention will be described in detail below in conjunction with the accompanying drawings, wherein the accompanying drawings constitute a part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, and are not used to limit the scope of the present invention.

[0050] In actual production, product development often involves multiple organizations, often integrating data from multiple sources to better predict and analyze quality. With the implementation of privacy regulations in countries around the world, organizations across multiple units, production lines, and processes in smart manufacturing are eager to protect their data assets, leading to a serious data silo problem. Data silos and the implementation of various privacy regulations make it even more difficult to integrate data from multiple sources for research. Existing technologies cannot accurately predict product quality while protecting data privacy.

[0051] To this end, this application proposes a method for product quality prediction of manufacturing production lines based on federated learning, which can jointly train product quality prediction models on multiple production lines without leaving the local area, thereby improving privacy while improving the accuracy of product quality prediction results.

[0052] A specific embodiment of the present application discloses a method for predicting product quality of a manufacturing production line based on federated learning, such as Figure 1 As shown, the following steps are included:

[0053] S1. Obtain sensor data from the product manufacturing process of each production line, perform feature extraction on the sensor data, and construct a local input feature matrix for each production line; each production line constructs a local training set based on the local input feature matrix and the product quality labels of the production line.

[0054] Sensors are divided into internal sensors and external sensors according to the objects being measured. Internal sensors mainly detect information such as position, speed, torque, temperature, and abnormal changes within the manufacturing production line. For example, speed sensors can be installed on the driving shaft. External sensors mainly detect the external environmental status of the manufacturing production line. They have corresponding contact types (tactile sensors, sliding sensors, pressure sensors) and non-contact types (visual sensors, ultrasonic ranging, laser ranging). The types of sensors in a production plant vary depending on the type of product produced. They are generally used to measure the factory's production resources and production volume, as well as to measure the performance of production machines, such as operating rate, failure rate, etc. The setting location varies from factory to factory and is set in various links of the production chain as appropriate.

[0055] For example, during the parts manufacturing process, the rotation of the motor, the movement of moving parts, and cutting will generate heat, and the temperature distribution is uneven, resulting in temperature differences, causing thermal deformation of the processing machine tools and affecting the processing accuracy of the parts. Therefore, temperature sensors can be installed on certain parts of the processing machine tools to collect temperature data.

[0056] For example, a position sensor is installed and configured on the inner side of the robot palm to detect the contact position between the robot palm and an object. Another example is that a speed sensor is installed on a rotating part to measure the rotation speed of the rotating part.

[0057] In order to improve the effectiveness of data, data preprocessing is required before feature extraction of sensor data. Preprocessing includes data deduplication, sparse data processing, data balancing, and data dimensionality reduction.

[0058] Exemplarily, data deduplication may be the deletion of duplicate sample data.

[0059] Exemplarily, sparse data processing involves removing sample data that is too sparse, such as deleting feature dimensions whose effective data volume is less than about 10% of the total samples, to ensure that the data is not too sparse.

[0060] During model training, data balancing is necessary to prevent the effects of data skew. For example, the data from each production line is balanced, with positive and negative examples extracted in a certain ratio, such as a 2:1 ratio. This ensures that the positive examples in the original dataset are retained while maintaining a high proportion of positive examples.

[0061] The Spearman correlation coefficient is used to calculate the correlation of different dimensions. The Spearman correlation coefficient is a non-parametric indicator that characterizes correlation. This method uses a monotonic equation to evaluate the correlation between two statistical variables. It does not require the distribution of the original variables and has a wide range of applications. Dimensions with high correlation contain relatively repeated information, and one of them is retained for training.

[0062] Specifically, the sensor data is subjected to feature extraction to construct a local input feature matrix, including:

[0063] Each production line extracts features from the sensor data and sends the extracted feature information to the server in an encrypted manner. For example, each production line may encode the extracted features in a specific manner, such as using a hash function to ensure that the feature code is recognizable while maintaining the anonymity of the features. Each production line sends the encoded features to the server in an encrypted manner.

[0064] The server obtains the common features of each production line and sends them to each production line in an encrypted manner. Specifically, after receiving the feature information sent by all production lines, the server selects the common features and sends the selected common features to each production line in an encrypted manner.

[0065] Each production line calculates the Gini coefficient of the common features and sends the Gini coefficients of the common features to the server. The server selects the M common features with the smallest Gini coefficients as the final features and sends them to each production line. Specifically, according to the CART method of the decision tree, the optimal feature is the feature with the smallest Gini coefficient. Therefore, each production line calculates the Gini coefficient for each common feature and sends the calculated Gini coefficient to the server. For each common feature, the server selects the maximum value of the calculation results of each production line as the final Gini coefficient of the common feature, selects the M common features with the smallest final Gini coefficients as the final features and sends them to each production line. Among them, M can be determined according to the actual training situation, for example, it can be determined according to the training speed.

[0066] Each production line constructs a local input feature matrix based on the final features. Specifically, the final features corresponding to each production line are arranged according to the size of the Gini coefficient to obtain the local input feature matrix.

[0067] S2. The server sends an initial product quality prediction model to each production line. Each production line trains the product quality prediction model based on the local training set using a federated training method of forward propagation, and encrypts the model parameters during the training process and sends them to the server. The server fuses all model parameters. Each production line updates the local product quality prediction model based on the fused model parameters. When the training requirements are met, the training is stopped to obtain a trained product quality prediction model.

[0068] Since the preceding processing and manufacturing process may affect the subsequent processing process during the product processing and manufacturing process, the product quality prediction model of an embodiment of the present invention includes an input layer, an LSTM network layer, a Linear layer, and an output layer. The LSTM network layer can realize state transfer, allowing the model to autonomously select the states that need to be remembered and learn the long-term dependencies in the processing and manufacturing process. The Linear layer is used to convert the output of the LSTM layer into a one-dimensional vector.

[0069] During implementation, the local training set can be divided into a training set and a test set as needed, which are used for model training and testing respectively. The initial model of each production line is the same. During the first training, each production line trains the model based on the training set and tests the model based on the test set. After the first training, each production line sends the hidden state and cell state of the LSTM network layer to the server in an encrypted manner. Each production line calculates the loss and update parameters of the product quality prediction model according to the loss function formula and the update parameter calculation formula, and encrypts the calculated update parameters and sends them to the server. The server fuses all the received update parameters, encrypts the fused parameters and sends them to each production line, and each production line updates the local product quality prediction model according to the fused parameters. Specifically, the loss function and update parameter calculation formulas are as follows:

[0070]

[0071]

[0072] θ′ i =θ i -Δθ i (3)

[0073] in, represents the quality label of the i-th production line in this training, represents the model output value of the i-th production line during this training, θ i Represents the model parameters of the current model of the i-th production line, loss i represents the loss function of the i-th production line, m represents the number of training samples for this training of the i-th production line, Δθ i represents the updated gradient, θ′ i Indicates the update parameters, Represents theta i Find the derivative.

[0074] Since the processing and manufacturing process of products on the upstream production line may affect the processing and manufacturing process of products on the downstream production line, during the model training process, it is necessary not only to learn the long-term dependencies in the processing of each production line itself, but also to consider the mutual dependencies between the processing and manufacturing processes of the upstream and downstream production lines. Therefore, each production line uses the federated training method of forward propagation based on the local training set to train the product quality prediction model and send the trained model parameters to the server, including:

[0075] S21. Each production line obtains the hidden states of all its upstream production lines at the previous moment from the server; the hidden states of the upstream production lines at the previous moment are aggregated with the hidden states of the current production line at the previous moment to obtain historical hidden state information. The historical hidden state information obtained by each production line includes not only the hidden state information of the previous training (i.e., the previous moment) of the current production line, but also the hidden state information of the previous training of all its upstream production lines. Aggregating the historical hidden state information not only learns the dependencies between the processing and manufacturing processes of the current production line, but also learns the impact of the processing and manufacturing processes of its upstream growth lines on the current production line. This comprehensively considers the interdependencies between the processing and manufacturing processes of upstream and downstream production lines, making the trained product quality prediction model more accurate and enabling more precise product quality prediction.

[0076] Specifically, the LSTM network layer introduces an attention mechanism to learn and weighted average the attention probability distribution value of the hidden state of the upstream production line at a certain moment, thereby selectively filtering out a small amount of important information and focusing on this important information. Specifically, the LSTM network layer includes an attention mechanism module, which is used to aggregate the hidden state of the upstream production line at a certain moment using the following formula:

[0077]

[0078]

[0079]

[0080] The LSTM network layer calculates the historical hidden state information according to the following formula:

[0081]

[0082] Where k represents the number of upstream production lines of the i-th production line, represents the hidden state of the j-th production line at time t-1, is the attention distribution, w and u represent the weight coefficient and bias of the attention mechanism module of the j-th production line, q is the query vector, s is the attention scoring function, ω represents the learning parameter, tanh is the activation function, represents the historical hidden state information of the i-th production line at time t-1, v represents the aggregated hidden state of the upstream production line, ω T represents the transpose of ω.

[0083] S22. Each production line obtains the cell state of the most recent upstream production line from the server; the cell state of the most recent upstream production line is aggregated with the cell state of the current production line to obtain historical cell state information. The historical cell state information obtained by each production line includes not only the cell information of the last training (i.e., the last moment) of the current production line, but also the cell state information of the last training of the most recent upstream production line, and the historical cell state information is aggregated. The cell state contains the long-term memory of the model. By aggregating the cell state of the current production line at the last moment and the cell state of the most recent production line at the last moment, the learning information of the upstream manufacturer can be effectively utilized, combined with the time-series production information, to improve the accuracy of product quality prediction.

[0084] Specifically, the cell state of the LSTM network layer at the previous moment in the product quality prediction model of production line i is transformed into The cell state of the closest upstream production line j at the previous moment Perform additive aggregation to obtain historical cell states in, represents the cell state of production line i at the previous moment, Indicates the cell state of the most upstream production line j of production line i at the previous moment.

[0085] S23. Based on the historical hidden state information, the historical cell state information and the local training set, calculate the hidden state, cell state of the LSTM network layer of the current production line and the output result of the product quality prediction model.

[0086] Specifically, the aggregated historical hidden state information and the current moment (time t) training data of production line i Input to the forget gate. The forget gate consists of a sigmoid neural network layer and a bitwise multiplication operation.

[0087]

[0088] where σ f is the activation function of the forget gate, W f and b f , respectively represent the weight coefficient and bias of the forget gate of the LSTM network layer of the i-th production line, Represents the output of the forget gate.

[0089] The input gate of the LSTM network layer includes a sigmoid layer and a tanh neural network layer, which are used to determine the new input information. and What information will be retained.

[0090]

[0091]

[0092] in and are the outputs of the sigmoid layer and the tanh neural network layer, σ I represents the activation function, W I , b I They represent the weight coefficient and bias of the sigmoid layer of the input gate of the LSTM network layer of the i-th production line, tanh represents the activation function, and W c ,b c They represent the weight coefficient and bias of the tanh neural network layer of the input gate of the LSTM network layer of the i-th production line respectively.

[0093] Update the cell state of the LSTM network layer at the current moment using the following formula:

[0094]

[0095] in, is the cell state of the LSTM network layer of the i-th production line at time t.

[0096] The output gate of the LSTM network layer works together with the tanh function and the bitwise multiplication operation to pass the cell state and input signal to the output end. According to the output of the output gate and the cell state, a new hidden state is obtained:

[0097]

[0098]

[0099] Right now

[0100] in, is the hidden state of the LSTM network layer of the i-th production line at time t, is the historical hidden state information of the i-th production line at time t-1, is the input data of the model of the i-th production line at time t, W o and b o are the weight coefficient and bias of the output gate of the LSTM network layer, is the output of the output gate, σ ois the sigmoid activation function, tanh is the tanh activation function, is the cell state of the LSTM network layer of the i-th production line at time t.

[0101] S24. Calculate the updated parameters of the product quality prediction model based on the output results and quality labels of the model quality prediction model; send the hidden state, cell state and updated parameters of the LSTM network layer of the current production line to the server.

[0102] Specifically, each production line calculates the update parameters of the local product quality prediction model according to formulas (1)-(3), and each production line sets the hidden state at time t to Cell state And the updated parameters θ′ obtained after training i Encrypted and sent to the server.

[0103] The server receives the hidden state and cell state sent by each production line and stores them, and fuses all the updated parameters received. Specifically, the server uses the following formula to fuse the model parameters:

[0104]

[0105] Among them, N i is the number of training samples in the local training set of the i-th production line, θ′ i is the updated parameter of the i-th production line, θ represents the fused parameter, and n represents the number of production lines.

[0106] The server encrypts the fused parameters and sends them to each production line. Each production line updates the local product quality prediction model based on the fused model parameters. That is, each production line updates the parameters of the local product quality prediction model to the fused parameters.

[0107] When the training requirements are met, the training ends. Optionally, the training requirements are met when the losses of all production lines no longer change significantly. After a training run, the losses of each production line can be calculated according to formula (1). If the losses of all production lines no longer change significantly, the training ends and a trained product quality prediction model is obtained. Otherwise, the next training run continues.

[0108] Optionally, the number of iterations can also be set. When the set number of iterations is reached, the training requirements are met and the training is stopped to obtain a trained product quality prediction model.

[0109] S3. Based on the trained product quality prediction model, perform quality prediction on the product to be predicted.

[0110] Each production line obtains data collected by sensors during the manufacturing process of the product to be predicted. Based on the final features used in training, feature extraction and feature dimensionality reduction are performed on the data, retaining only the final features used in training. Missing value processing and normalization preprocessing are performed to obtain a feature input matrix. The feature matrix of the product to be predicted is input into the trained product quality prediction model to obtain the prediction results of the product quality of this production line.

[0111] Compared with the existing technology, the manufacturing production line product quality prediction method based on federated learning provided by the present invention has the following beneficial effects:

[0112] 1. Each production line conducts model training locally. The training data does not need to be sent to the server. Only the model parameters are sent to the server in an encrypted manner. This allows multiple production lines to jointly train product quality prediction models without leaving the local area. This solves the data silo problem of cross-production line and cross-regional joint training, improves privacy, and improves the accuracy of product quality prediction results.

[0113] 2. By calculating the Gini coefficient of the features, the feature with the smallest Gini coefficient is selected as the final feature. The smaller the Gini coefficient, the better the feature, thereby screening out the most important features, making the established model more accurate, improving the accuracy of product quality prediction, and improving the efficiency of product quality prediction model training.

[0114] 3. By fusing the hidden state and cell state of the upstream production line with the hidden state and cell state of the current production line, we can learn the impact of the upstream production line's production process on the current production line's production process, making the trained product quality prediction model more accurate and improving the accuracy of the prediction.

[0115] 4. By introducing the attention mechanism, the attention weights of the hidden states of the upstream production lines of each production line are calculated, and then the most relevant upstream production line can be determined, ignoring the noise and redundancy therein, making the established model more accurate, improving the accuracy of product quality prediction, and improving the efficiency of product quality prediction model training.

[0116] 5. Use weighted averaging to fuse the update parameters, and assign different update parameter weights to different update parameters according to the number of training samples of each production line, so that the update of model parameters is more accurate.

[0117] Those skilled in the art will appreciate that all or part of the process steps of the above-described embodiments can be implemented by instructing related hardware through a computer program, and the program can be stored in a computer-readable storage medium, such as a magnetic disk, an optical disk, a read-only memory, or a random access memory.

[0118] The above description is only a preferred specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by any technician familiar with this technical field within the technical scope disclosed by the present invention should be covered by the scope of protection of the present invention.

Claims

1. A method for predicting product quality of a manufacturing production line based on federated learning, characterized in that: The following steps are involved: Acquire sensor data from the product manufacturing process on each production line, perform feature extraction on the sensor data, and construct a local input feature matrix for each production line. Each production line then constructs a local training set based on the local input feature matrix and the product quality labels of that production line. The server sends an initial product quality prediction model to each production line. Each production line trains the product quality prediction model using a federated forward propagation training method based on a local training set and encrypts the model parameters during the training process and sends them to the server. The server then fuses all model parameters. Each production line updates its local product quality prediction model based on the fused model parameters. When the training requirements are met, the training is stopped to obtain a trained product quality prediction model. Based on the trained product quality prediction model, quality prediction is performed on the product to be predicted; The product quality prediction model includes an LSTM network layer; the model parameters include the hidden state, cell state and update parameters of the LSTM network layer; Each production line trains the product quality prediction model using a federated training method of forward propagation based on a local training set, and sends the trained model parameters to the server, including: Each production line obtains the hidden states of all its upstream production lines at the previous moment from the server; the hidden states of the upstream production lines at the previous moment are aggregated with the hidden state of the current production line at the previous moment to obtain historical hidden state information; Each production line obtains the cell state of its most recent upstream production line from the server; aggregates the cell state of the most recent upstream production line with the cell state of the current production line to obtain historical cell state information; Based on the historical hidden state information, the historical cell state information and the local training set, the hidden state and cell state of the LSTM network layer of the current production line and the output result of the product quality prediction model are calculated; Calculate the updated parameters of the product quality prediction model based on the output results and quality labels of the product quality prediction model; send the hidden state, cell state and updated parameters of the LSTM network layer of the current production line to the server.

2. The method for predicting product quality of a manufacturing production line based on federated learning according to claim 1, characterized in that: The extracting features from the sensor data and constructing a local input feature matrix includes: Each production line extracts features from the sensor data and sends the extracted feature information to the server in an encrypted manner; the server obtains the common features of each production line and sends them to each production line in an encrypted manner; Each production line calculates the Gini coefficient of the common features and sends the Gini coefficients of the common features to the server. The server selects M common features with the smallest Gini coefficients as final features and sends them to each production line. Each production line constructs a local input feature matrix according to the final features.

3. The method for predicting product quality of a manufacturing production line based on federated learning according to claim 1, characterized in that: The LSTM network layer includes an attention mechanism module, which is used to aggregate the hidden state of the upstream production line at a certain moment using the following formula: The LSTM network layer calculates the historical hidden state information according to the following formula: Where k represents the number of upstream production lines of the i-th production line, represents the hidden state of the j-th production line at time t-1, is the attention distribution, w and u represent the weight coefficient and bias of the attention mechanism module of the j-th production line, q is the query vector, s is the attention scoring function, ω represents the learning parameter, tanh is the activation function, represents the historical hidden state information of the i-th production line at time t-1, v represents the aggregated hidden state of the upstream production line, ω T represents the transpose of ω.

4. The method for predicting product quality of a manufacturing production line based on federated learning according to claim 1, characterized in that: The cell state at the last moment of the closest upstream production line is aggregated with the cell state at the last moment of the current production line according to the following formula to obtain historical cell state information: in, represents the cell state of production line i at the previous moment, Indicates the cell state of the most upstream production line j of production line i at the previous moment.

5. The method for predicting product quality of a manufacturing production line based on federated learning according to claim 1, characterized in that: The cell state of the LSTM network layer of this production line at the current moment is calculated according to the following formula: Among them, f t i is the output of the forget gate of the LSTM network layer of the i-th production line at time t, is the historical cell state information of the LSTM network layer of the i-th production line at time t-1, and are the outputs of the input gate of the LSTM network layer of the i-th production line at time t, is the cell state of the LSTM network layer of the i-th production line at time t.

6. The method for predicting product quality of a manufacturing production line based on federated learning according to claim 1, characterized in that: The hidden state of the LSTM network layer of this production line at the current moment is calculated according to the following formula: in, is the hidden state of the LSTM network layer of the i-th production line at time t, is the historical hidden state information of the i-th production line at time t-1, is the input data of the model of the i-th production line at time t, W o and b o are the weight coefficient and bias of the output gate of the LSTM network layer of the i-th production line, σ o is the sigmoid activation function, tanh is the tanh activation function, is the cell state of the LSTM network layer of the i-th production line at time t.

7. The method for predicting product quality of a manufacturing production line based on federated learning according to claim 1, characterized in that: The updated parameters of the product quality prediction model are calculated using the following formula: θ′ i =θ i -Dth i in, represents the quality label of the i-th production line in this training, represents the model output value of the i-th production line during this training, θ i Represents the model parameters of the current model of the i-th production line, loss i represents the loss function of the i-th production line, m represents the number of training samples for this training of the i-th production line, Δθ i represents the updated gradient, θ′ i Indicates the update parameters, Represents theta i Find the derivative.

8. According to the method for predicting product quality of a manufacturing production line based on federated learning in claim 1, the server fuses the model parameters using the following formula: in, N i is the training sample size of the i-th production line, θ i ′ is the updated parameter of the i-th production line, θ represents the fused parameter, and n represents the number of production lines.

Images