A product assembly quality prediction method and system based on vertical federated learning

Abstract

The application discloses a product assembly quality prediction method and system based on longitudinal federal learning, wherein data samples of each participant are subjected to sample alignment under an encryption strategy, solving the problems of data decentralization, unevenness and scarcity. A multi-party parallel structure of the local model of each participant adopts a fixed data partition strategy to partition product assembly data, the encoder part first normalizes the partitioned data in layers, and features are extracted through a multi-head attention layer, so as to mine the correlation between each assembly production line and each equipment assembly data in the participant, and the purpose of enhancing the feature extraction capability of the model is achieved. A homomorphic encryption method of multi-party secure computation is used to perform gradient secure aggregation on the local model of each participant, and a global model is obtained, so that the goal of jointly training a high-precision assembly quality prediction model by using the data of multiple sub-plants is achieved, and the product service research and development auxiliary decision is realized by giving consideration to the trusted sharing and privacy protection of the multi-party difference roles.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of product assembly quality prediction, and in particular to a product assembly quality prediction method and system based on longitudinal federated learning. BACKGROUND

[0002] In many manufacturing scenarios, the production of a single product requires the participation of multiple manufacturers, and when an artificial intelligence model is used to predict the assembly quality of a product, the traditional artificial intelligence model usually needs to centrally process data from multiple manufacturers in the network, that is, the manufacturers should upload their own data to the central server to train the learning model. However, due to the high data secrecy requirement of some product services, these manufacturers are reluctant to disclose the data information generated during the production process. In this regard, although federated learning as a distributed machine learning technology can create a global model by learning from multiple decentralized edge clients, the feature data obtained from different production environments or detection devices for the same product is often different, which leads to different sample data features in different product assembly or detection environments, and the sample ID is the same and only the detection party has specific label data, which meets the sample data type of longitudinal federated learning. However, based on the characteristics of product assembly data, on the one hand, the degree of data centralization is low, and the data secrecy requirement of some enterprises is high, on the other hand, some data features are scarce or unevenly distributed. This will bring challenges to model training, and joint training of high-quality models by all participants requires the establishment of a corresponding suitable model. SUMMARY

[0003] The present application aims to overcome the shortcomings of the prior art and provide a product assembly quality prediction method based on longitudinal federated learning.

[0004] To achieve the above-mentioned purpose, the technical solution provided by the present application is as follows:

[0005] A product assembly quality prediction method based on longitudinal federated learning, comprising:

[0006] The data samples of each participant are aligned under an encryption strategy;

[0007] Each participant trains a local model using their own aligned data samples, and extracts and aggregates the data features generated by different devices in the corresponding participant based on the local model;

[0008] The local models of each participant are securely aggregated using a multi-party secure computation homomorphic encryption method to obtain a global model;

[0009] The data features generated by different devices of each participant extracted and aggregated are merged, and the global model is trained using the merged data features; finally, the trained global model is used to predict the product assembly quality.

[0010] Further, the data samples of each participant are aligned under an encryption strategy, including: using a homomorphic encryption method to perform a homomorphic encryption operation on the data samples of each participant, and then performing an alignment operation on the data samples in the state of ciphertext.

[0011] Further, the aligned data samples of each participant are subjected to two rounds of multi-party parallel structure in the local model, and data features are extracted and aggregated.

[0012] Further, the multi-party parallel structure uses a fixed data partitioning strategy to partition the product assembly data, the encoder part first normalizes the partitioned data, and then extracts features through a multi-head attention layer to mine the correlation between the assembly production lines and the assembly data of each device of the participants, so as to realize feature extraction.

[0013] Further, the fixed data partitioning strategy includes:

[0014] The jth key assembly machining equipment data with a size of Hj x Wj is partitioned, and the data is divided into Nj data blocks:

[0015] Nj=HjWj / patch_size;

[0016] patch_size=PHj·PWj;

[0017] Wherein, patch_size is the size of the data block, PHj and PWj are determined according to the machining characteristics of the jth key assembly machining equipment, and are the length and width of the data block respectively;

[0018] An elementclass of a learnable classification is added before the data partitioning sequence, and a data partitioning sequence with a length of Nj+1 is obtained;

[0019] After the device fixed data partitioning strategy is used for each key equipment data in the assembly line, the data can be input into the corresponding editor for feature extraction.

[0020] To achieve the above purpose, the application further provides a product assembly quality prediction system based on longitudinal federated learning, which is used to realize the product assembly quality prediction method based on longitudinal federated learning, and the system comprises a sample alignment module, a bottom module, a federated interaction module and a top module.

[0021] Wherein,

[0022] The sample alignment module is used for sample alignment of the data samples of each participant under an encryption strategy.

[0023] The bottom layer module is used for training of the local model of each participant based on the aligned data samples, extraction and aggregation of the data features generated by different devices in the corresponding participant based on the local model.

[0024] The federal interaction module is used for gradient secure aggregation of the local model of each participant by using a homomorphic encryption method of multi-party secure computation to obtain a global model.

[0025] The top layer module is used for merging of the extracted and aggregated data features generated by different devices in each participant, and training of the global model by using the merged data features. Finally, the product assembly quality is predicted by using the trained global model.

[0026] Compared with the prior art, the principles and advantages of the present application are as follows:

[0027] 1. In the data preprocessing stage, the data samples of each participant are aligned under an encryption strategy to solve the problems of non-centralization, non-uniformity and scarcity of data.

[0028] 2. The local model of each participant adopts a multi-party parallel structure, and the multi-party parallel structure adopts a fixed data partition strategy to partition the product assembly data. The encoder part first performs layer normalization on the partitioned data, and then performs feature extraction through a multi-head attention layer to mine the correlation between the assembly production lines and the assembly data of each device in the participant, so as to enhance the feature extraction capability of the model.

[0029] 3. The homomorphic encryption method of multi-party secure computation is used to perform gradient secure aggregation on the local model of each participant to obtain a global model, so as to realize the goal of jointly training a high-precision assembly quality prediction model by using the data of multiple sub-plants, and realize the product service research and development auxiliary decision which takes into account the trusted sharing and privacy protection of the multi-party difference roles. BRIEF DESCRIPTION OF DRAWINGS

[0030] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, a brief introduction to the services needed in the embodiments or the prior art description will be given below. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0031] Figure 1 The connection block diagram of the product assembly quality prediction system based on longitudinal federal learning of the present application;

[0032] Figure 2 This is a framework diagram of a local model used in a product assembly quality prediction system based on vertical federated learning according to the present invention;

[0033] Figure 3 Schematic diagram of the relationship between multi-party parallel structure and multi-head attention mechanism;

[0034] Figure 4 Schematic diagram of the operating mechanism of the fixed data partitioning strategy. DETAILED DESCRIPTION

[0035] The present invention will be further described below in conjunction with specific embodiments:

[0036] like Figure 1 As shown, the product assembly quality prediction system based on vertical federated learning described in this embodiment includes a sample alignment module, a bottom module, a federated interaction module, and a top module;

[0037] in,

[0038] The sample alignment module is used to align the data samples of each participant under the encryption strategy;

[0039] The underlying module obtains a local model of each participant based on the aligned data samples, and extracts and aggregates data features generated by different devices in the corresponding participant based on the local model;

[0040] The federated interaction module uses a homomorphic encryption method based on multi-party secure computation to perform gradient security aggregation on the local models of each participant to obtain a global model.

[0041] The top-level module merges the extracted and aggregated data features generated by different devices in each participant, and uses the merged data features to train the global model; finally, the trained global model is used to predict the product assembly quality.

[0042] Here’s how the system works:

[0043] In the manufacturing industry, product assembly often involves multiple assembly steps or multiple factories. Different parties possess different feature data, and the same product is processed by different factories, resulting in a high degree of sample overlap. However, the feature spaces of the samples may not be exactly the same, so appropriate methods are needed to address these differences. To this end, a sample alignment module is first used to align the data samples of each party under an encryption strategy. This involves using homomorphic encryption methods to homomorphically encrypt the data samples of each party, followed by alignment of the data samples in a ciphertext state.

[0044] Then, the bottom layer module trains the local model of each participant based on the aligned data samples, extracts and aggregates the data features generated by different devices in the corresponding participant based on the local model;

[0045] In this step, the framework of the local model is as shown in Figure 2 .

[0046] The data samples are first extracted by the encoder part, which extracts features from the product assembly quality data through the multi-head attention mechanism. The relationship between the multi-party parallel structure and the multi-head attention mechanism is as shown in Figure 3 . The main part of the local model and the global classifier of the local model can be simplified as a multi-head attention mechanism. The multi-head attention mechanism can be represented as a method of mapping the output vector Q with a group-key pair K-V. According to the characteristics of the key equipment in each assembly step, this embodiment adopts a fixed data partitioning strategy in a multi-party parallel manner to partition the assembly data. The encoder part first normalizes the partitioned data and extracts features through the multi-head attention layer, and then mines the correlation between the assembly production lines and the assembly data of each device in the participant, so as to realize the powerful feature extraction function of the neural network.

[0047] As shown in Figure 4 , the fixed data partitioning strategy includes:

[0048] The jth key assembly processing equipment data with a size of Hj×Wj is partitioned to divide the data into Nj data blocks:

[0049] Nj=HjWj / patch_size;

[0050] patch_size=PHj·PWj;

[0051] Wherein, patch_size is the size of the data block, PHj and PWj are determined according to the processing characteristics of the jth key assembly processing equipment, and are the length and width of the data block respectively;

[0052] An elementclass of a learnable classification is added in front of the data partitioning sequence, and then a data partitioning sequence with a length of Nj+1 is obtained;

[0053] After the device fixed data partitioning strategy is adopted for the data of each key equipment in the assembly line, the data can be input into the corresponding editor for feature extraction.

[0054] Then, the federal interactive module uses the homomorphic encryption method of multi-party secure computation to securely aggregate the gradients of the local models of each participant, and obtains the global model.

[0055] In this step, considering that some enterprises or manufacturers have higher requirements for the confidentiality of data, the model must be encrypted before aggregation. Multi-party secure computation in vertical federated learning is a computing model that emphasizes data privacy and security, which allows participants to cooperate in computing without exposing private information, so it is an effective method to handle privacy issues. After the multi-device features of each sub-factory, i.e. each participant, are aggregated in the respective local model, they are then transmitted into the federal interaction module, and the homomorphic encryption method of multi-party secure computation is used to securely aggregate the gradients of the local models of each participant to obtain the global model without sharing the original data, thereby achieving a higher level of data privacy and security.

[0056] Finally, the top module merges the data features generated by different devices of each participant extracted and aggregated, and uses the merged data features to train the global model; and uses the trained global model to predict the product assembly quality.

[0057] The above-described embodiments are only preferred embodiments of the present application, and do not limit the scope of the application. Any changes made in shape or principle according to the present application should be covered within the scope of protection of the present application.

Claims

1. A product assembly quality prediction method based on vertical federated learning, characterized in that: include: The data samples of each participant are aligned under the encryption strategy; Each participant obtains its own local model through training of its own aligned data samples, and extracts and aggregates the data features generated by different devices in the corresponding participant based on the local model; Adopting the homomorphic encryption method of multi-party secure computation to perform gradient secure aggregation on the local models of each participant to obtain the global model; The extracted and aggregated data features generated by different devices in each participant are merged and used to train a global model. Finally, the trained global model is used to predict product assembly quality. After alignment, the data samples of each participant go through two rounds of multi-party parallel structure in the local model to extract data features and aggregate the data features; The multi-party parallel structure uses a fixed data partitioning strategy to partition product assembly data. The encoder first performs layer normalization on the partitioned data and extracts features through a multi-head attention layer. It then mines the correlation between the assembly data of each assembly line and each device within the participating party to achieve feature extraction. Fixed data partitioning strategies include: Perform partitioning on the data of the jth key assembly processing equipment with a size of Hj×Wj, and divide the data into Nj data blocks: Nj = HjWj / patch_size; patch_size = PHj·PWj; Among them, patch_size is the size of the data block, PHj and PWj are determined according to the processing characteristics of the j-th key assembly processing equipment, and are the length and width of the data block respectively; Add a learnable classification information elementclass before the data partition sequence, and then obtain a data partition sequence with a length of Nj+1; After adopting the device-specific data partitioning strategy for each key device data in the assembly production line, the data can be input into the corresponding editor for feature extraction.

2. The method for predicting product assembly quality based on vertical federated learning according to claim 1, characterized in that: The data samples of each participant are aligned under the encryption strategy, including: using the homomorphic encryption method to perform homomorphic encryption operations on the data samples of each participant, and then aligning the data samples in the ciphertext state.

3. A product assembly quality prediction system based on vertical federated learning, used to implement the product assembly quality prediction method based on vertical federated learning according to any one of claims 1-2, characterized in that: It includes sample alignment module, bottom module, federation interaction module and top module; in, The sample alignment module is used to align the data samples of each participant under the encryption strategy; The underlying module obtains a local model of each participant based on the aligned data samples, and extracts and aggregates data features generated by different devices in the corresponding participant based on the local model; The federated interaction module uses a homomorphic encryption method based on multi-party secure computation to perform gradient security aggregation on the local models of each participant to obtain a global model. The top-level module merges the extracted and aggregated data features generated by different devices in each participant, and uses the merged data features to train the global model; finally, the trained global model is used to predict the product assembly quality.

Images