Digital twin factory data communication and privacy protection method

Abstract

The invention relates to a digital twinborn factory data communication and privacy protection method, which comprises the following steps: establishing an edge-cloud digital twinborn model which comprises an edge layer, a digital twinborn layer and a cloud layer; the edge layer comprises an edge server and IIOT equipment of a factory; the digital twin layer comprises a plurality of digital twin servers; the cloud layer comprises a cloud server; dividing the neural network model for classifying tasks into a shallow model and a deep model; and the cloud server receives real-time model update provided by the digital twin server, grouping and training the IIOT equipment based on the digital twin model updated in real time, weighting and aggregating local shallow model parameters encrypted by each IIOT equipment in a group by adopting a secure multi-party computing method, and performing parallel computing by the IIOT equipment by using the aggregated part. According to the method, data sharing among multiple pieces of workshop equipment can be realized while data privacy is protected, the production process is optimized, and the product quality is improved.

Description

Technical Field

[0001] The present application relates to the technical field of digital twin factory data security transmission, and in particular to a digital twin factory data communication and privacy protection method. Background Art

[0002] In actual smart factories, there are a large number of heterogeneous resource IIoT devices (such as robotic arms, CNC machine tools, monitoring equipment, etc.), and the data transmission among them often needs to cross regions and time zones, which involves the issue of data privacy protection. At the same time, the computing resources of the IIoT devices themselves will be occupied by encrypted calculations. Summary of the invention

[0003] Based on this, in order to balance the relationship between the efficiency of data transmission, the occupation of IIoT device computing resources, and the cost of data privacy protection, it is necessary to provide a digital twin factory data communication and privacy protection method based on federated learning, which includes: S1: Establish an edge-cloud digital twin model, wherein the edge-cloud digital twin model includes an edge layer, a digital twin layer, and a cloud layer; the edge layer includes an edge server and IIoT devices of a factory; the digital twin layer includes multiple digital twin servers; and the cloud layer includes a cloud server; The edge server is used to receive data collected by the IIoT device cluster of each factory, train the local model of the IIoT device, and transmit the status update of the IIoT device to the digital twin server; The digital twin server is used to update the digital twin model in real time according to the status update; S2: Divide the neural network model used for the classification task into a shallow model and a deep model; distribute the shallow model to the IIoT device and train it on the edge server, and train the deep model on the cloud server; S3: The cloud server receives the real-time model update provided by the digital twin server, and based on the real-time updated digital twin model, groups and trains the IIoT devices, and then uses a secure multi-party computing method to weight the encrypted local shallow model parameters of each IIoT device in the aggregation group. The IIoT devices use the aggregated parts for parallel computing.

[0004] Beneficial effects: This method can realize data sharing, collaborative training and real-time analysis among multiple workshop equipment while protecting data privacy, optimize the production process and improve product quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0005] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0006] Figure 1 This is a schematic diagram of the structure of a flexible display screen according to an embodiment of the present application. DETAILED DESCRIPTION

[0007] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are described in detail below in conjunction with the accompanying drawings. In the following description, many specific details are set forth to facilitate a full understanding of the present application. However, the present application can be implemented in many other ways different from those described herein, and those skilled in the art can make similar improvements without violating the connotation of the present application, so the present application is not limited by the specific embodiments disclosed below.

[0008] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0009] This embodiment provides a new digital twin factory data transmission (DT-ASFL) architecture based on federated learning, specifically a digital twin factory data communication and privacy protection method, which uses the asynchronous split federated learning algorithm (ASFL) to enable devices with limited computing resources to participate in the learning process by participating in the training part of the learning model. At the same time, digital twins (DT) are used to provide the real-time status of IIoT devices in the system, thereby enabling asynchronous model updates in split federated learning. Compared with existing methods, ASFL has better results.

[0010] like Figure 1 As shown, a digital twin factory data communication and privacy protection method includes: S1: Establish an edge-cloud digital twin model, wherein the edge-cloud digital twin model includes an edge layer, a digital twin layer, and a cloud layer; the edge layer includes an edge server and IIoT devices in the factory; the digital twin layer includes multiple digital twin servers; and the cloud layer includes a cloud server.

[0011] The edge server is used to receive data collected by the IIoT device cluster of each factory, train the local model of the IIoT device, and transmit the status update of the IIoT device to the digital twin server; The digital twin server is used to update the digital twin model in real time according to the status update; Specifically, the digital twin model is expressed as: ; in, Indicates t Moment i A digital twin model of a digital twin server; Indicates i edge servers; Indicates i The aggregate state of the edge servers; Indicates i The current training participation status of the edge servers; Indicates i Estimated computation time for edge servers; Indicates i The communication time between edge servers; The aggregation status of the edge server includes True / False. When the training in which the edge server participates is completed, the aggregation status of the edge server is True; when the edge server is still participating in training or has been aggregated or has not participated in training, the aggregation status of the edge server is False; The current training participation status of the edge server includes True / False. When the edge server is currently participating in training, the training participation status is True; when the edge server is not currently participating in training, the training participation status is False.

[0012] S2: Divide the neural network model used for the classification task into a shallow model and a deep model; distribute the shallow model to the IIoT device and train it on the edge server, and train the deep model on the cloud server.

[0013] In this embodiment, the size of the shallow model is smaller than that of the deep model.

[0014] Furthermore, the training process of the shallow model includes: Step 1: Initialize the local shallow model parameters of the IIoT devices involved in the training, the aggregation state of the edge server, and the time step of the current edge server; Step 2: Divide the data and corresponding labels of each IIoT device involved in the training into batches of preset size; Step 3: In each edge server, forward propagate the data of each batch of IIoT devices to obtain the intermediate output; Step 4: Send the intermediate output and corresponding labels to the cloud server; Step 5: The cloud server calculates the gradient based on the received labels and their corresponding true labels; Step 6: The IIoT devices participating in the training use the gradients for back propagation, update the local shallow model parameters, and update the aggregation state of the edge server to True.

[0015] Furthermore, the training process of the deep model includes: Step 1: Initialize the global shallow model parameters, training rounds, update time steps, the number of IIoT devices participating in the training, the set of IIoT devices currently participating in the training which is initially empty, the edge server, and a large prime number. Step 2: Randomly select IIoT devices in the edge server to update the digital twin model, and process each IIoT device in parallel; Step 3: receiving the intermediate output and the corresponding label, and performing forward propagation based on the intermediate output to obtain a first predicted label; Step 4: Calculate the loss function according to the first predicted label and the true label corresponding to the intermediate output, and perform back propagation calculation based on the loss function to obtain the second gradient; Step 5: Send the second gradient to the IIoT device, and the IIoT device updates the global shallow model parameters according to the second gradient.

[0016] The update formula of the global shallow model parameters is: ; ; ; in, represents the global shallow model parameters at the r+1th time step; represents the global shallow model parameters at the rth time step; represents the learning rate; represents the amount of data used by the pth IIoT device; N represents the amount of data used by all IIoT devices; P represents the number of IIoT devices; represents the first gradient; represents the loss function; represents the true label corresponding to the intermediate output of the p-th IIoT device; represents the first predicted label; represents partial derivative; Represents the second gradient.

[0017] In this embodiment, the training of the deep model also includes: When the loss function is differentiable, calculating the gradient of the loss function with respect to the global shallow model parameters; The gradient of the edge server is calculated by averaging the gradients corresponding to all IIoT devices participating in the training, and the global shallow model parameters of the current time step are updated based on the gradient of the edge server; Update the digital twin model.

[0018] S3: The cloud server receives the real-time model update provided by the digital twin server, and based on the real-time updated digital twin model, groups and trains the IIoT devices, and then uses a secure multi-party computing method to weight the encrypted local shallow model parameters of each IIoT device in the aggregation group. The IIoT devices use the aggregated parts for parallel computing.

[0019] In this scenario, the characteristics of the computing and communication resources of the device are constantly changing, and outliers are a key problem. Therefore, the Partitioning Around Medoids (PAM) algorithm is selected as the dynamic clustering scheduling system. The first advantage of the PAM algorithm is that it is robust to noise and outliers. Since the PAM algorithm uses medoids instead of means as clustering centers, it is not sensitive to outliers and noise in the data. This robustness is essential for our dynamic clustering scheduling system. The second advantage of the PAM algorithm is that it is designed to minimize the sum of the differences between each participant and the medoid selected for each cluster. In this scenario, devices with similar characteristics need to be grouped so that they can collaborate more effectively and generate updated global models. The PAM algorithm ensures that devices with similar characteristics are grouped into one group by minimizing the sum of the differences between each participant and the selected medoid.

[0020] Specifically, IIoT devices are grouped into: Step 1: The cloud server predefines the number of clusters as k, and identifies the IIoT devices corresponding to the edge servers currently participating in the training from the digital twin model; Step 2: The cloud server calculates the estimated execution time of the edge server currently participating in the training based on the collected estimated computing time and communication time of the edge server currently participating in the training; Step 3: randomly select k pseudo-center points from the estimated execution time, and calculate the distances from all non-pseudo-center points to any pseudo-center point; Step 4: In each cluster, exchange the quasi-center points with the non-quasi-center points, and calculate the costs before and after the exchange; if the cost after the exchange is less than the cost before the exchange, the exchange operation is retained, otherwise the exchange operation is canceled; Step 5: Repeat step 4 until all clusters are traversed and k groups of IIoT devices are obtained.

[0021] Furthermore, when all IIoT devices in the cluster with the smallest aggregation center complete training, the StartAggregation aggregation function is triggered on the edge server side, and the large prime number participating in the aggregation process and a group of IIoT devices are sent to the IIoT devices that are about to join the aggregation process.

[0022] Furthermore, the weighted aggregation process is performed using the additive secret sharing protocol as a cryptographic primitive. Additive secret sharing allows multiple parties to add their values ​​without revealing their true values. Assuming that there are n parties who want to perform an additive secret sharing scheme, the secret sharing scheme is expressed as: ; ; in, represents the secret sharing scheme function; z represents the secret; n represents the number of IIoT devices in the group; A random number representing the first IIoT device in the group; A random number representing the i-th IIoT device in the group; A random number representing the n-1th IIoT device in the group; A random number representing the nth IIoT device in the group; Q represents a large prime number.

[0023] Suppose there are three parties who want to join an additive secret sharing scheme. Each party has the required secrets u, t, and v. First, each party divides their secret into several parts. u is divided into three parts u = {u 1 , u 2 ,u 3}. t and v are also divided into three parts, that is, t= {t 1 ,t 2 ,t 3} and v = {v 1 ,v 2 ,v 3}. These shares are then exchanged with each other so that all parties receive a portion of the other secret. Each party then adds up the secret shares they receive to get m i , where iϵ{1, . . . , n}. Finally, to find the value of u + t + v, the sum shares are summarized as follows: .

[0024] The secure multi-party computing method comprises: Calculating the difference between the global shallow model parameters and the local shallow model parameters of each IIoT device; Use a secret sharing scheme to divide the difference into n parts, where n is the number of IIoT devices in the group; Send each secret shared value to other IIoT devices, and receive secret shared values ​​from other IIoT devices and store them in a set of secret shared values; Calculating a first shared value sum of all received secret shared values; Send the sum of the first shared values ​​of the first hidden layer in the neural network model that does not belong to the classification task to other IIoT devices, and receive the sum of the first shared values ​​of other IIoT devices and store them in a second secret shared value set; All the first shared value sums are added together to obtain the weighted aggregated shared value sum within the group to update the local first hidden layer parameters.

[0025] This method reduces the computational burden of resource-constrained devices by dividing the learning model into a shallow model and a deep model, with the shallow model trained on the client and the deep model trained on the cloud. By utilizing the latest advances in DT technology, a clustering mechanism based on edge device status and the estimated training time of each participant in the edge cloud DT space can be used to optimize the efficiency of the model aggregation process. By using secure multi-party computing technology (SMPC) to enhance privacy protection, the robustness of DT-ASFL against training data reconstruction attacks is enhanced, ensuring the privacy and security of data in the factory system. This method can achieve data sharing, collaborative training and real-time analysis among multiple workshop devices while protecting data privacy, optimizing the production process and improving product quality.

[0026] This method is used for data communication and privacy protection between IIOT devices in digital twin factories, enabling devices with limited computing power to participate in collaborative machine learning to improve factory operation and maintenance. This method takes into account the computing and communication capabilities of the equipment and uses digital twin technology to allow real-time monitoring and dynamic aggregation scheduling; the weighted aggregation process also helps with rapid model convergence and high model accuracy.

[0027] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0028] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be construed as limiting the scope of the patent application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent application shall be subject to the attached claims.

Claims

1. A digital twin factory data communication and privacy protection method, characterized in that: include: S1: Establish an edge-cloud digital twin model, wherein the edge-cloud digital twin model includes an edge layer, a digital twin layer, and a cloud layer; the edge layer includes an edge server and IIoT devices of a factory; the digital twin layer includes multiple digital twin servers; and the cloud layer includes a cloud server; The edge server is used to receive data collected by the IIoT device cluster of each factory, train the local model of the IIoT device, and transmit the status update of the IIoT device to the digital twin server; The digital twin server is used to update the digital twin model in real time according to the status update; S2: Divide the neural network model used for the classification task into a shallow model and a deep model; distribute the shallow model to the IIoT device and train it on the edge server, and train the deep model on the cloud server; S3: The cloud server receives the real-time model update provided by the digital twin server, and based on the real-time updated digital twin model, groups and trains the IIoT devices, and then uses a secure multi-party computing method to weight the encrypted local shallow model parameters of each IIoT device in the aggregation group. The IIoT devices use the aggregated parts for parallel computing.

2. The digital twin factory data communication and privacy protection method according to claim 1 is characterized in that: The digital twin model is expressed as: ； in, Indicates t Moment i A digital twin model of a digital twin server; Indicates i edge servers; Indicates i The aggregate state of the edge servers; Indicates i The current training participation status of the edge servers; Indicates i Estimated computation time for edge servers; Indicates i The communication time between edge servers; The aggregation status of the edge server includes True / False. When the training in which the edge server participates is completed, the aggregation status of the edge server is True; when the edge server is still participating in training or has been aggregated or has not participated in training, the aggregation status of the edge server is False; The current training participation status of the edge server includes True / False. When the edge server is currently participating in training, the training participation status is True; when the edge server is not currently participating in training, the training participation status is False.

3. The digital twin factory data communication and privacy protection method according to claim 1 is characterized in that: Grouping IIoT devices includes: Step 1: The cloud server predefines the number of clusters as k, and identifies the IIoT devices corresponding to the edge servers currently participating in the training from the digital twin model; Step 2: The cloud server calculates the estimated execution time of the edge server currently participating in the training based on the collected estimated computing time and communication time of the edge server currently participating in the training; Step 3: randomly select k pseudo-center points from the estimated execution time, and calculate the distances from all non-pseudo-center points to any pseudo-center point; Step 4: In each cluster, exchange the quasi-center points with the non-quasi-center points, and calculate the costs before and after the exchange; if the cost after the exchange is less than the cost before the exchange, the exchange operation is retained, otherwise the exchange operation is canceled; Step 5: Repeat step 4 until all clusters are traversed and k groups of IIoT devices are obtained.

4. The digital twin factory data communication and privacy protection method according to claim 1 is characterized in that: The size of shallow models is smaller than that of deep models.

5. The digital twin factory data communication and privacy protection method according to claim 3 is characterized in that: The training process of the shallow model includes: Step 1: Initialize the local shallow model parameters of the IIoT devices involved in the training, the aggregation state of the edge server, and the time step of the current edge server; Step 2: Divide the data and corresponding labels of each IIoT device involved in the training into batches of preset size; Step 3: In each edge server, forward propagate the data of each batch of IIoT devices to obtain the intermediate output; Step 4: Send the intermediate output and corresponding labels to the cloud server; Step 5: The cloud server calculates the gradient based on the received labels and their corresponding true labels; Step 6: The IIoT devices participating in the training use the gradients for back propagation, update the local shallow model parameters, and update the aggregation state of the edge server to True.

6. The digital twin factory data communication and privacy protection method according to claim 5 is characterized in that: The training process of the deep model includes: Step 1: Initialize the global shallow model parameters, training rounds, update time steps, the number of IIoT devices participating in the training, the set of IIoT devices currently participating in the training which is initially empty, the edge server, and a large prime number. Step 2: Randomly select IIoT devices in the edge server to update the digital twin model, and process each IIoT device in parallel; Step 3: receiving the intermediate output and the corresponding label, and performing forward propagation based on the intermediate output to obtain a first predicted label; Step 4: Calculate the loss function according to the first predicted label and the real label corresponding to the intermediate output, and perform back propagation calculation based on the loss function to obtain the second gradient; Step 5: Send the second gradient to the IIoT device, and the IIoT device updates the global shallow model parameters according to the second gradient.

7. The digital twin factory data communication and privacy protection method according to claim 6 is characterized in that: The training of deep models also includes: When the loss function is differentiable, calculating the gradient of the loss function with respect to the global shallow model parameters; The gradient of the edge server is calculated by averaging the gradients corresponding to all IIoT devices participating in the training, and the global shallow model parameters of the current time step are updated based on the gradient of the edge server; Update the digital twin model.

8. The digital twin factory data communication and privacy protection method according to claim 1, characterized in that S3 include: When all IIoT devices in the cluster with the smallest aggregation center complete training, the StartAggregation aggregation function is triggered on the edge server side, and the large prime number participating in the aggregation process and a group of IIoT devices are sent to the IIoT devices that are about to join the aggregation process.

9. The digital twin factory data communication and privacy protection method according to claim 6 is characterized in that: The secure multi-party computing method comprises: Calculating the difference between the global shallow model parameters and the local shallow model parameters of each IIoT device; Use a secret sharing scheme to divide the difference into n parts, where n is the number of IIoT devices in the group; Send each secret shared value to other IIoT devices, and receive secret shared values ​​from other IIoT devices and store them in a set of secret shared values; Calculating a first shared value sum of all received secret shared values; Send the sum of the first shared values ​​of the first hidden layer in the neural network model that does not belong to the classification task to other IIoT devices, and receive the sum of the first shared values ​​of other IIoT devices and store them in a second secret shared value set; All the first shared value sums are added together to obtain the weighted aggregated shared value sum within the group to update the local first hidden layer parameters.

10. The digital twin factory data communication and privacy protection method according to claim 9, characterized in that: The secret sharing scheme is expressed as: ； ； in, represents the secret sharing scheme function; z represents the secret; n represents the number of IIoT devices in the group; A random number representing the first IIoT device in the group; A random number representing the i-th IIoT device in the group; A random number representing the n-1th IIoT device in the group; A random number representing the nth IIoT device in the group; Q represents a large prime number.

Images