An industrial software resource sharing method based on key feature extraction

Abstract

The application provides an industrial software resource sharing method based on key feature extraction, which is divided into three stages: feature extraction stage, initial sharing stage and federal learning stage. In the feature extraction stage, the industrial software resource provider processes the local raw data by using a variational autoencoder (VAE), extracts robust features, and calculates key features. In the initial sharing stage, the participants add random noise to the extracted key features, encrypt them, and upload them to the blockchain to form a global shared feature set. In the federal learning stage, the participants combine the global feature set with the local features to train a unified model through distributed collaboration. To ensure privacy and security, homomorphic encryption technology is used to encrypt and aggregate the model parameters during updating, preventing privacy leakage.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of resource sharing, in particular industrial software resource sharing based on federated learning, and specifically to an industrial software resource sharing method based on key feature extraction. BACKGROUND

[0002] To protect the privacy and security of industrial software data and solve the problem of data silos, researchers have proposed the concept of federated learning. Federated learning is a distributed machine learning method that allows multiple participants to jointly train a model without directly exchanging industrial software data, thereby avoiding the risk of privacy leakage caused by centralized storage of industrial software data. In this way, each party can fully utilize the advantages of other participants' industrial software data while keeping the industrial software data localized, thereby improving the accuracy and robustness of the model as a whole.

[0003] However, in real industrial software data sharing scenarios, participants in federated learning often face the problem of industrial software data heterogeneity. Industrial software data heterogeneity refers to the significant differences in distribution, format, and features of industrial software data between participants, which can lead to a decline in model training effectiveness. Solving the problem of industrial software data heterogeneity requires designing flexible model architectures and algorithms for different industrial software data sources to ensure the effectiveness and fairness of federated learning. At the same time, it is necessary to consider the computational and communication overheads caused by industrial software data heterogeneity in system design to ensure the efficiency and scalability of the federated learning system.

[0004] Past research attempts to solve this problem have mainly focused on model parameters during local training [18,45] and global aggregation

[46] . However, these methods often result in excessive computational burden and high communication costs due to the over-parameterization of deep neural networks

[47] . Zhao

[19] opened up a new branch of research to solve the Non-IID problem by constructing cross-client shareable information. They shared a portion of the original data in advance before federated learning began to construct global IID data, and solved the client drift problem by supplementing client information, which greatly improved the model performance. However, sharing original industrial software data in advance obviously poses a privacy and security risk, which contradicts the original intention of federated learning. Jeong

[48] generated synthetic data based on the original data and shared this content in advance. This method avoids the privacy risks of directly sharing original industrial software data, but synthetic industrial software data is usually similar to the original industrial software data and still causes some degree of privacy leakage. Shin

[20] The data based on data statistical synthesis is shared to cope with additional privacy security problems, but even the new industrial software data based on industrial software data statistical synthesis contains a part of original industrial software data.

[49] The original data is involved in the sharing dilemma by collecting Logits statistical information to generate virtual data. However, research shows that malicious users can perform reverse deduction through feature inversion methods

[50] Therefore, the scheme cannot completely guarantee the privacy and security of the industrial software data of users. SUMMARY

[0005] The purpose of the present application is to propose an industrial software resource sharing method based on key feature extraction to solve the performance and privacy dilemma in federated learning.

[0006] The purpose of the present application is achieved as follows:

[0007] The local industrial software data of the federated learning industrial software data provider is subjected to key feature extraction by a variational autoencoder, and robust features and key features are extracted. Robust features refer to features that have little effect on the performance of the model and are commonly used to describe the characteristics of user local data. Key features refer to features that have a significant impact on the performance of the model, but may also contain sensitive information about individuals. These features are particularly important in the context of federated learning, as they are directly related to the effectiveness of the model and the privacy of industrial software data. In order to improve the performance of the model while protecting the privacy of participants, industrial software data providers can only select key features for sharing, while using differential privacy to protect the privacy of these features. In this way, federated learning participants can effectively utilize data from various industrial software data providers to enhance the performance of the model, maximizing the performance of the federated learning model without exposing user privacy.

[0008] The specific method is as follows:

[0009] An industrial software resource sharing method based on key feature extraction, comprising the following steps:

[0010] Step 1, the parties involved in federated learning first analyze the industrial software resource data set they hold, and extract the data summary information of the local industrial software data. Then the data summary information is encrypted and uploaded to the blockchain. These information should include the type, distribution, size and whether structured of industrial software data, but do not contain any personal or sensitive data;

[0011] Step 2, when the data requester generates an industrial software data request demand, sends the industrial software data request information to the blockchain system;

[0012] Step 3, based on the industrial software data digest information and industrial software data request information on the blockchain, the blockchain system screens out a suitable industrial software data provider set P to participate in federated learning through a similarity matching algorithm;

[0013] Step 4, the system forwards the industrial software data sharing request to each member in the industrial software data provider set P;

[0014] Step 5, each member in P uses local original industrial software data to extract key features of local industrial software data through the feature extraction method designed in this chapter, and uploads the key features with random noise to the blockchain;

[0015] Step 6, based on the key features of industrial software data uploaded by all members in P, a global shared set is constructed;

[0016] Step 7, each member in P downloads the global shared industrial software data feature set, and combines with its own local industrial software data to participate in model training, so as to reduce the data heterogeneity of each industrial software data provider;

[0017] Step 8, after each member in P trains the model locally, the model update is encrypted and uploaded to the blockchain;

[0018] Step 9, the industrial software data requester polls all the model updates uploaded by the participants, aggregates the model to form a new global model parameter, and distributes the aggregated global model parameter to all participants again until the model fitting or the number of iterations is reached.

[0019] Further, the key feature extraction in step 5 can be formalized as the following optimization problem:

[0020] min Z I(X;Y|Z), s.t. I(X;X-Z|Z) ≥ C.

[0021] Further, because I(X;Y|Z) = H(Y|Z) - H(Y|X,Z), and H(Y|X,Z) is a fixed value about X and Z, the optimization problem in step 5 can be further equivalent to:

[0022] min Z H(Y|Z), s.t. H(Z) ≤ H c

[0023] Further, the conditional entropy H(Y|Z) can be estimated by the expected logarithmic value of the following prediction probability p(y|Z):

[0024]

[0025] Further, q(X; θ) is explicitly used in step 5 and implicitly models the key features Z. Thus, the optimization problem is further equivalent to:

[0026]

[0027] Further, the encoder of a variational autoencoder (VAE) can be used as q(*) to extract latent features from the input industrial software data. The design of VAE ensures that these features are helpful for both accurate reconstruction of the industrial software data and keeping the latent space structured and meaningful. Through the optimization process, we are able to find a set of parameters θ that maximizes the efficiency of the industrial software data reconstruction or prediction task while satisfying certain constraints.

[0028] The positive effects of the present application are:

[0029] The present application reduces the heterogeneity of industrial software data among users before the formal data sharing begins by sharing a part of the features most relevant to the industrial software data expected by the users in advance. At the same time, the present application uses differential privacy and blockchain in the initial sharing process to ensure the security of the process, which is superior to other solutions in the current research. BRIEF DESCRIPTION OF DRAWINGS

[0030] Figure 1 is a flowchart of the industrial software resource sharing method based on key feature extraction.

[0031] Figure 2 is a model overall block diagram.

[0032] Figure 3 is a block design diagram.

[0033] Figure 4 is a model accuracy rate change graph with rounds.

[0034] Figure 5 is a model accuracy rate change graph with rounds in a highly heterogeneous environment. DETAILED DESCRIPTION

[0035] The specific embodiments of the present application will be further described in detail below with reference to the accompanying drawings.

[0036] As Figure 1As shown, the application uses a variational autoencoder to extract key features of local industrial software data, and shares part of the key features of local industrial software data before each node participates in formal federated learning. This aims to reduce the heterogeneity of industrial software data of each participating party in the federated learning process as much as possible while not exposing any local original industrial software data, thereby effectively improving the accuracy of the model and the convergence speed of the model, which includes the following steps:

[0037] Step 1, the parties participating in federated learning first analyze the industrial software resource data set they hold, and extract the data summary information of the local industrial software data. Then the data summary information is encrypted and uploaded to the blockchain. These information should include the type, distribution, size and whether structured of industrial software data, but does not contain any personal or sensitive data;

[0038] Step 2, when the data requester generates industrial software data request demand, send industrial software data request information to the blockchain system;

[0039] Step 3, the blockchain system selects a suitable industrial software data provider set P participating in federated learning based on the industrial software data summary information and industrial software data request information on the blockchain through a similarity matching algorithm;

[0040] Step 4, the system forwards the industrial software data sharing request to each member in the industrial software data provider set P;

[0041] Step 5, each member in P uses local original industrial software data to extract key features of local industrial software data through the feature extraction method designed in this chapter, and uploads the key features with random noise to the blockchain;

[0042] Step 6, construct a global shared set based on the key features of industrial software data uploaded by all members in P;

[0043] Step 7, each member in P downloads the global shared industrial software data feature set, and combines it with its own local industrial software data to participate in model training, in order to reduce the data heterogeneity of each industrial software data provider;

[0044] Step 8, each member in P uploads the model update after training the model locally and encrypts it to the blockchain;

[0045] Step 9, the industrial software data requester polls all the model updates uploaded by the participating parties, aggregates the model to form new global model parameters, and distributes the aggregated global model parameters to all participating parties again until the model fitting or reaches the iteration number.

[0046] Further, the key feature extraction in step 5 can be formalized as the following optimization problem:

[0047] min Z I(X; Y|Z), s.t. I(X; X-Z|Z) ≥ C.

[0048] Further, because of I(X; Y|Z) = H(Y|Z) - H(Y|X,Z), and H(Y|X,Z) is a fixed value on X and Z, the optimization problem in step 5 can be further equivalent to:

[0049] min Z H(Y|Z), s.t. H(Z) ≤ H c

[0050] Further, the conditional entropy H(Y|Z) can be estimated by the expected log value of the following prediction probability p(y|Z):

[0051]

[0052] Further, q(X; θ) is explicitly used in step 5 and Z is implicitly modeled. Thus the optimization problem is further equivalent to:

[0053]

[0054] Further, the encoder of variational autoencoder (VAE) can be used as q(*) to extract latent features from the input industrial software data. The design of VAE ensures that these features are helpful for both accurate reconstruction of industrial software data and maintaining in a meaningful and structured latent space. Through the optimization process, we can find a set of parameters θ, so that the extracted features maximize the efficiency of industrial software data reconstruction or prediction tasks while meeting certain constraints.

[0055] Figure 2 is the overall block diagram of the model.

[0056] Figure 3 is the block design diagram, which shows the block design of the blockchain technology relied on by the present application.

[0057] Figure 4 is the model accuracy rate change graph of the present application and other methods with communication rounds, in which the fifth column represents the present application. From Figure 4 it can be seen that the method of the present application can achieve faster convergence speed and higher model accuracy than other methods.

[0058] Figure 5 is the model accuracy rate change graph of the present application and other methods with communication rounds, in which the fifth column represents the present application. From Figure 5It can be seen that the method can realize faster convergence speed and higher model precision than other methods, especially in the case of highly heterogeneous industrial software data.

Claims

1. An industrial software resource sharing method based on key feature extraction, characterized in that, The method comprises the following steps: Step 1, the federal learning participants first analyze the industrial software resource data set held by themselves, extract the data summary information of the local industrial software data, and then upload the encrypted data summary information to the blockchain. These information should include the type, distribution, size and whether structured of the industrial software data, but do not contain any personal or sensitive data; Step 2, when the data requester generates industrial software data request demand, send industrial software data request information to the blockchain system; Step 3, the blockchain system screens out the industrial software data provider set P that meets the industrial software summary information and industrial software data request information demand through similarity matching algorithm based on the industrial software data summary information and industrial software data request information on the blockchain; Step 4, the system forwards the industrial software resource sharing request to each member in the industrial software data provider set P; Step 5, each member in P uses the local original industrial software data to extract the key features of the local industrial software data by using the feature extraction method of variational autoencoder (VAE) which can effectively compress the input data while retaining the useful information of the output data, and uploads the key features with random noise to the blockchain; Step 6, construct a global shared set based on the industrial software data key features uploaded by all members in P; Step 7, each member in P downloads the global shared industrial software data feature set and combines it with its own local industrial software data to participate in the training of the model, so as to reduce the data heterogeneity of each industrial software data provider; Step 8, each member in P uploads the model update after local training to the blockchain after encryption; Step 9, the industrial software data requester polls all the model updates uploaded by the participants, aggregates the model to form a new global model parameter, and distributes the aggregated global model parameter to all participants again until the model fitting or the number of iterations is reached; In step 1, the industrial software data generated by device i in time slot t can be segmented into applications with a segmentation ratio of λ. i (t)∈[0,1], where λ i (t)L bits of industrial software data are executed locally, (1-λ i (t))L bits of industrial software data are transmitted to the server for execution, and the application's segmentation ratio variable λ i (t) represents the decision variable for calculating unloading; The wireless transmission rate of the mobile device i on the uplink channel at time slot t where ω represents the channel bandwidth obtained by the device i, P i (t) represents the transmission power of the device i at time slot t, σ is the channel noise, H i (t) represents the channel gain of the device i at time slot t, which is related to the distance d i , H i (t) = g0(d i / d0) -n , wherein g0 is a channel attenuation coefficient, generally -40 dB, and n is a channel attenuation index, generally 4. The amount of computation of the partial application performed by the mobile device i in step 2 at time slot t is λ i (t) = LX, where X is the computational intensity of the application, i.e., the number of CPU cycles required to process one bit, and thus the local computation latency is where f i,l (t) is the CPU frequency of device i at time slot t, and accordingly, the local computation energy is E i,l (t) = kλ i (t) = LXf i,l 2 (t), where k is the energy required for a single CPU cycle, also called effective switching capacitance; The length of the part of the application executed by the mobile device i at the edge server in step 2 is (1-λ i (t))L, ignoring the delay of the application executed at the edge server and the delay of the downlink return of the application result, then the total delay of the mobile device i in time slot t to transmit the part of the application to the edge server for processing is equal to the uplink transmission delay The total energy consumption is equal to the uplink transmission energy consumption E i,c (t) = P i (t)·L i,c (t)·L The energy that mobile device i can harvest in time slot t in step 2 is where is the maximum energy that device i can harvest in time slot t, and e i (t) is the energy stored by mobile device i in time slot t, and B i,H (t) is the energy of mobile device i at the beginning of time slot t, and B i (t) is the energy of device i at the end of time slot t, and B i (t+1) is the energy of device i at the beginning of time slot t+1, then the energy queue equation is B i (t+1) = B i (t) - E i (t) + e i (t), where E i (t) represents the energy consumption of mobile device i in time slot t to process an application task, and is specifically represented as E i (t) = 1 (ζ i,t = 1, I i,d (t) = 0) (E i,l (t) + E i,c (t)), where ζ i,t is an indication variable of whether device i generates an application request at the beginning of time slot t, and when ζ i,t = 1, it means that device i generates an application request at the beginning of time slot t, otherwise, there is no application request, and I i,d (t) is a binary indication variable of whether the application will be discarded, and when I i,d (t) = 0, it means that the application will not be discarded, otherwise, the application will be discarded, and 1 (·) is an indication function, and · is a relational expression; when · is true, 1 (·) = 1, otherwise, 1 (·) = 0, so E i (t) represents the energy consumption of device i in time slot t when the generated application is not discarded. The execution cost of the plurality of mobile devices in a single time slot in step 3 is where φ is the execution cost of applying the discard, assuming the optimization variable set The average execution cost minimization problem P1 can be constructed as: Wherein, constraint C1 is the offloading decision constraint of the application, which ensures that the splitting ratio of the industrial software data splittable application is between 0 and 1; constraint C2 is the constraint of discarding indicator variable, which ensures that the application has only two states of discarding and not discarding; constraint C3 is the available energy constraint, which ensures that the energy obtained by the mobile device does not exceed the available energy of the current time slot; constraint C4 is the energy causality constraint, which ensures that the energy consumption of all devices in the current time slot under the calculation offloading does not exceed the remaining energy of the device in the current time slot; constraint C5 is the completion deadline constraint, which ensures that the maximum value of the completion time of the part of the application executed locally and the completion time of the part of the application executed on the server does not exceed the completion deadline of the application; constraint C6 is the transmission power constraint, which ensures that the transmission power of all devices when offloading is non-negative and does not exceed the maximum value of the transmission power; constraint C7 is the computing frequency constraint, which ensures that the computing frequency of all devices when performing local computing task is non-negative and does not exceed the maximum value of the computing frequency.

2. The industrial software resource sharing method based on key feature extraction according to claim 1, characterized in that, The time of the multi-user edge computing system in step 1 is divided by time slots, represented by T={1, 2, 3…}, the set of time slots, time slot t∈T, and each mobile device generates an industrial software data splittable application in each time slot according to a certain probability, represented by a two-tuple representing an industrial software data splittable application, where L represents the length of the input industrial software data of the application as L bits, representing the completion deadline of the application as seconds.

3. The industrial software resource sharing method based on key feature extraction according to claim 1, characterized in that, The Lyapunov drift plus penalty function in step 4 is where ΔL(t) is the Lyapunov drift, V is a control parameter constant, and C is a constant, is the virtual energy queue, which is specifically expressed as where θ i is a disturbance parameter, which is specifically expressed as where is the maximum energy consumption in theory applied to local execution and transmission to server execution, and is Therefore, the problem P1 can be reconstructed as a Lyapunov drift plus penalty minimization problem P2: Because P2 is an optimization problem without energy causality constraint and devices are independent of each other, problem P2 can be decomposed into multiple single-device single-time-slot optimization problems P3: Because problem P3 is a single-device single-time-slot computation offloading problem, problem P3 can be decomposed into two sub-problems: computation offloading sub-problem and energy acquisition sub-problem and solved respectively.

4. The industrial software resource sharing method based on key feature extraction according to claim 1, characterized in that, The energy acquisition sub-problem in step 5 is: This problem is a linear optimization problem, and also a convex optimization problem, so the optimal solution of the energy that device i can obtain at time slot t can be obtained is: Then the computation offloading sub-problem is constructed: The problem P3(b) is discussed in two cases: ζ i,t = 1, I i,d (t) = 1 and ζ i,t = 1, I i,d (t) = 0, i.e. the two cases of application of the discarded and not discarded are discussed respectively; When ζ i,t = 1, I i,d (t) = 1, it is easy to know that the minimum value of the objective function is Vφ. When ζ i,t = 1, I i,d (t) = 0, problem P3(b) can be transformed into problem P4: By analyzing problem P4, it is known that λ i (t) and f i,l (t) are coupled with each other, λ i (t) and P i (t) are also coupled with each other, but f i,l (t) and P i (t) are decoupled, so problem P4 can be decomposed into a two-stage problem: the first stage is a resource allocation problem, and the second stage is a resource allocation problem, which is further divided into a computing resource allocation sub-problem and a power allocation sub-problem; First, the computation resource allocation sub-problem P5 is constructed: Setting In any case, the problem P5 is a convex optimization problem when The optimal solution of CPU frequency can be obtained by using the first order condition of the existence of the optimal solution of convex function as follows: Then the power allocation sub-problem P6 is constructed: Setting In any case, it holds that The problem P6 is a convex optimization problem, which can be obtained from the constraint C5 first: wherein Again by the first order condition of the optimal solution of convex function, is the global optimal solution of the objective function on R, so the optimal solution of the transmission power can be obtained Before constructing the computation offloading decision problem P7, first substitute and into problem P4, and the objective function of problem P7 is obtained as follows: Then the computation offloading sub-problem is constructed: Thus, problem P7 containing only split proportion variables is obtained: Since the second derivative of the objective function is semi-positive definite, problem P7 is a convex optimization problem, so let g ′ (λ i (t)) = 0, the optimal solution of the split ratio variable λ i * (t) is obtained by using one-dimensional search algorithm such as Newton method.

Images