Industrial software resource sharing method based on key feature extraction

Abstract

The invention provides an industrial software resource sharing method based on key feature extraction. The method is divided into three stages: a feature extraction stage, a primary sharing stage and a federal learning stage. In the feature extraction stage, an industrial software resource provider processes local original data through a variational auto-encoder (VAE), robust features are extracted, and key features are calculated. In an initial sharing stage, a participant adds random noise to extracted key features, uploads the key features to a block chain after encryption, and forms a global sharing feature set. In the federation learning stage, the participants train the unified model through distributed cooperation in combination with the global feature set and the local features. In order to ensure privacy security, a homomorphic encryption technology is adopted for encryption aggregation when model parameters are updated, and privacy leakage is prevented.

Description

Technical Field

[0001] The present invention relates to the field of resource sharing, in particular to industrial software resource sharing based on federated learning, and specifically to an industrial software resource sharing method based on key feature extraction. Background Art

[0002] In order to protect the privacy and security of industrial software data and solve the problem of data silos, researchers have proposed the concept of federated learning. Federated learning is a distributed machine learning method that allows multiple participants to jointly train models without directly exchanging industrial software data, thereby avoiding the risk of privacy leakage caused by centralized data storage. In this way, each party can make full use of the industrial software data advantages of other participants while keeping the industrial software data localized, thereby improving the accuracy and robustness of the model as a whole.

[0003] However, in real industrial software data sharing scenarios, participants in federated learning often face the problem of industrial software data heterogeneity. Industrial software data heterogeneity refers to the significant differences in distribution, format, and characteristics of industrial software data among participants, which may lead to a decline in model training results. Solving the problem of industrial software data heterogeneity requires designing flexible model architectures and algorithms for different industrial software data sources to ensure the effectiveness and fairness of federated learning. At the same time, the computing and communication overhead caused by industrial software data heterogeneity also needs to be considered in system design to ensure the efficiency and scalability of the federated learning system.

[0004] Past attempts to address this problem have focused on local training [18,45] and global aggregation

[46] However, these methods often lead to excessive computational burden and high communication cost due to over-parameterization of deep neural networks.

[47] . Zhao

[19] A new branch of research on solving the Non-IID problem has been opened up - building cross-client sharable information. They build global IID data by sharing a portion of the original data in advance before the start of federated learning, and solve the client drift problem by supplementing customer information, which greatly improves the model performance. However, sharing the original industrial software data in advance obviously brings privacy and security risks, which is contrary to the original intention of federated learning design. Jeong

[48] Synthetic data is generated based on the original data and this part of the content is shared in advance. Although this method avoids the privacy risks brought by directly sharing the original industrial software data, the synthetic industrial software data is usually similar to the original industrial software data, which still leads to a certain degree of privacy leakage.

[20] It is proposed to share data based on statistical synthesis to deal with additional privacy and security issues, but even the new industrial software data based on statistical synthesis of industrial software data also contains a part of the original industrial software data.

[49] Attempts to generate virtual data by collecting Logits statistics to get rid of the dilemma of original data sharing. However, research shows that malicious users can reversely deduce through feature inversion methods.

[50] Therefore, this solution cannot fully guarantee the privacy and security of users’ industrial software data. Summary of the invention

[0005] The purpose of this invention is to propose an industrial software resource sharing method based on key feature extraction to solve the performance and privacy dilemma in federated learning.

[0006] The object of the present invention is achieved in this way:

[0007] The variational autoencoder is used to extract key features of the local industrial software data of the federated learning industrial software data provider, and the robust features and key features are extracted. Robust features refer to those features that have little impact on the performance of the model and are often used to describe the characteristics of the user's local data. Key features refer to those features that have a significant impact on the performance of the model but may also contain sensitive information about individuals. These features are particularly important in federated learning scenarios because they are directly related to the effect of the model and the privacy of industrial software data. In order to protect the privacy of participants while improving model performance, industrial software data providers can only select key features for sharing, and use differential privacy to ensure the privacy security of these features. In this way, federated learning participants can effectively use the data from various industrial software data providers to enhance the performance of the model, and maximize the performance of the federated learning model without exposing user privacy.

[0008] The specific method is as follows:

[0009] An industrial software resource sharing method based on key feature extraction comprises the following steps:

[0010] Step 1: All parties involved in federated learning first analyze the industrial software resource data sets they hold and extract the data summary information of the local industrial software data. The data summary information is then encrypted and uploaded to the blockchain. This information should include the type, distribution, size, and whether the industrial software data is structured, but does not contain any personal or sensitive data;

[0011] Step 2: When the data requester generates an industrial software data request demand, the industrial software data request information is sent to the blockchain system;

[0012] Step 3: Based on the industrial software data summary information and industrial software data request information on the blockchain, the blockchain system selects a suitable industrial software data provider set P to participate in federated learning through a similarity matching algorithm;

[0013] Step 4: The system forwards the industrial software data sharing request to each member of the industrial software data provider set P;

[0014] Step 5. Each member in P uses the local original industrial software data to extract the key features of the local industrial software data through the feature extraction method designed in this chapter, and then adds random noise to the key features and uploads them to the blockchain;

[0015] Step 6: Build a global shared set based on the key features of industrial software data uploaded by all members in P;

[0016] Step 7: Each member in P downloads the globally shared industrial software data feature set and participates in the model training in combination with its own local industrial software data to reduce the data heterogeneity of each industrial software data provider;

[0017] Step 8. After each member in P trains the model locally, they encrypt the model update and upload it to the blockchain;

[0018] Step 9: The industrial software data requester polls and collects the model updates uploaded by all participants, aggregates the models to form new global model parameters, and distributes the aggregated global model parameters to all participants again until the model is fitted or the number of iterations is reached.

[0019] Furthermore, the key feature extraction in step 5 can be formalized as the following optimization problem:

[0020] min Z I(X;Y|Z),stI(X;XZ|Z)≥C.

[0021] Furthermore, since I(X; Y|Z)=H(Y|Z)-H(Y|X,Z), and H(Y|X,Z) is a fixed value with respect to X and Z, the optimization problem in step 5 can be further equivalent to:

[0022] min Z H(Y|Z),stH(Z)≤H c

[0023] Furthermore, the conditional entropy H(Y|Z) can be estimated by the expected logarithm of the following predicted probability p(y|Z):

[0024]

[0025] Furthermore, in step 5, q(X;θ) is explicitly used and the key feature Z is implicitly modeled. Therefore, the optimization problem is further equivalent to:

[0026]

[0027] Furthermore, the encoder of the variational autoencoder (VAE) can be used as q(*) to extract latent features from the input industrial software data. The design of VAE ensures that these features can both contribute to the accurate reconstruction of industrial software data and remain in a meaningful and structured latent space. Through the optimization process, we are able to find a set of parameters θ such that the extracted features maximize the efficiency of the industrial software data reconstruction or prediction task while satisfying specific constraints.

[0028] The positive effects of the present invention are:

[0029] The present invention achieves the purpose of reducing the heterogeneity of industrial software data among users before the formal data sharing begins by sharing a part of the most relevant features of the industrial software data that users expect to obtain in advance. At the same time, this method uses differential privacy and blockchain to ensure the security of the process during the initial sharing process, which is better than other solutions currently under research. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 It is a flow chart of the industrial software resource sharing method based on key feature extraction.

[0031] Figure 2 This is the overall block diagram of the model.

[0032] Figure 3 It is a block design diagram.

[0033] Figure 4 This is a graph showing how model accuracy changes over rounds.

[0034] Figure 5 This is a graph showing how model accuracy changes over rounds in a highly heterogeneous environment. DETAILED DESCRIPTION

[0035] The specific implementation modes of the present invention are further described in detail below with reference to the accompanying drawings.

[0036] like Figure 1As shown, the present invention uses a variational autoencoder to extract key features of local industrial software data, and shares the key features of some local industrial software data before each node participates in formal federated learning. This is intended to reduce the heterogeneity of industrial software data of all parties involved in the federated learning process as much as possible without exposing any local original industrial software data, thereby effectively improving the accuracy of the model and the convergence speed of the model, and specifically includes the following steps:

[0037] Step 1: All parties involved in federated learning first analyze the industrial software resource data sets they hold and extract the data summary information of the local industrial software data. The data summary information is then encrypted and uploaded to the blockchain. This information should include the type, distribution, size, and whether the industrial software data is structured, but does not contain any personal or sensitive data;

[0038] Step 2: When the data requester generates an industrial software data request demand, the industrial software data request information is sent to the blockchain system;

[0039] Step 3: Based on the industrial software data summary information and industrial software data request information on the blockchain, the blockchain system selects a suitable industrial software data provider set P to participate in federated learning through a similarity matching algorithm;

[0040] Step 4: The system forwards the industrial software data sharing request to each member of the industrial software data provider set P;

[0041] Step 5. Each member in P uses the local original industrial software data to extract the key features of the local industrial software data through the feature extraction method designed in this chapter, and then adds random noise to the key features and uploads them to the blockchain;

[0042] Step 6: Build a global shared set based on the key features of industrial software data uploaded by all members in P;

[0043] Step 7: Each member in P downloads the globally shared industrial software data feature set and participates in the model training in combination with its own local industrial software data to reduce the data heterogeneity of each industrial software data provider;

[0044] Step 8. After each member in P trains the model locally, they encrypt the model update and upload it to the blockchain;

[0045] Step 9: The industrial software data requester polls and collects the model updates uploaded by all participants, aggregates the models to form new global model parameters, and distributes the aggregated global model parameters to all participants again until the model is fitted or the number of iterations is reached.

[0046] Furthermore, the key feature extraction in step 5 can be formalized as the following optimization problem:

[0047] min Z I(X;Y|Z),stI(X;XZ|Z)≥C.

[0048] Furthermore, since I(X; Y|Z)=H(Y|Z)-H(Y|X,Z), and H(Y|X,Z) is a fixed value with respect to X and Z, the optimization problem in step 5 can be further equivalent to:

[0049] min Z H(Y|Z),stH(Z)≤H c

[0050] Furthermore, the conditional entropy H(Y|Z) can be estimated by the expected logarithm of the following predicted probability p(y|Z):

[0051]

[0052] Furthermore, in step 5, q(X;θ) is explicitly used and the key feature Z is implicitly modeled. Therefore, the optimization problem is further equivalent to:

[0053]

[0054] Furthermore, the encoder of the variational autoencoder (VAE) can be used as q(*) to extract latent features from the input industrial software data. The design of VAE ensures that these features can both contribute to the accurate reconstruction of industrial software data and remain in a meaningful and structured latent space. Through the optimization process, we are able to find a set of parameters θ such that the extracted features maximize the efficiency of the industrial software data reconstruction or prediction task while satisfying specific constraints.

[0055] Figure 2 This is the overall block diagram of the model.

[0056] Figure 3 It is a block design diagram, which shows the block design of the blockchain technology on which the present invention relies.

[0057] Figure 4 This is a graph showing the variation of the model accuracy of the present invention and other methods with the number of communication rounds, where the fifth column represents the present invention. Figure 4 It can be seen that the method of the present invention can achieve faster convergence speed and higher model accuracy than other methods.

[0058] Figure 5 This is a graph showing the variation of the model accuracy of the present invention and other methods with the number of communication rounds, where the fifth column represents the present invention. Figure 5It can be seen that the method of the present invention can achieve faster convergence speed and higher model accuracy than other methods, especially when the industrial software data is highly heterogeneous.

Claims

1. An industrial software resource sharing method based on key feature extraction, characterized in that: The method comprises the following steps: Step 1: All parties involved in federated learning first analyze the industrial software resource data sets they hold and extract the data summary information of the local industrial software data; then encrypt the data summary information and upload it to the blockchain; this information should include the type, distribution, size, and whether the industrial software data is structured, but does not contain any personal or sensitive data; Step 2: When the data requester generates an industrial software data request demand, the industrial software data request information is sent to the blockchain system; Step 3: Based on the industrial software data summary information and industrial software data request information on the blockchain, the blockchain system selects a suitable industrial software data provider set P to participate in federated learning through a similarity matching algorithm; Step 4: The system forwards the industrial software resource sharing request to each member of the industrial software data provider set P; Step 5. Each member in P uses the local original industrial software data to extract the key features of the local industrial software data through the feature extraction method designed in this chapter, and then adds random noise to the key features and uploads them to the blockchain; Step 6: Build a global shared set based on the key features of industrial software data uploaded by all members in P; Step 7: Each member in P downloads the globally shared industrial software data feature set and participates in the model training in combination with its own local industrial software data to reduce the data heterogeneity of each industrial software data provider; Step 8. After each member in P trains the model locally, they encrypt the model update and upload it to the blockchain; Step 9: The industrial software data requester polls and collects the model updates uploaded by all participants, aggregates the models to form new global model parameters, and distributes the aggregated global model parameters to all participants again until the model is fitted or the number of iterations is reached.

2. According to the industrial software resource sharing method based on key feature extraction described in claim 1, it is characterized in that: In step 1, the time of the multi-user edge computing system is divided into time slots, and T = {1, 2, 3 ...} is used to represent the set of time slots. In time slot t∈T, each mobile device generates an industrial software data splittable application according to a certain probability in each time slot, and a two-tuple is used. represents an industrial software data splittable application, where L represents the length of the input industrial software data of the application, which is L bits. The deadline for completion of this application is The unit is seconds.

3. According to the industrial software resource sharing method based on key feature extraction described in claim 1, it is characterized in that: The industrial software data generated by device i in time slot t in step 1 can be divided into applications at a split ratio of λ i (t)∈[0,1], where λ i (t) L bits of industrial software data are executed locally, (1-λ i (t)) L bits of industrial software data are transmitted to the server for execution, and the application's segmentation ratio variable λ i (t) is the decision variable for calculating unloading.

4. According to the industrial software resource sharing method based on key feature extraction described in claim 1, it is characterized in that: The wireless transmission rate of the uplink channel of mobile device i in time slot t in step 1 is Where ω represents the channel bandwidth obtained by device i, P i (t) represents the transmission power of device i in time slot t, σ is the channel noise, H i (t) represents the channel gain of device i at time slot t, and distance d i Related, H i (t) = g0(d i / d0) -n , where g0 is the channel attenuation coefficient, generally -40dB, and n is the channel attenuation exponent, generally 4.

5. According to the industrial software resource sharing method based on key feature extraction described in claim 1, it is characterized in that: The computation amount of the partial application executed by mobile device i in time slot t in step 2 is λ i (t)LX, where X is the computational density of the application, that is, the number of CPU cycles required to process a unit bit. Therefore, the local computational latency is where f i,l (t) is the CPU frequency of device i in time slot t, and accordingly, the local computing energy consumption is E i,l (t) = kλ i (t)L i,l 2 (t), where k is the energy required for a single CPU cycle, also called the effective switching capacitance.

6. The industrial software resource sharing method based on key feature extraction according to claim 1 is characterized in that: In step 2, the length of the part of the application that mobile device i transmits to the edge server for execution at time slot t is (1-λ i (t))L, ignoring the delay of application execution in the edge server and the downlink return delay of application results, the total delay of mobile device i transmitting part of the application to the edge server for processing in time slot t is equal to the uplink transmission delay The total energy consumption is equal to the uplink transmission energy consumption E i,c (t) = P i (t)·L i,c (t).

7. An industrial software resource sharing method based on key feature extraction according to claim 1, characterized in that: The energy that mobile device i can obtain in time slot t in step 2 is: in is the maximum energy that device i can obtain in time slot t. In addition, the energy stored by mobile device i in time slot t is e i (t)≤E i,H (t); the energy of mobile device i at the beginning of time slot t is B i (t), the energy of device i at the end of time slot t is B i (t+1), then the energy queue equation can be obtained as B i (t+1)=B i (t)-E i (t)+e i (t), where E i (t) represents the energy consumption of mobile device i in processing application tasks in time slot t, which is specifically expressed as E i (t) = 1 (ζ i,t =1,I i,d (t)=0)(E i,l (t)+E i,c (t)), where ζ i,t is an indicator variable indicating whether device i generates an application request at the beginning of time slot t. i,t = 1, it means that device i generates an application request at the beginning of time slot t, otherwise there is no application request, I i,d (t) is a binary indicator variable indicating whether the application will be abandoned. i,d When (t) = 0, it means that the application will not be abandoned, otherwise, the application will be abandoned. 1(·) is the indicator function, and · is the relational expression; when · is true, 1(·) = 1, otherwise, 1(·) = 0, so E i (t) represents the energy consumption of device i when it does not abandon the generated application in time slot t.

8. An industrial software resource sharing method based on key feature extraction according to claim 1, characterized in that: The execution cost of multiple mobile devices in a single time slot in step 3 is Where φ is the execution cost of the application being abandoned, assuming that the optimization variable set Then the average execution cost minimization problem P1 can be constructed as: s.t.C1:0≤λ i (t)≤1,i∈N,t∈T C2:I i,d (t)∈{0,1},i∈N,t∈T C3:0≤e i (t)≤E i,H (t),i∈N,t∈T C4:E i (t)≤B i (t)<+∞,i∈N,t∈T Among them, constraint C1 is the application offloading decision constraint, which ensures that the split ratio of industrial software data splittable applications is between 0 and 1; constraint C2 is the constraint of the abandonment indicator variable, which ensures that the application has only two states: abandoned and not abandoned; constraint C3 is the constraint of available energy, which ensures that the amount of power obtained by the mobile device does not exceed the available energy in the current time slot; constraint C4 is the energy causal constraint, which ensures that the energy consumption generated by computing offloading of all devices in the current time slot does not exceed the remaining energy of the device in the current time slot; constraint C5 is the completion deadline constraint, which ensures that the maximum value of the completion time of some applications executed locally and the completion time of some applications executed on the server does not exceed the completion deadline of the application; constraint C6 is the transmission power constraint, which ensures that the transmission power of all devices when unloading is non-negative and does not exceed the maximum value of the transmission power; constraint C7 is the computing frequency constraint, which ensures that the computing frequency of all devices when performing local computing tasks is non-negative and does not exceed the maximum value of the computing frequency.

9. The industrial software resource sharing method based on key feature extraction according to claim 1 is characterized in that: The Lyapunov drift plus penalty function in step 4 is: Where ΔL(t) is the Lyapunov drift, V is a control parameter constant, and C is a constant. is a virtual energy queue, specifically expressed as where θ i is the disturbance parameter, which is specifically expressed as in is the theoretical maximum energy consumption of the application when it is executed locally and transmitted to the server for execution. Therefore, problem P1 can be reconstructed as Lyapunov drift plus penalty minimization problem P2: stC1,C2,C3,C5,C6,C7 Because P2 is an optimization problem that eliminates energy causal constraints and the devices are independent of each other, problem P2 can be decomposed into multiple single-device single-time slot optimization problems P3: stC1,C2,C3,C5,C6,C7 Because problem P3 is a computation offloading problem of a single device and a single time slot, problem P3 can be decomposed into two sub-problems: a computation offloading sub-problem and an energy acquisition sub-problem and solved separately.

10. An industrial software resource sharing method based on key feature extraction according to claim 1, characterized in that: The energy acquisition sub-problem in step 5 is: sC3 This problem is a linear optimization problem and also a convex optimization problem, so the optimal solution for the energy that device i can obtain in time slot t can be obtained: for: Then construct the computation offloading subproblem: stC1,C2,C5,C6,C7 Let’s discuss problem P3(b) in two cases: i,t =1,I i,d (t) = 1 and ζ i,t =1,I i,d (t) = 0, that is, the two cases of application being abandoned and not abandoned are discussed separately; When i,t =1,I i,d When (t) = 1, it is easy to know that the minimum value of the objective function is Vφ; When i,t =1,I i,d (t) = 0, problem P3(b) can be transformed into problem P4: stC1,C5,C6,C7 By analyzing problem P4, we know that λ i (t) and f i,l (t) are mutually coupled, λ i (t) and P i (t) are also coupled to each other, but f i,l (t) and P i (t) is uncoupled, so problem P4 can be decomposed into two stages: the first stage is the resource allocation problem, and the second stage is the resource allocation problem, which is divided into the computational resource allocation sub-problem and the power allocation sub-problem; First, construct the computing resource allocation subproblem P5: stC5,C7 set up In any case, when When , problem P5 is a convex optimization problem. By using the first-order condition for the existence of the optimal solution of the convex function, the optimal solution of the CPU frequency can be obtained as: Then construct the power allocation subproblem P6: stC5,C6 set up In any case, when When , problem P6 is a convex optimization problem. First, from the constraint condition C5, we can get: in According to the first-order condition for the existence of the optimal solution of a convex function, is the global optimal solution of the objective function on R, so the optimal solution of transmission power can be obtained Before constructing the computation offloading decision problem P7, and Substituting into problem P4, the objective function of problem P7 is: Then we get problem P7 which only contains the split ratio variable: stC1 Since the second-order derivative of the objective function is semi-positive definite, problem P7 is a convex optimization problem, so let g′(λ i (t)) = 0, and use a one-dimensional search algorithm such as Newton's method to obtain the optimal solution λ for the segmentation ratio variable i * (t).

Images