Anomaly detection model generation method, device and equipment supporting privacy protection

Abstract

The present invention discloses a method, device and electronic device for generating anomaly detection models supporting privacy protection. The method comprises: sending a preset global model and a preset key pair to multiple clients, wherein the preset key pair includes a preset public key and a preset secret key, receiving multiple encrypted local anomaly detection models fed back by multiple clients according to the preset global model and the preset public key, aggregating the multiple encrypted local anomaly detection models to generate a target global model, generating multiple encrypted sparse subnetworks corresponding to multiple clients one by one according to the target global model and the multiple encrypted local anomaly detection models, sending the multiple encrypted sparse subnetworks one by one to the corresponding multiple clients, realizing anomaly detection model interaction in a distributed learning process through the encrypted sparse subnetworks, and completing the training of the anomaly detection model under the premise of effectively protecting the privacy of the model.

Description

Technical Field

[0001] The present invention relates to the field of model training technology, and in particular to a method, device and equipment for generating anomaly detection models supporting privacy protection. Background Art

[0002] In recent years, as a common paradigm in distributed learning, people have shown increasing interest in the study of federated learning. However, there are still many problems to be solved in the application of federated learning, such as high communication costs, statistical heterogeneity, and communication security. In related technologies, the main method to improve communication efficiency in the federated learning process is to update the model parameters of the client, such as structured updates of model parameters, to reduce the communication cost in federated learning. However, this update method ignores the personalized performance of the client's local model, and cannot directly obtain the global model. In addition, the system log contains sensitive information about IoT devices or edge devices. There is a risk of privacy leakage during the data transmission process of federated learning. Summary of the invention

[0003] In response to the technical problem of privacy leakage caused by federated learning in the prior art, the present invention provides a method, device and equipment for generating an anomaly detection model that supports privacy protection.

[0004] To achieve the above objectives, the present invention is implemented through the following technical solutions:

[0005] According to a first aspect of an embodiment of the present invention, a method for generating an anomaly detection model supporting privacy protection is provided, which is applied to a cloud server, and the method includes:

[0006] Sending a preset global model and a preset key pair to multiple clients, wherein the preset key pair includes a preset public key and a preset secret key;

[0007] Receiving multiple encrypted local anomaly detection models fed back by the multiple clients according to the preset global model and the preset public key, wherein the multiple clients correspond one-to-one to the multiple encrypted local anomaly detection models;

[0008] Aggregating the multiple encrypted local anomaly detection models to generate a target global model;

[0009] Generate a plurality of encrypted sparse subnetworks corresponding one-to-one to the plurality of clients according to the target global model and the plurality of encrypted local anomaly detection models;

[0010] The multiple encrypted sparse subnetworks are sent one by one to the corresponding multiple clients, and the encrypted sparse subnetworks are used to instruct the clients to decrypt the encrypted sparse subnetworks according to the preset key, generate a target sparse subnetwork, and update the local anomaly detection model according to the target sparse subnetwork.

[0011] Optionally, generating a plurality of encrypted sparse subnetworks corresponding one-to-one to the plurality of clients according to the target global model and the plurality of encrypted local anomaly detection models comprises:

[0012] Generate a plurality of masks corresponding to the plurality of clients one by one according to the target global model and the plurality of encrypted local anomaly detection models;

[0013] According to the multiple masks and the target global model, the multiple encrypted sparse sub-networks corresponding to the multiple clients are generated one by one.

[0014] Optionally, generating the plurality of encrypted sparse subnetworks corresponding one-to-one to the plurality of clients according to the plurality of masks and the target global model comprises:

[0015] For any mask among the multiple masks, filter out model parameters corresponding to the encrypted sparse subnetwork from the target global model according to the mask;

[0016] According to the model parameters, the multiple encrypted sparse sub-networks corresponding to the multiple clients are generated.

[0017] Optionally, aggregating the multiple encrypted local anomaly detection models to generate a target global model includes:

[0018] Determining the weight quantity of the model parameters in each encrypted local anomaly detection model according to the multiple encrypted local anomaly detection models;

[0019] Determining weight masks of the respective encrypted local anomaly detection models according to the weight quantity;

[0020] The multiple encrypted local anomaly detection models are aggregated according to the weight mask to generate the target global model.

[0021] Optionally, aggregating the multiple encrypted local anomaly detection models according to the weight mask to generate the target global model includes:

[0022] According to the weight mask, the model parameters of each encrypted local anomaly detection model are screened to determine the model parameters whose corresponding weight values ​​are not zero as the target model parameters;

[0023] Aggregate multiple target model parameters corresponding to the multiple encrypted local anomaly detection models to generate the target global model.

[0024] According to a second aspect of an embodiment of the present invention, a method for generating an anomaly detection model supporting privacy protection is provided, the method being applied to a client, and the method comprising:

[0025] Receiving a preset global model and a preset key pair sent by a cloud server, wherein the preset key pair includes a preset public key and a preset secret key;

[0026] Pruning the preset global model through preset iterative pruning rules to generate a local anomaly detection model;

[0027] Encrypting the local anomaly detection model according to the preset public key to generate an encrypted local anomaly detection model;

[0028] Sending the encrypted local anomaly detection model to the cloud server, the encrypted local anomaly detection model is used to instruct the cloud server to aggregate the received multiple encrypted local anomaly detection models to generate a target global model, and generating multiple encrypted sparse subnetworks corresponding to the multiple clients one by one according to the target global model and the multiple local anomaly detection models;

[0029] Decrypting the encrypted sparse subnetwork fed back by the server according to the preset key to generate a target local anomaly detection model;

[0030] Iteratively train the target local anomaly detection model, and when the target local anomaly detection model is updated, send the updated target local anomaly detection model to the cloud server.

[0031] Optionally, pruning the preset global model by using a preset iterative pruning rule to generate a local anomaly detection model includes:

[0032] Iteratively training the preset global model according to the preset iterative pruning rule to generate an accuracy value of the preset global model;

[0033] When the accuracy value exceeds a set accuracy threshold, determining the number of iterative training times of the preset global model;

[0034] The preset local model is pruned according to the number of iterative training times and the preset pruning rate to generate the local anomaly detection model.

[0035] According to a third aspect of an embodiment of the present invention, there is provided a device for generating an anomaly detection model supporting privacy protection, which is applied to a cloud server, and the device includes:

[0036] A first sending module, configured to send a preset global model and a preset key pair to multiple clients, wherein the preset key pair includes a preset public key and a preset secret key;

[0037] A receiving module, configured to receive a plurality of encrypted local anomaly detection models fed back by the plurality of clients according to the preset global model and the preset public key, wherein the plurality of clients correspond one to one to the plurality of encrypted local anomaly detection models;

[0038] A first generating module, configured to aggregate the multiple encrypted local anomaly detection models to generate a target global model;

[0039] A second generating module, configured to generate a plurality of encrypted sparse sub-networks corresponding one-to-one to the plurality of clients according to the target global model and the plurality of encrypted local anomaly detection models;

[0040] The second sending module is used to send the multiple encrypted sparse subnetworks to the corresponding multiple clients one by one, and the encrypted sparse subnetworks are used to instruct the clients to decrypt the encrypted sparse subnetworks according to the preset key, generate a target sparse subnetwork, and update the local anomaly detection model according to the target sparse subnetwork.

[0041] According to a fourth aspect of an embodiment of the present invention, a device for generating an anomaly detection model supporting privacy protection is provided, which is applied to a client, and the device includes:

[0042] A first receiving module, configured to receive a preset global model and a preset key pair sent by a cloud server, wherein the preset key pair includes a preset public key and a preset secret key;

[0043] A first generation module prunes the preset global model through a preset iterative pruning rule to generate a local anomaly detection model;

[0044] An encryption module, used to encrypt the local anomaly detection model according to the preset public key to generate an encrypted local anomaly detection model;

[0045] A second generation module is used to send the encrypted local anomaly detection model to the cloud server, wherein the encrypted local anomaly detection model is used to instruct the cloud server to aggregate the received multiple encrypted local anomaly detection models to generate a target global model, and generate multiple encrypted sparse subnetworks corresponding to the multiple clients one by one according to the target global model and the multiple local anomaly detection models;

[0046] A third generating module is used to decrypt the encrypted sparse sub-network fed back by the server according to the preset key to generate a target local anomaly detection model;

[0047] The execution module is used to iteratively train the target local anomaly detection model and send the updated target local anomaly detection model to the cloud server when the target local anomaly detection model is updated.

[0048] According to a fifth aspect of an embodiment of the present invention, there is provided an electronic device, including:

[0049] a memory having a computer program stored thereon;

[0050] A processor is used to execute the computer program in the memory to implement the steps of the method for generating an anomaly detection model supporting privacy protection as described in any one of the first aspects of the present disclosure, or to implement the steps of the method for generating an anomaly detection model supporting privacy protection as described in any one of the second aspects of the present disclosure.

[0051] The present invention provides a method, device and electronic device for generating anomaly detection models supporting privacy protection. Compared with the prior art, it has the following beneficial effects:

[0052] In the above manner, a preset global model and a preset key pair are sent to multiple clients, wherein the preset key pair includes a preset public key and a preset secret key, and multiple encrypted local anomaly detection models fed back by multiple clients according to the preset global model and the preset public key are received, wherein multiple clients correspond to multiple encrypted local anomaly detection models one by one, multiple encrypted local anomaly detection models are aggregated to generate a target global model, multiple encrypted sparse subnetworks corresponding to multiple clients are generated according to the target global model and multiple encrypted local anomaly detection models, and multiple encrypted sparse subnetworks are sent to the corresponding multiple clients one by one, and the encrypted sparse subnetworks are used to instruct the client to decrypt the encrypted sparse subnetwork according to the preset key to generate the target sparse subnetwork, and the local anomaly detection model is updated according to the target sparse subnetwork. The lottery hypothesis is combined with federated learning, and after the target global model is generated by aggregating multiple encrypted local anomaly detection models, an encrypted sparse subnetwork is generated according to the encrypted local anomaly detection model and the target global model and sent to the corresponding multiple clients, and the interaction of anomaly detection models in the distributed learning process is realized through the encrypted sparse subnetwork, and the training of the anomaly detection model is completed under the premise of effectively protecting the privacy of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 The present invention provides a flowchart of a method for generating anomaly detection models supporting privacy protection.

[0054] Figure 2 The present invention is a flowchart of a method for generating a sparse subnetwork according to an exemplary embodiment.

[0055] Figure 3 The present invention is a flowchart of another method for generating anomaly detection models supporting privacy protection provided by the present invention.

[0056] Figure 4The present invention is a flowchart of another method for generating anomaly detection model supporting privacy protection provided by the present invention.

[0057] Figure 5 A block diagram of a device for generating anomaly detection models supporting privacy protection provided by the present invention.

[0058] Figure 6 A block diagram of another anomaly detection model generation device supporting privacy protection provided according to the present invention. DETAILED DESCRIPTION

[0059] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0060] Figure 1 For a flowchart of a method for generating anomaly detection models supporting privacy protection according to the present invention, please refer to Figure 1 The present invention provides a technical solution: the method comprises the following steps.

[0061] In step S11, a preset global model and a preset key pair are sent to multiple clients, where the preset key pair includes a preset public key and a preset secret key.

[0062] For example, in this embodiment, the cloud server and the client can exchange data through wireless communication. A preset global model is set in the cloud server. The client does not have an anomaly detection model installed in the initial state. The cloud server randomly selects a group of M clients to participate in federated learning, and broadcasts the preset global model to the fog nodes corresponding to the M clients. Among them, for clients with the same attributes, a fog node can be set to manage multiple clients. A fog node can correspond to multiple clients. The fog nodes can be divided according to the attributes of each client. For example, clients in the same city can be summarized in the same fog node for management according to the geographical attributes of the client.

[0063] It is worth mentioning that in this embodiment, the server generates a preset key pair, which includes a preset public key public_key and a preset key private_key. The public key is used for encryption and the private key is used for decryption, and the preset public key and the preset key are sent to each client. Each client uses the public key public_key to encrypt the local anomaly detection model, obtains the encrypted local anomaly detection model, and uploads the encrypted local anomaly model to the server.

[0064] In step S12, multiple encrypted local anomaly detection models fed back by multiple clients according to a preset global model and a preset public key are received, wherein the multiple clients correspond one-to-one to the multiple encrypted local anomaly detection models.

[0065] For example, this embodiment is applied to a cloud server, which updates and trains the preset global model in the cloud server by interacting with multiple clients set on the fog node side, and determines the target local anomaly detection model parameters of each client based on the trained target global model, sends the target local anomaly detection model parameters to the corresponding client, and updates the local anomaly detection model in the client based on the target local anomaly detection model parameters, thereby completing the federated learning process of the global model through the interaction of model parameters between the cloud server and the client. The cloud server can divide clients with the same attributes into the same fog node based on the attributes of the client, wherein the attribute can be the regional attribute of the client's location, and the clients in the same area are divided into the same fog node accordingly. There can be multiple fog nodes corresponding to the cloud server. After the fog node receives the preset global model sent by the cloud server, it trains the preset global model according to the daily operation parameters in the fog node to generate the local anomaly detection model corresponding to the fog node. The fog node sends the local anomaly detection model to the client. The fog node aggregates multiple local anomaly detection models sent by multiple clients under its jurisdiction to generate the local anomaly detection model corresponding to the fog node, and encrypts the local anomaly detection model according to the preset public key to generate an encrypted local anomaly detection model, which is sent to the cloud server. The cloud server establishes a communication connection with multiple clients and accepts multiple encrypted local anomaly detection models fed back by multiple clients.

[0066] In step S13, multiple encrypted local anomaly detection models are aggregated to generate a target global model.

[0067] For example, in this embodiment, after the cloud server receives the encrypted local anomaly detection models fed back by multiple clients, the multiple encrypted local anomaly detection models are aggregated to generate a target global model. It is worth mentioning that, considering that each client only acts on part of the weight parameters of the preset global model, after the cloud server receives the encrypted local anomaly detection models fed back by the client set, it cannot simply average the multiple encrypted local anomaly detection models received by the number of clients, but needs to aggregate the multiple encrypted local anomaly detection models according to the weight values ​​of each encrypted local anomaly detection model. For example, in this embodiment, the weight value of each encrypted local anomaly detection model can be determined by calculating the weight number of each encrypted local anomaly detection model, and multiple encrypted local anomaly detection models can be aggregated according to the weight value to generate a target global model.

[0068] Optionally, the above step S13 includes:

[0069] Determining the weight quantity of the model parameters in each encrypted local anomaly detection model according to the multiple encrypted local anomaly detection models;

[0070] Determining weight masks of the respective encrypted localized anomaly detection models according to the weight quantity;

[0071] The multiple encrypted local anomaly detection models are aggregated according to the weight mask to generate the target global model.

[0072] For example, in this embodiment, the number of weights of the encrypted local anomaly detection model is determined by calculation, and the weight mask of each encrypted local anomaly detection model is determined according to the weight number. For example, the weight mask corresponding to the encrypted local anomaly detection model can be determined in the following manner:

[0073]

[0074] Among them, the function Map each element X input into the matrix to the inverse. The mapping function can be:

[0075]

[0076] Among them, a 0 element is mapped to 0, so the model parameters corresponding to the 0 element in the encrypted local anomaly detection model are masked and will not contribute to the update of the preset global model during aggregation. To encrypt the weight values ​​of the corresponding elements in the local anomaly detection model, the parameters of multiple encrypted local anomaly detection models can be aggregated according to the weight mask to generate the target global model.

[0077] Optionally, in one implementation, the step of aggregating the multiple encrypted local anomaly detection models according to the weight mask to generate the target global model includes:

[0078] According to the weight mask, the model parameters of each encrypted local anomaly detection model are screened to determine the model parameters whose corresponding weight values ​​are not zero as the target model parameters;

[0079] Multiple target model parameters corresponding to the multiple encrypted local anomaly detection models are aggregated to generate a target global model.

[0080] For example, in this embodiment, the weight mask is used to represent the contribution of the corresponding client to the preset global model. In this embodiment, the weight value of the encrypted local anomaly detection model in the aggregation process is determined according to the weight mask, and the model parameters with a weight value of 0 are removed from the encrypted local anomaly detection model according to the weight value, thereby obtaining the target model parameters. The cloud server aggregates multiple target model parameters sent from each client to generate a target global model. It is worth mentioning that in this embodiment, the reciprocal value of each element can be determined by a mapping function, and the reciprocal value is judged. When the reciprocal is equal to 0, the weight value is 0, and the model parameters with a weight value not equal to 0 are determined from the encrypted local anomaly detection model as the target model parameters, and the various target model parameters are aggregated into the target global model of the cloud server.

[0081] In step S14, a plurality of encrypted sparse sub-networks corresponding one-to-one to the plurality of clients are generated according to the target global model and the plurality of encrypted local anomaly detection models.

[0082] For example, in this embodiment, after the target global model of the cloud server is aggregated and generated through the above steps, the encrypted local anomaly detection model sent by each client is pruned according to the target global model to generate a sparse subnetwork corresponding to each client, and the sparse subnetwork is encrypted according to the preset public key to generate an encrypted sparse subnetwork.

[0083] Figure 2 FIG. 1 is a flow chart of a method for generating a sparse subnetwork according to an exemplary embodiment. Figure 2 As shown, the above step S14 includes the following steps.

[0084] In step S21, a plurality of masks corresponding one-to-one to the plurality of clients are generated according to the target global model and the plurality of encrypted local anomaly detection models.

[0085] It should be noted that in federated learning, clients are able to share part of the local model, which can achieve better performance compared to federated aggregation. Pruning methods can eliminate unnecessary structures in the neural network so that federated learning can be more effectively performed by sharing pruned models. The lottery network hypothesis shows the effectiveness of pruning during initialization, rather than being developed for efficient inference computing and storage. Therefore, the lottery hypothesis is used for federated learning in this embodiment. The lottery hypothesis demonstrates a method for extracting sparse subnetworks from the base network. Through iterative training, pruning and initialization processes, the mask of the corresponding client is determined from the client. The mask can be used to prune the global model, and the sparse subnetwork generated after pruning is sent to the corresponding client, and fed back to the corresponding client through the client.

[0086] In step S22, a plurality of encrypted sparse sub-networks corresponding one-to-one to the plurality of clients are generated according to the plurality of masks and the target global model.

[0087] For example, in this embodiment, the sparse subnetwork is formally LTNs (Lottery Ticket Networks), which can be represented as m⊙θ, where m∈{0,1} |θ| is the mask on the base network, θ and ⊙ are element-wise products. For each client k and its individual dataset D k , obtain the specific LTN, θ and K, and determine the mask m corresponding to the client through the above parameters k , in order to realize the formulation of the client LTN network, and to perform federated learning through the customized LTN. In this embodiment, the encrypted local anomaly detection model fed back by the client is compared with the target global model to determine the personalized mask corresponding to each client, and the target global model is extracted through the mask, thereby deriving the sparse sub-network corresponding to each client in the target global model, and the sparse sub-network is used to perform model training on the encrypted local anomaly detection model corresponding to the client.

[0088] Optionally, in one implementation, the above step S22 includes:

[0089] For any mask among the multiple masks, filter out model parameters corresponding to the sparse sub-network from the target global model according to the mask;

[0090] According to the model parameters, a sparse sub-network corresponding to the client is generated.

[0091] For example, in this embodiment, the cloud server coordinates joint learning through the mask determined in the above steps. Specifically, the encrypted local anomaly detection model fed back by the client determines the model parameters corresponding to the client from the model parameters corresponding to the target global model generated after aggregation, and based on the model parameters, a sparse sub-network corresponding to the client is extracted from the target global model through a preset derivation algorithm.

[0092] Through the above method, the encrypted local anomaly detection model is compared with the aggregated target global model to determine the mask of each client, and the model parameters of the target global model are pruned according to the mask to generate a sparse sub-network of each client, thereby reducing the transmission of invalid model parameters in the federated learning process and saving the communication cost in the federated learning process.

[0093] In step S15, multiple encrypted sparse subnetworks are sent one by one to the corresponding multiple clients, and the encrypted sparse subnetworks are used to instruct the clients to decrypt the encrypted sparse subnetworks according to a preset key, generate a target sparse subnetwork, and update the local anomaly detection model according to the target sparse subnetwork.

[0094] By way of example, in this embodiment, the cloud server sends the encrypted sparse subnetwork corresponding to the client to the client, and the client feeds back the encrypted sparse subnetwork to the corresponding client. The client decrypts the encrypted sparse subnetwork according to the preset key to generate the target sparse subnetwork, and iteratively trains the local anomaly detection model through the target sparse subnetwork to update the local anomaly detection model, so that the local anomaly detection model better meets the client's usage requirements. By way of example, after the client receives the sparse subnetwork, it can prune and iterate the local anomaly detection model based on the sparse subnetwork, train a corresponding number of times on the local data set to complete multiple iterations of the preset model, and upload the local anomaly detection model to the corresponding client when the number of iterations reaches the preset number threshold, or after the local anomaly detection model completes the iteration, and the client evaluates the local anomaly detection model on the local verification set. After the verification accuracy reaches the set accuracy threshold, the client issues a stop training instruction to the client, and prunes the local anomaly detection model and feeds it back to the cloud server. In this embodiment, a distributed model training method is adopted, whereby the client uploads the trained model parameters to the cloud server in an encrypted manner. The cloud server aggregates the model parameters uploaded by multiple clients to generate a global model and sends it to the corresponding clients. The client then trains the global model based on local personalized parameters. Through the distributed model training between the client and the cloud server, the model obtained in the client can better meet the personalized needs of the client.

[0095] In the above manner, a preset global model and a preset key pair are sent to multiple clients, wherein the preset key pair includes a preset public key and a preset secret key, and multiple encrypted local anomaly detection models fed back by multiple clients according to the preset global model and the preset public key are received, wherein multiple clients correspond to multiple encrypted local anomaly detection models one by one, multiple encrypted local anomaly detection models are aggregated to generate a target global model, multiple encrypted sparse subnetworks corresponding to multiple clients are generated according to the target global model and multiple encrypted local anomaly detection models, and multiple encrypted sparse subnetworks are sent to the corresponding multiple clients one by one, and the encrypted sparse subnetworks are used to instruct the client to decrypt the encrypted sparse subnetwork according to the preset key to generate the target sparse subnetwork, and the local anomaly detection model is updated according to the target sparse subnetwork. Thus, the lottery hypothesis is combined with federated learning, and after the target global model is generated by aggregating multiple encrypted local anomaly detection models, an encrypted sparse subnetwork is generated according to the encrypted local anomaly detection model and the target global model and sent to the corresponding multiple clients, and the interaction of anomaly detection models in the distributed learning process is realized through the encrypted sparse subnetwork, and the training of the anomaly detection model is completed under the premise of effectively protecting the privacy of the model.

[0096] Figure 3 For a flowchart of another method for generating anomaly detection models supporting privacy protection according to the present invention, please refer to Figure 3 , the method is applied in the client and includes the following steps.

[0097] In step S31, a preset global model and a preset key pair sent by a cloud server are received, wherein the preset key pair includes a preset public key and a preset secret key.

[0098] For example, this embodiment is applied to the client, and the way in which the client receives the preset global model and the preset key pair sent by the cloud server is the same as in the above step S11, and can refer to the above step S11, which will not be repeated here.

[0099] In step S32, the preset global model is pruned by preset iterative pruning rules to generate a local anomaly detection model.

[0100] By way of example, this embodiment is applied to a client, which prunes a received preset global model through preset iterative pruning rules to generate a local anomaly detection model corresponding to the client.

[0101] Optionally, in one implementation, the above step S32 includes:

[0102] Iteratively training the preset global model according to the preset iterative pruning rule to generate an accuracy value of the preset global model;

[0103] When the accuracy value exceeds a set accuracy threshold, determining the number of iterative training times of the preset global model;

[0104] The preset local model is pruned according to the number of iterative training times and the preset pruning rate to generate the local anomaly detection model.

[0105] For example, in this embodiment, after the client receives the sparse sub-network, it can prune and iterate the local anomaly detection model based on the sparse sub-network, train a corresponding number of times on the local data set to complete multiple iterations of the preset model, and when the number of iterations reaches a preset threshold, or after the local anomaly detection model completes the iteration, upload the local anomaly detection model to the corresponding client, and the client evaluates the local anomaly detection model on the local verification set. After the verification accuracy reaches the set accuracy threshold, the client issues a stop training instruction to the client, and prunes the local anomaly detection model and feeds it back to the cloud server.

[0106] In step S33, the local anomaly detection model is encrypted according to a preset public key to generate an encrypted local anomaly detection model.

[0107] For example, in this embodiment, the client encrypts the local anomaly detection model by using a preset public key to generate an encrypted local anomaly detection model. For example, the cloud server generates a preset key pair and sends it to each client: c1, c2, ..., c n ; The preset key pair includes a preset public key public_key and a preset private key private_key, the public key is used for encryption, and the private key is used for decryption; each client c k Use the public key public_key to encrypt the local anomaly detection model m k , get the encrypted local anomaly model E pub (m k ), and upload the encrypted local anomaly model to the cloud server; the cloud server uses Add pub Function Processing E pub (m1), E pub (m2),…,E pub (m n ), obtain the target global model of this round, that is, The target global model of this round is sent to each client to be used as the model for the next round of local training; each client c k After receiving the target global model sent by the cloud server, use the private_key key to decrypt the global model and obtain the target local anomaly detection model: In this embodiment, the homomorphic encryption technology can ensure that the decryption process of the above steps meets The above steps enable the client model to perform additive averaging operations in an encrypted state, thereby ensuring the security of transmission between the local model and the global model.

[0108] In step S34, the encrypted local anomaly detection model is sent to the cloud server, and the encrypted local anomaly detection model is used to instruct the cloud server to aggregate the received multiple encrypted local anomaly detection models to generate a target global model, and generate multiple encrypted sparse subnetworks corresponding to multiple clients one by one according to the target global model and the multiple local anomaly detection models;

[0109] For example, in this embodiment, the encrypted local anomaly detection model generated by the above steps is sent to the cloud server, and the cloud server aggregates the received multiple encrypted local anomaly detection models to generate a target global model, and generates multiple encrypted sparse sub-networks corresponding to the multiple clients one by one based on the target global model and the multiple encrypted local anomaly detection models. The way in which the cloud server generates encrypted sparse sub-networks through the received encrypted local anomaly detection models can refer to the above steps S12 and S13 and will not be repeated here.

[0110] In step S35, the encrypted sparse subnetwork fed back by the server is decrypted according to the preset key to generate a target local anomaly detection model.

[0111] For example, in this embodiment, the cloud server aggregates multiple encrypted local anomaly detection models fed back by multiple clients to generate a target global model, and compares the target global model with the encrypted local anomaly detection model corresponding to each client, generates an encrypted sparse subnetwork corresponding to each client, and sends the encrypted sparse subnetwork to the corresponding client. The client decrypts the encrypted sparse subnetwork according to a preset key to generate a target local anomaly detection model.

[0112] In step S35, the target local anomaly detection model is iteratively trained, and when the target local anomaly detection model is updated, the updated target local anomaly detection model is sent to the cloud server.

[0113] For example, in this embodiment, the client first initializes the preset global model and the corresponding mask, and then in the i-th pruning iteration process, the client trains a preset number of rounds on the local data set, and prunes the preset global model at a preset pruning rate to complete the iterative training of the preset global model, and feeds back the encrypted local anomaly detection model generated after training to the cloud server, so that the cloud server aggregates multiple local global models sent by multiple clients to generate a target global model, and replaces the pre-reprinted preset global model with the target global model, thereby updating the global model in the cloud server.

[0114] In the above manner, a preset global model and a preset key pair sent by a cloud server are received, wherein the preset key pair includes a preset public key and a preset secret key. The preset global model is pruned according to a preset iterative pruning rule to generate a local anomaly detection model. The local anomaly detection model is encrypted according to a preset public key to generate an encrypted local anomaly detection model. The encrypted local anomaly detection model is sent to the cloud server. The encrypted local anomaly detection model is used to instruct the cloud server to aggregate multiple received encrypted local anomaly detection models to generate a target global model. Multiple encrypted sparse subnetworks corresponding to multiple clients are generated according to the target global model and the multiple local anomaly detection models. The encrypted sparse subnetwork fed back by the server is decrypted according to the preset key to generate a target local anomaly detection model. The target local anomaly detection model is iteratively trained. When the target local anomaly detection model is updated, the updated target local anomaly detection model is sent to the cloud server. In this way, the lottery hypothesis is combined with federated learning. After generating the target global model by aggregating multiple encrypted local anomaly detection models, an encrypted sparse subnetwork is generated according to the encrypted local anomaly detection model and the target global model and sent to the client. The encrypted sparse subnetwork is used to realize the interaction of anomaly detection models in the distributed learning process, and the training of the anomaly detection model is completed under the premise of effectively protecting the privacy of the model.

[0115] Figure 4 For a flowchart of another method for generating anomaly detection models supporting privacy protection provided by the present invention, please refer to Figure 4 The present invention provides a technical solution: the method comprises the following steps.

[0116] 1. The cloud server exports LTN by shielding the global model;

[0117] 2. The cloud server sends LTN to the corresponding client;

[0118] 3. The client performs client update according to LTN;

[0119] 4. Upload the personalized anomaly detection model to the cloud server;

[0120] 5. The cloud server aggregates the personalized anomaly detection models according to the masks to update the global model.

[0121] In the above way, the lottery hypothesis is combined with federated learning, and the global model is updated by sending masks, which reduces the computational cost and communication cost in the federated learning process.

[0122] Figure 5 For a block diagram of a device for generating anomaly detection models supporting privacy protection according to the present invention, see Figure 5, applied to a cloud server, the device 100 , comprises: a first sending module 110 , a receiving module 120 , a first generating module 130 , a second generating module 140 and a second sending module 150 .

[0123] A first sending module 110 is used to send a preset global model and a preset key pair to multiple clients, wherein the preset key pair includes a preset public key and a preset secret key;

[0124] A receiving module 120, configured to receive a plurality of encrypted local anomaly detection models fed back by the plurality of clients according to the preset global model and the preset public key, wherein the plurality of clients correspond one to one to the plurality of encrypted local anomaly detection models;

[0125] A first generating module 130, configured to aggregate the multiple encrypted local anomaly detection models to generate a target global model;

[0126] A second generating module 140 is used to generate a plurality of encrypted sparse sub-networks corresponding to the plurality of clients one by one according to the target global model and the plurality of encrypted local anomaly detection models;

[0127] The second sending module 150 is used to send the multiple encrypted sparse subnetworks to the corresponding multiple clients one by one, and the encrypted sparse subnetworks are used to instruct the client to decrypt the encrypted sparse subnetwork according to the preset key, generate a target sparse subnetwork, and update the local anomaly detection model according to the target sparse subnetwork.

[0128] Optionally, the second generating module 140 includes:

[0129] A first generating submodule, configured to generate a plurality of masks corresponding to the plurality of clients one by one according to the target global model and the plurality of encrypted local anomaly detection models;

[0130] The second generating submodule is used to generate the plurality of encrypted sparse sub-networks corresponding to the plurality of clients one by one according to the plurality of masks and the target global model.

[0131] Optionally, the second generation submodule includes:

[0132] A screening unit, configured to screen out a model parameter corresponding to the encrypted sparse subnetwork from the target global model according to any mask among the multiple masks;

[0133] A generating unit is used to generate the multiple encrypted sparse sub-networks corresponding to the multiple clients according to the model parameters.

[0134] Optionally, the first generating module 130 includes:

[0135] A first determination submodule, configured to determine the weight quantity of the model parameters in each encrypted local anomaly detection model according to the multiple encrypted local anomaly detection models;

[0136] A second determination submodule, configured to determine a weight mask of each encrypted local anomaly detection model according to the weight quantity;

[0137] The third generating submodule is used to aggregate the multiple encrypted local anomaly detection models according to the weight mask to generate the target global model.

[0138] Optionally, the third generation submodule is used to:

[0139] According to the weight mask, the model parameters of each encrypted local anomaly detection model are screened to determine the model parameters whose corresponding weight values ​​are not zero as the target model parameters;

[0140] Aggregate multiple target model parameters corresponding to the multiple encrypted local anomaly detection models to generate the target global model.

[0141] Figure 6 For a block diagram of another anomaly detection model generation device supporting privacy protection provided by the present invention, please refer to Figure 6 , applied to the client, the device 200, comprises: a first receiving module 210, a first generating module 220, an encryption module 230, a second generating module 240, a third generating module 250 and an execution module 260.

[0142] A first receiving module 210 is used to receive a preset global model and a preset key pair sent by a cloud server, wherein the preset key pair includes a preset public key and a preset secret key;

[0143] A first generating module 220 prunes the preset global model by using preset iterative pruning rules to generate a local anomaly detection model;

[0144] An encryption module 230, configured to encrypt the local anomaly detection model according to the preset public key to generate an encrypted local anomaly detection model;

[0145] A second generation module 240 is used to send the encrypted local anomaly detection model to the cloud server, wherein the encrypted local anomaly detection model is used to instruct the cloud server to aggregate the received multiple encrypted local anomaly detection models to generate a target global model, and generate multiple encrypted sparse subnetworks corresponding to the multiple clients one by one according to the target global model and the multiple local anomaly detection models;

[0146] A third generating module 250 is used to decrypt the encrypted sparse sub-network fed back by the server according to the preset key to generate a target local anomaly detection model;

[0147] The execution module 260 is used to iteratively train the target local anomaly detection model, and when the target local anomaly detection model is updated, send the updated target local anomaly detection model to the cloud server.

[0148] Optionally, the first generating module 220 is configured to:

[0149] Iteratively training the preset global model according to the preset iterative pruning rule to generate an accuracy value of the preset global model;

[0150] When the accuracy value exceeds a set accuracy threshold, determining the number of iterative training times of the preset global model;

[0151] The preset local model is pruned according to the number of iterative training times and the preset pruning rate to generate the local anomaly detection model.

[0152] Based on the same inventive concept, this embodiment further provides an electronic device, including:

[0153] a memory having a computer program stored thereon;

[0154] A processor is used to execute the computer program in the memory to implement the steps of the method for generating an anomaly detection model supporting privacy protection in any one of the above solutions.

[0155] Based on the above ideal embodiments of this application, the relevant staff can make various changes and modifications without departing from the technical concept of this application through the above description. The technical scope of this application is not limited to the content in the specification, and its technical scope must be determined according to the scope of the claims.

[0156] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for generating anomaly detection models supporting privacy protection, characterized in that: Applied to a cloud server, the method comprises: Sending a preset global model and a preset key pair to multiple clients, wherein the preset key pair includes a preset public key and a preset secret key; Receiving multiple encrypted local anomaly detection models fed back by the multiple clients according to the preset global model and the preset public key, wherein the multiple clients correspond one-to-one to the multiple encrypted local anomaly detection models; Aggregating the multiple encrypted local anomaly detection models to generate a target global model; Generate a plurality of encrypted sparse subnetworks corresponding one-to-one to the plurality of clients according to the target global model and the plurality of encrypted local anomaly detection models; Sending the multiple encrypted sparse subnetworks to the corresponding multiple clients one by one, the encrypted sparse subnetworks are used to instruct the clients to decrypt the encrypted sparse subnetworks according to the preset key, generate a target sparse subnetwork, and update the local anomaly detection model according to the target sparse subnetwork; The step of generating a plurality of encrypted sparse subnetworks corresponding to the plurality of clients one by one according to the target global model and the plurality of encrypted local anomaly detection models comprises: Generate a plurality of masks corresponding to the plurality of clients one by one according to the target global model and the plurality of encrypted local anomaly detection models; Generate the plurality of encrypted sparse subnetworks corresponding one-to-one to the plurality of clients according to the plurality of masks and the target global model; The step of generating the plurality of encrypted sparse subnetworks corresponding one-to-one to the plurality of clients according to the plurality of masks and the target global model comprises: For any mask among the multiple masks, filter out model parameters corresponding to the encrypted sparse subnetwork from the target global model according to the mask; According to the model parameters, the multiple encrypted sparse sub-networks corresponding to the multiple clients are generated.

2. The method according to claim 1, characterized in that The aggregating the multiple encrypted local anomaly detection models to generate a target global model includes: Determining the weight quantity of the model parameters in each encrypted local anomaly detection model according to the multiple encrypted local anomaly detection models; Determining weight masks of the respective encrypted local anomaly detection models according to the weight quantity; The multiple encrypted local anomaly detection models are aggregated according to the weight mask to generate the target global model.

3. The method according to claim 2, characterized in that The aggregating the multiple encrypted local anomaly detection models according to the weight mask to generate the target global model includes: According to the weight mask, the model parameters of each encrypted local anomaly detection model are screened to determine the model parameters whose corresponding weight values ​​are not zero as the target model parameters; Aggregate multiple target model parameters corresponding to the multiple encrypted local anomaly detection models to generate the target global model.

4. A method for generating anomaly detection models supporting privacy protection, characterized in that: Applied to a client, the method comprises: Receiving a preset global model and a preset key pair sent by a cloud server, wherein the preset key pair includes a preset public key and a preset secret key; Pruning the preset global model through preset iterative pruning rules to generate a local anomaly detection model; Encrypting the local anomaly detection model according to the preset public key to generate an encrypted local anomaly detection model; Sending the encrypted local anomaly detection model to the cloud server, the encrypted local anomaly detection model is used to instruct the cloud server to aggregate the received multiple encrypted local anomaly detection models to generate a target global model, and generating multiple encrypted sparse subnetworks corresponding to multiple clients one by one according to the target global model and the multiple local anomaly detection models; Decrypting the encrypted sparse subnetwork fed back by the server according to the preset key to generate a target local anomaly detection model; Iteratively training the target local anomaly detection model, and when the target local anomaly detection model is updated, sending the updated target local anomaly detection model to the cloud server; The step of generating a plurality of encrypted sparse subnetworks corresponding to the plurality of clients one by one according to the target global model and the plurality of encrypted local anomaly detection models comprises: Generate a plurality of masks corresponding to the plurality of clients one by one according to the target global model and the plurality of encrypted local anomaly detection models; Generate the plurality of encrypted sparse subnetworks corresponding one-to-one to the plurality of clients according to the plurality of masks and the target global model; The step of generating the plurality of encrypted sparse subnetworks corresponding one-to-one to the plurality of clients according to the plurality of masks and the target global model comprises: For any mask among the multiple masks, filter out model parameters corresponding to the encrypted sparse subnetwork from the target global model according to the mask; According to the model parameters, the multiple encrypted sparse sub-networks corresponding to the multiple clients are generated.

5. The method according to claim 4, characterized in that The method of pruning the preset global model by using a preset iterative pruning rule to generate a local anomaly detection model includes: Iteratively training the preset global model according to the preset iterative pruning rule to generate an accuracy value of the preset global model; When the accuracy value exceeds a set accuracy threshold, determining the number of iterative training times of the preset global model; The preset global model is pruned according to the number of iterative training times and the preset pruning rate to generate the local anomaly detection model.

6. A device for generating anomaly detection models supporting privacy protection, characterized in that: Applied to a cloud server, the device comprises: A first sending module, configured to send a preset global model and a preset key pair to multiple clients, wherein the preset key pair includes a preset public key and a preset secret key; A receiving module, configured to receive a plurality of encrypted local anomaly detection models fed back by the plurality of clients according to the preset global model and the preset public key, wherein the plurality of clients correspond one to one to the plurality of encrypted local anomaly detection models; A first generating module, configured to aggregate the multiple encrypted local anomaly detection models to generate a target global model; A second generating module, configured to generate a plurality of encrypted sparse sub-networks corresponding one-to-one to the plurality of clients according to the target global model and the plurality of encrypted local anomaly detection models; A second sending module is used to send the multiple encrypted sparse subnetworks to the corresponding multiple clients one by one, and the encrypted sparse subnetwork is used to instruct the client to decrypt the encrypted sparse subnetwork according to the preset key, generate a target sparse subnetwork, and update the local anomaly detection model according to the target sparse subnetwork; The step of generating a plurality of encrypted sparse subnetworks corresponding to the plurality of clients one by one according to the target global model and the plurality of encrypted local anomaly detection models comprises: Generate a plurality of masks corresponding to the plurality of clients one by one according to the target global model and the plurality of encrypted local anomaly detection models; Generate the plurality of encrypted sparse subnetworks corresponding one-to-one to the plurality of clients according to the plurality of masks and the target global model; The step of generating the plurality of encrypted sparse subnetworks corresponding one-to-one to the plurality of clients according to the plurality of masks and the target global model comprises: For any mask among the multiple masks, filter out model parameters corresponding to the encrypted sparse subnetwork from the target global model according to the mask; According to the model parameters, the multiple encrypted sparse sub-networks corresponding to the multiple clients are generated.

7. A device for generating anomaly detection models supporting privacy protection, characterized in that: Applied to a client, the device comprises: A first receiving module, configured to receive a preset global model and a preset key pair sent by a cloud server, wherein the preset key pair includes a preset public key and a preset secret key; A first generation module prunes the preset global model through a preset iterative pruning rule to generate a local anomaly detection model; An encryption module, used to encrypt the local anomaly detection model according to the preset public key to generate an encrypted local anomaly detection model; A second generation module is used to send the encrypted local anomaly detection model to the cloud server, wherein the encrypted local anomaly detection model is used to instruct the cloud server to aggregate the received multiple encrypted local anomaly detection models to generate a target global model, and generate multiple encrypted sparse subnetworks corresponding to multiple clients one by one according to the target global model and the multiple local anomaly detection models; A third generating module is used to decrypt the encrypted sparse sub-network fed back by the server according to the preset key to generate a target local anomaly detection model; an execution module, configured to iteratively train the target local anomaly detection model, and, if the target local anomaly detection model is updated, send the updated target local anomaly detection model to the cloud server; The step of generating a plurality of encrypted sparse subnetworks corresponding to the plurality of clients one by one according to the target global model and the plurality of encrypted local anomaly detection models comprises: Generate a plurality of masks corresponding to the plurality of clients one by one according to the target global model and the plurality of encrypted local anomaly detection models; Generate the plurality of encrypted sparse subnetworks corresponding one-to-one to the plurality of clients according to the plurality of masks and the target global model; The step of generating the plurality of encrypted sparse subnetworks corresponding one-to-one to the plurality of clients according to the plurality of masks and the target global model comprises: For any mask among the multiple masks, filter out model parameters corresponding to the encrypted sparse subnetwork from the target global model according to the mask; According to the model parameters, the multiple encrypted sparse sub-networks corresponding to the multiple clients are generated.

8. An electronic device, characterized in that: include: a memory having a computer program stored thereon; A processor, configured to execute the computer program in the memory to implement the steps of the method for generating an anomaly detection model supporting privacy protection as described in any one of claims 1 to 3, or to implement the steps of the method for generating an anomaly detection model supporting privacy protection as described in any one of claims 4 to 5.

Images