Model updating method and system based on active attack

Abstract

The present disclosure provides a model updating method based on active attack, comprising: receiving a model updating request from a user end; backing up a neural network model based on the model updating request to obtain a backup model; using an invertible transformation to obfuscate the neural network model based on the model updating request to obtain an obfuscated model; sending one of the obfuscated model and the backup model to the user end; receiving a model gradient of a local model trained by the user end based on the sent model and local samples; and updating the backup model based on the received model gradient to achieve model updating.

Description

TECHNICAL FIELD

[0001] The present disclosure mainly relates to federated learning, and in particular to privacy protection in federated learning. BACKGROUND

[0002] With the application of large-scale deep learning and the increasing emphasis on data privacy protection at home and abroad, federated learning has become one of the main ways for multi-party cooperation to train deep learning models.

[0003] However, due to the distributed nature and heterogeneous computing characteristics of federated learning, it is difficult to establish a strict trust relationship with all participants. There may be a type of attacker lurking between participants who try to steal the data privacy of other participants from the slow update of the global model.

[0004] Therefore, there is a need in the art for a solution that can defend against data privacy theft based on model updates or parameter updates. SUMMARY

[0005] To solve the above technical problems, the present disclosure provides a model update scheme based on active attack, which actively changes the global model by randomly constructing reversible model confusion, thereby actively attacking the attacker's model restoration through the confused model to defend against data privacy theft.

[0006] In an embodiment of the present disclosure, a model update method based on active attack is provided, comprising: receiving a model update request from a user end; backing up a neural network model based on the model update request to obtain a backup model; using a reversible transformation to confuse the neural network model based on the model update request to obtain a confused model; sending one of the confused model and the backup model to the user end; receiving a model gradient of a local model trained by the user end based on the sent model and a local sample; and updating the backup model based on the received model gradient to achieve model update.

[0007] In another embodiment of the present disclosure, the reversible transformation includes a transformation of part of the parameters of the neural network model by shifting, substitution, linear transformation, etc.

[0008] In yet another embodiment of the present disclosure, using a reversible transformation to confuse the neural network model based on the model update request includes using the reversible transformation to confuse part of the modules of the neural network model.

[0009] In another embodiment of the present disclosure, the part of the modules of the neural network model confused using the reversible transformation can be one, multiple, or all modules of the neural network model.

[0010] In yet another embodiment of the present disclosure, using a reversible transformation to confuse the neural network model based on the model update request includes using the reversible transformation to confuse the fully connected layer of the neural network model.

[0011] In another embodiment of the disclosure, obfuscating the neural network model using the invertible transformation based on the model update request comprises obfuscating convolutional layers of the neural network model using the invertible transformation.

[0012] In yet another embodiment of the disclosure, obfuscating the neural network model using the invertible transformation based on the model update request comprises obfuscating fully connected layers and convolutional layers of the neural network model using the invertible transformation.

[0013] In another embodiment of the disclosure, sending one of the obfuscated model and the backup model to the user end comprises sending the obfuscated model and the backup model in turns.

[0014] In yet another embodiment of the disclosure, sending one of the obfuscated model and the backup model to the user end comprises sending the obfuscated model only once at intervals, and otherwise sending the backup model.

[0015] In another embodiment of the disclosure, the intervals are variable depending on application scenarios.

[0016] In yet another embodiment of the disclosure, sending one of the obfuscated model and the backup model to the user end comprises always sending the obfuscated model.

[0017] In another embodiment of the disclosure, updating the backup model based on the received model gradient to achieve the model update comprises: if the received model gradient is calculated based on the obfuscated model, converting the received model gradient into a gradient applicable to the backup model through an inverse transformation.

[0018] In yet another embodiment of the disclosure, updating the backup model based on the received model gradient to achieve the model update comprises: if the received model gradient is calculated based on the backup model, directly updating the backup model with the received model gradient.

[0019] In an embodiment of the disclosure, there is provided a model update system based on active attacks, comprising: a transceiver module that receives a model update request from a user end; a model acquisition module that backs up a neural network model based on the model update request to obtain a backup model, and obfuscates the neural network model using an invertible transformation based on the model update request to obtain an obfuscated model; wherein the transceiver module sends one of the obfuscated model and the backup model to the user end, and receives a model gradient of a local model trained by the user end based on the sent model and local samples; and a model update module that updates the backup model based on the received model gradient to achieve the model update.

[0020] In an embodiment of the disclosure, there is provided a computer-readable storage medium storing instructions that, when executed, cause a machine to perform the method as described above.

[0021] This summary is provided to introduce some concepts of the following detailed description in a simplified form. This summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter. BRIEF DESCRIPTION OF DRAWINGS

[0022] The above summary of the present disclosure and the following detailed description of the present disclosure will be better understood when read in conjunction with the appended drawings. It is to be noted that the figures are merely schematic and are not drawn to scale. It should also be noted that the figures are only intended to facilitate the understanding of the present application. In the drawings, like reference numerals refer to like elements.

[0023] Figure 1 is a schematic diagram illustrating a data privacy theft process in federated learning;

[0024] Figure 2 is a flowchart illustrating a model updating method based on active attacks according to an embodiment of the present disclosure;

[0025] Figure 3 is a schematic diagram illustrating a model updating process based on active attacks according to an embodiment of the present disclosure;

[0026] Figure 4 is a schematic diagram illustrating a modular reversible transformation process in a convolutional neural network according to an embodiment of the present disclosure;

[0027] Figure 5 is a schematic diagram illustrating a reversible transformation process for the same type of convolution kernel in a convolutional neural network according to an embodiment of the present disclosure;

[0028] Figure 6 is a schematic diagram illustrating a reversible transformation process for different types of convolution kernel in a convolutional neural network according to an embodiment of the present disclosure;

[0029] Figure 7 is a block diagram illustrating a model updating system based on active attacks according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0030] In order to make the above objectives, features and advantages of the present disclosure more apparent, the specific embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings.

[0031] In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present disclosure. However, it will be apparent to one skilled in the art that the present disclosure can be practiced without the specific details presented herein. In other instances, well-known methods, procedures, components, and circuits have not been described in detail as not to unnecessarily obscure aspects of the present disclosure.

[0032] With the application of large-scale deep learning and the increasing emphasis on data privacy protection at home and abroad, federated learning has become one of the main ways for multiple parties to cooperatively train a deep learning model. Federated learning can break through the limitations of data silos to some extent, helping multiple participants collaboratively train a deep learning model without exchanging their private data. However, due to its distributed nature and heterogeneous computing characteristics, it is difficult to establish a strict trust relationship with all participants in federated learning.

[0033] There may be a class of attackers who lurk among the participants and try to steal the data privacy of other participants from the slow updates of the global model. Such attackers can restore some information about the training data through the short-term changes in model data such as model parameters, partial model gradients, etc.

[0034] The present disclosure provides a model update scheme based on active attacks, which prevents the privacy theft behavior of attackers by actively constructing a series of reversible destructive attacks or backdoor attacks. In other words, the model update scheme based on active attacks of the present disclosure is to defend against data privacy theft by actively attacking the model restoration of the attacker through a confused model.

[0035] In the present disclosure, the specific description of the scheme will mainly be described with a convolutional neural network as an example. Those skilled in the art can understand that the model update scheme based on active attacks of the present disclosure is applicable to various federated learning application scenarios, and is not limited to the update of convolutional neural network models.

[0036] Figure 1 is a schematic diagram showing the data privacy theft process in federated learning.

[0037] With the increasing requirements for data itself and the increasingly prominent data privacy problem, federated learning as a distributed learning provides an attractive architecture for breaking down the overall machine learning workflow into modular units. The federated learning model can provide a certain degree of privacy for participating users through data minimization, i.e. the original user data never leaves the device, and only the model update (e.g. gradient update) will be sent to the central server. Compared with the original data, these model updates do not contain additional information about the user at all, and are usually significantly less than the original data. Thus, by making the model training on the user side and the global model on the server side, not only the generalization ability of the model is improved, but also the sensitive information is effectively protected.

[0038] Although federated learning can provide significant practical privacy improvement compared to centralized training of all data, there may still be privacy leakage, such as inferring the training samples held by a user from the known previous model and the gradient update of the user.

[0039] As shown in Figure 1 , user data is not shared throughout the system network, only intermediate calculation results such as model gradient or parameter update are transmitted. Specifically, the gradient of the local model is uploaded from the user end to the server end, and the parameter update of the global model is issued from the server end to the user end. Ideally, any such application is considered secure because the amount of information contained in the gradient is less than the original data.

[0040] However, when an attacker lurks in the system network as one of the user ends, or the attacker steals one of the user ends, the attacker can infer the training sample of the user end, the global model, and the like from the local model, the gradient update, and the sequential parameter issuance of the server end. For example, the attacker can save the latest model parameters at each step, compare them with the previously saved models, and restore part of the training data information from the changes in the model through training of a generative adversarial network (GAN), model flipping attack, gradient data restoration, and the like. Such an attack scheme is difficult to defend against with existing capabilities for federated learning, and even cannot be detected, because such an attacker does not affect the integrity of the system.

[0041] In the art, there are some defense schemes against privacy leakage. For example, a scheme based on homomorphic encryption uses homomorphic encryption to protect the transmission of gradients / models, preventing the server and external attackers from obtaining information about the gradients and models, thereby protecting data privacy. However, it cannot defend against privacy theft attacks from cooperative participants, has large computational overhead, and has difficulty in multi-user key negotiation.

[0042] As another example, a scheme based on differential privacy uses differential privacy to protect the gradient, preventing the server and external attackers from obtaining information about the gradient and model, thereby protecting data privacy. However, it can damage the model performance, and the definition of privacy may not align with the privacy that needs to be protected.

[0043] For another example, a scheme based on secure multi-party computation uses secure multi-party computation technology in conjunction with a double server / multi-server scheme, which has strong privacy protection capability. However, it is difficult to calculate, and approximate calculation is needed for most neural network models.

[0044] And a scheme based on data sharing shares encrypted data through homomorphic encryption and the like, and then calculates on the cloud. However, in this scheme, data leaves the local device uncontrollably.

[0045] Therefore, there is a need in the art for a defense scheme against difficult-to-detect data privacy theft. The model update scheme based on active attack of the present disclosure replaces passive defense with active attack, prevents attacks through reversible model confusion, and ensures normal model training.

[0046] Figure 2 FIG. 2 is a flowchart illustrating a model updating method 200 based on an active attack according to an embodiment of the present disclosure.

[0047] Those skilled in the art can understand that, in a federated learning application scenario, before formally starting training, the server first distributes an initial model to each participant (i.e., a distributed client or user end, hereinafter referred to as a user end), and then each user end respectively trains the obtained model according to a local data set. Next, each user end uploads the model parameters obtained by local training to the server in an encrypted manner. The server aggregates all model gradients, and then transmits the aggregated global model parameters to each user end in an encrypted manner.

[0048] At 202, a model updating request from a user end is received.

[0049] The distributed user end needs to upload the model parameters after training in the distributed training process, at which time the model updating request is sent to the server. The server receives the model updating request.

[0050] At 204, a neural network model is backed up based on the model updating request to obtain a backup model.

[0051] The server backs up the current global neural network model based on the received model updating request.

[0052] In an embodiment of the present disclosure, to obtain the backup model, the parameters required by the model can be directly extracted and saved.

[0053] In another embodiment of the present disclosure, to save performance overhead, only the latest part of the parameters can be saved by comparing the differences between the previous backup model and the current model to obtain the current backup model. At the same time, a periodic synchronization method is used to avoid excessive network overhead, such as synchronization once after training multiple epochs (uploading interval or frequency).

[0054] At 206, a reversible transformation is used to obfuscate the neural network model based on the model updating request to obtain an obfuscated model.

[0055] The reversible transformation is to transform part of the parameters of the model by means of shifting, substitution, linear transformation, etc., while ensuring the consistency of the overall output of the model and the recoverability of the calculated gradient.

[0056] After receiving the model updating request, the server can transform the neural network model by means of the reversible transformation, and obfuscate the neural network model to different degrees to obtain the obfuscated model. In different application scenarios, the requirements for privacy protection are different, so the degree of obfuscation of the neural network model is different.

[0057] In the following, reference will be made toFigure 4 A process of modular reversible transformation of a neural network model is depicted according to an embodiment of the present disclosure. Also, reference will be made to Figure 5 and Figure 6 A process of reversible transformation in a convolutional neural network for the same type or different type of convolution kernel is depicted according to an embodiment of the present disclosure.

[0058] At 208, one of the obfuscated model and the backup model is sent to the user end.

[0059] As mentioned before, in different application scenarios, the requirement of privacy protection is different, resulting in different obfuscation degrees of the neural network model. Also, depending on different privacy protection requirements, the model can be sent to the user end in different ways.

[0060] In an embodiment of the present disclosure, when the requirement of privacy protection is high, the obfuscated model can always be sent to the user end. Further, the obfuscated model obfuscated in different reversible transformation ways can be selected to send.

[0061] By recording the random seed of the obfuscation of the model sent to each user end last time, different obfuscated models can be sent to the user end each time. However, considering the cost, sending the backup model to most user ends once can save a lot of performance overhead.

[0062] In another embodiment of the present disclosure, when the requirement of privacy protection is medium, the obfuscated model or the backup model can be sent to the user end. Further, the obfuscated model and the backup model can be selected to send alternately.

[0063] In yet another embodiment of the present disclosure, when the requirement of privacy protection is low, the obfuscated model can be selected to send at certain intervals, while the backup model is sent at other update times. The interval can be predefined or preset by the system.

[0064] At 210, the model gradient of the local model trained by the user end based on the sent model and the local sample is received.

[0065] After receiving the model sent by the service end, the user end trains the local model using the model and the local sample at the user end, and calculates the model gradient of the local model after the training is completed, and then uploads to the service end.

[0066] Stochastic Gradient Descent (SGD) is the most commonly used optimization method for training deep learning models. Specifically, it is to calculate (and cache) the output value of each node in the forward propagation manner first, and then calculate the partial derivative of the loss function value with respect to each parameter in the backward propagation traversal graph. The current deep learning framework can basically realize automatic solution of gradient.

[0067] Those skilled in the art can understand that the calculation method of the model gradient includes numerical differentiation, symbolic differentiation, forward mode and reverse mode. Among them, the reverse mode is to use the BP algorithm, or in other words, to use the chain rule, and only one forward process and reverse process are needed to calculate the derivative or gradient of all parameters, which is especially suitable for the calculation of the gradient of the neural network model with a large number of training parameters. The calculation process of the model gradient is not described here.

[0068] At 212, the backup model is updated based on the received model gradient to achieve model updating.

[0069] Based on whether the model transmitted to the user end is a confusion model or a backup model, the received model gradient can be processed differently by the server to update the backup model and achieve model updating.

[0070] In an embodiment of the present disclosure, since the model transmitted to the user end is a confusion model, the server equivalently converts the gradient calculated based on the confusion model into the gradient on the backup model through inverse transformation, and then uses the converted gradient to update the backup model.

[0071] In another embodiment of the present disclosure, since the model transmitted to the user end is a backup model, the server directly updates the backup model with the gradient obtained from the backup model.

[0072] That is, model updating is always performed only on the backup model.

[0073] Under the protection of the model updating scheme based on active attack in the present disclosure, the model obtained by the attacker from the server is significantly different each time, because the model itself is reversibly transformed, so it is impossible for the attacker to restore the training data by comparing the difference between two models. At the same time, since the operation of the server on the model is reversible and does not affect the output, the server can losslessly update using the gradient returned by the user end.

[0074] Therefore, the model updating method based on active attack in the present disclosure actively confuses the global model, that is, a series of reversible destructive attacks or backdoor attacks are randomly constructed, so that the attacker cannot steal the data privacy of other participants or user ends based on the slow updating of the global model. That is, the model updating method based on active attack in the present disclosure is to defend against data privacy theft by actively attacking the model restoration of the attacker through confusion of the model.

[0075] Figure 3is a schematic diagram illustrating a model updating process based on active attack according to an embodiment of the present disclosure.

[0076] In the embodiment of the present disclosure as shown in Figure 3 In the embodiment of the present disclosure as shown in, the server maintains a global model, and each user end saves the latest model by synchronizing the model parameters at each iteration. Meanwhile, the model gradient of the local model calculated using the local sample data is returned to the server. After receiving the model gradient, the server updates the global model.

[0077] In the embodiment of the present disclosure as shown in Figure 3 In the embodiment of the present disclosure as shown in, the server obfuscates the global model by reversible transformation and distributes the obfuscated model. The user end trains based on the distributed obfuscated model and the local sample, and the gradient of the local model after training is calculated and sent to the server. The gradient returned to the server is used to update the global model by inverse transformation.

[0078] In this process, the global model is backed up before being obfuscated, that is, a backup model. The model gradient of the local model returned by the user end is also applied to the backup model after inverse transformation to obtain the update of the global model.

[0079] Since the obfuscated model is difficult to obtain the real update of the global model even if it is obtained by the attacker, the possibility of the attacker obtaining the data privacy of other user ends is eliminated. At the same time, the obfuscated model is obtained by reversible transformation, and the server itself knows the way of the reversible transformation, that is, when the model gradient update of the user end is received, the inverse transformation is used to restore and update the backup global model. That is, such active attack does not affect the update of the global model.

[0080] Therefore, the model updating scheme based on active attack of the present disclosure has high feasibility. By replacing passive defense with active attack, the obfuscated model obtained by reversible transformation is sent to the attacker as a participant, and the model gradient obtained is restored by inverse transformation to be applied to the global model update, thereby combining privacy protection and efficiency guarantee.

[0081] In the present disclosure, the model updating scheme based on active attack will be described in detail taking a convolutional neural network as an example. However, those skilled in the art can understand that the scheme of the present disclosure is not limited to the application scene of federated learning of convolutional neural network.

[0082] Figure 4 is a schematic diagram illustrating a modular reversible transformation process of a convolutional neural network according to an embodiment of the present disclosure.

[0083] As known to those skilled in the art, human beings are hierarchical and abstract in recognizing an image, first understanding color and brightness, then local detail features such as edges, corners, straight lines, and next more complex information and structures such as textures, geometrical shapes, and finally forming a concept of the whole object. The process of convolutional neural networks automatically learning features of an image at various levels through convolution and pooling operations is also the same.

[0084] As Figure 4 As shown in the above figure, signals from the retina first reach the primary visual cortex, i.e. the V1 cortex, through the visual center (LGN). Simple neurons in the V1 cortex are sensitive to image signals of a specific direction. After processing by the V1 cortex, the signals are transmitted to the V2 cortex. The V2 cortex represents edge and contour information as simple shapes, which are then processed by neurons in the V4 cortex, which are sensitive to color information. Complex objects are finally represented via the lateral occipital cortex (LOC), the IT cortex (inferior temporal cortex) at the end of the channel.

[0085] As Figure 4 As shown in the following figure, it can be composed of multiple convolutional layers, each of which contains multiple convolution kernels, with which the entire image is scanned from left to right and from top to bottom to obtain output data called feature maps. The convolutional neural network can be represented as a modular neural network, including an input module, a direction information extraction module, a contour information extraction module, a color information extraction module, an information aggregation module, an output module, and the like.

[0086] In an embodiment of the present disclosure, the modular reversible transformation of the convolutional neural network can reversibly transform one or more modules, such as one or more of the direction information extraction module, the contour information extraction module, the color information extraction module, or the information aggregation module.

[0087] Those skilled in the art can understand that the model updating scheme based on active attack of the present disclosure can also apply reversible transformation to other neural network models, which will not be described here.

[0088] A typical convolutional neural network is composed of convolutional layers, pooling layers, and fully connected layers. Taking the application of convolutional neural networks in machine vision as an example, it completes the extraction and abstraction of features of an image at each level through the combination of multiple convolutional layers and pooling layers.

[0089] In one embodiment of this disclosure, the reversible transformation process of a convolutional neural network can use reversible transformation to obfuscate the convolutional layers of the neural network model, that is, to perform a reversible transformation on the convolutional layers. Convolutional neural networks extract image features through convolutional kernels. For example, to extract edge information from an image, operators such as Sobel, Roberts, and Prewitt can be used, each with different convolutional kernel matrices. In different application scenarios, different convolutional layers of a convolutional neural network can use the same type of convolutional kernel or different types of convolutional kernels. The reversible transformation process of the convolutional neural network in this embodiment of the disclosure can perform a reversible transformation on convolutional kernels of the same type or different types.

[0090] Figure 5 This is a schematic diagram illustrating a reversible transformation process for the same type of convolutional kernel in a convolutional neural network according to an embodiment of the present disclosure.

[0091] like Figure 5 As shown in the image above, the input feature map of the convolutional layer is 3-channel (RGB). Correspondingly, the convolutional kernel (e.g., a 3x3 kernel) is also 3-channel. During the convolution operation, the features of the corresponding channel are convolved with the kernel of each channel, and then the channel values ​​at the same location are summed to obtain a single-channel feature map. Figure 5 In one embodiment, there are four convolutional kernels of the same type. Each convolutional kernel produces a single-channel output feature map, and the four convolutional kernels produce a total of four channels of output feature maps.

[0092] One approach to performing reversible transformations on convolutional layers is to perform reversible transformations on multiple convolutional kernels of the same type. For example... Figure 5 As shown in the diagram below, the reversible transformation of convolution is as follows: the first convolutional layer changes the order of four convolutional kernels of the same type, and the second convolutional layer correspondingly changes the order of the weights of each convolutional kernel, while recording the original order and the changed order as backups. Specifically, this reversible transformation of convolution first changes the order of the output channels (out_channels) of the first convolutional layer, and then correspondingly changes the order of the input channels (in_channels) of the second convolutional layer.

[0093] In a particular embodiment of this disclosure, in order to reduce computational costs, the convolutional layer does not consider the order of input features, thereby allowing the order of the convolutional kernels to be arbitrarily adjusted.

[0094] Figure 6 This is a schematic diagram illustrating a reversible transformation process for different types of convolutional kernels in a convolutional neural network according to another embodiment of the present disclosure.

[0095] In another embodiment of the present disclosure, the order of the different types of convolution kernels can be changed for different convolution layers that employ different types of convolution kernels. For example, the different types of convolution kernels can refer to 1 x 1 convolution kernels, 3 x 3 convolution kernels, 5 x 5 convolution kernels, and so on. Alternatively, the different types of convolution kernels can refer to Sobel operator, Roberts operator, Prewitt operator, and so on. In such a scenario, as shown in FIG. 8, the convolution kernels can be sequentially moved up, sequentially moved down, or shuffled. Figure 6

[0096] Another approach to reversibly transform the convolution layer is to reversibly transform one or more of the plurality of different types of convolution kernel matrices, i.e., to shift, permute, linearly transform, and so on the convolution kernel matrices themselves. Again, the original convolution kernel matrices and the altered convolution kernel matrices need to be recorded as backups.

[0097] Those skilled in the art can understand that the approach to reversibly transform the convolution layer is not limited to the above approaches. The approach to reversibly transform employed by the model updating scheme based on active attack of the present disclosure can encompass all feasible reversible transformations.

[0098] In another embodiment of the present disclosure, the reversible transformation process of the convolutional neural network can use reversible transformation to confuse the fully connected layer of the neural network model, i.e., reversibly transform the fully connected layer. For example, the fully connected network layer can be formulated as

[0099] y = W * x + b

[0100] At this time, only a specific W can be used so that there is W = W1 * W2, and then this layer of full connection can be disassembled into a 2-layer equivalent full connection nest. In particular, if W1 is a full-rank matrix, the gradient of the training process W2 can be mapped back to the original parameters one by one. By replacing different W1, the network model can be changed to obtain different confusion models.

[0101] Those skilled in the art can understand that for different neural network models, reversible transformation can be performed for different layers according to their different architectures, which will not be described here.

[0102] Figure 7 FIG. 7 is a block diagram illustrating a model updating system 700 based on active attack according to an embodiment of the present disclosure.

[0103] The model updating system 700 based on active attack according to an embodiment of the present disclosure includes a transceiver module 702, a model obtaining module 706, and a model updating module 708.

[0104] ​The transceiver module 702 receives a model update request from the user client. During distributed training, the distributed user client needs to upload the trained model parameters, at which point it sends a model update request to the server. The transceiver module 702 receives this model update request.

[0105] The model acquisition module 706 backs up the neural network model based on the model update request to obtain the backup model, and uses a reversible transformation to confuse the neural network model based on the model update request to obtain the confused model.

[0106] The model acquisition module 706 updates the current global neural network model in global backup based on the model received by the transceiver module 702.

[0107] Upon receiving a model update request, the model acquisition module 706 can transform the neural network model through reversible transformations to obfuscate it to varying degrees, thereby obtaining an obfuscated model. Different application scenarios have different privacy protection requirements, resulting in varying degrees of obfuscation of the neural network model.

[0108] The transceiver module 702 sends one of the obfuscated model and the backup model to the user terminal, and receives the model gradient of the local model trained by the user terminal based on the sent model and the local samples.

[0109] As mentioned earlier, privacy protection requirements vary in different application scenarios, resulting in varying degrees of obfuscation of neural network models. Similarly, depending on different privacy protection requirements, the transceiver module 702 can send the model to the user terminal in different ways.

[0110] In one embodiment of this disclosure, when privacy protection requirements are high, the transceiver module 702 can always send an obfuscation model to the user terminal. Furthermore, the transceiver module 702 can selectively send obfuscation models obfuscated using different reversible transformation methods.

[0111] By having the model acquisition module 706 record the random seed for model obfuscation sent by each user terminal in the previous transmission, the transceiver module 702 can send a different obfuscated model to that user terminal each time. However, for cost considerations, sending a backup model to most user terminals at once would save a lot of performance overhead.

[0112] In another embodiment of this disclosure, when privacy protection requirements are stringent, the transceiver module 702 may send an obfuscated model or a backup model to the user terminal. Furthermore, it may be possible to send the obfuscated model and the backup model alternately.

[0113] In yet another embodiment of this disclosure, when privacy protection requirements are low, the transceiver module 702 may choose to send the obfuscated model at certain intervals, while sending the backup model at other update times. This interval can be predefined or preset by the system.

[0114] After receiving the model sent by the server, the user end trains a local model using the model and local samples, and calculates the model gradient of the local model after training, and then uploads it to the server. The transceiver module 702 receives the model gradient of the local model trained by the user end based on the sent model and local samples.

[0115] The model updating module 708 updates the backup model based on the received model gradient to achieve model updating.

[0116] Based on whether the model sent to the user end is a confusion model or a backup model, the received model gradient can be processed differently by the model updating module 708 for updating the backup model and achieving model updating.

[0117] In an embodiment of the present disclosure, since the model sent to the user end is a confusion model, the model updating module 708 equivalently converts the gradient calculated based on the confusion model into the gradient on the backup model through inverse transformation, and then uses the converted gradient to update the backup model again.

[0118] In another embodiment of the present disclosure, since the model sent to the user end is a backup model, the model updating module 708 directly updates the backup model with the gradient obtained from the backup model.

[0119] That is, model updating is always only performed on the backup model.

[0120] Therefore, the model updating system based on active attack of the present disclosure has high feasibility, by replacing passive defense with active attack, sending a confusion model confused by reversible transformation to the attacker as a participant, while restoring the obtained model gradient through inverse transformation to apply to global model updating, thereby combining privacy protection and efficiency guarantee. That is, the model updating system based on active attack of the present disclosure is to defend against data privacy theft in the way of actively attacking the model restoration of the attacker through the confusion model.

[0121] The various steps and modules of the active attack based model updating methods and systems described above can be implemented in hardware, software, or a combination thereof. If implemented in hardware, the various illustrative blocks, modules, and circuits described in connection with the present disclosure can be implemented or performed with a general purpose processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other programmable logic component, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. A general purpose processor can be a microprocessor, controller, microcontroller, or state machine, etc. If implemented in software, the various illustrative blocks, modules, and circuits described in connection with the present disclosure can be stored in or transmitted over, as one or more instructions or code on a computer-readable medium. The software module(s) of the present disclosure can be a single thread of code, or multiple co-threads of code, able to run simultaneously. The software module(s) can reside on a storage medium, memory, or a processor to name a few. The storage medium can be removable or non-removable, and includes, but is not limited to, RAM, flash memory, ROM, EPROM, EEPROM, registers, hard disk, a removable disk, a CD-ROM, a cloud storage, or any other storage medium known in the art. The storage medium can be coupled to the processor such that the processor can read information from, and write information to, the storage medium. In the alternative, the storage medium can be integral to the processor. Further, a software-based embodiment of at least one software module can be loaded onto a

[0122] It is also noted that the embodiments can be described as a process which is depicted as a flowchart, a flow diagram, a structure diagram, or a block diagram. Although a flowchart can describe the operations as a sequential process, many of the operations can be performed in parallel or concurrently. In addition, the order of the operations can be re-arranged.

[0123] The disclosed systems, devices, and systems should not be limited in any way by the above description. Rather, the present disclosure encompasses all novel and non-obvious features and aspects of the various disclosed embodiments, alone and in various combinations and sub-combinations with one another. The disclosed systems, devices, and systems are not limited to any particular aspect or feature or combination of them, nor do any of the disclosed embodiments require the presence of any particular advantage or solve a particular or all technical problems.

[0124] The embodiments of the present application described above with reference to the drawings are merely exemplary and are not intended to limit the present application to the specific embodiments described. The above-described embodiments are merely illustrative, and are not intended to limit the present application to the specific embodiments described. Many modifications can be made by those skilled in the art without departing from the spirit and scope of the present application, and the scope of protection of the present application is not limited to the specific embodiments described above.

Claims

1. A model update method based on active attack, comprising: Receive model update requests from the user's client; Based on the model update request, a backup neural network model is requested to obtain the backup model; Based on the model update request, a reversible transformation is used to obfuscate the neural network model to obtain an obfuscated model; Based on different privacy protection requirements, one of the obfuscation model and the backup model is sent to the user terminal; Receive the model gradient of the local model trained by the user terminal based on the sent model and local samples; and The backup model is updated based on the received model gradient to achieve model update.

2. The method as described in claim 1, wherein the reversible transformation includes transforming some parameters of the neural network model through shifting, substitution, or linear transformation.

3. The method of claim 1, wherein using a reversible transformation to obfuscate the neural network model based on the model update request includes a portion of the module that uses a reversible transformation to obfuscate the neural network model.

4. The method of claim 3, wherein the partial module of the neural network model using reversible transformation obfuscation is one or more modules of the neural network model.

5. The method of claim 1, wherein using a reversible transformation to obfuscate the neural network model based on the model update request includes obfuscating the fully connected layers of the neural network model using a reversible transformation.

6. The method of claim 1, wherein using a reversible transformation to obfuscate the neural network model based on the model update request includes obfuscating the convolutional layers of the neural network model using a reversible transformation.

7. The method of claim 1, wherein using a reversible transformation to obfuscate the neural network model based on the model update request includes using a reversible transformation to obfuscate the fully connected layers and convolutional layers of the neural network model.

8. The method of claim 1, wherein sending one of the obfuscation model and the backup model to the user terminal based on different privacy protection requirements comprises: When privacy protection requirements are moderate, the obfuscated model and the backup model are sent alternately.

9. The method of claim 1, wherein sending one of the obfuscation model and the backup model to the user terminal based on different privacy protection requirements comprises: When privacy protection requirements are low, the obfuscated model is sent only once at intervals; otherwise, the backup model is sent.

10. The method of claim 9, wherein the interval is variable depending on the application scenario.

11. The method of claim 1, wherein sending one of the obfuscation model and the backup model to the user terminal based on different privacy protection requirements comprises: When privacy protection requirements are high, the obfuscation model is always sent.

12. The method of claim 1, wherein updating the backup model based on the received model gradient to achieve model update comprises: If the received model gradient is calculated based on the confusing model, the received model gradient is transformed into a model gradient suitable for the backup model through an inverse transformation, and the backup model is updated with the transformed model gradient.

13. The method of claim 1, wherein updating the backup model based on the received model gradient to achieve model update comprises: If the received model gradient is calculated based on the backup model, then the backup model is directly updated using the received model gradient.

14. A model update system based on active attack, comprising: The transceiver module receives model update requests from the user's client. The model acquisition module backs up the neural network model based on the model update request to obtain the backup model, and uses a reversible transformation to obfuscate the neural network model based on the model update request to obtain the obfuscated model. The transceiver module sends one of the obfuscated model and the backup model to the user terminal based on different privacy protection requirements, and receives the model gradient of the local model trained by the user terminal based on the sent model and local samples; and The model update module updates the backup model based on the received model gradient to achieve model update.

15. A computer-readable storage medium storing instructions that, when executed, cause a machine to perform the method as claimed in any one of claims 1-13.

Images