Controllable data privacy enhancement method for deep learning inference services

Abstract

The present invention discloses a controllable data privacy enhancement method for deep learning reasoning services, which realizes data enhancement of the servo engine by designing a dynamic routing neural network model, and designs a multi-layer training framework for step-by-step training to achieve different levels of data protection functions. The dynamic routing neural network has multiple information bottlenecks of different sizes and bottleneck selection based on gates. In the multi-level training framework, according to the optimization target expression, the information reduction loss is calculated and trained step by step. This method can meet the different protection needs of users in the actual use of deep learning reasoning services. It takes user data and enhancement level as input, and enhances the data in a controllable data information reduction manner, which can effectively prevent secondary reasoning attacks and reconstruction attacks.

Description

Technical Field

[0001] The present invention relates to the technical field of data privacy protection, and in particular to a controllable data privacy enhancement method for deep learning reasoning services. Background Art

[0002] Deep learning inference service is a smart web service. The service provider trains the deep learning model in the cloud and makes the access to the model public in the form of an online application program interface, so that ordinary users can also use deep learning technology. However, to enjoy such services, users must send their private data to a remote server. With the rapid development of this service, the problem of user data privacy leakage has also been exposed.

[0003] There are already some solutions for protecting user data in deep learning reasoning services. Solutions based on homomorphic encryption, secure multi-party computing, and trusted execution environments have too high overhead costs and require special hardware support, making them difficult to deploy and run well in actual application scenarios. Solutions that achieve data protection by deleting sensitive information based on rules are limited by the algorithm's demand for data. The algorithms of such solutions usually require prior knowledge and clear privacy data attribute information. Although data enhancement solutions from the perspective of information theory cannot achieve the best data protection effect, they are more flexible to use and deploy, and receive more attention in actual application scenarios. Such solutions focus on balancing the contradiction between privacy protection and service practicality, but existing implementation methods do not take into account user data preferences, resulting in users with different needs only enjoying the same level of protection. Summary of the invention

[0004] The present invention provides a controllable data privacy enhancement method for deep learning reasoning services, which implements multi-level data enhancement through a dynamic routing neural network and designs a multi-layer training framework for step-by-step training to achieve different levels of data protection functions.

[0005] To achieve the above object, the technical solution adopted by the present invention is:

[0006] The controllable data privacy enhancement method for deep learning inference service includes the following steps:

[0007] S1. Design a model: Enhance multi-level data through the model; the model is a dynamic routing neural network model, and its architecture includes an encoder, a gating, an information bottleneck, and a decoder; the steps are as follows:

[0008] S1.1. Input relevant data in the deep learning inference server, and extract and encode the information of the relevant data into semantic features through an encoder; the relevant data includes variable X and related variable Y, which affect privacy protection and inference performance respectively;

[0009] S1.2. Design multiple data enhancement levels, each level corresponds to a gating network, an information bottleneck and a decoder. According to the data enhancement level selected by the user, the gating network is connected to the corresponding information bottleneck and decoder;

[0010] S1.3. Through the information bottleneck corresponding to the level selected by the user, the relevant data in the deep learning reasoning service is compressed, the relevant data is retained, and sensitive attributes are removed. The compressed semantic features are converted into bits to achieve data enhancement at the selected level;

[0011] S1.4. Decode the compressed features into the same size and format as the relevant data in step S1.1 through the decoder to ensure that the output data has the same size and format as the original data, so as not to hinder the normal application of deep learning reasoning service.

[0012] S2. Model training: Design a multi-level information reduction loss and train the model in sequence according to the number of features required at different levels, so as to stabilize the data enhancement effects at different levels; the steps are as follows:

[0013] S2.1. Determine the optimization target and design a multi-level information reduction loss based on the expression of the optimization target;

[0014] S2.2. Determine the specific algorithm. Determine the distance measurement algorithm according to the specific reasoning task of the target scenario, determine the entropy measurement algorithm according to the format and type of the relevant data actually input, substitute the distance measurement algorithm and entropy measurement algorithm into the multi-level information reduction loss, determine the model to minimize the target loss function, and use this function as the optimization target function for step-by-step training;

[0015] S2.3, adopting the step-by-step training method, the information bottleneck and decoder of each level are trained by minimizing the objective loss function, starting from the lowest enhancement level to the highest enhancement level, and each time the trained parameters are input into the next level for data enhancement.

[0016] As a further preferred embodiment of the present invention, the encoder consists of six convolutional layers and two ordinary ResBlocks networks for extracting semantic features; wherein the ResBlocks network includes two convolutional layers and a shortcut connection layer between the input and output, the shortcut connection performs recognition mapping and adds the connected output to the output of the convolutional layer.

[0017] As a further preferred embodiment of the present invention, for n data enhancement levels, Gn gated networks are designed accordingly. If the user selects level i to enhance data privacy, gate Gi is opened and other gates are closed; wherein 1≤i≤n.

[0018] As a further preferred embodiment of the present invention, if the user selects level i to enhance data privacy, after the gate Gi is opened, the data extracted by the encoder for semantic features passes through the i-th information bottleneck IB. i to compress.

[0019] As a further preferred embodiment of the present invention, in step S1.4, the decoder decodes the compressed features through four transposed convolutional layers and one convolutional layer.

[0020] As a further preferred embodiment of the present invention, in step S2.1, the optimization target is determined to be information reduction loss, and the expression is expressed as:

[0021] ;

[0022] in, represents the deep learning inference service, X and X’ i They represent the original data and the data after level i enhanced protection, Represents the distance between two variables;

[0023] In the scenario where the data of the system submodules is considered missing and the entire network architecture is trained as an end-to-end task, only one optimization target is used as the training loss, and the privacy constraints are formulated by weighting and relaxing the target through joint entropy. According to the expression of the optimization target, the multi-level information reduction loss is finally designed as:

[0024] ,

[0025] in, It is a hyperparameter selected according to the enhancement level, between 0 and 1. represents the distance between two variables, and H( ) represents the element entropy, whose measurement depends on the format of the input data.

[0026] As a further preferred embodiment of the present invention, the dynamic routing neural network model described in step S1 determines the specific optimization algorithm and training method in step S2.2, and the dynamic routing neural network model is defined as:

[0027] ;

[0028] in, Represents the entire network, Represents the parameters involved in network training; Stem represents the encoder, IB represents n information bottlenecks, Dec represents n decoders, Stem, IB, Dec Construct a complete network sequence, using the same parameters ;

[0029] Determine the distance measurement algorithm and entropy measurement algorithm and substitute them into the multi-level information reduction loss. When facing image tasks, the mean square error algorithm is used. The loss function of the model optimization target is:

[0030] ,

[0031] in, It is a hyperparameter selected according to the enhancement level, between 0 and 1. represents the mean square error, represents the deep learning inference service, X and X’ i They represent the original data and the data after level i enhanced protection, refers to the model optimization function at level i, Directly taking the minimum value can minimize the entropy of the protection result.

[0032] As a further preference of the present invention, in step S2.3, the step-by-step training method is to train from the lowest enhancement level to the highest enhancement level in sequence, and the fixed encoder is additionally trained in the first round. All features used in the data enhancement of the second to n levels are subsets of the data enhancement of the first level. In each round, the information bottleneck and decoder of the corresponding level are trained according to the loss function, and the trained model parameters are frozen to enter the training of the next level of data enhancement. The protection strength of two adjacent enhancement levels is related to the information difference between the two adjacent levels, and the amount of information compressed by the i-th level enhancement is greater than the amount of information compressed by the (i+1)th to nth level enhancements. The features used for the subsequent level enhancements are all subsets of the features of the previous level, so step-by-step training can ensure the stability of training.

[0033] Compared with the prior art, the present invention has the following advantages or beneficial effects:

[0034] The present invention provides a controllable data privacy enhancement method for deep learning reasoning services, which can meet the different protection needs of users in the actual use of deep learning reasoning services. It takes user data and enhancement level as input, and enhances the data in a controllable data information reduction manner, which can effectively prevent secondary reasoning attacks and reconstruction attacks.

[0035] Compared with solutions based on homomorphic encryption, secure multi-party computing, and trusted execution environments, the design concept of this solution does not change the training structure of the existing deep learning reasoning service, and is distributed to the user side in the form of a plug-in. Therefore, this solution is not limited by hardware devices, has low deployment overhead, and is more flexible in deployment and use. Compared with the solution that designs algorithms to achieve data protection by deleting sensitive information, this solution uses the idea of ​​data enhancement. The algorithm itself has no requirements for the specific attributes of prior data and sensitive information, and has a wider range of applications. Compared with existing data enhancement solutions of the same type, unlike the disadvantage of general data enhancement solutions that do not consider user preferences, this solution adopts a multi-level controllable data privacy enhancement solution, which supports users to perform different levels of data protection according to their own preferences, while protecting privacy data to the greatest extent possible without affecting service performance as much as possible, and can meet the diverse needs of different users. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 A neural network model architecture for dynamic routing;

[0037] Figure 2 Experimental results for data reconstruction attacks. DETAILED DESCRIPTION

[0038] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0039] This embodiment provides a dynamic routing neural network to achieve multi-level data enhancement, and designs a multi-layer training framework for step-by-step training to achieve different levels of data protection functions. The dynamic routing neural network has multiple information bottlenecks of different sizes and gate-based bottleneck selection. In the multi-level training framework, according to the optimization target expression, the information reduction loss is calculated and trained step by step.

[0040] The dynamic routing neural network model consists of an encoder, a gating, an information bottleneck, and a decoder. The operation process of the entire network is as follows:

[0041] S1.1. Input the data in the deep learning inference server, and use the encoder to extract features from the input data and encode them into semantic features. In the dynamic routing neural network, the information of variable X and related variable Y in the input data is considered, both of which affect privacy protection and reasoning performance respectively. First, the information of the input data must be encoded. For high-dimensional data, in order to solve the problem that it is difficult to effectively split, this embodiment chooses to use a feature extractor to encode the data into semantic features. Our encoder Stem consists of six convolutional layers and two ordinary ResBlocks networks for extracting semantic features. ResBlock has two convolutional layers plus a shortcut connection layer between the input and output. The shortcut connection performs recognition mapping and adds the connected output to the output of the convolutional layer.

[0042] S1.2. Design multiple data enhancement levels. Each level corresponds to a gating network, an information bottleneck, and a decoder. According to the data enhancement level selected by the user, the gating network is connected to the corresponding information bottleneck and decoder. For n data enhancement levels, Gn gating networks are designed accordingly. If the user selects level i to enhance data privacy, the gate Gi is opened, and other gates are closed. At the same time, only the subsequent information bottleneck and decoder of this level work.

[0043] S1.3, Dynamic Routing Neural Network consists of a set of information bottlenecks for squeezing data related to deep learning inference services. These information bottlenecks will compress the relevant variable X, retain the information it carries about another relevant variable Y, and remove sensitive data that is expected to be protected. For n data enhancement levels, there are n information bottlenecks. For the i-th information bottleneck IB i For blocks, if the user selects level 𝑖 to enhance data privacy, after the gate Gi is opened, the data extracted by the encoder with semantic features passes through IB i , where the relevant data of the deep learning reasoning service is compressed and sensitive attributes are removed. i Use a convolutional layer to reduce the number of channels of the feature. Different bottlenecks can convert features into different numbers of bits to achieve different enhancements, so different IB i Blocks have different numbers of output channels.

[0044] S1.4, decode the compressed features into the same size and format as the input data in step S1.1 through the decoder. Decoder Dec i For each information bottleneck setting, the compressed features are decoded to the same size and format as the input data. i Four transposed convolutional layers and one convolutional layer are used to decode the compressed features.

[0045] The structure of the entire network is as follows Figure 1As shown, in the complete process, for the data in the deep learning inference service, it is first encoded into semantic features through feature extraction, and then connected to the corresponding information bottleneck and decoder by the gated network according to the data enhancement level selected by the user. In the information bottleneck, the features are converted into different numbers of bits to achieve data enhancement at this level, and then the corresponding decoder restores the size and format of the data. The specific network parameter settings of each substructure are shown in Table 1. Conv2D, ReLU, BN, ResBlock, Tanh, etc. are the network layers used. In the input and output formats, w and h represent the width and height of the input image, sc, dc and ibc are the number of input channels, and the parameters k represent the kernel size, s represent the training stride size, and p represent the padding size. Different IB i The blocks have different numbers of output channels, and the channel sizes of the encoder and decoder are hyperparameters determined by the deep learning inference serving task of the target scenario.

[0046] Table 1

[0047]

[0048] In this embodiment, the data enhancement at level 1 retains the most information, that is, it has the opportunity to achieve the maximum inference accuracy, and the data enhancement at level n removes the most information, that is, it has the best privacy enhancement effect. Finally, the expression of data enhancement is:

[0049] ,

[0050] in, Indicates whether to execute the IB of this block i and Dec i .

[0051] During the model training process, our optimization goal is to reduce information loss, expressed as The system's training process adopts level-by-level training, starting from level 1 enhancement, and training is performed sequentially according to the number of features required at different levels, so as to achieve a stable multi-level enhancement effect. Specifically, the model training is divided into two parts: designing a multi-level information reduction loss and level-by-level training.

[0052] In this embodiment, the training data of each system submodule is missing, and the network architecture of the system (including feature extractor Stem encoder, information bottleneck IB i and decoderDec i ) will be trained as an end-to-end task, requiring only one optimization objective as the training loss. According to the expression of the optimization objective , design the multi-level information reduction loss as follows:

[0053] ,

[0054] By using the weighted sum method, the optimization objective is relaxed first. Privacy protection constraints are formulated by joint entropy. Here we set .

[0055] Then, the privacy constraints are formulated using the entropy value of the enhanced data, and the weighted privacy objectives are directly added as part of the optimization objectives to determine the training objective function adopted in the system network. The final design is mainly divided into two algorithms: distance metric algorithm , element entropy algorithm . Distance Metric Deep learning inference service tasks depending on the target scenario, entropy measurement Depends on the format (type) of the input data. When the target scenario is an image classification task, it is recommended to use the mean square error ( ) is used to measure the distance to characterize data entropy.

[0056] Finally, the model network is defined as , the model minimizes the objective loss function: .

[0057] In order to ensure the stability of training, the model adopts a step-by-step training method, starting from the lowest enhancement level (i.e., the highest data utility) to the highest enhancement level. In the training of n enhancement levels, each round of training trains the corresponding information bottleneck IB by minimizing the objective loss function i and decoderDec i , and freeze the obtained parameters to enter the next round of training with higher-level data enhancement. In particular, during the first level of training, Results of training the fixed encoder Stem. , all features used in data augmentation levels 2 to n are subsets of data augmentation level 1. Depending on the total number of augmentation levels n, the hyperparameter Choose uniformly between 0 and 1.

[0058] The present invention is verified by the following experiments. First, the effectiveness of the data augmentation scheme is evaluated by studying the average value of the inference accuracy and the maximum entropy of the augmented data. CelebA, Cifar and ImageNet are used as data sets. A device equipped with four NVIDIA GeForce RTX3090 GPUs is used for offline training of the network. The experimental parameter settings are shown in Table 2 below, where sc, ibc, and dc are the channel number parameters of the network structure consistent with Table 1. It is a hyperparameter selected according to the enhancement level and is between 0 and 1.

[0059] Table 2

[0060]

[0061] Table 3 lists the data enhancement effects using different enhancement levels. The experimental results show that the effectiveness of this scheme for protected images decreases as the level increases, and the effectiveness of privacy protection is comparable to the baseline. This experiment cannot intuitively reflect the data enhancement effect. This scheme conducted two more specific tests to further analyze the enhancement effect of this scheme in the face of secondary reasoning attacks and reconstruction attacks.

[0062] Table 3

[0063]

[0064] The experimental results of the secondary reasoning attack are shown in Table 4. Overall, the enhancement effect of this scheme is better than the baseline method. On some data sets, this scheme can achieve the goal of successfully preventing secondary reasoning attacks by sacrificing only 0.01 or 0.02 accuracy. This scheme provides better protection than the baseline at both level 1 and level 2 data enhancement, and the accuracy is also higher than the baseline.

[0065] Table 4

[0066]

[0067] The experimental results of data reconstruction attack are as follows Figure 2 As shown in Figure 2, in general, as the enhancement level increases, the effect of the enhancement becomes more obvious. In other words, the higher the enhancement level, the worse the result of the reconstruction attack.

[0068] The above shows and describes the basic principles, main features and advantages of the present invention. It should be understood by those skilled in the art that the present invention is not limited to the above embodiments. The above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention to be protected. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. A controllable data privacy enhancement method for deep learning inference services, characterized in that: The following steps are involved: S1. Design a model: Enhance multi-level data through the model; the model is a dynamic routing neural network model, and its architecture includes an encoder, a gating, an information bottleneck, and a decoder; the steps are as follows: S1.

1. Input relevant data in the deep learning inference server, and extract features from the relevant data and encode them into semantic features through an encoder; the relevant data includes variable X and related variable Y, which affect privacy protection and inference performance respectively; S1.

2. Design multiple data enhancement levels, each level corresponds to a gating network, an information bottleneck and a decoder. According to the data enhancement level selected by the user, the gating network is connected to the corresponding information bottleneck and decoder; S1.

3. Through the information bottleneck corresponding to the level selected by the user, the relevant data in the deep learning reasoning service is compressed, the relevant data is retained, and sensitive attributes are removed. The compressed semantic features are converted into bits to achieve data enhancement at the selected level; S1.4, decoding the compressed features into the same size and format as the relevant data in step S1.1 through a decoder; S2. Model training: Design a multi-level information reduction loss and train the model in sequence according to the number of features required at different levels, so as to stabilize the data enhancement effects at different levels; the steps are as follows: S2.

1. Determine the optimization target and design a multi-level information reduction loss based on the expression of the optimization target; S2.

2. Determine the specific algorithm. Determine the distance measurement algorithm according to the specific reasoning task of the target scenario, determine the entropy measurement algorithm according to the format and type of the relevant data actually input, substitute the distance measurement algorithm and entropy measurement algorithm into the multi-level information reduction loss, determine the model to minimize the target loss function, and use this function as the optimization target function for step-by-step training; S2.3, adopting the step-by-step training method, the information bottleneck and decoder of each level are trained by minimizing the objective loss function, starting from the lowest enhancement level to the highest enhancement level, and each time the trained parameters are input into the next level for data enhancement.

2. The controllable data privacy enhancement method for deep learning reasoning services according to claim 1, characterized in that: In step S1, the encoder is composed of six convolutional layers and two ordinary ResBlocks networks for extracting semantic features; wherein the ResBlocks network includes two convolutional layers and a shortcut connection layer between input and output, the shortcut connection performs recognition mapping and adds the connected output to the output of the convolutional layer.

3. The controllable data privacy enhancement method for deep learning reasoning services according to claim 1, characterized in that: In step S1.2, for n data enhancement levels, Gn gated networks are designed accordingly. If the user chooses level i to enhance data privacy, gate Gi is opened and other gates are closed; where 1≤i≤n.

4. The controllable data privacy enhancement method for deep learning reasoning services according to claim 1, characterized in that: In step S1.3, if the user selects level i to enhance data privacy, after gate Gi is opened, the data extracted by the encoder passes through the i-th information bottleneck IB. i to compress.

5. The controllable data privacy enhancement method for deep learning reasoning services according to claim 1, characterized in that: In step S1.4, the decoder decodes the compressed features through four transposed convolutional layers and one convolutional layer.

6. The controllable data privacy enhancement method for deep learning reasoning services according to claim 1, characterized in that: In step S2.1, the optimization goal is determined to be information loss reduction, and the expression is expressed as: ； in, represents the deep learning inference service, X and X’ i They represent the original data and the data after level i enhanced protection, Represents the distance between two variables; An optimization target is used as the training loss, and the target is relaxed by weighting and entropy, and the privacy constraint is formulated by joint entropy. According to the expression of the optimization target, the multi-level information reduction loss is finally designed as: ； in, is a hyperparameter selected according to the level of enhancement, ranging from 0 to 1, and H( ) represents the element entropy, whose measurement depends on the format of the input data.

7. The controllable data privacy enhancement method for deep learning reasoning services according to claim 1, characterized in that: The specific steps of step S2.2 are: Define the dynamic routing neural network model as: ； in, Represents the entire network, Represents the parameters involved in network training; Stem represents the encoder, IB represents n information bottlenecks, Dec represents n decoders, Stem, IB, Dec Construct a complete network sequence, using the same parameters ; Determine the distance measurement algorithm and entropy measurement algorithm and substitute them into the multi-level information reduction loss. When facing image tasks, the mean square error algorithm is used. The loss function of the model optimization target is: ； in, It is a hyperparameter selected according to the enhancement level, ranging from 0 to 1. represents the mean square error, represents the deep learning inference service, X and X’ i They represent the original data and the data after level i enhanced protection, Represents the model optimization function of the i-th level.

8. The controllable data privacy enhancement method for deep learning reasoning services according to claim 1, characterized in that: In step S2.3, the step-by-step training method is to start from the lowest enhancement level to the highest enhancement level in sequence, train the information bottleneck and decoder of the corresponding level by minimizing the target loss function, freeze the trained data, and enter the next level of data enhancement training.

Images