Image data protection method, system and electronic device based on split learning

Abstract

The application provides a split learning-based image data protection method and system and electronic equipment, and relates to the technical field of information security. The method first acquires original image data and determines initial embedding features corresponding to the original image data. Then, the original image data is superimposed with preset noise data to obtain a perturbed image, and perturbed embedding features corresponding to the perturbed image are determined. The preset noise data is generated by a trained generator. The initial embedding features and the perturbed embedding features are input into a trained discriminator to obtain feature similarity results, and the perturbed embedding features corresponding to the feature similarity results meeting a preset threshold are aggregated. The method reconstructs the image data before each participant uploads the pulverized data to the server, and improves the privacy of the image data on the premise of avoiding data reconstruction failure.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of information security, and in particular to an image data protection method and system based on split learning and an electronic device. BACKGROUND

[0002] Split learning is a common image data protection technology in the field of deep learning. In the split learning process, first, sub-models are allocated to each participant, and a server model is distributed to the server side. Second, each participant extracts features from the local data set using their own sub-model to obtain shattered data and uploads it to the server. Then, the server aggregates the shattered data uploaded by each participant. Finally, the server model and the sub-model parameter information of each participant are updated by back propagation. However, in the process of split learning, an attacker can steal the original data of the participants by introducing a malicious discriminator and an autoencoder, which damages the privacy of the entire split learning framework. SUMMARY

[0003] Therefore, the present application aims to provide an image data protection method and system based on split learning, which reconstructs image data before each participant uploads shattered data to the server, and improves the privacy of image data while avoiding data reconstruction failure.

[0004] In a first aspect, an image data protection method based on split learning is provided, which comprises:

[0005] Obtaining original image data and determining initial embedding features corresponding to the original image data;

[0006] Superimposing the original image data with preset noise data to obtain a perturbed image, and determining perturbed embedding features corresponding to the perturbed image; wherein the preset noise data is generated by a trained generator;

[0007] Inputting the initial embedding features and the perturbed embedding features into a trained discriminator to obtain feature similarity results, and aggregating the perturbed embedding features corresponding to the feature similarity results that meet a preset threshold.

[0008] In some embodiments, the training process of the generator comprises:

[0009] Selecting noise {z 1 ,z 2 ,…z m} in the prior distribution P prior (x) as the noise vector of the generator G(ω);

[0010] Inputting the noise vector into the initialized generator, and the generator forwards the noise vector to output noise data Pc and reconstruct the generator;

[0011] superimposing the original image data I c with the preset noise data P c to obtain a perturbed image I p , and using the perturbed image I p and the target perturbed image I t to construct a loss function; the loss function is:

[0012]

[0013] wherein, loss is the loss function of the generator; M is a client sub-model; D is a discriminator model; m is the number of samples;

[0014] The loss value of the generator is calculated using the loss function, and the training is stopped when the loss value meets a preset threshold.

[0015] In some embodiments, after superimposing the original image data with the preset noise data to obtain the perturbed image, the method further comprises: performing a preprocessing operation on the perturbed image; wherein the preprocessing operation on the perturbed image comprises:

[0016] obtaining the perturbed image I p ;

[0017] performing a normalization operation on the perturbed image I p using a linear normalization formula to constrain the pixel values of the perturbed image; wherein the linear normalization formula is:

[0018]

[0019] In some embodiments, the training process of the discriminator comprises:

[0020] extracting features from the original image data I C and the perturbed image I p using the sub-model M(θ) respectively, to obtain initial embedding features F C and perturbed embedding features F p ;

[0021] inputting the initial embedding features F C and the perturbed embedding features F p to the initialized discriminator D(ψ); and setting the corresponding label of the perturbed embedding features F p to 0 and the corresponding label of the initial embedding features F C to 1;

[0022] The loss value of the discriminator is calculated by using a preset loss function, and the training is stopped when the loss value meets a preset threshold; wherein the loss function is:

[0023]

[0024] Wherein, loss is the loss function of the discriminator.

[0025] In some embodiments, the initial embedding feature and the perturbation embedding feature are input into the trained discriminator to obtain a feature similarity result, including:

[0026] The initial embedding feature F C and the perturbation embedding feature F p are obtained.

[0027] The initial embedding feature F C and the perturbation embedding feature F p uploaded by each client in the server are aggregated by using data splicing and / or data averaging.

[0028] The aggregated perturbation embedding feature F p is forward propagated to obtain a feature similarity result, and the parameters of the server and the client are updated in real time.

[0029] In some embodiments, after the original image data is superimposed with the preset noise data to obtain the perturbation image, it further includes:

[0030] The perturbation image is denoised to generate a denoised image of the perturbation image.

[0031] The denoised image is normalized to obtain a normalized calculation result; the normalized calculation result is used to determine the perturbation embedding feature corresponding to the perturbation image.

[0032] In some embodiments, the perturbation embedding features corresponding to the feature similarity results meeting the preset threshold are aggregated, including:

[0033] It is judged whether the feature similarity result meets the data de-sensitization relationship in the noise data.

[0034] If not, the noise data determined by the generator is superimposed with the preset target de-sensitization data to generate a de-sensitized image, and the feature similarity result of the de-sensitized image is obtained by using the discriminator, until the feature similarity result of the de-sensitized image meets the data de-sensitization relationship, the perturbation embedding feature corresponding to the feature similarity result of the de-sensitized image is aggregated.

[0035] In a second aspect, the embodiments of the present application provide an image data protection system based on split learning, which comprises:

[0036] An initial embedding feature acquisition unit is configured to acquire original image data and determine initial embedding features corresponding to the original image data;

[0037] A perturbation embedding feature acquisition unit is configured to superimpose the original image data and preset noise data to obtain a perturbation image, and determine perturbation embedding features corresponding to the perturbation image, wherein the preset noise data is generated by a trained generator.

[0038] An image data aggregation unit is configured to input the initial embedding features and the perturbation embedding features into a trained discriminator to obtain feature similarity results, and aggregate the perturbation embedding features corresponding to the feature similarity results satisfying a preset threshold.

[0039] In a third aspect, an electronic device is provided, which includes a processor and a storage device; the storage device stores a computer program, and the computer program performs the steps of the split learning-based image data protection method provided in the first aspect when executed by the processor.

[0040] In a fourth aspect, a computer readable storage medium is provided, which stores a computer program, and the computer program implements the steps of the split learning-based image data protection method provided in the first aspect when executed by a processor.

[0041] The embodiments of the present application have the following beneficial effects: The embodiments of the present application provide a split learning-based image data protection method, system and electronic device. The method first acquires original image data and determines initial embedding features corresponding to the original image data. Then, the original image data is superimposed with preset noise data to obtain a perturbation image, and perturbation embedding features corresponding to the perturbation image are determined. The preset noise data is generated by a trained generator. The initial embedding features and the perturbation embedding features are input into a trained discriminator to obtain feature similarity results, and the perturbation embedding features corresponding to the feature similarity results satisfying a preset threshold are aggregated. The method reconstructs the image data before the data is uploaded to the server by each participant, and improves the privacy of the image data while avoiding data reconstruction failure.

[0042] Other features and advantages of the present application will be further described in the following description, and will become apparent from the description, or will be learned through practice of the present application. The objects and other advantages of the present application will be realized and achieved by the structure particularly pointed out in the description, claims, and drawings.

[0043] In order to make the above objects, features and advantages of the present application more apparent, the following preferred embodiments are specifically described with reference to the accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS

[0044] In order to more clearly illustrate the technical solutions in the specific embodiments or prior art of the present application, the drawings required to be used in the description of the specific embodiments or prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0045] Figure 1 A flowchart of an image data protection method based on split learning provided for an embodiment of the present application;

[0046] Figure 2 A training flowchart of a generator in an image data protection method based on split learning provided for an embodiment of the present application;

[0047] Figure 3 A training flowchart of a discriminator in an image data protection method based on split learning provided for an embodiment of the present application;

[0048] Figure 4 A method flowchart after superimposing original image data and preset noise data to obtain a perturbed image in an image data protection method based on split learning provided for an embodiment of the present application;

[0049] Figure 5 Another method flowchart after superimposing original image data and preset noise data to obtain a perturbed image in an image data protection method based on split learning provided for an embodiment of the present application;

[0050] Figure 6 A flowchart of inputting initial embedding features and perturbed embedding features into a trained discriminator to obtain feature similarity results in an image data protection method based on split learning provided for an embodiment of the present application;

[0051] Figure 7 A flowchart of aggregating perturbed embedding features corresponding to feature similarity results satisfying a preset threshold in an image data protection method based on split learning provided for an embodiment of the present application;

[0052] Figure 8 A flowchart of another image data protection method based on split learning provided for an embodiment of the present application;

[0053] Figure 9 A structural schematic diagram of an image data protection system based on split learning provided for an embodiment of the present application;

[0054] Figure 10A structural schematic diagram of an electronic device provided by an embodiment of the present application.

[0055] Icon:

[0056] 910-initial embedding feature acquisition unit; 920-perturbation embedding feature acquisition unit; 930-image data aggregation unit; 101-processor; 102-memory; 103-bus; 104-communication interface. DETAILED DESCRIPTION

[0057] In order to make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme of the present application will be described clearly and completely below with reference to the drawings. Obviously, the described embodiments are some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0058] Split learning is a common image data protection technology in the field of deep learning. In the split learning process, first, sub-models are allocated to each participant, and a server model is distributed to the server side. Secondly, each participant extracts features from the local data set using their own sub-model to obtain shattered data and uploads it to the server. Then, the server aggregates the shattered data uploaded by each participant, and finally, the server model and the sub-model parameter information of each participant are updated by back propagation.

[0059] However, there is still a risk of data privacy leakage in the process of split learning. Specifically, malicious attackers can introduce malicious discriminators and autoencoders to steal the original data of participants. The initiator of this inference attack is usually a malicious server or a participant. Specifically, the malicious server iteratively trains an attack model and a normal classification model during training, and the decoder obtained after training can directly recover the original data of a victim. This inference attack on the original data of the benign client damages the privacy of the entire split learning framework.

[0060] To solve the above problems, the present application provides an image data protection method, system and electronic device based on split learning. The method reconstructs image data before each participant uploads the shattered data to the server, and improves the privacy of image data while avoiding data reconstruction failure.

[0061] In order to facilitate the understanding of the present embodiment, first, a split learning-based image data protection method disclosed by the present embodiment will be described in detail. The flowchart of the method is shown in Figure 1 , which includes:

[0062] Step S101, obtain original image data, and determine initial embedding features corresponding to the original image data.

[0063] The original image data in this step is digital image data, which can be obtained from a related image data set or directly obtained from a preset camera. The image data can include digital images or related setting parameters of digital images, such as quantity, label, etc.

[0064] The determination process of the initial embedding features is a feature acquisition process, which is obtained by extracting the features contained in the original image data. The feature extraction process can be extracted by using the corresponding sub-models in the split learning framework. The embedding features extracted by using these sub-models on the local data set are the initial embedding features. In a popular way, these initial embedding features are clean embedding feature vectors.

[0065] Step S102, superimpose the original image data and the preset noise data to obtain a perturbed image, and determine perturbed embedding features corresponding to the perturbed image; wherein the preset noise data is generated by a trained generator.

[0066] The preset noise data is the output data of the generator, and the generator in this embodiment is a generation model in the field of generative adversarial networks. Correspondingly, the discriminator model in the generative adversarial network is the discriminator. The preset noise data is obtained by calculating the initial noise using the generator. The initial noise can be obtained according to the average variance of the pixel values of the original image data and by setting a corresponding random noise. Generally, the distribution rule of the noise can adopt uniform distribution and normal distribution, and the initial noise is obtained by calculating the mean square error of the image pixel values.

[0067] The initial noise is input into the generator and forward propagated to obtain the corresponding noise data. After scaling the image size of the noise data to the same size as that of the original image data, the perturbed image is generated by superimposing the two. After obtaining the perturbed image, the feature extraction is performed using the related sub-model to obtain the perturbed embedding features corresponding to the perturbed image.

[0068] Step S103, input the initial embedding features and the perturbed embedding features into the trained discriminator to obtain feature similarity results, and aggregate the perturbed embedding features corresponding to the feature similarity results satisfying the preset threshold.

[0069] After obtaining the initial embedding feature and the perturbed embedding feature, the two are input into the discriminator for discrimination to obtain the similarity result of the features. The discriminator is a discrimination model in the generative adversarial network, which is used to judge the similarity of the initial embedding feature and the perturbed embedding feature. Using the generative adversarial network can make the perturbed image data and the original image data have similar embedding feature space distribution, and at the same time the perturbed image is difficult to distinguish from the original image with the naked eye.

[0070] After obtaining the feature similarity result, the perturbed embedding feature corresponding to the feature similarity result satisfying the preset threshold is aggregated. The aggregation process occurs on the server side, and the server aggregates the perturbed embedding features uploaded by each client by splicing or averaging, which is also called the pulverized data process. The splicing method is to splice the embedding information uploaded by the client, that is, to splice all feature vector information by splicing operation to obtain the embedding information after aggregation; the averaging method is to weight and sum all client embedding feature information to obtain the feature vector information.

[0071] The perturbed embedding feature corresponding to the feature similarity result satisfying the preset threshold is aggregated, and the pulverized data after aggregation is used as the input data of the server model. The server model is forward propagated, and the parameters of the server model and the parameters of the client sub-model are updated by calculating the loss function and back propagation.

[0072] The above image data protection method based on split learning fully considers the mechanism that the split learning framework is vulnerable to inference attacks launched by attackers to leak original data information. Through the split learning framework based on the generative adversarial network, each participant reconstructs the image data before uploading the pulverized data to the server, and improves the privacy of the image data while avoiding data reconstruction failure. The entire operation process does not depend on any third party and is completed by the client independently, further ensuring the privacy of the image data.

[0073] Since the generator and the discriminator are components in the generative adversarial network, the generative adversarial network and the split learning related parameters need to be initialized. The initialization process can set the model related parameters, such as: initializing the split learning training round number R and the local participant training round number T using the generative adversarial network, the server network structure and its initialization parameters, the embedding vector dimension N uploaded by the participant, the model learning rate r, the split learning loss function and the generative adversarial network loss function model, etc.

[0074] Each client i loads local data to the local device, and automatically matches the client data using data alignment technology. The local client selects a feature sub-model M(θ) for a specific data set, such as a convolutional neural network for the MNIST data set. In addition, different model modalities can be selected according to the current data modality, such as a graph convolutional neural network as a sub-model of the client for network data. Then the sub-model M(θ) is used to perform forward propagation on the local raw image data I c Extract features, that is, the model completes forward propagation to obtain clean embedding feature vectors F c .

[0075] Specifically, in some embodiments, the training process of the generator, as shown in Figure 2 , includes:

[0076] Step S201, selecting noise {z prior ,z 1 ,z 2 ,…z m} in the prior distribution P

[0077] Initialize the noise as the data of the generator G(ω), the client first quantifies the complexity of the local image data, specifically calculates the average variance of the pixel values of the local raw image; then set the variance range of the random noise according to the variance, and the distribution of the noise mainly has uniform distribution and normal distribution.

[0078] Specifically, the formula for quantifying the pixel value mean square error is: Where ΔI i,j is the difference value of the i,j pixel points.

[0079] Step S202, input the noise vector into the initialized generator, and the generator outputs noise data P c through forward propagation, and reconstructs the generator.

[0080] After the initial noise is input into the generator, the specific noise data P c is output through forward propagation, and a size scaling operation is performed, so that the output data P c and the original image data I c have the same size.

[0081] Step S203, superimpose the original image data I c and the noise data P c to obtain the perturbed image I p , and use the perturbed image I p and the target perturbed image I t to construct a loss function.

[0082] The original image data I c is superimposed with the noise data P c to obtain the perturbed image I p . The process is implemented by the formula I p = P c + I c . Then, the loss function is constructed using the perturbed image I p and the target perturbed image I t . Specifically, the target perturbed image can be collectively referred to as a target desensitized image, and the construction process of the loss function is constructed by the norm of the pixel value, for example, the user end constructs the loss function using the L2 norm. Meanwhile, the discriminator is introduced to update the generator model parameters.

[0083] Specifically, the loss function is:

[0084]

[0085] where loss is the loss function of the generator; M is the client sub-model; D is the discriminator model; and m is the sample number.

[0086] In step S204, the loss value of the generator is calculated using the loss function, and the training is stopped when the loss value meets the preset threshold.

[0087] In some embodiments, the training process of the discriminator, as shown in Figure 3 , includes:

[0088] In step S301, the sub-model M(θ) is used to extract features from the original image data I C and the perturbed image I p , respectively, to obtain the initial embedding feature F C and the perturbed embedding feature F p .

[0089] The sub-model M(θ) is used to extract features from the original image data I C and the perturbed image I p , respectively, i.e., the sub-model completes the forward propagation process, thereby obtaining the embedding feature vector F c of the perturbed image and the embedding feature vector F p corresponding to the original image.

[0090] In step S302, the initial embedding feature F c and the perturbed embedding feature F p are input into the initialized discriminator D(ψ); and the corresponding label of the perturbed embedding feature F p is set to 0, and the corresponding label of the initial embedding feature F C is set to 1.

[0091] The discriminator D(ψ) in this step implements a binary classification task, specifically, classifying the perturbation embedding feature F p The corresponding label is set to 0, and the initial embedding feature F C The corresponding label is set to 1.

[0092] Step S303, calculate the loss value of the discriminator using the preset loss function, and stop training when the loss value meets the preset threshold; wherein the loss function is:

[0093]

[0094] Wherein, loss is the loss function of the discriminator.

[0095] Whether the training is completed is determined by the comparison relationship between the loss value and the preset threshold. After several training, until the discriminator D(ψ) and the generator G(ω) are converged, it can be considered that the above training is completed.

[0096] In some embodiments, after superimposing the original image data and the preset noise data to obtain the perturbation image, a preprocessing operation needs to be performed on the perturbation image. Specifically, the method further includes: performing a preprocessing operation on the perturbation image; the above process is as shown in Figure 4 , which includes:

[0097] Step S401, obtaining a perturbation image I p .

[0098] Step S402, performing a linear normalization operation on the perturbation image I p using a linear normalization formula to constrain the pixel value of the perturbation image; wherein the linear normalization formula is:

[0099]

[0100] The above step performs a pruning operation on the perturbation image, and then performs a normalization operation to constrain the pixel value of the perturbation image to between 0 and 1, which is beneficial to improve the processing speed.

[0101] In some embodiments, the perturbation image also needs to be denoised. Specifically, after superimposing the original image data and the preset noise data to obtain the perturbation image, the method as shown in Figure 5 , further includes:

[0102] Step S501, performing a denoising process on the perturbation image to generate a denoised image of the perturbation image.

[0103] The denoising process can use a corresponding denoising algorithm to denoise the perturbation image, thereby reducing the problem that the features brought by the noise in the perturbation image are not obvious, and finally generating a denoised image of the perturbation image.

[0104] Step S502, the noise-reduced image is normalized to obtain a normalized result; the normalized result is used to determine the perturbation embedding feature corresponding to the perturbation image.

[0105] The normalization calculation in this step can be the same as the normalization operation in step S402, which will not be repeated here.

[0106] In some embodiments, the initial embedding feature and the perturbation embedding feature are input into the trained discriminator to obtain a feature similarity result, as shown in formula (1), which includes: Figure 6

[0107] Step S601, obtaining an initial embedding feature F C and a perturbation embedding feature F p .

[0108] Step S602, using data splicing and / or data averaging to aggregate the initial embedding feature F C and the perturbation embedding feature F p uploaded by each client in the server.

[0109] The above process is implemented in the server, specifically, the server uses splicing and / or averaging to aggregate the initial embedding feature F C and the perturbation embedding feature F p uploaded by each client in the server, which is also called pulverized data.

[0110] Step S603, performing forward propagation on the aggregated perturbation embedding feature F p to obtain a feature similarity result, and updating the parameters of the server and the client in real time.

[0111] The pulverized data F p is input as the input data of the server model, and the server model is forward propagated, and the parameters of the server model are updated by reverse propagation of the loss function subsequently; similarly, the initial embedding feature F C is forward propagated to finally update the parameters of the client sub-model, which will not be repeated here.

[0112] In some embodiments, the perturbation embedding features corresponding to the feature similarity results satisfying the preset threshold are aggregated, as shown in formula (2), which includes: Figure 7

[0113] Step S701, determining whether the feature similarity result satisfies the data de-sensitization relationship in the noise data.

[0114] ​​Step S702, if not satisfied, the noise data determined by the generator is superimposed with the preset target desensitization data to generate a desensitized image, and the feature similarity result of the desensitized image is obtained by using the discriminator until the feature similarity result of the desensitized image satisfies the data desensitization relationship, and then the perturbation embedding features corresponding to the feature similarity result of the desensitized image are aggregated.

[0115] Before the perturbation embedding features are aggregated, it is necessary to determine whether the feature similarity result in the discriminator satisfies the preset data desensitization relationship. Specifically, the data desensitization relationship can be that the feature similarity result of the target desensitized image I t The data, that is, the data of the target perturbed image. If the result output by the discriminator is not desensitized, the generator is used again to generate a specific noise, which is then superimposed with the original image data, and then the feature extraction process is performed, and the feature similarity is judged by the discriminator until the result output by the discriminator is desensitized, and then the perturbation embedding features corresponding to the feature similarity result of the desensitized image are aggregated. The above process is shown in Figure 8 Another flowchart of the image data protection method based on split learning is shown in FIG. 6, and details are not repeated here.

[0116] It can be known from the image data protection method based on split learning provided in the above embodiments that the method can desensitize the data by adding perturbation noise to the original image while ensuring the usability of the data. After using the generative adversarial network, the perturbed image and the original image can have similar embedding feature space distribution, and the perturbed image can be difficult to distinguish from the original sample in the naked eye. Specifically, the generator is used to generate perturbation noise, so that the perturbed image superimposed with the noise and the target desensitized image are as similar as possible; at the same time, the embedding feature vector obtained by inputting the image superimposed with the noise into the client sub-model is as similar as possible to the embedding feature vector of the original image. Using this method, all perturbation encryption operations of the client can be completed locally on the client, so that the entire perturbation encryption operation has privacy, avoiding privacy leakage in the intermediate operation process; the perturbed image generated by the client by adding perturbation noise and the original image do not have similarity in the naked eye, so even if a malicious attacker steals the data of the participants by using the generative adversarial network technology, the undisturbed image cannot be obtained, thus ensuring the privacy of the original image; moreover, the perturbed image and the original image participating in the training in the client have related feature space distribution, thus ensuring the usability of the perturbed data. It can be seen that the above method reconstructs the image data before each participant sends the pulverized data to the server, and improves the privacy of the image data on the premise of avoiding data reconstruction failure.

[0117] Corresponding to the above embodiment of the image data protection method based on split learning, the present embodiment also provides an image data protection system based on split learning, as shown in Figure 9 The system comprises:

[0118] The initial embedding feature acquisition unit 910 is configured to acquire original image data and determine initial embedding features corresponding to the original image data.

[0119] The perturbation embedding feature acquisition unit 920 is configured to superimpose the original image data with preset noise data to obtain a perturbation image, and determine perturbation embedding features corresponding to the perturbation image, wherein the preset noise data is generated by a trained generator.

[0120] The image data aggregation unit 930 is configured to input the initial embedding features and the perturbation embedding features into a trained discriminator to obtain feature similarity results, and aggregate the perturbation embedding features corresponding to the feature similarity results satisfying a preset threshold.

[0121] The image data protection system based on split learning provided in the embodiments has the same implementation principle and technical effects as the image data protection method based on split learning, and for brevity, the part not mentioned in the embodiments can refer to the corresponding content in the foregoing method embodiments.

[0122] The embodiment also provides an electronic device, and a structural schematic diagram of the electronic device is shown in Figure 10 The device includes a processor 101 and a memory 102; wherein the memory 102 is configured to store one or more computer instructions, and the one or more computer instructions are executed by the processor to implement the image data protection method based on split learning.

[0123] Figure 10 The server shown in the figure also includes a bus 103 and a communication interface 104, and the processor 101, the communication interface 104 and the memory 102 are connected through the bus 103.

[0124] The memory 102 can include a high-speed random access memory (RAM, Random Access Memory), and can also include a non-volatile memory, for example, at least one disk memory. The bus 103 can be an ISA bus, a PCI bus or an EISA bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For the convenience of representation, Figure 10 only one bidirectional arrow is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0125] The communication interface 104 is configured to connect with at least one user terminal and other network units through a network interface, and send the encapsulated IPv4 packet or IPv4 packet to the user terminal through the network interface.

[0126] The processor 101 can be an integrated circuit chip having a processing capability of signals. In the implementation process, each step of the above method can be completed by the integrated logic circuit of hardware in the processor 101 or the instruction in the form of software. The processor 101 described above can be a general processor, including a central processing unit (CPU), a network processor (NP), etc.; can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. Each method, step and logic block diagram disclosed in the embodiments of the present disclosure can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor. The steps of the method disclosed in combination with the embodiments of the present disclosure can be directly embodied as a hardware code processor to execute, or be executed by a combination of hardware and software modules in the code processor. The software module can be located in a random access memory, a flash memory, a read only memory, a programmable read only memory or an electrically erasable programmable memory, a register, etc. The storage medium in the art. The storage medium is located in the memory 102, and the processor 101 reads the information in the memory 102, and combines the hardware to complete the steps of the method of the above embodiment.

[0127] The embodiment of the present application further provides a computer readable storage medium, and the computer readable storage medium stores a computer program. When the computer program is run by a processor, the steps of the method of the above embodiment are executed.

[0128] In several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented by other ways. The above-described device embodiments are only schematic, for example, the division of units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some communication interface, device or unit, and can be electrical, mechanical or other forms.

[0129] The units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, i.e. may be located in one place, or may be distributed on multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0130] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit.

[0131] If the function is realized in the form of a software function unit and sold or used as an independent product, it can be stored in a non-volatile computer readable storage medium executable by a processor. Based on this understanding, the technical solutions of the present application or the part of the present application that essentially contributes to the prior art or the part of the technical solutions can be embodied in the form of a software product, which is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disk, and various program code storage media.

[0132] Finally, it should be noted that: the above embodiments are only specific embodiments of the present application, used to illustrate the technical solutions of the present application, and not to limit it, the protection scope of the present application is not limited to this, although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand: any person skilled in the art within the technical range disclosed by the present application, still can modify or easily think of changes to the technical solutions recorded in the foregoing embodiments, or make equivalent replacement to part of the technical features; and these modifications, changes or replacements do not make the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be covered in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. An image data protection method based on split learning, characterized in that, The method includes: Acquire the original image data and determine the initial embedding features corresponding to the original image data; The original image data is superimposed with preset noise data to obtain a perturbed image, and the perturbed embedding features corresponding to the perturbed image are determined; wherein, the preset noise data is generated by a pre-trained generator; The initial embedding features and the perturbation embedding features are input into the trained discriminator to obtain feature similarity results, and the perturbation embedding features corresponding to the feature similarity results that meet the preset threshold are aggregated. The training process of the generator includes: In the prior distribution Selecting noise As the generator The noise vector; The noise vector is input into the initialized generator, which then propagates the noise vector forward and outputs noise data. And reconstruct the generator; The original image data With the noise data The perturbation image is obtained by superimposing the images. and using perturbation images With target perturbation image Construct a loss function; the loss function is: ， in, The loss function of the generator; For client-side sub-models; For the discriminator model; The number of samples; The loss value of the generator is calculated using the loss function, and training stops when the loss value meets a preset threshold. The training process of the discriminator includes: Using sub-models In the original image data respectively and the perturbation image Extract features to obtain initial embedding features. and perturbation embedding features ; The initial embedded features and the perturbation embedding feature Input to the initialized discriminator In; and embed the perturbation into the feature. The corresponding label is set to 0, and the initial embedded feature The corresponding label is set to 1; The loss value of the discriminator is calculated using a preset loss function, and training stops when the loss value meets a preset threshold; wherein, the loss function is: ， in, The loss function of the discriminator; Aggregating the perturbation embedding features corresponding to the feature similarity results that meet a preset threshold includes: Determine whether the feature similarity result satisfies the data desensitization relationship in the noisy data; If the conditions are not met, the noise data determined by the generator is superimposed with the preset target desensitized data to generate a desensitized image, and the feature similarity result of the desensitized image is obtained by the discriminator until the feature similarity result of the desensitized image satisfies the data desensitization relationship. Then, the perturbation embedding features corresponding to the feature similarity result of the desensitized image are aggregated.

2. The image data protection method based on split learning according to claim 1, characterized in that, After superimposing the original image data with preset noise data to obtain a perturbed image, the method further includes: performing a preprocessing operation on the perturbed image; wherein, the preprocessing operation on the perturbed image includes: Acquire the disturbance image ; The perturbed image is normalized using a linear normalization formula, thus normalizing the perturbed image. The pixel values ​​are constrained; wherein, the linear normalization formula is: 。 3. The image data protection method based on split learning according to claim 1, characterized in that, The initial embedding features and the perturbation embedding features are input into the trained discriminator to obtain feature similarity results, including: Obtain the initial embedding features and the perturbation embedding feature ; The initial embedded features uploaded by each client on the server are processed using data concatenation and / or data averaging. and the perturbation embedding feature Perform aggregation; The already aggregated perturbation embedding features The feature similarity results are obtained through forward propagation, and the parameters of the server and the client are updated in real time.

4. The image data protection method based on split learning according to claim 1, characterized in that, After superimposing the original image data with preset noise data to obtain the perturbed image, the process further includes: The disturbed image is denoised to generate a denoised image of the disturbed image; The denoised image is normalized to obtain a normalized result; the normalized result is used to determine the perturbation embedding feature corresponding to the perturbation image.

5. An image data protection system based on split learning, characterized in that, The system includes: An initial embedding feature acquisition unit is used to acquire the original image data and determine the initial embedding features corresponding to the original image data; The perturbation embedding feature acquisition unit is used to superimpose the original image data with preset noise data to obtain a perturbation image, and determine the perturbation embedding features corresponding to the perturbation image; wherein, the preset noise data is generated by a trained generator; An image data aggregation unit is used to input the initial embedding features and the perturbation embedding features into a trained discriminator to obtain feature similarity results, and to aggregate the perturbation embedding features corresponding to the feature similarity results that meet a preset threshold; During the training process of the generator in the perturbation embedding feature acquisition unit, it is also used to: in the prior distribution Selecting noise As the generator The noise vector is input into the initialized generator, which then propagates the noise vector forward and outputs noise data. The generator is then reconstructed; the original image data is then processed. With the noise data The perturbation image is obtained by superimposing the images. and using perturbation images With target perturbation image Construct a loss function; the loss function is: ， in, The loss function of the generator; For client-side sub-models; For the discriminator model; The number of samples is given; the loss value of the generator is calculated using the loss function, and training stops when the loss value meets a preset threshold. The discriminator in the image data aggregation unit is also used during training to: utilize sub-models In the original image data respectively and the perturbation image Extract features to obtain initial embedding features. and perturbation embedding features ; the initial embedding features and the perturbation embedding feature Input to the initialized discriminator In; and embed the perturbation into the feature. The corresponding label is set to 0, and the initial embedded feature The corresponding label is set to 1; the loss value of the discriminator is calculated using a preset loss function, and training stops when the loss value meets a preset threshold; wherein, the loss function is: ， in, The loss function of the discriminator; In the process of aggregating the perturbation embedding features corresponding to the feature similarity results that meet the preset threshold, the image data aggregation unit is further configured to: determine whether the feature similarity results satisfy the data desensitization relationship in the noise data; if not, use the noise data determined by the generator to superimpose the preset target desensitization data to generate a desensitized image, and use the discriminator to obtain the feature similarity results of the desensitized image, until the feature similarity results of the desensitized image satisfy the data desensitization relationship, and then aggregate the perturbation embedding features corresponding to the feature similarity results of the desensitized image.

6. An electronic device, characterized in that, include: Processors and storage devices; The storage device stores a computer program that, when executed by the processor, implements the steps of the image data protection method based on split learning as described in any one of claims 1 to 4.

7. A computer-readable storage medium storing a computer program thereon, characterized in that, The computer program, when executed by a processor, implements the steps of the image data protection method based on split learning as described in any one of claims 1 to 4.

Images