A privacy protection method, device, equipment and medium for image reconstruction and generation

Abstract

The present invention relates to the field of image generation, and discloses a method, apparatus, device and medium for privacy-protected image reconstruction and generation. For each round of training, the original image is input into the encoding layer to obtain the original image features. The original image features are input into the embedding layer to obtain discrete image features. The discrete image features are respectively input into the decoding layer and the regression layer to obtain the reconstructed image and the autoregressive image features. The first loss value is determined according to the original image and the reconstructed image, and the decoding layer, the embedding layer and the encoding layer are trained according to the first loss value and the preset differential privacy noise perturbation parameter. The second loss value is determined according to the discrete image features and the autoregressive image features, and the autoregressive layer is trained. In response to a generation and / or reconstruction request for a target image, the privacy protection model that has completed training is used to determine the target image and return it, reducing the number of times of accessing the original image, reducing the privacy budget of differential privacy, and improving the security of the original image.

Description

Technical Field

[0001] The present invention relates to the field of image generation, and particularly to an image reconstruction and generation method, apparatus, device, and medium for privacy protection. Background Art

[0002] Currently, with the rapid development of technologies such as big data, cloud computing, and artificial intelligence, generative models are increasingly widely used in fields such as image generation and data augmentation. However, training a generative model requires a large amount of training data, and this training data may contain sensitive information, posing a risk of privacy leakage when using this training data.

[0003] To further improve the privacy security of this training data, differential privacy is usually introduced into the generative model to achieve privacy enhancement of the generative model. In particular, when using a variational autoencoder generative model, the training data needs to be called multiple times to complete one round of training of the generative model. And the more frequently the training data is called, the greater the risk of privacy leakage when using this training data. At the same time, the differential privacy mechanism introduced to protect data privacy also leads to an increase in privacy budget due to the multiple calls to the training data.

[0004] Therefore, the present application provides an image reconstruction and generation method, apparatus, device, and medium for privacy protection. Summary of the Invention

[0005] The purpose of the present invention is to provide an image reconstruction and generation method, apparatus, device, and medium for privacy protection, which can train the model without multiple calls to the training data and can protect the privacy information in the target image and reduce the privacy budget when introducing the differential privacy mechanism to generate or reconstruct the target image.

[0006] To achieve the above purpose, the technical solution adopted by the present invention is as follows:

[0007] An image reconstruction and generation method for privacy protection, using a privacy protection model to be trained, where the privacy protection model to be trained includes an encoding layer, an embedding layer, a decoding layer, and an autoregressive layer, and includes the following steps:

[0008] S100: For each round of training of the privacy protection model to be trained, obtain an original image and input the original image into the encoding layer to obtain original image features;

[0009] S101: Input the original image features into the embedding layer to obtain discrete image features;

[0010] S102: Input the discrete image features into the decoding layer and the autoregressive layer respectively to obtain a reconstructed image and autoregressive image features respectively;

[0011] S103: Determine a first loss value based on the original image and the reconstructed image, and train the decoding layer, the embedding layer, and the encoding layer according to the first loss value and a preset differential privacy noise perturbation parameter; determine a second loss value based on the discrete image features and the autoregressive image features, and train the autoregressive layer according to the second loss value and the differential privacy noise perturbation parameter;

[0012] S104: In response to a generation and / or reconstruction request for a target image, determine the target image and return it through the trained privacy protection model.

[0013] According to the above technical means, in each round of training of the privacy protection model in the present invention, the encoding layer, the embedding layer, the decoding layer, and the autoregressive layer are trained simultaneously. It is not necessary to train the decoding layer, the embedding layer, and the encoding layer separately first, and then repeatedly input the original image to train the autoregressive layer. During the process of training the model, the number of times of accessing the original image is reduced, and the security of the original image is improved. Moreover, during the training process of the model, the differential privacy mechanism is introduced. By reducing the number of times of accessing the original image, the privacy budget of differential privacy is reduced, and the amount of noise injection is increased, further protecting the privacy of the original image. It can achieve the same model performance as when no noise is introduced under the premise of reducing the privacy budget, and can significantly shorten the training time and improve the training efficiency.

[0014] Further, in S103, training the decoding layer, the embedding layer, and the encoding layer according to the first loss value and a preset differential privacy noise perturbation parameter specifically includes:

[0015] Determine the gradient of the decoding layer according to the first loss value and the preset differential privacy noise perturbation parameter, and adjust the parameters of the decoding layer according to the gradient of the decoding layer;

[0016] Determine the gradient of the embedding layer according to the adjusted parameters of the decoding layer and the gradient transmission relationship between the decoding layer and the embedding layer, and adjust the parameters of the embedding layer according to the gradient of the embedding layer;

[0017] Determine the gradient of the encoding layer according to the adjusted parameters of the embedding layer and the gradient transmission relationship between the embedding layer and the encoding layer, and adjust the parameters of the encoding layer according to the gradient of the encoding layer.

[0018] According to the above technical means, during the backpropagation process of the privacy protection model, when determining the gradient, the differential privacy stochastic gradient descent method is introduced to perturb the gradient, and the encoding layer, embedding layer, and decoding layer are trained according to the perturbed gradient, which can enable the privacy protection model to reduce privacy-sensitive information during the process of reconstructing the target image, ensure the privacy security of the output image, and improve the performance of the privacy protection model.

[0019] Further, training the autoregressive layer according to the second loss value and the differential privacy noise perturbation parameter in S103 specifically includes:

[0020] Determine the gradient of the autoregressive layer according to the second loss value and the differential privacy noise perturbation parameter, and adjust the parameters of the autoregressive layer according to the gradient of the autoregressive layer.

[0021] According to the above technical means, during the backpropagation process of the privacy protection model, when determining the gradient, the differential privacy stochastic gradient descent method is introduced to perturb the gradient, and the autoregressive layer is trained according to the perturbed gradient, which can enable the privacy protection model to generate a target image with privacy protection during the process of generating an artificial synthetic image.

[0022] Further, determining the target image through the trained privacy protection model in S104 specifically includes:

[0023] In response to the reconstruction request of the target image, input the image to be reconstructed into the encoding layer of the trained privacy protection model to obtain the image features of the image to be reconstructed;

[0024] Input the image features of the image to be reconstructed into the embedding layer of the trained privacy protection model to obtain the discrete image features of the image to be reconstructed;

[0025] Input the discrete image features of the image to be reconstructed into the decoding layer of the trained privacy protection model to obtain the target image.

[0026] According to the above technical means, the trained privacy protection model reconstructs the image to be reconstructed input into the model into a target image without carrying privacy information, avoiding the problem of being able to identify privacy information through the target image.

[0027] Further, determining the target image through the trained privacy protection model in S104 specifically includes:

[0028] In response to the generation request of the target image, determine the noise information from the preset random noise distribution sequence;

[0029] Input the noise information into the autoregressive layer of the trained privacy protection model to obtain the autoregressive image features of the noise information;

[0030] Input the autoregressive image features of the noise information into the decoding layer of the trained privacy protection model to obtain the target image.

[0031] According to the above technical means, through the trained privacy protection model, an artificial synthetic target image without privacy information is generated according to the noise information input into the model. Moreover, since the noise information is randomly selected unpredictable random information, the generated target image has extremely high privacy.

[0032] Further, the format of the discrete image features is the same as that of the autoregressive image features.

[0033] Further, before inputting the discrete image features into the autoregressive layer in S102, the method further includes:

[0034] Construct an initialized autoregressive layer according to the format of the image features output by the embedding layer and the format of the image features input into the decoding layer.

[0035] This specification provides an image reconstruction and generation device for privacy protection, which adopts a privacy protection model to be trained. The privacy protection model to be trained includes an encoding layer, an embedding layer, a decoding layer, and an autoregressive layer, and includes:

[0036] An encoding module, for each round of training of the privacy protection model to be trained, obtain the original image and input the original image into the encoding layer to obtain the original image features;

[0037] An embedding module, for inputting the original image features into the embedding layer to obtain discrete image features;

[0038] A decoding autoregressive module, for inputting the discrete image features into the decoding layer and the autoregressive layer respectively to obtain a reconstructed image and autoregressive image features respectively;

[0039] A training module, for determining a first loss value according to the original image and the reconstructed image, and training the decoding layer, the embedding layer, and the encoding layer according to the first loss value and a preset differential privacy noise perturbation parameter; determining a second loss value according to the discrete image features and the autoregressive image features, and training the autoregressive layer according to the second loss value and the differential privacy noise perturbation parameter;

[0040] A determination module, configured to determine a target image and return it through a trained privacy protection model in response to a generation and / or reconstruction request for the target image.

[0041] This specification provides a computer-readable storage medium storing a computer program, which when executed by a processor implements the above-mentioned image reconstruction and generation method for privacy protection.

[0042] This specification provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, it implements the image reconstruction and generation method for privacy protection.

[0043] Beneficial effects achieved by the present invention:

[0044] In each round of training of the privacy protection model in the present invention, the encoding layer, the embedding layer, the decoding layer, and the autoregressive layer are all trained. There is no need to separately train the decoding layer, the embedding layer, and the encoding layer first, and then repeatedly input the original image to train the autoregressive layer. During the process of training the model, the number of times of accessing the original image is reduced, improving the security of the original image. Moreover, during the training process of the model, a differential privacy mechanism is introduced, and the differential privacy stochastic gradient descent method is used to train the model. By reducing the number of times of accessing the original image, the privacy budget of differential privacy is reduced, ensuring the privacy of the original image. It can achieve almost the same model performance as when no noise is introduced and when the differential privacy mechanism is introduced but the traditional training method is used, while significantly shortening the training time and improving the training efficiency on the premise of reducing the privacy budget. Description of the Drawings

[0045] Figure 1 It is a schematic flowchart of an image reconstruction and generation method for privacy protection provided by an embodiment of this specification;

[0046] Figure 2 It is a schematic structural diagram of a privacy protection model provided by this specification;

[0047] Figure 3 It is a schematic diagram of a reconstructed image provided by this specification;

[0048] Figure 4 It is a schematic diagram of a generated image provided by this specification;

[0049] Figure 5 It is a schematic diagram of an image reconstruction and generation device for privacy protection provided by this specification;

[0050] Figure 6 It is provided by this specification corresponding to Figure 1 a schematic structural diagram of an electronic device.

[0051] The accompanying drawings are only for illustrative purposes and should not be construed as limiting the present patent; for better illustration of this embodiment, some components in the accompanying drawings may be omitted, enlarged or reduced, which does not represent the size of the actual product; for those skilled in the art, it is understandable that some well-known structures and their descriptions in the accompanying drawings may be omitted; the same or similar reference numerals correspond to the same or similar components; the terms used to describe the positional relationship in the accompanying drawings are only for illustrative purposes and should not be construed as limiting the present patent. Detailed implementation manners

[0052] It should be noted that, without conflict, the embodiments in the present application and the technical features in the embodiments can be combined with each other. The detailed descriptions in the specific embodiments should be understood as the explanatory illustrations of the gist of the present application and should not be regarded as an improper limitation to the present application.

[0053] To make the objectives, technical solutions and advantages of the embodiments of the present application clearer, the following will further describe the specific technical solutions of the present application in detail with reference to the accompanying drawings in the embodiments of the present application. The following embodiments are used to illustrate the present application but are not intended to limit the scope of the present application.

[0054] In the embodiments of the present application, the term "including", "comprising" or any other variant thereof is intended to cover non-exclusive inclusion, such that a process, method, article or device including a series of elements not only includes those elements but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the phrase "including an..." does not exclude the presence of additional identical elements in the process, method, article or device including such element.

[0055] The following will introduce and describe the technical solutions of the present invention in detail with reference to specific accompanying drawings.

[0056] Figure 1 The flowchart of a method for privacy-protected image reconstruction and generation provided for the embodiments of this specification includes the following steps:

[0057] S100: For each round of training of the privacy protection model to be trained, input the original image into the encoding layer to obtain the original image features.

[0058] In the process of privacy - protected image reconstruction and generation in this specification, it involves the processing of image data. In the embodiments of this specification, the server can execute the process of privacy - protected image reconstruction and generation. Of course, this specification does not limit which device implements the process of privacy - protected image reconstruction and generation, and devices such as personal computers and mobile terminals can also be used for privacy - protected image reconstruction and generation. For the convenience of description, the server is used as the execution entity for illustration below.

[0059] Figure 2 It is a schematic structural diagram of a privacy - protection model provided by this specification. As Figure 2 shown, the privacy - protection model includes an encoding layer, an embedding layer, a decoding layer, and an autoregressive layer. The result output by the encoding layer can be sent to the embedding layer, and the result output by the embedding layer can be sent to the autoregressive layer and the decoding layer respectively. The result output by the autoregressive layer can be sent to the decoding layer.

[0060] It should be noted that the privacy - protection model in this specification can be a model that combines a Vector Quantized Variational AutoEncoder (VQ - VAE) model and a Pixel Convolutional Neural Networks (PixelCNN) model. Among them, the encoding layer, the embedding layer, and the decoding layer can be regarded as three components of the VQ - VAE model, and the autoregressive layer can be regarded as the PixelCNN model.

[0061] In one or more embodiments of this specification, the training method of the privacy - protection model is as follows. For each round of training of the privacy - protection model to be trained, the server can obtain the original images used to train the privacy - protection model and input the original images into the encoding layer of the privacy - protection model to be trained to obtain the original image features of the original images output by the encoding layer.

[0062] S101: Input the original image features into the embedding layer to obtain discrete image features.

[0063] In one or more embodiments of this specification, the server further inputs the original image features into the embedding layer to obtain the discrete image features output by the embedding layer. Among them, the embedding layer represents the codebook mechanism in the privacy protection model, and the image features can be encoded into discrete vectors through the embedding layer. In the embodiments of this specification, the original image features of the original image are mapped into a discrete vector (i.e., discrete image features) by using the embedding layer. Equivalently, the embedding layer has a codebook mechanism, and the original image features can find the corresponding discrete vectors in the embedding layer (equivalent to the codebook). Through this codebook mechanism, the image information existing in the original image features can be replaced with discrete vectors. However, the discrete image features after being mapped by the embedding layer can still represent the original image, and the original image represented by the discrete image features does not have privacy information.

[0064] S102: Input the discrete image features into the decoding layer and the regression layer respectively to obtain a reconstructed image and autoregressive image features.

[0065] After that, in one or more embodiments of this specification, the server can input the discrete image features into the decoding layer and the autoregressive layer respectively to obtain the reconstructed image output by the decoding layer and the autoregressive image features output by the autoregressive layer.

[0066] S103: Determine a first loss value according to the original image and the reconstructed image, and train the decoding layer, the embedding layer, and the encoding layer according to the first loss value and a preset differential privacy noise perturbation parameter; determine a second loss value according to the discrete image features and the autoregressive image features, and train the autoregressive layer according to the second loss value and the differential privacy noise perturbation parameter.

[0067] In this specification, when an image needs to be reconstructed, the discrete image features need to be decoded by a decoder to obtain a reconstructed image. When an image needs to be synthesized, in order to make the synthesized image have the same feature distribution as the original image, the discrete image features output by the embedding layer are used as the input of the autoregressive layer during the training process. Moreover, in order to make the synthesized image match the image features input to the autoregressive layer, the image features input to the autoregressive layer and the image features output by the autoregressive layer should be as consistent as possible. The higher the consistency between the two, the more the synthesized image matches the image features input to the autoregressive layer.

[0068] Then, during the backpropagation process of the privacy protection model, the server can determine the first loss value based on the difference between the original image and the reconstructed image. And according to the first loss value and the preset differential privacy noise perturbation parameter, train the decoding layer, the embedding layer, and the encoding layer, where the difference between the original image and the reconstructed image is positively correlated with the first loss value.

[0069] Specifically, the server can determine the gradient of the decoding layer according to the first loss value and the preset differential privacy noise perturbation parameter, that is, introduce differential privacy noise during the process of calculating the gradient, and adjust the parameters of the decoding layer according to the gradient of the decoding layer. Then, according to the adjusted parameters of the decoding layer and the gradient transmission relationship between the decoding layer and the embedding layer, determine the gradient of the embedding layer, and adjust the parameters of the embedding layer according to the gradient of the embedding layer. After that, the server determines the gradient of the encoding layer according to the adjusted parameters of the embedding layer and the gradient transmission relationship between the embedding layer and the encoding layer, and adjusts the parameters of the encoding layer according to the gradient of the encoding layer. The preset noise mechanism of the Gaussian distribution is a noise curve in a normal distribution, the ordinate of the curve is the probability value, and the probability of each noise being selected in the noise curve is the probability value corresponding to that noise.

[0070] While training the decoding layer, the embedding layer, and the encoding layer, in one or more embodiments of this specification, the server can also determine the second loss value according to the difference between the discrete image features and the autoregressive image features output by the autoregressive layer, and train the autoregressive layer according to the second loss value and the preset differential privacy noise perturbation parameter.

[0071] Specifically, the server determines the gradient of the autoregressive layer according to the second loss value and the preset differential privacy noise perturbation parameter, and adjusts the parameters of the autoregressive layer according to the gradient of the autoregressive layer.

[0072] It should be noted that in each round of training of the privacy protection model, the encoding layer, the embedding layer, the decoding layer, and the autoregressive layer will be trained. Moreover, whether the training between the encoding layer, the embedding layer, and the decoding layer is completed does not affect the training of the autoregressive layer. The training of the autoregressive layer can be executed synchronously with the training between the encoding layer, the embedding layer, and the decoding layer.

[0073] S104: In response to a generation and / or reconstruction request for a target image, determine the target image through the trained privacy protection model and return it.

[0074] In one or more embodiments of this specification, after the server finishes training the privacy protection model, it can, in response to a generation and / or reconstruction request for a target image, determine the target image through the trained privacy protection model and return it.

[0075] Specifically, in response to a reconstruction request for a target image, the server inputs the image to be reconstructed into the encoding layer of the trained privacy protection model to obtain the image features of the image to be reconstructed. Then, the server inputs the image features of the image to be reconstructed into the embedding layer of the trained privacy protection model to obtain the discrete image features of the image to be reconstructed. After that, the server can input the discrete image features of the image to be reconstructed into the decoding layer of the trained privacy protection model to obtain the target image.

[0076] Figure 3 This is a schematic diagram of reconstructing an image provided in this specification. As Figure 3 shown, the solid lines in the figure represent the data transmission routes when reconstructing the image. The dashed lines in the figure represent the routes where no data transmission occurs when reconstructing the image. The image to be reconstructed is input into the encoding layer of the privacy protection model, and then passes through the embedding layer and the decoding layer in sequence to obtain the reconstructed target image output by the decoding layer.

[0077] In response to a generation request for a target image, the server determines noise information from a preset sequence of random noise distributions. The server inputs the noise information into the autoregressive layer of the trained privacy protection model to obtain the autoregressive image features of the noise information. The server inputs the autoregressive image features of the noise information into the decoding layer of the trained privacy protection model to obtain the target image.

[0078] Through the trained privacy protection model, an artificially synthesized target image without privacy information is generated based on the noise information input into the model, enabling the privacy protection model to generate a random image (target image) with privacy protection through discrete vectors sampled from the sequence of random noise distributions during the generation of the artificial synthesized image. Moreover, since the noise information is randomly selected and unpredictable random information, the generated target image has extremely high privacy.

[0079] Figure 4 This is a schematic diagram of generating an image provided in this specification. As Figure 4 shown, the solid lines in the figure represent the data transmission routes when generating the target image. The dashed lines in the figure represent the routes where no data transmission occurs when generating the target image. The noise information is input into the autoregressive layer of the privacy protection model and passes through the decoding layer to obtain the generated target image output by the decoding layer.

[0080] In addition, in one or more embodiments of this specification, since the discrete image features output by the embedding layer can be input into the decoding layer for decoding into an image, and the autoregressive image features output by the autoregressive layer can also be input into the decoding layer for decoding into an image, in order to be able to be input into the decoding layer, the format of the discrete image features output by the embedding layer is the same as the format of the autoregressive image features output by the autoregressive layer.

[0081] In one or more embodiments of the present specification, when constructing an initialized privacy protection model to be trained, since the privacy protection model can be a model combining a VQ-VAE model and a PixelCNN model, during the process of combining the autoregressive layer representing the PixelCNN model with the three components of the VQ-VAE model (the encoding layer, the embedding layer, and the decoding layer), the server can construct an initialized autoregressive layer according to the format of the image features output by the embedding layer and the format of the image features input to the decoding layer. So that the discrete image features output by the embedding layer can be input into the autoregressive layer, and the autoregressive image features output by the autoregressive layer can be input into the decoding layer.

[0082] In each round of training of the privacy protection model in the present invention, the encoding layer, the embedding layer, the decoding layer, and the autoregressive layer are respectively trained. There is no need to separately train the decoding layer, the embedding layer, and the encoding layer first, and then repeatedly input the original image to train the autoregressive layer. During the process of training the model, the number of times of accessing the original image is reduced, and the security of the original image is improved. Moreover, during the training process of the model, a differential privacy mechanism is introduced, and the differential privacy gradient descent method is used to train the model. By adjusting the training method, the number of times of accessing the original image is reduced, thereby reducing the privacy budget required for the differential privacy mechanism. Further, the privacy of the original image is guaranteed. It can achieve consistent model performance with that without introducing noise under the premise of reducing the privacy budget, and can significantly shorten the training time and improve the training efficiency.

[0083] During the backpropagation process of the privacy protection model, when determining the gradient, a differential privacy noise perturbation parameter is introduced to perturb the gradient, and the decoding layer, the embedding layer, and the encoding layer are trained according to the perturbed gradient. This can ensure that even when the privacy protection model is interfered by noise during the process of reconstructing the target image, the privacy security of the output image can be ensured, and the performance of the privacy protection model is improved.

[0084] It is also possible to introduce a noise perturbation parameter when determining the gradient of the autoregressive layer, perturb the gradient, and train the autoregressive layer according to the perturbed gradient. This can ensure that even when the privacy protection model is interfered by noise during the process of generating the target image, the privacy security of the output target image can be ensured, and the performance of the privacy protection model is improved.

[0085] Through the trained privacy protection model, the image to be reconstructed input to the model is reconstructed into a target image without privacy information, avoiding the problem of being able to identify privacy information through the target image. And through the trained privacy protection model, an artificially synthesized target image without privacy information is generated according to the noise information input to the model. Moreover, since the noise information is randomly selected and unpredictable random information, the generated target image has extremely high privacy.

[0086] The above is a method for privacy - protected image reconstruction and generation provided by one or more embodiments of this specification. Based on the same idea, this specification also provides a corresponding device for privacy - protected image reconstruction and generation. As Figure 5 shown, a privacy - protected model to be trained is adopted. The privacy - protected model to be trained includes an encoding layer, an embedding layer, a decoding layer, and an autoregressive layer.

[0087] An encoding module 500 is configured to, for each round of training of the privacy - protected model to be trained, obtain an original image, input the original image into the encoding layer, and obtain original image features.

[0088] An embedding module 501 is configured to input the original image features into the embedding layer to obtain discrete image features.

[0089] A decoding and regression module 502 is configured to input the discrete image features into the decoding layer and the autoregressive layer respectively to obtain a reconstructed image and autoregressive image features.

[0090] A training module 503 is configured to determine a first loss value according to the original image and the reconstructed image, and train the decoding layer, the embedding layer, and the encoding layer according to the first loss value and a preset differential privacy noise perturbation parameter; determine a second loss value according to the discrete image features and the autoregressive image features, and train the autoregressive layer according to the second loss value and the differential privacy noise perturbation parameter.

[0091] A determination module 504 is configured to, in response to a request for generating and / or reconstructing a target image, determine the target image through the trained privacy - protected model and return it.

[0092] This specification also provides a computer - readable storage medium. The storage medium stores a computer program, and the computer program can be used to execute the Figure 1 privacy - protected image reconstruction and generation method provided above.

[0093] This specification also provides Figure 6 a schematic structural diagram of the electronic device shown in. As Figure 6 shown, at the hardware level, the electronic device includes a processor, an internal bus, a network interface, a memory, and a non - volatile memory. Of course, it may also include other hardware required for other services. The processor reads the corresponding computer program from the non - volatile memory into the memory and then runs it to implement the Figure 1 privacy - protected image reconstruction and generation method described above.

[0094] Of course, in addition to the software implementation, this specification does not exclude other implementation manners, such as logic devices or the combination of software and hardware, etc. That is to say, the execution subject of the following processing flow is not limited to each logic unit, and may also be hardware or logic devices.

[0095] The serial numbers of the embodiments of the present application above are only for description and do not represent the superiority or inferiority of the embodiments. The above are only the preferred embodiments of the present application, and do not limit the patent scope of the present application accordingly. Any equivalent structure or equivalent process transformation made by using the content of the specification and drawings of the present application, or directly or indirectly applied in other related technical fields, shall be equally included in the patent protection scope of the present application.

Claims

1. A method for privacy - protected image reconstruction and generation, characterized in that, Using a privacy protection model to be trained, the privacy protection model to be trained includes an encoding layer, an embedding layer, a decoding layer, and an autoregressive layer, and includes the following steps: S100: For each round of training of the privacy protection model to be trained, obtain an original image, and input the original image into the encoding layer to obtain original image features; S101: Input the original image features into the embedding layer to obtain discrete image features; S102: Input the discrete image features into the decoding layer and the autoregressive layer respectively to obtain a reconstructed image and autoregressive image features respectively; S103: Determine a first loss value according to the original image and the reconstructed image, and train the decoding layer, the embedding layer, and the encoding layer according to the first loss value and a preset differential privacy noise perturbation parameter; determine a second loss value according to the discrete image features and the autoregressive image features, and train the autoregressive layer according to the second loss value and the differential privacy noise perturbation parameter; S104: In response to a generation and / or reconstruction request for a target image, determine the target image through the trained privacy protection model and return it; Wherein, determining the target image through the trained privacy protection model includes: In the case of responding to the reconstruction request for the target image, input the image to be reconstructed into the encoding layer of the trained privacy protection model to obtain the image features of the image to be reconstructed; input the image features of the image to be reconstructed into the embedding layer of the trained privacy protection model to obtain the discrete image features of the image to be reconstructed; input the discrete image features of the image to be reconstructed into the decoding layer of the trained privacy protection model to obtain the target image, or; In the case of responding to the generation request for the target image, determine noise information from a preset sequence of random noise distributions; input the noise information into the autoregressive layer of the trained privacy protection model to obtain the autoregressive image features of the noise information; input the autoregressive image features of the noise information into the decoding layer of the trained privacy protection model to obtain the target image.

2. The privacy protection image reconstruction and generation method according to claim 1, characterized in that In S103, training the decoding layer, the embedding layer, and the encoding layer according to the first loss value and the preset differential privacy noise perturbation parameter specifically includes: Determine the gradient of the decoding layer according to the first loss value and the preset differential privacy noise perturbation parameter, and adjust the parameters of the decoding layer according to the gradient of the decoding layer; According to the adjusted parameters of the decoding layer and the gradient transmission relationship between the decoding layer and the embedding layer, determine the gradient of the embedding layer, and adjust the parameters of the embedding layer according to the gradient of the embedding layer; According to the adjusted parameters of the embedding layer and the gradient transmission relationship between the embedding layer and the encoding layer, determine the gradient of the encoding layer, and adjust the parameters of the encoding layer according to the gradient of the encoding layer.

3. A method for privacy-protected image reconstruction and generation according to claim 1, characterized in that, In S103, training the autoregressive layer according to the second loss value and the differential privacy noise perturbation parameter specifically includes: Determine the gradient of the autoregressive layer according to the second loss value and the differential privacy noise perturbation parameter, and adjust the parameters of the autoregressive layer according to the gradient of the autoregressive layer.

4. A method for privacy - protected image reconstruction and generation according to claim 1, characterized in that, The format of the discrete image feature is the same as that of the autoregressive image feature.

5. A method for privacy - protected image reconstruction and generation according to claim 1, characterized in that, Before inputting the discrete image feature into the autoregressive layer in S102, the method further includes: Construct an initialized autoregressive layer according to the format of the image feature output by the embedding layer and the format of the image feature input into the decoding layer.

6. An image reconstruction and generation device for privacy protection, characterized in that, Using a privacy protection model to be trained, the privacy protection model to be trained includes an encoding layer, an embedding layer, a decoding layer, and an autoregressive layer, including: An encoding module, configured to obtain an original image for each round of training of the privacy protection model to be trained, and input the original image into the encoding layer to obtain an original image feature; An embedding module, configured to input the original image feature into the embedding layer to obtain a discrete image feature; A decoding autoregressive module, configured to input the discrete image feature into the decoding layer and the autoregressive layer respectively to obtain a reconstructed image and an autoregressive image feature respectively; A training module, configured to determine a first loss value according to the original image and the reconstructed image, and train the decoding layer, the embedding layer, and the encoding layer according to the first loss value and a preset differential privacy noise perturbation parameter; determine a second loss value according to the discrete image feature and the autoregressive image feature, and train the autoregressive layer according to the second loss value and the differential privacy noise perturbation parameter; A determining module, configured to, in response to a generation and / or reconstruction request for a target image, determine the target image through the trained privacy protection model and return it; Wherein, determining the target image through the trained privacy protection model includes: In the case of responding to the reconstruction request for the target image, input the image to be reconstructed into the encoding layer in the trained privacy protection model to obtain the image feature of the image to be reconstructed; input the image feature of the image to be reconstructed into the embedding layer in the trained privacy protection model to obtain the discrete image feature of the image to be reconstructed; input the discrete image feature of the image to be reconstructed into the decoding layer in the trained privacy protection model to obtain the target image, or; In the case of responding to the generation request for the target image, determine noise information from a preset random noise distribution sequence; input the noise information into the autoregressive layer in the trained privacy protection model to obtain the autoregressive image feature of the noise information; input the autoregressive image feature of the noise information into the decoding layer in the trained privacy protection model to obtain the target image.

7. A computer-readable storage medium, the storage medium stores a computer program, and when the computer program is executed by a processor, the method described in any one of claims 1 to 5 above is implemented.

8. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein when the processor executes the program, the method according to any one of claims 1 to 5 above is implemented.

Images