Privacy protection image reconstruction and generation method and device, equipment and medium

Abstract

The invention relates to the field of image generation, and discloses a privacy protection image reconstruction and generation method and device, equipment and a medium, aiming at each round of training, an original image is input into a coding layer, and original image features are obtained. And inputting the original image features into the embedded layer to obtain discrete image features. And respectively inputting the discrete image features into a decoding layer and a regression layer to obtain reconstructed image features and autoregression image features. And determining a first loss value according to the original image and the reconstructed image, and training a decoding layer, an embedding layer and a coding layer according to the first loss value and a preset differential privacy noise disturbance parameter. And determining a second loss value according to the discrete image features and the autoregression image features, and training an autoregression layer. And in response to the generation and / or reconstruction request of the target image, the target image is determined and returned through the trained privacy protection model, so that the number of times of accessing the original image is reduced, the privacy budget of differential privacy is reduced, and the security of the original image is improved.

Description

Technical Field

[0001] The present invention relates to the field of image generation, and in particular to a privacy-protected image reconstruction and generation method, device, equipment and medium. Background Art

[0002] At present, with the rapid development of technologies such as big data, cloud computing, and artificial intelligence, generative models are increasingly being used in areas such as image generation and data augmentation. However, training generative models requires a large amount of training data, which may contain sensitive information, and there is a risk of privacy leakage when using these training data.

[0003] In order to further improve the privacy security of these training data, differential privacy is usually introduced into the generative model to achieve privacy enhancement of the generative model. In particular, when using a variational autoencoder to generate a model, it is necessary to call the training data multiple times to complete a round of training for the generative model. The more frequently the training data is called, the greater the risk of privacy leakage when using these training data. At the same time, the differential privacy mechanism introduced to protect data privacy also increases the privacy budget due to multiple calls to the training data.

[0004] To this end, the present application provides a privacy-preserving image reconstruction and generation method, device, equipment and medium. Summary of the invention

[0005] The purpose of the present invention is to provide a privacy-preserving image reconstruction and generation method, device, equipment and medium, which, when introducing a differential privacy mechanism to generate or reconstruct a target image, can achieve the purpose of training a model without calling training data multiple times and protecting the privacy information in the target image and reducing the privacy budget.

[0006] In order to achieve the above object, the technical solution adopted by the present invention is as follows: A privacy-preserving image reconstruction and generation method adopts a privacy-preserving model to be trained, wherein the privacy-preserving model to be trained includes an encoding layer, an embedding layer, a decoding layer, and an autoregressive layer, and includes the following steps: S100: For each round of training of the privacy protection model to be trained, an original image is obtained, and the original image is input into the encoding layer to obtain original image features; S101: Inputting the original image features into the embedding layer to obtain discrete image features; S102: Inputting the discrete image features into the decoding layer and the autoregressive layer respectively to obtain a reconstructed image and an autoregressive image feature respectively; S103: determining a first loss value according to the original image and the reconstructed image, and training the decoding layer, the embedding layer, and the encoding layer according to the first loss value and a preset differential privacy noise perturbation parameter; determining a second loss value according to the discrete image feature and the autoregressive image feature, and training the autoregressive layer according to the second loss value and the differential privacy noise perturbation parameter; S104: In response to a request for generating and / or reconstructing a target image, the target image is determined and returned through the trained privacy protection model.

[0007] According to the above technical means, in each round of training of the privacy protection model in the present invention, the coding layer, embedding layer, decoding layer and autoregressive layer are trained simultaneously. There is no need to train the decoding layer, embedding layer and coding layer separately first, and then repeatedly input the original image to train the autoregressive layer. In the process of training the model, the number of accesses to the original image is reduced, and the security of the original image is improved. In addition, in the process of training the model, a differential privacy mechanism is introduced to reduce the number of accesses to the original image, reduce the privacy budget of the differential privacy, increase the amount of noise injection, and further protect the privacy of the original image. It is possible to achieve the same model performance as when no noise is introduced under the premise of reducing the privacy budget, and can greatly shorten the training time and improve the training efficiency.

[0008] Furthermore, in S103, the decoding layer, the embedding layer, and the encoding layer are trained according to the first loss value and the preset differential privacy noise perturbation parameter, specifically including: Determine the gradient of the decoding layer according to the first loss value and a preset differential privacy noise perturbation parameter, and adjust the parameters of the decoding layer according to the gradient of the decoding layer; Determine the gradient of the embedding layer according to the adjusted parameters of the decoding layer and the gradient transmission relationship between the decoding layer and the embedding layer, and adjust the parameters of the embedding layer according to the gradient of the embedding layer; According to the adjusted parameters of the embedding layer and the gradient transmission relationship between the embedding layer and the coding layer, the gradient of the coding layer is determined, and the parameters of the coding layer are adjusted according to the gradient of the coding layer.

[0009] According to the above technical means, during the back propagation process of the privacy protection model, when determining the gradient, the differential privacy stochastic gradient descent method is introduced to apply perturbations to the gradients. The encoding layer, embedding layer, and decoding layer are trained according to the perturbed gradients. This enables the privacy protection model to reduce privacy-sensitive information during the reconstruction of the target image, ensure the privacy security of the output image, and improve the performance of the privacy protection model.

[0010] Furthermore, in S103, the autoregressive layer is trained according to the second loss value and the differential privacy noise disturbance parameter, specifically including: According to the second loss value and the differential privacy noise perturbation parameter, the gradient of the autoregressive layer is determined, and the parameters of the autoregressive layer are adjusted according to the gradient of the autoregressive layer.

[0011] According to the above technical means, during the back propagation process of the privacy protection model, when determining the gradient, the differential privacy stochastic gradient descent method is introduced to apply perturbations to the gradients, and the autoregressive layer is trained according to the perturbed gradients. This enables the privacy protection model to generate target images with privacy protection during the process of generating artificial synthetic images.

[0012] Furthermore, in S104, the target image is determined by the privacy protection model trained, which specifically includes: In response to a reconstruction request of the target image, inputting the image to be reconstructed into a coding layer in a trained privacy protection model to obtain image features of the image to be reconstructed; Inputting the image features of the image to be reconstructed into the embedding layer of the trained privacy protection model to obtain discrete image features of the image to be reconstructed; The discrete image features of the image to be reconstructed are input into the decoding layer of the trained privacy protection model to obtain a target image.

[0013] According to the above technical means, the image to be reconstructed input into the model is reconstructed into a target image that does not carry privacy information through the trained privacy protection model, thereby avoiding the problem of being able to identify privacy information through the target image.

[0014] Furthermore, in S104, the target image is determined by the privacy protection model trained, which specifically includes: In response to a request for generating the target image, determining noise information from a preset random noise distribution sequence; Inputting the noise information into the autoregressive layer of the trained privacy protection model to obtain the autoregressive image features of the noise information; The autoregressive image features of the noise information are input into a decoding layer in the trained privacy protection model to obtain a target image.

[0015] According to the above technical means, through the trained privacy protection model, an artificially synthesized target image that does not carry privacy information is generated according to the noise information of the input model. Moreover, since the noise information is randomly selected and unpredictable random information, the generated target image has extremely high privacy.

[0016] Furthermore, the format of the discrete image feature is the same as the format of the autoregressive image feature.

[0017] Furthermore, before inputting the discrete image features into the autoregressive layer in S102, the method further includes: An initialized autoregressive layer is constructed according to the format of the image features output by the embedding layer and the format of the image features input to the decoding layer.

[0018] This specification provides a privacy-preserving image reconstruction and generation device, which uses a privacy-preserving model to be trained, wherein the privacy-preserving model to be trained includes a coding layer, an embedding layer, a decoding layer, and an autoregressive layer, including: An encoding module, used for acquiring an original image for each round of training of the privacy protection model to be trained, and inputting the original image into the encoding layer to obtain original image features; An embedding module, used for inputting the original image features into the embedding layer to obtain discrete image features; A decoding autoregressive module, used to input the discrete image features into the decoding layer and the autoregressive layer respectively, to obtain a reconstructed image and an autoregressive image feature respectively; A training module, configured to determine a first loss value according to the original image and the reconstructed image, and train the decoding layer, the embedding layer, and the encoding layer according to the first loss value and a preset differential privacy noise perturbation parameter; determine a second loss value according to the discrete image feature and the autoregressive image feature, and train the autoregressive layer according to the second loss value and the differential privacy noise perturbation parameter; The determination module is used to respond to a request for generating and / or reconstructing a target image, determine the target image through a trained privacy protection model, and return the target image.

[0019] This specification provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned privacy-protected image reconstruction and generation method is implemented.

[0020] This specification provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements a privacy-protected image reconstruction and generation method when executing the program.

[0021] Beneficial effects achieved by the present invention: In each round of training of the privacy protection model in the present invention, the coding layer, embedding layer, decoding layer and autoregressive layer are trained. There is no need to train the decoding layer, embedding layer and coding layer separately before repeatedly inputting the original image to train the autoregressive layer. In the process of training the model, the number of times the original image is accessed is reduced, thereby improving the security of the original image. In addition, in the process of training the model, a differential privacy mechanism is introduced, and the model is trained using the differential privacy stochastic gradient descent method. By reducing the number of times the original image is accessed, the privacy budget of the differential privacy is reduced, thereby ensuring the privacy of the original image. Under the premise of reducing the privacy budget, it is possible to obtain a model performance that is almost the same as when no noise is introduced and when the differential privacy mechanism is introduced but the traditional training method is used, and it is possible to significantly shorten the training time and improve the training efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 A flowchart of a privacy-protected image reconstruction and generation method provided in an embodiment of this specification; Figure 2 A schematic diagram of the structure of a privacy protection model provided in this manual; Figure 3 A schematic diagram of reconstructing an image provided in this specification; Figure 4 A schematic diagram of generating an image provided in this specification; Figure 5 A schematic diagram of a privacy-protected image reconstruction and generation device provided in this specification; Figure 6 A method corresponding to the Figure 1 Schematic diagram of the structure of an electronic device.

[0023] The drawings are only used for illustrative purposes and should not be construed as limitations on this patent. In order to better illustrate this embodiment, some parts of the drawings may be omitted, enlarged or reduced, and do not represent the size of the actual product. For those skilled in the art, it is understandable that some well-known structures and their descriptions in the drawings may be omitted. The same or similar numbers correspond to the same or similar parts. The terms describing the positional relationship in the drawings are only used for illustrative purposes and should not be construed as limitations on this patent. DETAILED DESCRIPTION

[0024] It should be noted that, in the absence of conflict, the embodiments and technical features in the embodiments of the present application can be combined with each other, and the detailed descriptions in the specific embodiments should be understood as explanations of the purpose of the present application and should not be regarded as improper limitations on the present application.

[0025] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the specific technical solution of the present application will be further described in detail below in conjunction with the drawings in the embodiments of the present application. The following embodiments are used to illustrate the present application, but are not used to limit the scope of the present application.

[0026] In the embodiments of the present application, the terms "include", "comprises" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device including the element.

[0027] The technical solution of the present invention is described in detail below in conjunction with the specific drawings.

[0028] Figure 1 A flowchart of a privacy-preserving image reconstruction and generation method provided in an embodiment of this specification includes the following steps: S100: For each round of training of the privacy protection model to be trained, the original image is input into the encoding layer to obtain the original image features.

[0029] In this specification, the process of image reconstruction and generation based on privacy protection involves the processing of image data. In the embodiments of this specification, the process of image reconstruction and generation based on privacy protection can be performed by a server. Of course, this specification does not limit the device used to perform the process of image reconstruction and generation based on privacy protection. For example, devices such as personal computers and mobile terminals can also be used to perform image reconstruction and generation based on privacy protection. For the convenience of description, the following description is based on the server as the execution subject.

[0030] Figure 2 This is a schematic diagram of the privacy protection model provided in this manual. Figure 2 As shown, the privacy protection model includes an encoding layer, an embedding layer, a decoding layer and an autoregressive layer. The output result of the encoding layer can be sent to the embedding layer, the output result of the embedding layer can be sent to the autoregressive layer and the decoding layer respectively, and the output result of the autoregressive layer can be sent to the decoding layer.

[0031] It is worth noting that the privacy protection model in this specification can be a model that combines a Vector Quantized Variational AutoEncoder (VQ-VAE) model with a Pixel Convolutional Neural Networks (PixelCNN) model, in which the encoding layer, embedding layer, and decoding layer can be regarded as the three components of the VQ-VAE model, and the autoregressive layer can be regarded as the PixelCNN model.

[0032] In one or more embodiments of the present specification, the training method of the privacy protection model is described as follows: for each round of training of the privacy protection model to be trained, the server can obtain the original image used to train the privacy protection model, and input the original image into the encoding layer in the privacy protection model to be trained to obtain the original image features of the original image output by the encoding layer.

[0033] S101: Input the original image features into the embedding layer to obtain discrete image features.

[0034] In one or more embodiments of the present specification, the server then inputs the original image features into the embedding layer to obtain discrete image features output by the embedding layer. Among them, the embedding layer represents a codebook mechanism in the privacy protection model, and the image features can be encoded into discrete vectors through the embedding layer. In the embodiments of the present specification, the original image features of the original image are mapped into a discrete vector (i.e., discrete image features) by using an embedding layer. It is equivalent to that the embedding layer has a codebook mechanism, and the original image features can find the corresponding discrete vectors in the embedding layer (equivalent to the codebook). Through this codebook mechanism, the image information existing in the original image features can be replaced with discrete vectors, but the discrete image features after mapping by the embedding layer can still represent the original image, and the original image represented by the discrete image features does not have privacy information.

[0035] S102: Input the discrete image features into the decoding layer and the regression layer respectively to obtain reconstructed image and autoregressive image features respectively.

[0036] Afterwards, in one or more embodiments of the present specification, the server may input the discrete image features into the decoding layer and the autoregressive layer respectively to obtain the reconstructed image output by the decoding layer and the autoregressive image features output by the autoregressive layer.

[0037] S103: Determine a first loss value according to the original image and the reconstructed image, and train the decoding layer, the embedding layer, and the encoding layer according to the first loss value and the preset differential privacy noise perturbation parameter; determine a second loss value according to the discrete image features and the autoregressive image features, and train the autoregressive layer according to the second loss value and the differential privacy noise perturbation parameter.

[0038] In this specification, when an image needs to be reconstructed, the discrete image features need to be decoded by a decoder to obtain a reconstructed image. When an image needs to be synthesized, in order to make the synthesized image have the same feature distribution as the original image, the discrete image features output by the embedding layer are used as the input of the autoregressive layer during the training process, and in order to make the synthesized image match the image features input to the autoregressive layer, the image features input to the autoregressive layer and the image features output by the autoregressive layer should be as consistent as possible, and the higher the consistency between the two, the more the synthesized image matches the image features input to the autoregressive layer.

[0039] Then, in the back propagation process of the privacy protection model, the server can determine the first loss value according to the difference between the original image and the reconstructed image, and train the decoding layer, the embedding layer, and the encoding layer according to the first loss value and the preset differential privacy noise perturbation parameter, wherein the difference between the original image and the reconstructed image is positively correlated with the first loss value.

[0040] Specifically, the server can determine the gradient of the decoding layer according to the first loss value and the preset differential privacy noise perturbation parameters, that is, introduce differential privacy noise in the process of calculating the gradient, and adjust the parameters of the decoding layer according to the gradient of the decoding layer. Then, according to the adjusted parameters of the decoding layer and the gradient transmission relationship between the decoding layer and the embedding layer, the gradient of the embedding layer is determined, and the parameters of the embedding layer are adjusted according to the gradient of the embedding layer. After that, the server determines the gradient of the encoding layer according to the adjusted parameters of the embedding layer and the gradient transmission relationship between the embedding layer and the encoding layer, and adjusts the parameters of the encoding layer according to the gradient of the encoding layer. The preset Gaussian distribution noise mechanism is a noise curve with a normal distribution. The ordinate of the curve is the probability value, and the probability of each noise in the noise curve being selected is the probability value corresponding to the noise.

[0041] While training the decoding layer, embedding layer, and encoding layer, in one or more embodiments of the present specification, the server may also determine a second loss value based on the difference between the discrete image features and the autoregressive image features output by the autoregressive layer, and train the autoregressive layer based on the second loss value and preset differential privacy noise perturbation parameters.

[0042] Specifically, the server determines the gradient of the autoregressive layer according to the second loss value and the preset differential privacy noise perturbation parameter, and adjusts the parameters of the autoregressive layer according to the gradient of the autoregressive layer.

[0043] It is worth noting that in each round of training of the privacy protection model, the encoding layer, embedding layer, decoding layer and autoregressive layer are trained. Moreover, whether the training between the encoding layer, embedding layer and decoding layer is completed does not affect the training of the autoregressive layer. The training of the autoregressive layer can be performed synchronously with the training between the encoding layer, embedding layer and decoding layer.

[0044] S104: In response to a request for generating and / or reconstructing a target image, the target image is determined and returned through the trained privacy protection model.

[0045] In one or more embodiments of the present specification, after the server has trained the privacy protection model, it can respond to a request for generating and / or reconstructing a target image, determine the target image through the trained privacy protection model, and return it.

[0046] Specifically, in response to a reconstruction request of a target image, the server inputs the image to be reconstructed into the encoding layer of the trained privacy protection model to obtain image features of the image to be reconstructed. The image features of the image to be reconstructed are then input into the embedding layer of the trained privacy protection model to obtain discrete image features of the image to be reconstructed. Afterwards, the server may input the discrete image features of the image to be reconstructed into the decoding layer of the trained privacy protection model to obtain the target image.

[0047] Figure 3 This is a schematic diagram of reconstructing an image provided in this specification. Figure 3 As shown in the figure, the solid line represents the data transmission route when reconstructing the image. The dotted line in the figure represents the route where no data transmission is performed when reconstructing the image. The image to be reconstructed is input into the encoding layer of the privacy protection model, and passes through the embedding layer and the decoding layer in sequence to obtain the target image reconstructed by the decoding layer output.

[0048] In response to a request to generate a target image, the server determines noise information from a preset random noise distribution sequence. The noise information is input into an autoregressive layer in a trained privacy-preserving model to obtain an autoregressive image feature of the noise information. The autoregressive image feature of the noise information is input into a decoding layer in a trained privacy-preserving model to obtain a target image.

[0049] The trained privacy-preserving model generates an artificially synthesized target image that does not carry privacy information based on the noise information of the input model. In the process of generating artificially synthesized images, the privacy-preserving model can generate a random image (target image) with privacy protection through a discrete vector sampled from a random noise distribution sequence. In addition, since the noise information is randomly selected and unpredictable random information, the generated target image has extremely high privacy.

[0050] Figure 4 This is a schematic diagram of generating an image provided in this specification. Figure 4 As shown in the figure, the solid line in the figure represents the data transmission route when generating the target image. The dotted line in the figure represents the route where no data transmission is performed when generating the target image. The noise information is input into the autoregressive layer in the privacy protection model, and the target image generated by the decoding layer output is obtained after passing through the decoding layer.

[0051] In addition, in one or more embodiments of the present specification, since the discrete image features output by the embedding layer can be input to the decoding layer and decoded into an image, and the autoregressive image features output by the autoregressive layer can also be input to the decoding layer and decoded into an image, in order to be able to be input into the decoding layer, the format of the discrete image features output by the embedding layer is the same as the format of the autoregressive image features output by the autoregressive layer.

[0052] In one or more embodiments of the present specification, when constructing an initialized privacy-preserving model to be trained, since the privacy-preserving model may be a model combining a VQ-VAE model and a PixelCNN model, in the process of combining the autoregressive layer representing the PixelCNN model with the three components of the VQ-VAE model (encoding layer, embedding layer, and decoding layer), the server may construct an initialized autoregressive layer according to the format of the image features output by the embedding layer and the format of the image features input to the decoding layer, so that the discrete image features output by the embedding layer can be input to the autoregressive layer, and the autoregressive image features output by the autoregressive layer can be input to the decoding layer.

[0053] In each round of training of the privacy protection model in the present invention, the coding layer, embedding layer, decoding layer and autoregressive layer are trained separately. It is not necessary to train the decoding layer, embedding layer and coding layer separately first, and then repeatedly input the original image to train the autoregressive layer. In the process of training the model, the number of times the original image is accessed is reduced, and the security of the original image is improved. In addition, in the process of training the model, a differential privacy mechanism is introduced, and the model is trained using the differential privacy gradient descent method. By adjusting the training method, the number of times the original image is accessed is reduced, thereby reducing the privacy budget required for the differential privacy mechanism, and further ensuring the privacy of the original image. It is possible to achieve a model performance consistent with that when no noise is introduced while reducing the privacy budget, and it is possible to significantly shorten the training time and improve the training efficiency.

[0054] During the back propagation process of the privacy protection model, the differential privacy noise perturbation parameter is introduced when determining the gradient, and the gradient is perturbed. The decoding layer, embedding layer, and encoding layer are trained according to the perturbed gradient. This allows the privacy protection model to ensure the privacy security of the output image even if it is interfered by noise during the reconstruction of the target image, thereby improving the performance of the privacy protection model.

[0055] It is also possible to introduce noise perturbation parameters when determining the gradient of the autoregressive layer, perturb the gradient, and train the autoregressive layer according to the perturbed gradient. This allows the privacy protection model to ensure the privacy security of the output target image even if it is interfered by noise during the target image generation process, thereby improving the performance of the privacy protection model.

[0056] Through the trained privacy protection model, the image to be reconstructed of the input model is reconstructed into a target image that does not carry privacy information, avoiding the problem of identifying privacy information through the target image. And through the trained privacy protection model, an artificially synthesized target image that does not carry privacy information is generated based on the noise information of the input model. Moreover, since the noise information is randomly selected and unpredictable random information, the generated target image has extremely high privacy.

[0057] The above is a privacy-preserving image reconstruction and generation method provided by one or more embodiments of this specification. Based on the same idea, this specification also provides a corresponding privacy-preserving image reconstruction and generation device, such as Figure 5 As shown, a privacy protection model to be trained is adopted, and the privacy protection model to be trained includes an encoding layer, an embedding layer, a decoding layer and an autoregressive layer.

[0058] The encoding module 500 is used to obtain an original image for each round of training of the privacy protection model to be trained, and input the original image into the encoding layer to obtain original image features; An embedding module 501, used for inputting the original image features into the embedding layer to obtain discrete image features; A decoding and regression module 502, used to input the discrete image features into the decoding layer and the autoregressive layer respectively to obtain a reconstructed image and an autoregressive image feature; A training module 503 is used to determine a first loss value according to the original image and the reconstructed image, and train the decoding layer, the embedding layer, and the encoding layer according to the first loss value and a preset differential privacy noise perturbation parameter; determine a second loss value according to the discrete image feature and the autoregressive image feature, and train the autoregressive layer according to the second loss value and the differential privacy noise perturbation parameter; The determination module 504 is used to respond to a request for generating and / or reconstructing a target image, determine the target image through the trained privacy protection model, and return it.

[0059] This specification also provides a computer-readable storage medium, which stores a computer program, which can be used to execute the above Figure 1 A privacy-preserving image reconstruction and generation method is provided.

[0060] This manual also provides Figure 6 The schematic structure diagram of the electronic device shown in FIG. Figure 6 As shown, at the hardware level, the electronic device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory, and may also include other hardware required for the business. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to achieve the above Figure 1 The privacy-preserving image reconstruction and generation method.

[0061] Of course, in addition to software implementation, this specification does not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc., that is to say, the executor of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0062] The serial numbers of the embodiments of the present application are only for description and do not represent the advantages and disadvantages of the embodiments. The above are only preferred embodiments of the present application, and do not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made by using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A privacy-preserving image reconstruction and generation method, characterized in that: Using a privacy protection model to be trained, the privacy protection model to be trained includes an encoding layer, an embedding layer, a decoding layer and an autoregressive layer, and includes the following steps: S100: For each round of training of the privacy protection model to be trained, an original image is obtained, and the original image is input into the encoding layer to obtain original image features; S101: Inputting the original image features into the embedding layer to obtain discrete image features; S102: Inputting the discrete image features into the decoding layer and the autoregressive layer respectively to obtain a reconstructed image and an autoregressive image feature respectively; S103: determining a first loss value according to the original image and the reconstructed image, and training the decoding layer, the embedding layer, and the encoding layer according to the first loss value and a preset differential privacy noise perturbation parameter; determining a second loss value according to the discrete image feature and the autoregressive image feature, and training the autoregressive layer according to the second loss value and the differential privacy noise perturbation parameter; S104: In response to a request for generating and / or reconstructing a target image, the target image is determined and returned through the trained privacy protection model.

2. The privacy-preserving image reconstruction and generation method according to claim 1, characterized in that: In S103, the decoding layer, the embedding layer, and the encoding layer are trained according to the first loss value and the preset differential privacy noise disturbance parameter, specifically including: Determine the gradient of the decoding layer according to the first loss value and a preset differential privacy noise perturbation parameter, and adjust the parameters of the decoding layer according to the gradient of the decoding layer; Determine the gradient of the embedding layer according to the adjusted parameters of the decoding layer and the gradient transmission relationship between the decoding layer and the embedding layer, and adjust the parameters of the embedding layer according to the gradient of the embedding layer; According to the adjusted parameters of the embedding layer and the gradient transmission relationship between the embedding layer and the coding layer, the gradient of the coding layer is determined, and the parameters of the coding layer are adjusted according to the gradient of the coding layer.

3. The privacy-preserving image reconstruction and generation method according to claim 1, characterized in that: In S103, the autoregressive layer is trained according to the second loss value and the differential privacy noise disturbance parameter, specifically including: According to the second loss value and the differential privacy noise perturbation parameter, the gradient of the autoregressive layer is determined, and the parameters of the autoregressive layer are adjusted according to the gradient of the autoregressive layer.

4. The privacy-preserving image reconstruction and generation method according to claim 1, characterized in that: In S104, the privacy protection model completed through training determines the target image, which specifically includes: In response to a reconstruction request of the target image, inputting the image to be reconstructed into a coding layer in a trained privacy protection model to obtain image features of the image to be reconstructed; Inputting the image features of the image to be reconstructed into the embedding layer of the trained privacy protection model to obtain discrete image features of the image to be reconstructed; The discrete image features of the image to be reconstructed are input into a decoding layer in the trained privacy protection model to obtain a target image.

5. The privacy-preserving image reconstruction and generation method according to claim 1, characterized in that: In S104, the privacy protection model completed through training determines the target image, which specifically includes: In response to a request for generating the target image, determining noise information from a preset random noise distribution sequence; Inputting the noise information into the autoregressive layer of the trained privacy protection model to obtain the autoregressive image features of the noise information; The autoregressive image features of the noise information are input into a decoding layer in the trained privacy protection model to obtain a target image.

6. The privacy-preserving image reconstruction and generation method according to claim 1, characterized in that: The format of the discrete image features is the same as the format of the autoregressive image features.

7. The privacy-preserving image reconstruction and generation method according to claim 1, characterized in that: Before inputting the discrete image features into the autoregressive layer in S102, the method further includes: An initialized autoregressive layer is constructed according to the format of the image features output by the embedding layer and the format of the image features input to the decoding layer.

8. A privacy-preserving image reconstruction and generation device, characterized in that: A privacy protection model to be trained is adopted, wherein the privacy protection model to be trained includes an encoding layer, an embedding layer, a decoding layer, and an autoregressive layer, including: An encoding module, used for acquiring an original image for each round of training of the privacy protection model to be trained, and inputting the original image into the encoding layer to obtain original image features; An embedding module, used for inputting the original image features into the embedding layer to obtain discrete image features; A decoding autoregressive module, used to input the discrete image features into the decoding layer and the autoregressive layer respectively, to obtain a reconstructed image and an autoregressive image feature respectively; A training module, configured to determine a first loss value according to the original image and the reconstructed image, and train the decoding layer, the embedding layer, and the encoding layer according to the first loss value and a preset differential privacy noise perturbation parameter; determine a second loss value according to the discrete image feature and the autoregressive image feature, and train the autoregressive layer according to the second loss value and the differential privacy noise perturbation parameter; The determination module is used to respond to a request for generating and / or reconstructing a target image, determine the target image through a trained privacy protection model, and return the target image.

9. A computer-readable storage medium, wherein the storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method according to any one of claims 1 to 7 when executing the program.

Images