Thumbnail keeping blind extraction method, device and equipment for privacy information of encrypted images

Abstract

The present application relates to a blind extraction method, device and equipment for privacy information of thumbnail-preserved encrypted images, which can provide an emergency solution for users who have passed authentication and legal authorization in abnormal situations such as when the key cannot be obtained, and improve the security redundancy. By using the ciphertext image samples generated by the thumbnail-preserved encryption method and the corresponding plaintext image samples as training samples, and inputting them into the privacy information extraction network constructed based on the generative adversarial network, the discriminator and the generator therein are alternately trained; during the training process, the generator generates predicted privacy information according to the ciphertext image samples, and after using the ciphertext image samples and the predicted privacy information to update the parameters of the discriminator, the parameters of the generator are updated by using the discriminator. The user inputs the ciphertext image to be processed into the generator in the trained privacy information extraction network, and decrypts the ciphertext image and extracts the privacy information in a key-free manner.

Description

Technical Field

[0001] This application relates to the field of image security technology, and in particular to a blind extraction method, device and equipment for privacy information of thumbnail-preserved encrypted images. Background Art

[0002] With the popularization of the Internet and various electronic devices, people have captured a large number of images. Due to the limited storage space of electronic devices and the convenience of cloud storage, more and more users choose to save a large number of images on cloud servers, but the security of cloud images has become one of the key issues that people worry about. In order to improve the security of cloud images and prevent the leakage of privacy information in images, researchers have proposed a series of image encryption algorithms, such as traditional image encryption (the ciphertext image is a noise-like image), content-based ciphertext image retrieval, region of interest image encryption, etc. However, these image encryption methods cannot well balance the privacy and usability of ciphertext images. To solve this problem, researchers have proposed a thumbnail-preserved encryption scheme.

[0003] The thumbnail-preserved encryption scheme erases the detailed information of the image while keeping the thumbnail of the ciphertext image the same or approximate to the thumbnail of the original image. The image owner with prior knowledge can identify the content through the thumbnail of the ciphertext image. Therefore, this type of scheme better balances the privacy and usability of ciphertext images. To ensure security, this type of scheme usually needs to use relevant information of the image (such as generation time, name, etc.) to generate keys through a dynamic update mechanism. When these information appear abnormal loss or incorrect modification during the image transmission, storage and processing, the decryption party cannot obtain the correct key and decrypt the ciphertext image. Summary of the Invention

[0004] Based on this, in view of the above technical problems, it is necessary to provide a blind extraction method, device and equipment for privacy information of thumbnail-preserved encrypted images without keys for users who have passed authentication and legal authorization in abnormal situations.

[0005] A blind extraction method for privacy information of thumbnail-preserved encrypted images, the method includes:

[0006] Obtain a training sample set, where the training sample set includes multiple ciphertext image samples related to a certain type of target and corresponding plaintext image samples, and the ciphertext image samples are generated according to the plaintext image samples by using a thumbnail-preserved encryption method;

[0007] Use the ciphertext image samples and the corresponding plaintext image samples in the training sample set as training samples, input them into the privacy information extraction network constructed based on the generative adversarial network, and alternately train the discriminator and the generator in the privacy information extraction network to obtain a trained privacy information extraction network. During the iterative process of training the privacy information extraction network, the generator generates predicted privacy information according to the ciphertext image samples, uses the ciphertext image samples and the predicted privacy information to update the parameters in the discriminator, and then uses the discriminator with updated parameters to update the parameters in the generator;

[0008] Obtain the ciphertext image to be extracted with privacy information, and input the ciphertext image into the generator in the trained privacy information extraction network to obtain the privacy information of the ciphertext image.

[0009] In one embodiment, the generator includes a downsampling convolutional block, a residual block, and an upsampling transposed convolutional block, and corresponding batch normalization operations and activation functions are also set in each block.

[0010] In one embodiment, after the discriminator downsamples the input data, it extracts multi-scale features and performs true / false discrimination according to the features of different sizes.

[0011] In one embodiment, when updating the discriminator:

[0012] The discriminator calculates the true discrimination loss according to the ciphertext image sample and the plaintext image sample;

[0013] The discriminator calculates the false discrimination loss according to the ciphertext image sample and the predicted privacy information;

[0014] Update the parameters of the discriminator according to the true discrimination loss and the false discrimination loss to obtain the discriminator with updated parameters.

[0015] In one embodiment, the discriminator uses a Markov discriminator.

[0016] In one embodiment, a target recognition network is further included in the privacy information extraction network. When updating the generator:

[0017] Use the target recognition network to perform target recognition according to the predicted privacy information and the plaintext image sample respectively, and calculate the target recognition loss according to the recognition results;

[0018] Calculate the structural similarity loss and the L1 norm loss according to the predicted privacy information and the plaintext image sample;

[0019] Using the discriminator after updating the parameters, calculate the true judgment loss according to the ciphertext image sample and the predicted privacy information;

[0020] Update the parameters of the generator according to the target recognition loss, structural similarity loss, L1 norm loss, and true judgment loss.

[0021] In one embodiment, the target in the ciphertext image sample is a human face.

[0022] This application also provides a blind extraction device for the privacy information of thumbnail-preserved encrypted images. The device includes:

[0023] A training sample set acquisition module, configured to acquire a training sample set, where the training sample set includes multiple ciphertext image samples related to a certain type of target and corresponding plaintext image samples, and the ciphertext image samples are generated according to the plaintext image samples by using a thumbnail-preserved encryption method;

[0024] A privacy information extraction network training module, configured to use the ciphertext image samples and the corresponding plaintext image samples in the training sample set as training samples, input them into a privacy information extraction network constructed based on a generative adversarial network, alternately train the discriminator and the generator in the privacy information extraction network, and obtain a trained privacy information extraction network. During the iterative process of training the privacy information extraction network, the generator generates predicted privacy information according to the ciphertext image samples, updates the parameters in the discriminator by using the ciphertext image samples and the predicted privacy information, and then updates the parameters in the generator by using the discriminator with updated parameters;

[0025] A privacy information blind extraction module, configured to obtain a ciphertext image for which privacy information is to be extracted, input the ciphertext image into the generator in the trained privacy information extraction network, and obtain the privacy information of the ciphertext image.

[0026] A computer device includes a memory and a processor. When the processor executes the computer program, the following steps are implemented:

[0027] Acquire a training sample set, where the training sample set includes multiple ciphertext image samples related to a certain type of target and corresponding plaintext image samples, and the ciphertext image samples are generated according to the plaintext image samples by using a thumbnail-preserved encryption method;

[0028] Use the ciphertext image samples and the corresponding plaintext image samples in the training sample set as training samples, and input them into the privacy information extraction network constructed based on the generative adversarial network. Alternately train the discriminator and the generator in the privacy information extraction network to obtain a trained privacy information extraction network. During the iterative process of training the privacy information extraction network, the generator generates predicted privacy information according to the ciphertext image samples, uses the ciphertext image samples and the predicted privacy information to update the parameters in the discriminator, and then uses the discriminator with updated parameters to update the parameters in the generator;

[0029] Obtain the ciphertext image for which privacy information is to be extracted, and input the ciphertext image into the generator in the trained privacy information extraction network to obtain the privacy information of the ciphertext image.

[0030] A computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0031] Obtain a training sample set, which includes multiple ciphertext image samples related to a certain type of target and the corresponding plaintext image samples. The ciphertext image samples are generated according to the plaintext image samples using the thumbnail retention encryption method;

[0032] Use the ciphertext image samples and the corresponding plaintext image samples in the training sample set as training samples, and input them into the privacy information extraction network constructed based on the generative adversarial network. Alternately train the discriminator and the generator in the privacy information extraction network to obtain a trained privacy information extraction network. During the iterative process of training the privacy information extraction network, the generator generates predicted privacy information according to the ciphertext image samples, uses the ciphertext image samples and the predicted privacy information to update the parameters in the discriminator, and then uses the discriminator with updated parameters to update the parameters in the generator;

[0033] Obtain the ciphertext image for which privacy information is to be extracted, and input the ciphertext image into the generator in the trained privacy information extraction network to obtain the privacy information of the ciphertext image.

[0034] The above thumbnail-preserving blind extraction method, device, and equipment for privacy information of encrypted images use the ciphertext image samples and corresponding plaintext image samples in the training sample set generated by the thumbnail-preserving encryption method as training samples, and input them into the privacy information extraction network constructed based on the generative adversarial network. The discriminator and generator in it are alternately trained. During the iterative training process, the generator generates predicted privacy information according to the ciphertext image samples. After using the ciphertext image samples and the predicted privacy information to update the parameters in the discriminator, the parameters in the generator are updated using it. The ciphertext image to be extracted for privacy information is input into the generator in the trained privacy information extraction network, so that users who have passed authentication and legal authorization can decrypt the ciphertext image and extract privacy information in the case of abnormal situations such as being unable to obtain the key, improving the security redundancy of the thumbnail-preserving encryption scheme. Brief Description of the Drawings

[0035] Figure 1 It is a schematic flowchart of the blind extraction method for privacy information of thumbnail-preserving encrypted images in one embodiment;

[0036] Figure 2 It is a schematic structural diagram of the discriminator in one embodiment;

[0037] Figure 3 It is a schematic training diagram of the privacy information extraction network in one embodiment;

[0038] Figure 4 It is a structural block diagram of the blind extraction device for privacy information of thumbnail-preserving encrypted images in one embodiment;

[0039] Figure 5 It is an internal structural diagram of a computer device in one embodiment. Detailed Embodiments

[0040] In order to make the purpose, technical solution, and advantages of this application clearer, the following further details this application in combination with the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain this application and are not used to limit this application.

[0041] Aiming at the problems that in the process of transmission, storage, and processing of thumbnail-preserving encrypted images, abnormal situations such as information loss and incorrect modification may occur, and it is impossible to generate a key through the key update mechanism to correctly decrypt the image and extract privacy information. In this application, as Figure 1 shown, a key-free blind extraction method for privacy information of thumbnail-preserving encrypted images is provided for users who have passed authentication and legal authorization in case of emergency, including the following steps:

[0042] Step S100: Obtain a training sample set, which includes multiple encrypted image samples related to a certain type of target and corresponding plaintext image samples. The encrypted image samples are generated from the plaintext image samples using the thumbnail-preserving encryption method.

[0043] Step S110: Use the encrypted image samples and the corresponding plaintext image samples in the training sample set as training samples and input them into a privacy information extraction network constructed based on a generative adversarial network. Alternately train the discriminator and the generator in the privacy information extraction network to obtain a trained privacy information extraction network. During the iterative process of training the privacy information extraction network, the generator generates predicted privacy information based on the encrypted image samples, uses the encrypted image samples and the predicted privacy information to update the parameters in the discriminator, and then uses the discriminator with updated parameters to update the parameters in the generator.

[0044] Step S120: Obtain an encrypted image for which privacy information extraction is to be performed, and input the encrypted image into the generator in the trained privacy information extraction network to obtain the privacy information of the encrypted image.

[0045] In this application, a blind extraction method for privacy information of thumbnail-preserving encrypted images without a key is proposed. This method takes the deep learning GAN model, i.e., the generative adversarial network, as the core, and combines a residual network and a multi-scale feature discriminator to optimize it. Inputting the encrypted image into the generator network model can process the encrypted image and extract the privacy information therein. This method can provide an emergency solution for users who have passed authentication and legal authorization in the case where the key cannot be normally generated, and improve the security redundancy of the thumbnail-preserving encryption scheme.

[0046] The application scenario of this method can be that a user encrypts an image using the thumbnail-preserving encryption method to protect their privacy and uploads the encrypted image to a certain cloud storage platform. However, during the transmission, storage, and processing of the encrypted image, due to certain reasons, abnormal situations such as missing image information or incorrect modification occur, resulting in the inability to obtain the correct decryption key. After passing authentication and legal authorization, the user can use this method to decrypt the encrypted image and extract the privacy information, thereby improving the security redundancy of the thumbnail-preserving encryption scheme.

[0047] Furthermore, this method can also be applied to assist law enforcement. Under the authorization of security organs or judicial organs, it can assist in cracking encrypted information related to public security.

[0048] In step S100, since this method mainly processes the ciphertext image encrypted by the thumbnail-preserving encryption method, when constructing the training sample set, first, according to multiple plaintext image samples of a certain type of target, the ciphertext image samples obtained by encrypting using the thumbnail-preserving encryption method are used to construct the training sample set based on the ciphertext image samples and the corresponding plaintext image samples, where the plaintext image samples serve as the ground truth labels.

[0049] In this embodiment, the target type in the image can be a face, an animal, a specific item such as a vehicle, a building, a daily necessity, etc. In this article, the face target is taken as an example for illustration.

[0050] Specifically, several images are selected from the images in the face image dataset FFHQ (Flickr-Faces-HQ) and scaled to a size of 3×256×256 pixels as the training set trainY. The training set trainY belongs to the Y domain of the plaintext image samples, and the images in the training set trainY are encrypted by thumbnail-preserving encryption to generate the training set trainX. The training set trainX belongs to the X domain of the ciphertext image samples, and then the training sample set is constructed based on the training set trainY and the training set trainX.

[0051] In one embodiment, there are 20,000 images in both trainX and trainY, and the image size is 3×256×256. The TPE-ADE encryption method is used for the plaintext image samples, and each JPEG plaintext image sample is encrypted according to a thumbnail block size of 16×16 to form the dataset trainX required for model training.

[0052] In step S110, the ciphertext image samples and the corresponding plaintext image samples in the training sample set are used as training samples and input into the privacy information extraction network for training.

[0053] In this embodiment, the privacy information extraction network consists of a generator G and a discriminator D. The generator G includes a downsampling convolutional block, a residual block, and an upsampling transposed convolutional block, and corresponding batch normalization operations and activation functions are also set in each block. After the discriminator D downsamples the input data, it extracts multi-scale features and makes a true / false discrimination based on the features of different sizes.

[0054] Specifically, the discriminator D downsamples the input image, outputs and calculates the losses for the convolution results at different levels of sampling, that is, performs multi-scale feature discrimination. For example, if the image X passes through the convolutional layer 1 to obtain X1, and X1 passes through the convolutional layer 2 to obtain X2, then both X1 and X2 are output for calculating the loss of the discriminator D. At the same time, in this method, the ciphertext image samples also need to be input into the discriminator to extract their features, enhancing the ability of the discriminator D to counter the generator G, thereby improving the ability of the generator G to extract private information.

[0055] As Figure 2 shown, it is the specific network structure of the discriminator D, whose input is the ciphertext image sample (i.e., the ciphertext image in Figure 2 ), and the corresponding plaintext image sample or predicted private information (i.e., the generated image or plaintext image in Figure 2 ). First, the two input data are concatenated to obtain 6-channel image data (i.e., the 6-channel picture data in Figure 2 ), and then this 6-channel image data is downsampled. The input of the downsampling is 6 channels, the convolutional kernel is 4×4, the stride is 2, and the output is 64 channels. The data after downsampling passes through 4 multi-scale feature extraction units in sequence. The inputs of these 4 different-scale feature extraction units are 64 channels, 128 channels, 256 channels, and 512 channels respectively. The size of the convolutional kernel is 4×4. The stride of the first two-scale feature extraction units is 2, and the stride of the last two-scale feature extraction units is 1. The outputs are 128 channels, 256 channels, 512 channels, and 1 channel respectively. The convolutional results of the outputs are four sizes of 128×63×63, 256×30×30, 512×27×27, and 1×26×26 respectively. Then, discrimination is performed based on the multi-scale features extracted by these four different-scale feature extraction units.

[0056] In this embodiment, in the blind extraction network of private information, first, the generator G generates the predicted private information fakeY according to the ciphertext image sample realX, using the following formula:

[0057] ;

[0058] Next, the parameters in the discriminator D are initially updated using the adversarial loss. The adversarial loss is a loss function that constrains the discriminator D and includes the false discrimination loss and the true discrimination loss. Among them, the discriminator D calculates the true discrimination loss according to the ciphertext image sample realX and the plaintext image sample , and calculates the false discrimination loss according to the ciphertext image sample realX and the predicted private information fakeY.

[0059] Furthermore, the discriminator D uses the adversarial loss function for the plaintext image sample Identify with the predicted privacy information fakeY, and the process is expressed as:

[0060] ;

[0061] In the above formula, represents The discrimination result obtained by inputting into the discriminator D, where true represents 1 and false represents 0, and the result is When it means that the discriminator D believes that belongs to the plaintext image sample Y domain, and when the result is false, it means that the discriminator D believes that realY does not belong to the plaintext image sample Y domain. represents splicing the images in the channel dimension. represents calculating the cross-entropy loss.

[0062] In this embodiment, a Markov discriminator is adopted in the discriminator D, and its output is an n×n matrix. When calculating the adversarial loss, the true discrimination loss is the loss calculated with the all-1 matrix, and the false discrimination loss is the loss calculated with the all-0 matrix.

[0063] Furthermore, after initially updating the parameters of the discriminator D using the adversarial loss function then use the discriminator D with updated parameters to update the parameters in the generator G.

[0064] In this embodiment, the privacy information extraction network further includes an object recognition network. When updating the generator: use the object recognition network to perform object recognition according to the predicted privacy information and the plaintext image sample respectively, calculate the object recognition loss according to the recognition result, calculate the structural similarity loss and the L1 norm loss according to the predicted privacy information and the plaintext image sample, use the discriminator D with updated parameters to calculate the true discrimination loss according to the ciphertext image sample and the predicted privacy information, and finally update the parameters of the generator according to the object recognition loss, the structural similarity loss, the L1 norm loss, and the true discrimination loss.

[0065] Specifically, the true discrimination loss calculated by using the discriminator D with updated parameters according to the ciphertext image sample and the predicted privacy information, that is, use the discriminator D with updated parameters to judge whether the image generated by the generator G belongs to the Y domain, so as to constrain whether the privacy information generated by the generator G belongs to the Y domain. is expressed as:

[0066] ;

[0067] Furthermore, it also uses the structural similarity loss and the L1 norm loss The constructed perceptual loss function constrains the generator G, that is, by comparing the plaintext image sample realY and the generated predicted privacy information fakeY, so as to constrain the generator G to generate images with richer textures and more privacy information. The formula is as follows:

[0068] ;

[0069] ;

[0070] In the above formula, represents calculating the L1 norm distance, represents calculating the structural similarity, and its value of 1 means the same, and 0 means completely different.

[0071] Furthermore, the perceptual loss function is expressed as:

[0072] ;

[0073] In the above formula, and are constant coefficients, and their value ranges are between 0 and 100.

[0074] Furthermore, the object recognition loss L ID is a loss function for constraining the generator G. That is, the plaintext image sample realY and the generated predicted privacy information fakeY are input into the object recognition network, so as to constrain the privacy information in the image generated by the generator G to be the same as the privacy information in the real image.

[0075] In this embodiment, taking the object as a human face as an example, that is, the object recognition network is a face recognition model, which is expressed as:

[0076] ;

[0077] In the above formula, recognizer(realY) and recognizer(fakeY) respectively represent inputting realY and fakeY into the face recognition model recognizer to obtain their respective identity feature vectors.

[0078] In this embodiment, the face recognition model used is SEResNet50, refers to calculating the cosine similarity.

[0079] It should be noted here that when the object in the ciphertext image is not a human face, a network that can recognize the object information can be used.

[0080] In this embodiment, the loss for the discriminator D is expressed as:

[0081] ;

[0082] Further, for the loss of the generator, it is expressed as:

[0083] ;

[0084] In the above formula, and are constant coefficients, and their value ranges are between 0 and 100.

[0085] In step S120, the trained generator G performs blind extraction of privacy information on the encrypted ciphertext image of the thumbnail. The ciphertext image for which privacy information needs to be extracted is input into the trained generator G. First, the convolutional neural network is used to extract features from the image, and the feature matrix is passed through the residual block. Then, the transposed convolutional layer is used to decode the feature matrix, and finally, an image with a size of 3×256×256 in the generated plaintext domain is obtained. That is, under the condition of no key, the ciphertext image is processed and the privacy information therein is extracted.

[0086] In this embodiment, since the thumbnail-preserving encryption method includes various other methods in addition to the TPE-ADE encryption method proposed above, at this time, for each specific method, a corresponding generator G can be trained, that is, in the training dataset, the corresponding ciphertext image samples are generated based on the plaintext image samples by using the corresponding thumbnail-preserving encryption method.

[0087] As Figure 3 shown, it is a schematic diagram for training the privacy information extraction network.

[0088] In the above-mentioned blind extraction method of privacy information from thumbnail-preserving encrypted images, in the case of abnormal situations where the key cannot be obtained, an emergency solution can be provided for users who have passed authentication and legal authorization, improving the security redundancy. By using the ciphertext image samples and the corresponding plaintext image samples in the training sample set generated by the thumbnail-preserving encryption method as training samples, they are input into the privacy information extraction network constructed based on the generative adversarial network, and the discriminator and generator in the privacy information extraction network are alternately trained. During the iterative training process, the generator generates predicted privacy information based on the ciphertext image samples. After using the ciphertext image samples and the predicted privacy information to update the parameters in the discriminator, the parameters in the generator are updated using it. The user can input the ciphertext image to be processed into the generator in the trained privacy information extraction network, and decrypt the ciphertext image and extract the privacy information in a key-free manner. The privacy information extracted by this method includes face contours, eye features, hairstyles, etc. Compared with the ciphertext image, it can intuitively reflect more detailed privacy information, making the predicted privacy information very close to the original plaintext image sample in visual perception and enhancing the usability of the predicted privacy information.

[0089] It should be understood that although Figure 1 each step in the flowchart is shown in sequence according to the indication of the arrow, these steps are not necessarily executed in the order indicated by the arrow. Unless otherwise clearly stated in this article, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 1 at least a part of the steps in

[0090] In one embodiment, as Figure 4 shown, a blind extraction device for privacy information of a thumbnail-preserved encrypted image is provided, including: a training sample set acquisition module 200, a privacy information extraction network training module 210, and a privacy information blind extraction module 220, where:

[0091] The training sample set acquisition module 200 is used to acquire a training sample set, where the training sample set includes multiple ciphertext image samples related to a certain type of target and corresponding plaintext image samples, and the ciphertext image samples are generated according to the plaintext image samples by using a thumbnail-preserved encryption method;

[0092] The privacy information extraction network training module 210 is used to use the ciphertext image samples and the corresponding plaintext image samples in the training sample set as training samples, input them into a privacy information extraction network constructed based on a generative adversarial network, alternately train the discriminator and the generator in the privacy information extraction network, and obtain a trained privacy information extraction network. During the iterative process of training the privacy information extraction network, the generator generates predicted privacy information according to the ciphertext image samples, uses the ciphertext image samples and the predicted privacy information to update the parameters in the discriminator, and then uses the discriminator with updated parameters to update the parameters in the generator;

[0093] The privacy information blind extraction module 220 is used to acquire a ciphertext image to be extracted with privacy information, input the ciphertext image into the generator in the trained privacy information extraction network, and obtain the privacy information of the ciphertext image.

[0094] For the specific limitations of the blind extraction device for preserving the privacy information of encrypted images in thumbnails, reference may be made to the limitations of the blind extraction method for preserving the privacy information of encrypted images in thumbnails in the foregoing text, which will not be elaborated here. Each module in the above-mentioned blind extraction device for preserving the privacy information of encrypted images in thumbnails can be implemented in whole or in part by software, hardware, and their combination. The above-mentioned modules can be embedded in the processor of the computer device in hardware form or be independent of it, or be stored in the memory of the computer device in software form, so as to facilitate the processor to call and execute the operations corresponding to each of the above modules.

[0095] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structural diagram may be as Figure 5 shown. The computer device includes a processor, a memory, a network interface, a display screen, and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, it implements a blind extraction method for preserving the privacy information of encrypted images in thumbnails. The display screen of the computer device may be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device may be a touch layer covering the display screen, or a button, a trackball, or a touchpad provided on the outer shell of the computer device, or an external keyboard, touchpad, or mouse, etc.

[0096] Those skilled in the art can understand that Figure 5 the structure shown in

[0097] is only a block diagram of some structures related to the solution of this application, and does not constitute a limitation on the computer device to which the solution of this application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0098] Obtain a training sample set, where the training sample set includes multiple ciphertext image samples related to a certain type of target and corresponding plaintext image samples, and the ciphertext image samples are generated according to the plaintext image samples by using the thumbnail-preserving encryption method;

[0099] Use the ciphertext image samples and the corresponding plaintext image samples in the training sample set as training samples, and input them into the privacy information extraction network constructed based on the generative adversarial network. Alternately train the discriminator and the generator in the privacy information extraction network to obtain a trained privacy information extraction network. During the iterative process of training the privacy information extraction network, the generator generates predicted privacy information based on the ciphertext image samples, uses the ciphertext image samples and the predicted privacy information to update the parameters in the discriminator, and then uses the discriminator with updated parameters to update the parameters in the generator;

[0100] Obtain the ciphertext image for which privacy information is to be extracted, and input the ciphertext image into the generator in the trained privacy information extraction network to obtain the privacy information of the ciphertext image.

[0101] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0102] Obtain a training sample set, where the training sample set includes multiple ciphertext image samples related to a certain type of target and the corresponding plaintext image samples, and the ciphertext image samples are generated according to the plaintext image samples using the thumbnail retention encryption method;

[0103] Use the ciphertext image samples and the corresponding plaintext image samples in the training sample set as training samples, and input them into the privacy information extraction network constructed based on the generative adversarial network. Alternately train the discriminator and the generator in the privacy information extraction network to obtain a trained privacy information extraction network. During the iterative process of training the privacy information extraction network, the generator generates predicted privacy information based on the ciphertext image samples, uses the ciphertext image samples and the predicted privacy information to update the parameters in the discriminator, and then uses the discriminator with updated parameters to update the parameters in the generator;

[0104] Obtain the ciphertext image for which privacy information is to be extracted, and input the ciphertext image into the generator in the trained privacy information extraction network to obtain the privacy information of the ciphertext image.

[0105] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, storage, database, or other medium used in the various embodiments provided in the present application can include non-volatile and / or volatile memories. Non-volatile memories can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memories can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and Rambus dynamic RAM (RDRAM), etc.

[0106] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope described in this specification.

[0107] The above-described embodiments merely represent several implementation manners of the present application. The description thereof is relatively specific and detailed, but it should not be construed as a limitation on the scope of the invention. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. A method for blindly extracting private information of encrypted images by keeping thumbnails, characterized in that: The method comprises: Acquire a training sample set, wherein the training sample set includes a plurality of ciphertext image samples and corresponding plaintext image samples related to a certain type of target, wherein the ciphertext image samples are generated according to the plaintext image samples by using a thumbnail preservation encryption method; The ciphertext image samples and the corresponding plaintext image samples in the training sample set are used as training samples and input into a private information extraction network constructed based on a generative adversarial network. The discriminator and the generator in the private information extraction network are alternately trained to obtain a trained private information extraction network. In the iterative process of training the private information extraction network, the generator generates predicted private information according to the ciphertext image samples, and the true judgment loss and the false judgment loss are calculated using the ciphertext image samples, the predicted private information and the corresponding plaintext image samples, and the parameters in the discriminator are updated according to the true judgment loss and the false judgment loss. The process is expressed as follows: In the above formula, represents the adversarial loss function, represents a ciphertext image sample, represents a plaintext image sample, Indicates that the generator is based on the ciphertext image sample Generate prediction privacy information, Represents the discriminator's judgment result, true represents 1, false represents 0, and the result is When the discriminator D thinks belongs to the plaintext image sample Y domain. When the result is false, it means that the discriminator D believes that realY does not belong to the plaintext image sample Y domain. Indicates that the images are concatenated in the channel dimension. Indicates the calculation of cross entropy loss; Then, the parameters in the generator are updated by using the discriminator with updated parameters. The private information extraction network also includes a target recognition network. The process is expressed as follows: using the target recognition network to perform target recognition according to the predicted private information and the plaintext image sample, and calculating the target recognition loss according to the recognition result, calculating the structural similarity loss and the L1 norm loss according to the predicted private information and the plaintext image sample, using the discriminator with updated parameters to calculate the true judgment loss according to the ciphertext image sample and the predicted private information, and updating the parameters of the generator according to the target recognition loss, the structural similarity loss, the L1 norm loss and the true judgment loss; Obtain a ciphertext image for which private information extraction is to be performed, input the ciphertext image into a generator in the trained private information extraction network, and obtain the private information of the ciphertext image.

2. The method for blindly extracting private information of an encrypted image by retaining a thumbnail according to claim 1, characterized in that: The generator includes a downsampling convolution block, a residual block, and an upsampling deconvolution block, wherein each of the blocks is also provided with a corresponding batch normalization operation and an activation function.

3. The method for blindly extracting private information of an encrypted image by retaining a thumbnail according to claim 2, characterized in that: After downsampling the input data, the discriminator extracts multi-scale features and performs true and false discrimination based on features of different sizes.

4. The method for blindly extracting private information of an encrypted image by retaining a thumbnail according to claim 3, characterized in that: The discriminator adopts a Markov discriminator.

5. The method for blindly extracting private information of an encrypted image with thumbnails preserved according to any one of claims 1 to 4, characterized in that: The target in the ciphertext image sample is a human face.

6. A device for blindly extracting private information of an encrypted image with thumbnail preservation, characterized in that: The device comprises: A training sample set acquisition module, used to acquire a training sample set, wherein the training sample set includes a plurality of ciphertext image samples and corresponding plaintext image samples related to a certain type of target, wherein the ciphertext image samples are generated from the plaintext image samples using a thumbnail preservation encryption method; The private information extraction network training module is used to input the ciphertext image samples and the corresponding plaintext image samples in the training sample set as training samples into the private information extraction network constructed based on the generative adversarial network, alternately train the discriminator and the generator in the private information extraction network to obtain a trained private information extraction network. In the iterative process of training the private information extraction network, the generator generates predicted private information according to the ciphertext image samples, calculates the true judgment loss and the false judgment loss by using the ciphertext image samples, and updates the parameters in the discriminator according to the true judgment loss and the false judgment loss. The process is expressed as follows: In the above formula, represents the adversarial loss function, represents a ciphertext image sample, represents a plaintext image sample, Indicates that the generator is based on the ciphertext image sample Generate prediction privacy information, Represents the discriminator's judgment result, true represents 1, false represents 0, and the result is When the discriminator D thinks belongs to the plaintext image sample Y domain. When the result is false, it means that the discriminator D believes that realY does not belong to the plaintext image sample Y domain. Indicates that the images are concatenated in the channel dimension. Indicates the calculation of cross entropy loss; Then, the parameters in the generator are updated by using the discriminator with updated parameters. The private information extraction network also includes a target recognition network. The process is expressed as follows: using the target recognition network to perform target recognition according to the predicted private information and the plaintext image sample, and calculating the target recognition loss according to the recognition result, calculating the structural similarity loss and the L1 norm loss according to the predicted private information and the plaintext image sample, using the discriminator with updated parameters to calculate the true judgment loss according to the ciphertext image sample and the predicted private information, and updating the parameters of the generator according to the target recognition loss, the structural similarity loss, the L1 norm loss and the true judgment loss; The private information blind extraction module is used to obtain a ciphertext image for private information extraction, input the ciphertext image into the generator in the trained private information extraction network, and obtain the private information of the ciphertext image.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 4 are implemented.

Images