Visual security image encryption method, device and equipment based on generative adversarial network

Abstract

The invention discloses a visual security image encryption method, device and equipment based on a generative adversarial network. The method comprises the following steps: constructing a model training framework comprising an encoder, a decoder, a discriminator and a differentiable noise layer; acquiring a training data set comprising a plurality of natural images, selecting secret images and carrier images from the training data set, and dynamically distributing the carrier image for each secret image by adopting a random pairing strategy to form a training sample pair; based on the training sample pair and by adopting a preset loss function as an optimization target, training an encoder, a decoder, a discriminator and a differentiable noise layer through a multi-stage progressive training strategy to obtain an encoder model and a decoder model; inputting the to-be-encrypted image and the selected carrier image into an encoder model deployed at a sending end for encryption, and generating a secure secret-carrying image; and inputting the received secure secret-carrying image into a decoder model deployed at a receiving end for decryption to obtain a decrypted image consistent with the to-be-encrypted image.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a visual security image encryption method, apparatus, and device based on generative adversarial networks. Background Technology

[0002] With the rapid development of the internet and multimedia technologies, images have become one of the main carriers of information transmission and storage, leading to increasingly prominent issues related to image data security. Traditional image encryption methods (such as chaotic encryption, AES, and DNA encoding) theoretically offer high security by converting the original image into a noise-like ciphertext image. However, these ciphertext images exhibit obvious random noise characteristics visually, making them easily identified as "encrypted data" by attackers during transmission and storage, thus vulnerable to targeted interception or attacks, failing to achieve truly covert communication. To overcome this problem, image steganography is applied to image encryption, embedding secret information into the carrier image to generate a visually natural ciphertext image. However, existing image steganography methods generally suffer from limited embedding capacity, making it difficult to achieve full-image encryption of the same size as the original image; simultaneously, they are weakly resistant to channel attacks in real-world environments such as JPEG compression, Gaussian noise, and cropping / scaling, allowing attackers to easily destroy hidden information by slightly manipulating the ciphertext image. In recent years, image encryption based on generative adversarial networks has been proposed, realizing an end-to-end deep learning-driven encryption framework. However, problems still exist, such as insufficient optimization of ciphertext visual features and inadequate verification of resistance to attacks in real-world environments. In summary, existing methods face irreconcilable contradictions among the three core objectives of visual naturalness, high decryption quality, and strong attack resistance, failing to achieve coordinated optimization and global optimum. Summary of the Invention

[0003] In view of this, the purpose of this invention is to propose a visual security image encryption method, apparatus and device based on generative adversarial networks, which aims to solve the problem that existing image encryption methods are unable to simultaneously achieve visual concealment, high decryption quality and strong anti-attack capability.

[0004] To achieve the above objectives, the present invention provides a visually secure image encryption method based on generative adversarial networks, the method comprising: Construct a model training framework that includes an encoder, decoder, discriminator, and differentiable noise layer; A training dataset consisting of multiple natural images is obtained. Secret images and carrier images are selected from the training dataset, and a random pairing strategy is used to dynamically assign a carrier image to each secret image to form a training sample pair. Based on the training sample pairs and using a preset loss function as the optimization objective, the encoder, decoder, discriminator, and differentiable noise layer are trained through a multi-stage progressive training strategy to obtain the encoder model and decoder model. The image to be encrypted and the selected carrier image are input into the encoder model deployed at the sending end for encryption to generate a secure encrypted carrier image; The received secure encrypted image is input into the decoder model deployed at the receiving end for decryption, resulting in a decrypted image that is identical to the image to be encrypted.

[0005] To achieve the above objectives, the present invention also provides a visual security image encryption device based on generative adversarial networks, the device comprising: Framework building unit, used to build a model training framework including encoder, decoder, discriminator and differentiable noise layer; The sample pairing unit acquires a training dataset including multiple natural images, selects secret images and carrier images from the training dataset, and dynamically assigns a carrier image to each secret image using a random pairing strategy to form a training sample pair. The model training unit is used to train the encoder, decoder, discriminator and differentiable noise layer based on the training sample pairs and using a preset loss function as the optimization objective, through a multi-stage progressive training strategy, to obtain the encoder model and decoder model. An image encryption unit is used to encrypt the image to be encrypted and the selected carrier image by inputting them into the encoder model deployed at the transmitting end, thereby generating a secure carrier image; The image decryption unit is used to input the received secure encrypted image into the decoder model deployed at the receiving end for decryption, so as to obtain a decrypted image that is consistent with the image to be encrypted.

[0006] To achieve the above objectives, the present invention also proposes a visual security image encryption device based on generative adversarial networks, comprising a processor, a memory, and a computer program stored in the memory, wherein the computer program is executed by the processor to implement the steps of a visual security image encryption method based on generative adversarial networks as described in the above embodiments.

[0007] To achieve the above objectives, the present invention also proposes a computer-readable storage medium storing a computer program that is executed by a processor to implement the steps of a visually secure image encryption method based on a generative adversarial network as described in the above embodiments.

[0008] Beneficial effects: The above scheme constructs a model training framework including an encoder, decoder, discriminator, and differentiable noise layer. It then employs a random pairing strategy and a multi-stage progressive training strategy to systematically train this framework, ultimately obtaining encoder and decoder models that can be directly used for encryption and decryption. This achieves an end-to-end image encryption and decryption process, enabling the complete embedding of the image to be encrypted into any selected carrier image to generate a visually natural secure carrier image, achieving true visual covert communication. Furthermore, it can recover the original secret image with high fidelity from potentially distorted secure carrier images. Thus, it simultaneously ensures the visual naturalness of the encrypted image, the high quality of the decrypted image, and strong robustness against attacks in real-world environments, overcoming the bottlenecks of existing technologies in multi-objective optimization.

[0009] By further defining the encoder using a U-Net architecture, the decoder using a mirror-symmetric structure, the discriminator using a PatchGAN architecture, the differentiable noise layer containing multiple differentiable attack sublayers, and the specific implementation of skip connections, the encoder can fully integrate high- and low-level features to generate high-quality secret-carrying images. The decoder can accurately recover secret information, and the discriminator effectively guides the generator to improve image realism through the discrimination of local receptive fields, greatly enhancing visual concealment. The differentiable noise layer ensures end-to-end differentiability of the entire training process, enabling the simulation of adversarial attacks to effectively improve the robustness of the decoder through gradient backpropagation. This improves the model's expressive power and training stability, laying a solid architectural foundation for subsequent progressive training.

[0010] In each training batch, a secret image is randomly selected and a carrier image is independently and randomly assigned, allowing the secret image and carrier image to be the same image, and the images are re-randomized between different rounds. This strategy avoids the model's memorization of specific image pairs, forcing the encoder to learn a general mapping ability to embed any secret image into any carrier image. Simultaneously, the role-swapping mechanism enhances the model's ability to represent image features universally. Furthermore, allowing the same image as both the secret and carrier image enables the model to learn self-embedding and self-extraction capabilities, further improving the generalization and robustness of the training. The limited random pairing strategy significantly reduces the risk of overfitting, enhances the model's ability to handle unseen image pairs in practical applications, and improves practicality and reliability.

[0011] By clearly dividing the training process into four stages—basic embedding, adversarial training, robust training, and joint fine-tuning—this phased progressive training decomposes the complex multi-objective optimization problem into ordered, focused sub-tasks. This allows the network to robustly master information embedding, visual realism, and anti-interference recovery capabilities step by step, ultimately achieving an optimal balance in the joint fine-tuning stage. This avoids the pattern collapse or optimization difficulties that easily occur when directly performing complex joint training. In other words, this progressive training strategy systematically resolves the contradiction between visual naturalness, high decryption quality, and strong anti-attack capability, avoiding the training instability or local optima problems that may result from direct multi-objective joint optimization. Meanwhile, the composite loss function is key to driving the training at each stage. Reconstruction loss and carrier fidelity loss ensure decryption accuracy and visual similarity, adversarial loss drives visual naturalness, and perceptual loss improves the visual quality of the decrypted image at a high-level semantic level. The synergistic effect of these losses constrains the model from multiple dimensions, including pixel-level, feature-level, and adversarial aspects, enabling the finally trained encoder and decoder to achieve comprehensive performance of high concealment, high fidelity, and high robustness in practical applications.

[0012] On the encryption end, plug-and-play full-image encryption is implemented, which can quickly generate highly concealed encrypted images without complex manual parameter adjustments. On the decryption end, even if the image is subjected to common channel attacks during image transmission, it can effectively resist image quality degradation caused by actual network transmission, storage compression, malicious tampering, etc., and ensure that high-quality original secret information can still be reliably recovered under various complex conditions. This greatly improves the stability and reliability of the entire encrypted communication scenario and can be widely used in public channel environments with security risks, such as social networks, cloud storage, and medical image transmission. Attached Figure Description

[0013] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0014] Figure 1 This is a flowchart illustrating a visual security image encryption method based on generative adversarial networks, as provided in an embodiment of the present invention.

[0015] Figure 2 This is a schematic diagram of the overall process of a visual security image encryption method provided in an embodiment of the present invention.

[0016] Figure 3 This is a schematic diagram of the network architecture of an encoder provided in an embodiment of the present invention.

[0017] Figure 4 This is a schematic diagram of the network architecture of a decoder provided in an embodiment of the present invention.

[0018] Figure 5 This is a schematic diagram of the network architecture of a discriminator provided in an embodiment of the present invention.

[0019] Figure 6 This is a schematic diagram of the network architecture of a differentiable noise layer provided in an embodiment of the present invention.

[0020] Figure 7 This is a schematic diagram of a visual security image encryption device based on generative adversarial networks, provided in an embodiment of the present invention.

[0021] The realization of the invention's objective, its functional characteristics, and advantages will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to represent selected embodiments of the invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] The present invention will be described in detail below with reference to the embodiments.

[0024] Reference Figure 1 The diagram shows a flowchart of a visual security image encryption method based on generative adversarial networks according to an embodiment of the present invention. Specifically, the method includes: S11 constructs a model training framework that includes an encoder, decoder, discriminator, and a differentiable noise layer.

[0025] Furthermore, in step S11, the construction of the model training framework, which includes an encoder, a decoder, a discriminator, and a differentiable noise layer, includes: S11-1, The encoder adopts a U-Net architecture, including a downsampling path and an upsampling path. The feature maps of each level of the downsampling path are passed to the corresponding level of the upsampling path through skip connections and then spliced ​​and fused along the channel dimension. S11-2, The decoder forms a mirror-symmetric structure with the encoder, and receives the feature map of the corresponding layer of the encoder through a skip connection; S11-3, the discriminator adopts the PatchGAN architecture. After the input image is processed by multiple convolutions and instance normalization, it outputs a local region probability map. Each local region probability map corresponds to the authenticity score of a local receptive field in the input image. S11-4, the differentiable noise layer includes at least one of a JPEG compression simulation layer, a Gaussian noise layer, a cropping and scaling layer, and a combined attack layer. The JPEG compression simulation layer uses a differentiable JPEG approximation algorithm. The noise variance in the Gaussian noise layer is adaptively adjusted during training as a trainable parameter. The cropping and scaling layer scales randomly cropped image regions back to their original size using a differentiable interpolation algorithm. The combined attack layer is used to randomly combine multiple attack methods.

[0026] Furthermore, in step S11-1, the step of transferring the feature maps of each level of the downsampling path to the corresponding level of the upsampling path through skip connections and performing channel-dimensional splicing and fusion includes: S11-1-1, embed residual connection blocks in the downsampling path, and in each level of the downsampling path, retain a copy of the feature map after processing by the residual connection block as a feature map copy, and pass it to the corresponding level in the upsampling path through cross-layer connection; S11-1-2, in the upsampling path, when each layer restores the image resolution through transposed convolution or interpolation, the input feature map from the previous layer and the feature map copy of the corresponding downsampling layer passed through the cross-layer connection are concatenated and fused in the channel dimension, and the concatenated feature map is input into the convolutional layer of the corresponding layer for processing.

[0027] The encoder's input is a multi-channel tensor formed by concatenating the secret image and the carrier image along the channel dimension; the encoder's output layer uses the Tanh activation function to map the output values ​​to the range [-1, 1].

[0028] Reference Figure 2 As shown in the figure. This embodiment specifically constructs a generative adversarial network (GAN) including an encoder, decoder, discriminator, and differentiable noise layer as a model training framework. Wherein, as... Figure 3As shown, the encoder adopts a U-Net architecture. Its input is a six-channel input tensor formed by concatenating the secret image and the carrier image in the channel dimension, so as to simultaneously process the secret information to be embedded and the carrier image that provides visual reference. The encoder includes a downsampling path and an upsampling path. The downsampling path extracts multi-scale features of the image step by step through multiple convolutions. The typical configuration is that the number of output channels increases from 64 to 128, 256, and 512 layer by layer. Each convolutional layer uses a 3×3 convolutional kernel, a stride of 1, and padding of 1 to keep the feature map size unchanged. Residual connection blocks are embedded in the downsampling path. Typically, 4 to 8 residual blocks are stacked. Each residual block contains two convolutional layers and skip connections. At each level, a copy of the feature map processed by the residual block is kept and passed to the corresponding level in the upsampling path through cross-layer connections. The upsampling path progressively restores image resolution through transposed convolution or interpolation operations. When restoring size at each layer, it first receives the input feature map from the previous layer. Then, this input is concatenated and fused with a copy of the feature map from the corresponding downsampling layer passed through cross-layer connections along the channel dimension. The concatenated feature map is then input into the convolutional layer of that layer for processing, thereby fusing high-level semantic information with low-level detail information. The encoder's output layer uses the Tanh activation function, whose mathematical expression is: The output values ​​are mapped to the range [-1,1] to generate a three-channel image.

[0029] Reference Figure 4 As shown, the decoder and encoder form a mirror-symmetric structure. The input layer receives a three-channel encrypted image, with the format completely consistent with the encrypted image output by the encoder. Internally, the decoder extracts features through multiple stacked convolutional layers, each using a 3×3 convolutional kernel. It progressively extracts the hidden information embedded in the image in a way imperceptible to the human eye, transforming shallow pixel features into high-level semantic representations through layer-by-layer abstraction. Subsequently, the upsampling layer uses transposed convolution or interpolation operations (such as bilinear interpolation) to progressively enlarge the feature map, restoring it to the original image size. Transposed convolution uses learnable parameters to achieve upsampling for better image detail reconstruction, while interpolation is more computationally efficient and can be chosen according to actual needs. The decoder receives feature maps from the corresponding layers of the encoder through skip connections and performs channel-level concatenation and fusion, enabling the decoding process to simultaneously utilize both low-level detail information (such as texture and edges) and high-level semantic information extracted by the encoder, thus more accurately restoring the original image. The decoder's output layer also uses the Tanh activation function, mapping feature values ​​to […]. Within the range of 1,1], a three-channel decrypted image is generated, whose pixel value distribution is completely consistent with the normalized original secret image. The entire decoder is trained end-to-end to learn how to accurately extract the embedded secret information from the secret image. Even after attacks such as JPEG compression, Gaussian noise, and cropping / scaling simulated by a differentiable noise layer, it can still recover the original secret image with high quality, making the pixel values ​​of the decrypted image highly consistent with the original secret image, with minimal loss of visual quality.

[0030] Reference Figure 5 As shown, the discriminator adopts the PatchGAN architecture. The input image is either a real natural image or a densely encoded image generated by the encoder, with a fixed size of 256×256 pixels. The convolutional layers use 4×4 kernels with a stride of 2, and the number of channels increases progressively from 64 to 128, 256, and 512, enabling the network to capture multi-level information from local texture to global structure. Each convolutional layer is followed by instance normalization (InstanceNorm), whose mathematical expression is: Where x is the input feature map, μ and σ 2 These are the mean and variance for each channel of each sample, respectively. It is a small constant (e.g., 10). 8 To prevent division by zero, the output y is multiplied by a learnable scaling parameter γ and then added to the offset parameter β. The activation function used is LeakyReLU, expressed as f(x) = x when x ≥ 0, and f(x) = αx when x < 0, where α = 0.2. After multiple convolutions, the output layer generates a 30×30 probability map. Each value corresponds to a realism score for a local receptive field (approximately 70×70 pixels) in the input image. Values ​​closer to 1 indicate that the region is more likely to be a real, natural image, while values ​​closer to 0 indicate that it is more likely to be a generated, dense image. This local discrimination mechanism forces the encoder not only to deceive the discriminator as a whole but also to achieve visual realism in each local region, thereby generating detailed, dense images without local anomalies.

[0031] Reference Figure 6As shown, the differentiable noise layer is located between the encoder and decoder to simulate various real-world environmental attacks that may be encountered during image transmission and storage, thereby improving the robustness of the decoder. It includes at least one of the following: a JPEG compression simulation layer, a Gaussian noise layer, a cropping and scaling layer, and a combined attack layer. The JPEG compression simulation layer employs a differentiable JPEG approximation algorithm to convert the image from RGB to YCbCr color space, performs block-wise discrete cosine transform (DCT) on the luminance and chrominance components, simulates lossy compression through differentiable quantization operations, recovers the image through inverse quantization and inverse discrete cosine transform (IDCT), and finally converts it back to the RGB color space. The entire process supports gradient backpropagation. The Gaussian noise layer adds Gaussian noise with learnable variance to the image. The noise variance is adaptively adjusted during training as a trainable parameter. The noise is added by using Gaussian noise with a mean of 0 and a variance of σ. 2 Gaussian distributed random noise is added to the image pixels, and σ can be dynamically changed at different training stages to simulate different levels of channel interference. The cropping and scaling layer performs random cropping on the image. First, a starting coordinate and cropping size are randomly selected on the image to crop a local region of the image. Then, a differentiable interpolation algorithm (such as bilinear interpolation) is used to scale the cropped region back to the original size to simulate the situation where the receiver still needs to recover the complete information after the image is partially truncated or its size is changed during transmission. The decoder can recover the complete secret image from distorted images with partially lost information. Its core lies in the encoder's use of a distributed redundant embedding strategy, spreading the secret image's information throughout the entire image region, ensuring that each local region contains partial clues about the whole secret image. Simultaneously, the decoder extracts global contextual information from the preserved regions through convolutional layers with progressively larger receptive fields. It uses natural image statistical priors learned during training (such as texture repeatability, structural symmetry, and color coherence) to reasonably infer the content of the lost regions. Multi-scale feature fusion allows shallow detail recovery and deep semantic guidance to mutually verify and complement each other. Finally, under the local realism constraint of the discriminator, the recovered image is ensured to be visually natural and realistic. The combined attack layer randomly combines various attack methods, such as JPEG compression followed by Gaussian noise, or cropping and scaling followed by JPEG compression, or even simultaneously superimposing multiple attacks, enabling the decoder to accurately extract secret information under complex and varied distortion conditions. Through this end-to-end differentiable noise layer design, the decoder gradually learns invariant characteristics to various real-world attacks, significantly improving the system's robustness in practical applications. Through the above-mentioned architectural design, the entire model training framework enables the encoder to embed the secret image into the carrier image in a way that is imperceptible to the human eye and generate a visually natural secret image. The decoder can accurately recover the original secret image from the distorted secret image that may have undergone various attacks in real environments. The discriminator forces the encoder to generate a more realistic secret image through local region discrimination.

[0032] S12, obtain a training dataset including multiple natural images, select secret images and carrier images from the training dataset, and use a random pairing strategy to dynamically assign a carrier image to each secret image to form a training sample pair.

[0033] Furthermore, in step S12, the step of dynamically assigning carrier images to each secret image using a random pairing strategy to form training sample pairs includes: In each training batch, K images are randomly selected from the training dataset as secret images, and a carrier image is independently and randomly selected from the training dataset for each secret image to obtain K training sample pairs; wherein the secret image and the corresponding carrier image are different images or the same image, and the sample pairs are re-randomly paired between different training rounds.

[0034] The acquisition of the training dataset, which includes multiple natural images, includes: The natural images in the training dataset are divided into a secret image set, a carrier image set, and a real image set. The secret image set is used to provide secret images, the carrier image set is used to provide carrier images, and the real image set is used to train a discriminator to distinguish between real images and generated images.

[0035] In this embodiment, a large-scale, diverse natural image dataset is collected. Public datasets such as ImageNet and COCO can be used to ensure that the dataset contains at least 100,000 images, covering various categories such as people, landscapes, buildings, animals, and objects, to guarantee the breadth and representativeness of the data distribution. All images undergo uniform preprocessing: bilinear interpolation or a more advanced downsampling algorithm is used to adjust them to the same size, typically 256×256 pixels; then the pixel values ​​are normalized to the [-1, 1] interval, using the normalization formula: , where x original The original pixel values ​​(range 0~255) are used for this transformation. This transformation ensures that the data distribution perfectly matches the output range of the Tanh activation function in the encoder output layer, which is beneficial to the stability and convergence speed of network training. The preprocessed image dataset is divided into three subsets: a secret image set, used to simulate images requiring encryption protection; a carrier image set, serving as a visual camouflage template, whose pixel values ​​do not carry secret information but only provide visual appearance reference; and a real image set, specifically used for training the discriminator network to distinguish between real and generated images.

[0036] During training, a random pairing strategy is used to dynamically assign carrier images to each secret image to form training sample pairs. Specifically, in each training batch, K images are randomly selected from the entire dataset as the secret image batch S={s1,s2,…,s…} K}, and simultaneously for each secret image s i A carrier image c is randomly selected independently from the dataset. i Allow s i With c i For the same image, form K training pairs {(s1,c1),(s2,c2),…,(s... K ,c K In the next training round, images are randomly reselected to ensure that each image pair combination is different. Since the extraction of secret images and carrier images are independent, the same image may appear in multiple roles during training: in one batch, it may be paired as a secret image with another carrier image; in another batch, it may be paired as a carrier image with other secret images; and in some batches, it may simultaneously act as its own secret and carrier (i.e., self-pairing). This role-swapping mechanism requires the model to simultaneously learn to retain sufficient information for the decoder to recover when acting as a secret image, and to provide visual appearance reference without carrying secret information when acting as a carrier image. The two tasks share underlying feature representations but have different optimization objectives, thus forcing the model to learn more fundamental feature representations rather than memorizing specific combinations. Mathematically, the optimization objective of the model under fixed pairings is... The model may degenerate into memorizing n specific pairs; while the expected loss under random pairing is The model learns a general mapping function that minimizes the expected loss. The optimal solution does not depend on specific image pairs. From an information theory perspective, the entropy H of a fixed-pair dataset... fixed =0, while the entropy of the randomly paired dataset is... (N is the dataset size), higher entropy forces the model to learn more fundamental feature representations. Through the above random pairing strategy, the mapping function f(s,c) learned by the encoder has symmetry and interchangeability with respect to the secret image and the carrier image. Even when encountering combinations of secret images and carrier images that never appeared simultaneously during training, the model can still maintain good embedding and extraction performance during the testing phase, and the generalization error is significantly reduced. In addition, all images are loaded into the network according to a preset batch size (such as 16 or 32) to ensure efficient and consistent data loading.

[0037] S13, based on the training sample pairs and using a preset loss function as the optimization objective, the encoder, decoder, discriminator and differentiable noise layer are trained through a multi-stage progressive training strategy until the model converges, thus obtaining the encoder model and decoder model.

[0038] Furthermore, in step S13, the training of the encoder, decoder, discriminator, and differentiable noise layer using a multi-stage progressive training strategy includes the following stages executed sequentially: S13-1, In the first training phase, the network parameters of the discriminator are fixed, and the encoder and decoder are trained by optimizing the reconstruction loss and the carrier fidelity loss; S13-2, in the second training phase, a discriminator is introduced, and the encoder, decoder and discriminator are alternately trained through joint optimization of adversarial loss, reconstruction loss and carrier fidelity loss. S13-3, in the third training phase, a differentiable noise layer is inserted between the encoder and decoder, and the encoder and decoder are trained by optimizing the reconstruction loss under the condition that the differentiable noise layer applies distortion processing to the dense image. In S13-4, the fourth training phase, the encoder, decoder, discriminator, and differentiable noise layer are jointly trained end-to-end by jointly optimizing adversarial loss, reconstruction loss, perception loss, and carrier fidelity loss.

[0039] The encoder is trained to embed information of the secret image into the entire region of the secret image in a distributed and redundant manner; the decoder is trained to recover the complete secret image from the region retained after cropping and scaling by extracting global context information within the retained region.

[0040] In the third training phase, when the differentiable noise layer applies distortion processing to the densely packed image, a progressively increasing difficulty strategy is adopted, including: Training begins with simulating a single, mild attack type, and as training progresses, the attack intensity and complexity of the distortion processing are gradually increased. The attack types include one or more combinations of JPEG compression simulation, Gaussian noise, cropping and scaling, and Gaussian blur.

[0041] Furthermore, the reconstruction loss is used to calculate the pixel differences between the decrypted image and the secret image; The carrier fidelity loss is used to calculate the pixel difference between the carrier image and the carrier image; wherein, the reconstruction loss and the carrier fidelity loss are calculated using L1 or L2 loss functions; The adversarial loss is calculated based on the discriminator's output and is used to drive the encoder to generate visually realistic encrypted images; The perceptual loss is used to improve the perceptual quality of the decrypted image. The perceptual loss is obtained by calculating the difference between the feature maps of the decrypted image and the secret image in the feature space of the pre-trained VGG network.

[0042] In this embodiment, based on the constructed training sample pairs, and with a preset composite loss function as the optimization objective, a four-stage progressive training strategy is used to jointly train the encoder, decoder, discriminator, and differentiable noise layer until the model converges, ultimately obtaining the trained encoder and decoder models. Specific training parameters are configured as follows: batch size is set to 16 or 32 (dynamically adjusted according to GPU memory), initial learning rate is set to 0.0002, the optimizer uses the Adam algorithm, and momentum parameters β1=0.5 and β2=0.999. The composite loss function consists of adversarial loss L... adv Reconstruction loss L recon Perceived loss L percep and carrier fidelity loss L fidelity The weighted composition has a total loss function of L. total =λ adv L adv +λ recon L recon +λ percep L percep +λ fidelity L fidelity Each weight is set as λ. adv =1、λ recon =10、λ percep =5、λ fidelity =5. Joint optimization during training enables the encoder, decoder, and discriminator to co-evolve, ultimately achieving the optimal balance between visual naturalness, decryption accuracy, and attack resistance. Among these, the reconstruction loss L... recon Used to calculate the pixel difference between the decrypted image and the original secret image, carrier fidelity loss L fidelity This is used to calculate the pixel difference between the encrypted image and the original carrier image, both employing either L1 or L2 loss functions; adversarial loss L... adv The output probability calculation based on the discriminator is used to drive the encoder to generate visually realistic dense images; perceptual loss L percep The feature map difference between the decrypted image and the secret image is calculated in the feature space of the pre-trained VGG network, which is used to improve the perceptual quality of the decrypted image.

[0043] The four-stage progressive training will be performed sequentially: First training phase (basic embedding training): Discriminator network parameters are fixed; only reconstruction loss and carrier fidelity loss are optimized, i.e., L... stage1 =λ recon L recon +λ fidelity L fidelityThe encoder and decoder are trained for 50 epochs. This phase allows the encoder to initially learn to embed secret images into the carrier image in an invisible manner, while the decoder simultaneously learns to extract secret information from the carrier image. After each training epoch, model checkpoints are saved to provide a reliable initial model for subsequent adversarial training.

[0044] The second training phase (adversarial training) introduces a discriminator that engages in alternating adversarial training against the encoder and decoder. In each training iteration, the discriminator parameters are first updated to better distinguish between real natural images and encoder-generated spoofed images; then, the generator (encoder) parameters are updated to ensure that the generated spoofed images can deceive the discriminator, while simultaneously maintaining reconstruction quality in conjunction with the decoder. The optimization objective is L... stage2 =λ adv L adv +λ recon L recon +λ fidelity L fidelity The training lasted for 100 epochs. Through adversarial training, the visual quality of the densely packed images was significantly improved, and the distribution difference between them and real natural images continued to narrow.

[0045] The third training phase (robust training) involves inserting a differentiable noise layer between the encoder and decoder to simulate attacks in a real-world environment. A progressively increasing difficulty strategy is employed: training begins with a mild, single attack type (such as mild Gaussian noise), gradually increasing the attack intensity and complexity as training progresses. Attack types include one or more combinations of JPEG compression simulation, Gaussian noise, cropping / scaling, and Gaussian blur. In each training batch, the system randomly selects the attack type and its parameters (e.g., JPEG compression followed by Gaussian noise, or cropping / scaling followed by JPEG compression, or even multiple attacks simultaneously) to prevent the decoder from overfitting to specific attack patterns. The optimization objective is to maintain decoding accuracy under noisy conditions. The loss function remains primarily reconstruction loss, while adversarial loss and carrier fidelity loss can be combined to maintain the visual quality of the carrier image. This phase lasts for 100 epochs. Through this phase, the decoder gradually learns invariant characteristics to various real-world attacks, significantly improving the system's robustness in practical applications.

[0046] The fourth training phase (joint fine-tuning) involves end-to-end joint training of the encoder, decoder, discriminator, and differentiable noise layer to achieve global optimization. The complete composite loss function L is used. total =λ adv L adv +λ recon L recon +λ percep L percep +λ fidelity Lfidelity The model simultaneously optimizes four objectives: visual naturalness, decryption accuracy, perceptual quality, and carrier fidelity. A progressively decreasing learning rate strategy is employed, multiplying by 0.5 every 20 epochs. The training epochs are set to 50 epochs. After each epoch, the model performance is evaluated using a validation set. Key metrics include the PSNR and SSIM between the decrypted image and the original secret image, the discriminator score between the secret image and the real natural image, and the decryption quality after attack simulation. After training, the model with the best overall performance on the validation set is selected as the final deployment version.

[0047] Through the aforementioned multi-stage progressive training, the encoder is trained to embed information from the secret image into the entire region of the carrier image in a distributed and redundant manner, ensuring that each local region contains partial clues about the entire secret image. The decoder is trained to recover the complete secret image from the regions retained after cropping and scaling of the carrier image by extracting global contextual information within the retained regions (utilizing convolutional layers with progressively larger receptive fields and natural image statistical priors). The entire training process systematically resolves the contradiction between visual naturalness, high decryption quality, and strong anti-attack capability. The resulting encoder and decoder models can simultaneously meet the core requirements of high-quality image encryption, visually covert transmission, and robust anti-interference in practical applications.

[0048] S14, the image to be encrypted and the selected carrier image are input into the encoder model deployed at the sending end for encryption to generate a secure carrier image.

[0049] Furthermore, in step S14, encrypting the image to be encrypted and the selected carrier image into the encoder model deployed at the sending end to generate a secure carrier image includes: The encoder model automatically embeds the complete secret information of the image to be encrypted into the selected carrier image, generating a secure encrypted image that is visually highly similar to the selected carrier image.

[0050] In this embodiment, the optimal encoder model, obtained after joint fine-tuning in the fourth training phase, is deployed at the encryption transmitter. This model is capable of embedding a secret image into any carrier image and generating a visually natural encrypted image. During the actual encryption process, the transmitter acquires the secret image to be encrypted (i.e., the image to be encrypted) and any selected carrier image. The two do not need to be related; the carrier image can be any ordinary natural image (such as a landscape photo, a portrait photo, etc.) without special screening or preprocessing. The image to be encrypted and the selected carrier image are simultaneously input into the deployed encoder model for forward inference. The encoder model automatically embeds the complete secret information of the image to be encrypted into the selected carrier image in a way imperceptible to the human eye, generating a secure encrypted image that is visually highly similar to the selected carrier image. The entire process is completed within milliseconds, supporting real-time encryption requirements. The generated secure encrypted image is extremely close to the original carrier image at the pixel level; the human eye cannot distinguish the difference, let alone perceive the hidden secret information, thus achieving true visual security encryption. Finally, the generated secure encrypted image is transmitted to the receiver through a public channel (such as the Internet, cloud storage, social media, or any open network that may have security risks). Because the secure image carrying the secret is presented as an ordinary, natural image, it will not attract the attention or interception of attackers, thus ensuring the covert transmission of confidential information. The embedding process of the encoder model does not require manual design of embedding rules or selection of embedding positions; it is completed automatically by the trained network parameters, achieving convenient plug-and-play deployment.

[0051] S15, the received secure encrypted image is input into the decoder model deployed at the receiving end for decryption to obtain a decrypted image that is consistent with the image to be encrypted.

[0052] Furthermore, in step S15, the step of inputting the received secure encrypted image into the decoder model deployed at the receiving end for decryption to obtain a decrypted image consistent with the image to be encrypted includes: The decoder model extracts the embedded complete secret information from a secure encrypted image that has been distorted by at least one of the attack methods of JPEG compression, Gaussian noise, and cropping / scaling, and then restores the image through upsampling and feature reconstruction to obtain the decrypted image.

[0053] In this embodiment, the optimal decoder model obtained after joint fine-tuning in the fourth training phase is deployed at the receiver. This model, robustly trained with differentiable noise layers, is capable of accurately extracting secret information from various distorted images. The receiver obtains the secure, secret-carrying image transmitted via a public channel. This image may have undergone various real-world attacks during transmission, such as JPEG compression on social media platforms, Gaussian noise interference introduced by network transmission, and cropping or resizing by malicious attackers. These distortions can lead to a decrease in image visual quality. The receiver inputs the potentially distorted secure, secret-carrying image into the deployed decoder model for forward inference. The decoder model automatically extracts the embedded complete secret information from the image implicitly and gradually restores the original image content through upsampling and feature reconstruction. Specifically, the decoder utilizes its mirror-symmetric network structure to extract distributed hidden features from the image through multiple stacked convolutional layers. Then, it uses transposed convolution or interpolation operations to progressively enlarge the feature maps. Simultaneously, it combines the skip connection feature maps from the corresponding layers of the encoder for channel-dimensional stitching and fusion, thereby fusing low-level detail information with high-level semantic information to achieve accurate image restoration. The entire decryption process is completed within milliseconds, supporting real-time decryption requirements. The decrypted image output by the decoder is highly consistent with the original image to be encrypted at the pixel level, with a peak signal-to-noise ratio (PSNR) typically exceeding 36dB and a structural similarity index (SSIM) exceeding 0.98. The human eye can hardly distinguish the decrypted image from the original, ensuring the complete recovery of the secret information. Even after the secure encrypted image has undergone severe attacks such as JPEG compression (e.g., quality factor 50), Gaussian noise (moderate variance), and cropping / scaling (e.g., cropping 25% and then scaling back to the original size), the robustly trained decoder still maintains high-quality decryption results with minimal visual quality loss and complete preservation of key information. This is thanks to the distributed redundant embedding strategy developed by the encoder during training—diffusing the secret information throughout the entire image region—and the decoder's extraction of global contextual information from the preserved regions through convolutional layers with gradually increasing receptive fields, and the reasonable inference of the content of lost or distorted regions using natural image statistical priors.

[0054] Furthermore, the method also includes: encapsulating the trained encoder and decoder models into standardized API services, deploying them in a microservice architecture to support high-concurrency calls and batch processing, and configuring key management, access control, and log monitoring functions.

[0055] In this embodiment, the encoder and decoder models provide services externally through RESTful APIs or gRPC interfaces, supporting high-concurrency calls and horizontal scaling. The number of service instances can be automatically increased or decreased based on actual load. The API service design supports batch encryption and decryption processing, allowing clients to submit multiple pairs of secret images and carrier images at once. The server automatically performs parallel inference processing, significantly improving the throughput and efficiency of large-scale image processing and meeting enterprise-level application requirements. The system provides a complete key management interface, including model version management, access key generation and rotation, and call permission control, ensuring that only authorized parties can call the encryption and decryption services, preventing the model from being illegally abused. Simultaneously, comprehensive log monitoring functions are integrated, recording key information such as the time, request parameters, processing results, and performance time of each API call, and storing and analyzing this information through a centralized log management system. The monitoring system provides real-time statistics on core indicators such as service call volume, response latency, success rate, and error rate, and sets alarm thresholds to promptly notify operations personnel when service anomalies or performance degradation occur. Log data can also be used for security auditing, tracing detailed information of each encryption and decryption operation to meet compliance requirements. The entire system supports containerized deployment and can run in private, public, or hybrid cloud environments. It features production-grade characteristics such as automatic scaling, load balancing, and failover, providing reliable, secure, and high-performance image encryption and decryption services for clients such as financial institutions, medical institutions, and government agencies.

[0056] Reference Figure 7 The diagram shown is a structural schematic of a visual security image encryption device based on generative adversarial networks according to an embodiment of the present invention.

[0057] In this embodiment, the device 20 includes: Framework building unit 21 is used to build a model training framework including an encoder, decoder, discriminator and differentiable noise layer; The sample pairing unit 22 acquires a training dataset including multiple natural images, selects secret images and carrier images from the training dataset, and dynamically assigns a carrier image to each secret image using a random pairing strategy to form a training sample pair. The model training unit 23 is used to train the encoder, decoder, discriminator and differentiable noise layer based on the training sample pairs and using a preset loss function as the optimization objective, through a multi-stage progressive training strategy, to obtain the encoder model and decoder model. Image encryption unit 24 is used to encrypt the image to be encrypted and the selected carrier image by inputting them into the encoder model deployed at the transmitting end to generate a secure carrier image; The image decryption unit 25 is used to input the received secure encrypted image into the decoder model deployed at the receiving end for decryption, so as to obtain a decrypted image that is consistent with the image to be encrypted.

[0058] Each unit module of the device 20 can execute the corresponding steps in the above method embodiment, so the details of each unit module will not be elaborated here. Please refer to the description of the corresponding steps above for details.

[0059] This invention also provides a visual security image encryption device based on generative adversarial networks (GANs). This device includes the visual security image encryption apparatus based on GANs described above, wherein the visual security image encryption apparatus based on GANs can employ... Figure 7 The structure of the embodiment, correspondingly, can be executed Figure 1 The technical solutions of the method embodiments shown are similar in implementation principle and technical effect. For details, please refer to the relevant records in the above embodiments, which will not be repeated here.

[0060] The device includes: a mobile phone, digital camera, or tablet computer, or other device with a camera function; or a device with an image processing function; or a device with an image display function. The device may include components such as a memory, processor, input unit, display unit, and power supply.

[0061] The memory can be used to store software programs and modules. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory. The memory can mainly include a program storage area and a data storage area. The program storage area can store the operating system, applications required for at least one function, etc.; the data storage area can store data created according to the use of the device, etc. In addition, the memory can include high-speed random access memory, and can also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory can also include a memory controller to provide access to the memory for the processor and input units.

[0062] The input unit can be used to receive input numerical, character, or image information, and to generate keyboard, mouse, joystick, optical, or trackball signal inputs related to user settings and function control. Specifically, in addition to a camera, the input unit of this embodiment may also include a touch-sensitive surface (e.g., a touch screen) and other input devices.

[0063] The display unit can be used to display information input by the user or information provided to the user, as well as various graphical user interfaces of the device. These graphical user interfaces can be composed of graphics, text, icons, video, and any combination thereof. The display unit may include a display panel, optionally configured as an LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode), or other similar display panel. Furthermore, a touch-sensitive surface may cover the display panel. When the touch-sensitive surface detects a touch operation on or near it, it transmits the information to the processor to determine the type of touch event. Subsequently, the processor provides corresponding visual output on the display panel based on the type of touch event.

[0064] This invention also provides a computer-readable storage medium, which may be a computer-readable storage medium included in the memory described in the above embodiments; or it may be a standalone computer-readable storage medium not assembled into a device. The computer-readable storage medium stores at least one instruction, which is loaded and executed by a processor to implement... Figure 1 The illustrated method is a visually secure image encryption method based on generative adversarial networks. The computer-readable storage medium can be a read-only memory, a hard disk, or an optical disk, etc.

[0065] This invention also provides a computer program product, including a computer program / instructions, which are loaded and executed by a processor to implement... Figure 1 This paper presents a visual security image encryption method based on generative adversarial networks.

[0066] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the device embodiments, equipment embodiments, and storage medium embodiments, since they are basically similar to the method embodiments, the descriptions are relatively simple, and relevant parts can be referred to the descriptions in the method embodiments.

[0067] Furthermore, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0068] The foregoing description illustrates and describes preferred embodiments of the present invention. It should be understood that the present invention is not limited to the forms disclosed herein and should not be construed as excluding other embodiments. It can be used in various other combinations, modifications, and environments, and can be altered within the scope of the inventive concept by means of the foregoing teachings or techniques or knowledge in related fields. Any modifications and variations made by those skilled in the art that do not depart from the spirit and scope of the present invention should be within the protection scope of the appended claims.

Claims

1. A visually secure image encryption method based on generative adversarial networks, characterized in that, The method includes: Construct a model training framework that includes an encoder, decoder, discriminator, and differentiable noise layer; A training dataset consisting of multiple natural images is obtained. Secret images and carrier images are selected from the training dataset, and a random pairing strategy is used to dynamically assign a carrier image to each secret image to form a training sample pair. Based on the training sample pairs and using a preset loss function as the optimization objective, the encoder, decoder, discriminator, and differentiable noise layer are trained through a multi-stage progressive training strategy to obtain the encoder model and decoder model; wherein, the multi-stage progressive training strategy for the encoder, decoder, discriminator, and differentiable noise layer includes the following stages executed sequentially: In the first training phase, the network parameters of the discriminator are fixed, and the encoder and decoder are trained by optimizing the reconstruction loss and the carrier fidelity loss. In the second training phase, a discriminator is introduced, and the encoder, decoder and discriminator are alternately trained by jointly optimizing the adversarial loss, reconstruction loss and carrier fidelity loss. In the third training phase, a differentiable noise layer is inserted between the encoder and the decoder. Under the condition that the differentiable noise layer applies distortion processing to the dense image, the encoder and decoder are trained by optimizing the reconstruction loss. In the fourth training phase, the encoder, decoder, discriminator, and differentiable noise layer are jointly trained end-to-end by jointly optimizing the adversarial loss, reconstruction loss, perception loss, and carrier fidelity loss. The image to be encrypted and the selected carrier image are input into the encoder model deployed at the sending end for encryption to generate a secure encrypted carrier image; The received secure encrypted image is input into the decoder model deployed at the receiving end for decryption, resulting in a decrypted image that is identical to the image to be encrypted.

2. The visual security image encryption method based on generative adversarial networks according to claim 1, characterized in that, The constructed model training framework includes an encoder, decoder, discriminator, and differentiable noise layer, comprising: The encoder adopts a U-Net architecture, which includes a downsampling path and an upsampling path. The feature maps of each level of the downsampling path are passed to the corresponding level of the upsampling path through skip connections and then spliced ​​and fused along the channel dimension. The decoder forms a mirror-symmetric structure with the encoder and receives the feature map of the corresponding layer of the encoder through a skip connection; The discriminator adopts the PatchGAN architecture. After the input image is processed by multiple convolutions and instance normalization, it outputs a local region probability map. Each local region probability map corresponds to the authenticity score of a local receptive field in the input image. The differentiable noise layer includes at least one of a JPEG compression simulation layer, a Gaussian noise layer, a cropping and scaling layer, and a combined attack layer. The JPEG compression simulation layer uses a differentiable JPEG approximation algorithm. The noise variance in the Gaussian noise layer is adaptively adjusted during training as a trainable parameter. The cropping and scaling layer scales randomly cropped image regions back to their original size using a differentiable interpolation algorithm. The combined attack layer is used to randomly combine multiple attack methods.

3. The visual security image encryption method based on generative adversarial networks according to claim 2, characterized in that, The step of passing feature maps from each level of the downsampling path to the corresponding level of the upsampling path via skip connections and then performing channel-dimensional concatenation and fusion includes: Residual join blocks are embedded in the downsampling path, and in each level of the downsampling path, a copy of the feature map processed by the residual join block is retained as a feature map copy and passed to the corresponding level in the upsampling path through cross-layer connections. In the upsampling path, when each layer restores the image resolution through transposed convolution or interpolation, the input feature map from the previous layer and the feature map copy of the corresponding downsampling layer passed through the cross-layer connection are concatenated and fused in the channel dimension, and the concatenated feature map is input into the convolutional layer of the corresponding layer for processing.

4. The visual security image encryption method based on generative adversarial networks according to claim 1, characterized in that, The method of dynamically assigning carrier images to each secret image using a random pairing strategy to form training sample pairs includes: In each training batch, K images are randomly selected from the training dataset as secret images, and a carrier image is independently and randomly selected from the training dataset for each secret image to obtain K training sample pairs; wherein the secret image and the corresponding carrier image are different images or the same image, and the sample pairs are re-randomly paired between different training rounds.

5. A visually secure image encryption method based on generative adversarial networks according to claim 1, characterized in that, The reconstruction loss is used to calculate the pixel differences between the decrypted image and the secret image; The carrier fidelity loss is used to calculate the pixel difference between the carrier image and the carrier image; wherein, the reconstruction loss and the carrier fidelity loss are calculated using L1 or L2 loss functions; The adversarial loss is calculated based on the discriminator's output and is used to drive the encoder to generate visually realistic encrypted images; The perceptual loss is used to improve the perceptual quality of the decrypted image. The perceptual loss is obtained by calculating the difference between the feature maps of the decrypted image and the secret image in the feature space of the pre-trained VGG network.

6. The visual security image encryption method based on generative adversarial networks according to claim 1, characterized in that, The step of encrypting the image to be encrypted and the selected carrier image by inputting them into the encoder model deployed at the transmitting end to generate a secure encrypted carrier image includes: The encoder model automatically embeds the complete secret information of the image to be encrypted into the selected carrier image, generating a secure encrypted image that is visually highly similar to the selected carrier image.

7. The visual security image encryption method based on generative adversarial networks according to claim 1, characterized in that, The step of inputting the received secure encrypted image into the decoder model deployed at the receiving end for decryption to obtain a decrypted image consistent with the image to be encrypted includes: The decoder model extracts the embedded complete secret information from a secure encrypted image that has been distorted by at least one of the attack methods, such as JPEG compression, Gaussian noise, and cropping / scaling, and then restores the image through upsampling and feature reconstruction to obtain the decrypted image.

8. A visually secure image encryption device based on generative adversarial networks, characterized in that, The apparatus for using a visually secure image encryption method based on generative adversarial networks according to any one of claims 1-7 includes: Framework building unit, used to build a model training framework including encoder, decoder, discriminator and differentiable noise layer; The sample pairing unit acquires a training dataset including multiple natural images, selects secret images and carrier images from the training dataset, and dynamically assigns a carrier image to each secret image using a random pairing strategy to form a training sample pair. The model training unit is used to train the encoder, decoder, discriminator and differentiable noise layer based on the training sample pairs and using a preset loss function as the optimization objective, through a multi-stage progressive training strategy, to obtain the encoder model and decoder model. An image encryption unit is used to encrypt the image to be encrypted and the selected carrier image by inputting them into the encoder model deployed at the transmitting end, thereby generating a secure carrier image; The image decryption unit is used to input the received secure encrypted image into the decoder model deployed at the receiving end for decryption, so as to obtain a decrypted image that is consistent with the image to be encrypted.

9. A visual security image encryption device based on generative adversarial networks, characterized in that, The device includes a processor, a memory, and a computer program stored in the memory, which, when executed by the processor, implements the steps of a visually secure image encryption method based on a generative adversarial network as described in any one of claims 1 to 7.

Images