Generative steganography method based on deep neural network
Through the generative steganography method of deep neural network, combined with image generator and information extractor, the problems of insufficient information hiding accuracy and image quality in the prior art are solved, and efficient and safe information hiding and extraction are achieved, with resistance to Attack ability.
Patent Information
- Application Number
- CN202411017357.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-26
- Publication Date
- 2024-11-29
AI Technical Summary
The existing non-general steganography system based on deep neural networks has insufficient accuracy in information hiding and extraction, and the generated dense images are not of high quality, which is easily detected by steganography analysis methods.
Using a generative steganography method based on deep neural networks, the image generator, information extractor, discriminator and steganography analyzer is used to generate dense images using mapping network, style controller and synthetic network, and extract information through information The module and the binary module extract hidden information, combine the adversarial loss and regularization terms for model training to improve image quality and attack resistance.
It realizes the ability to efficiently hide and extract information, ensures accurate interpretation of information and image quality, and has the ability to resist JPEG compression and scaling attacks, which significantly improves the security and reliability of information hiding.
Smart Images

Figure CN119048322A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of image processing technology, and in particular to a generative steganography method based on a deep neural network. Background Art
[0002] The non-generative steganography system based on deep neural networks hides secret information on a carrier image, and there are obvious differences between the generated secret images and the carrier images, which makes them easily detected by steganalysis methods.
[0003] At present, secret information detection can be achieved through the generative steganography method based on GAN. It specifically adopts the dictionary mapping method, first mapping the secret information to the input conditional variables or hidden variables of GAN, and then synthesizing the secret image through the image generator. For example, the secret information is mapped to the category label of the data set, or the secret information is hidden in the input hidden variable. However, due to the limitation of the number of image labels or the dimension of the hidden variable, these methods can only hide tens to hundreds of bits in each image, and the accuracy of extracting the hidden information is relatively low. If a generative steganography scheme based on a reversible flow model is used, the information hiding capability can be increased to 1Bpp, and the accuracy of extracting the hidden information can be increased to 100%. However, due to the limitation of the flow model capability, the quality of the secret image needs to be improved. Summary of the invention
[0004] The present invention provides a generative steganography method based on deep neural network, aiming to effectively solve the above technical problems.
[0005] According to a first aspect of the present invention, the present invention provides a generative steganography method based on a deep neural network, the method comprising:
[0006] Acquire a training image and a generative steganographic model, wherein the generative steganographic model includes an image generator, an information extractor, a discriminator, and a steganalyzer;
[0007] Inputting hidden variables and random noise into the image generator to obtain a non-confidential image; at the same time, inputting the hidden variables and secret information into the image generator to obtain a confidential image;
[0008] Inputting the non-confidential image and the training image, or the confidential image and the training image, into the discriminator to discriminate the authenticity of the non-confidential image and the confidential image, and obtaining a first discrimination result;
[0009] Inputting the non-confidential image and the confidential image into the steganalyzer to determine whether the image contains secret information, and obtaining a second determination result;
[0010] Inputting the encrypted image into the information extractor to obtain hidden information;
[0011] Calculate the total loss according to the non-confidential image, the confidential image, the first discrimination result, the second discrimination result, the hidden information and the corresponding label information;
[0012] The generative steganography model is trained and updated based on the total loss to obtain a trained generative steganography model.
[0013] Furthermore, after obtaining the trained generative steganographic model, the method further comprises:
[0014] Input the carrier image into the trained image generator to obtain the secret image;
[0015] Inputting the secret image into a trained information extractor to obtain hidden information;
[0016] The trained image generator is used by a carrier image sender, and the trained information extractor is used by a carrier image receiver.
[0017] Further, the image generator includes a mapping network, a style controller and a synthesis network;
[0018] The step of inputting hidden variables and random noise into the image generator to obtain a non-confidential image; and simultaneously inputting the hidden variables and secret information into the image generator to obtain a confidential image, comprises:
[0019] Mapping the latent variables to a latent variable space using the mapping network to obtain intermediate variables;
[0020] Inputting the intermediate variables into a plurality of style controllers to obtain a plurality of style information;
[0021] The style information and the random noise are input into the synthesis network to obtain a non-confidential image; or the style information and the secret information are input into the synthesis network to obtain a confidential image.
[0022] Furthermore, the information extractor includes a plurality of information extraction modules, a plurality of point convolution and a binarization module connected in sequence;
[0023] The step of inputting the encrypted image into the information extractor to obtain the hidden information comprises:
[0024] Using the multiple information extraction modules to extract features from the encrypted image to obtain a feature map;
[0025] The feature map is convolved using the multiple point convolutions, and a binarization operation is performed using the binarization module to obtain the hidden information.
[0026] Further, the total loss includes generator loss, discriminator loss, analyzer loss and extractor loss;
[0027] The generator loss is calculated based on a first adversarial loss between the image generator and the discriminator, a second adversarial loss between the image generator and the steganalyzer, and a first regularization term;
[0028] The discriminator loss is calculated according to the first adversarial loss and the second regularization term;
[0029] The analyzer loss is calculated based on the encrypted image, its corresponding first label, the non-encrypted image and its corresponding second label;
[0030] The extractor loss is calculated based on the encrypted image, the secret information and the hidden information.
[0031] Furthermore, the generator loss is calculated by the generator loss formula:
[0032]
[0033] In the formula, Loss G is the generator loss, is the first counter-loss, is the second confrontation loss, is the first regularization term, λ 1 and λ 2 is a hyperparameter, p z 、p n and p x are the probability distributions of hidden variables, random noise, and training images, respectively. z is the hidden variable, x is the training image, n is the random noise, and x n is a randomly generated Gaussian noise image, w is an intermediate latent variable generated by the randomly sampled latent variable z through the mapping network, and J w T is x n The Jacobian matrix with respect to w; η is ‖J w T ‖ 2 The sliding average of , d is the secret information, p dis the probability distribution of secret information, G(z,d) is the secret image obtained by processing the hidden variable z and the secret information d using the image generator G, D(G(z,d)) is the first discrimination result obtained by using the discriminator D to discriminate the secret image, D(x) is the first discrimination result obtained by using the discriminator D to discriminate the real image, S(G(z,d)) is the second discrimination result obtained by using the steganalyzer S to discriminate the secret image, G(z,n) is the non-secret image obtained by using the image generator G to process the hidden variable z and the random noise n input, S(G(z,n)) is the second discrimination result obtained by using the steganalyzer S to discriminate the non-secret image;
[0034] The discriminator loss is calculated by the discriminant loss formula:
[0035]
[0036] In the formula, Loss D is the discriminator loss, α is a hyperparameter, is the second regularization term;
[0037] The analyzer loss is calculated by the analyzer loss formula:
[0038]
[0039] In the formula, Loss S is the analyzer loss, y 1 and 2 are the first label and the second label respectively; the extractor loss is calculated by the extraction loss formula:
[0040]
[0041] In the formula, Loss data is the extractor loss, F is the feature map output by the information extractor, noise~N(0,0.01 2 ) is the noise that satisfies Gaussian distribution.
[0042] Furthermore, the mapping network includes a plurality of fully connected layers connected in sequence.
[0043] According to a second aspect of the present invention, the present invention further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the above method.
[0044] According to a third aspect of the present invention, the present invention further provides a computer program product, comprising a computer program, wherein the computer program implements the above method when executed by a processor.
[0045] According to a fourth aspect of the present invention, the present invention further provides a storage medium on which a computer program is stored, and when the program is executed by a processor, the method as described above is implemented.
[0046] Through one or more of the above embodiments of the present invention, at least the following technical effects can be achieved: the generation of confidential images and the extraction of hidden information can be realized through the image generator, information extractor, discriminator and steganalyzer, and the hidden information can be embedded and extracted efficiently. It also performs well in terms of extraction accuracy, ensuring the accurate interpretation of information, and has excellent performance in image quality, and can generate high-quality and highly realistic images, so that the hidden information and the original image are integrated. It is also significantly excellent in terms of security. After multiple rounds of attack training, the network can not only cope with common attack forms, such as JPEG compression attacks and scaling attacks, but also maintain a high degree of resistance to steganalysis during information transmission, providing a reliable solution for generative information hiding, with high application value and promotion potential. BRIEF DESCRIPTION OF THE DRAWINGS
[0047] The technical solutions and other beneficial effects of the present invention will be made apparent by describing in detail the specific embodiments of the present invention in conjunction with the accompanying drawings.
[0048] Figure 1 A flow chart of a generative steganography method based on a deep neural network provided in an embodiment of the present invention;
[0049] Figure 2 A schematic diagram of the structure of the generative steganography provided by an embodiment of the present invention;
[0050] Figure 3 A schematic diagram of the structure of an image generator provided by an embodiment of the present invention;
[0051] Figure 4 A schematic diagram of the structure of an information extractor provided by an embodiment of the present invention;
[0052] Figure 5 A schematic diagram of the structure of a steganalyzer provided in an embodiment of the present invention. DETAILED DESCRIPTION
[0053] The technical scheme in the embodiment of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiment of the present invention. Obviously, the described embodiment is only a part of the embodiment of the present invention, not all of the embodiments. Based on the embodiment of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.
[0054] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the term "and / or" herein is merely a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B may represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " herein, unless otherwise specified, generally indicates that the associated objects before and after are in an "or" relationship.
[0055] A generative steganography method based on a deep neural network provided by the present invention is introduced below with reference to the accompanying drawings.
[0056] See also Figure 1 , a generative steganography method based on deep neural network, comprising the following steps:
[0057] S101, obtaining a training image and a generative steganographic model, wherein the generative steganographic model includes an image generator, an information extractor, a discriminator, and a steganalyzer.
[0058] In this step, in order to effectively train the neural network in the project, when selecting and downloading the dataset, considering the diversity, scale and quality required by the project, three datasets are selected in this embodiment: LSUN, CelebA and FFHQ, with the scales of 300,000, 200,000 and 70,000 images respectively. These datasets each have unique features and rich content, and can provide rich and diverse training samples.
[0059] In addition, in order to ensure that the quality of the data meets the standards required for training, the above training set is strictly verified, cleaned and preprocessed to obtain training images for model training.
[0060] Among them, the backbone network of the generative steganography model is a Style GAN-2 synthesis network, such as Figure 2 As shown in the figure, it is composed of four sub-networks, namely image generator (G), information extractor (E), discriminator (D) and steganalyzer (S), which are used to generate secret images, extract hidden information, improve image quality and improve anti-steganalysis performance respectively.
[0061] Among them, the image generator G can simultaneously generate a secret image (Stego Image x s ) and non-confidential images (Cover Image x c ). The input of the image generator G includes one of the hidden variables z, random noise n or secret information d. When (z, n) are input to the image generator G at the same time, G can synthesize a non-confidential image x c ; When input (z, d) generates a secret image x s .
[0062] The secret image and the training image, or the non-secret image and the training image are sent to the discriminator D for true / false judgment respectively, and the secret image and the non-secret image are sent to the steganalyzer S to determine whether it is a secret image. D and S are used as dual discriminators of G, which can improve the visual quality of the synthesized image and the indistinguishability between the secret image and the non-secret image. Finally, the generated secret image is sent to the information extractor E to obtain the hidden information d′.
[0063] S102, inputting hidden variables and random noise into the image generator to obtain a non-confidential image; at the same time, inputting the hidden variables and secret information into the image generator to obtain a confidential image.
[0064] Wherein, the image generator is as follows: Figure 3 As shown, it includes a Mapping network, a style controller and a Synthesis network. The mapping network includes a plurality of fully connected layers connected in sequence.
[0065] The mapping network decouples the input latent variable z through a fully connected layer group to generate an intermediate variable w, which is then sent to AdaIN (adaptive instance normalization, i.e. style controller) to control the image style.
[0066] The step of inputting hidden variables and random noise into the image generator to obtain a non-confidential image; and simultaneously inputting the hidden variables and secret information into the image generator to obtain a confidential image, comprises:
[0067] Mapping the latent variables to a latent variable space using the mapping network to obtain intermediate variables;
[0068] Inputting the intermediate variables into a plurality of style controllers to obtain a plurality of style information;
[0069] The style information and the random noise are input into the synthesis network to obtain a non-confidential image; or the style information and the secret information are input into the synthesis network to obtain a confidential image.
[0070] The synthetic network includes a secret module (SecretBlock) and multiple common modules, etc.
[0071] The secret module is used to synthesize secret / non-secret images simultaneously. Specifically, the secret module includes three 1×1 convolutional layers (activation function is Leaky Relu), three 3×3 convolutional layers, two upsampling operations, four information fusion operations and a low-pass filter. The connections of each network layer are as follows: Figure 3As shown. The random noise tensor n (~N(0,σ2), with a shape of 1×H×W) is fused into the convolutional feature map through the secret module to generate a non-confidential image, and the secret information d is also fused into the convolutional feature map through the secret module to generate a confidential image. It should be noted that in this embodiment, the secret module is placed at the end of the synthetic network, but it can also be adjusted to different network layers according to the embedding rate. Preferably, it is placed in the third-to-last layer of the synthetic network, and it can also be set to multiple layers according to demand. This application does not limit this. Among them, placing the secret module at the end of the synthetic network can achieve a high embedding rate, and placing the secret module in the third-to-last layer of the synthetic network can improve robustness.
[0072] In some other embodiments of the present application, the style controller may also be a demodulation module (Weight Demodulation), which is also used to control the image content. Specifically, the demodulation module includes two steps: modulation (Mod) and demodulation (DeMod):
[0073] w' ijk =s i ·w ijk ,
[0074]
[0075] In the formula, w ijk is the i-th randomly initialized feature map weight, s i is the scale of scaling, which is obtained after the intermediate latent variable w passes through the fully connected layer, w' ijk is the modulated weight, w'' iijk is the demodulated weight, is the L2 norm of the modulated weight, ∈=1e-6, ∈ is a constant, which can avoid the situation where the divisor is 0.
[0076] Based on the network structure of the above image generator, through the hierarchical gradient decay technique training strategy, the generator can synthesize confidential / non-confidential images that are difficult to distinguish between the naked eye and steganalysis tools. The sender of information can send confidential / non-confidential images at the same time to deceive the opponent. The receiver of information can accurately extract the hidden information from the confidential image through the extractor E, but cannot obtain any meaningful information from the non-confidential image.
[0077] S103: Input the non-confidential image and the training image, or the confidential image and the training image, into the discriminator to discriminate the authenticity of the non-confidential image and the confidential image, and obtain a first discrimination result.
[0078] It should be noted that, based on the training images, the discriminator is used to discriminate the secret image and the real image respectively to obtain two first discrimination results, and the two first discrimination results are independently generated and correspond to the secret image and the real image respectively.
[0079] S104, inputting the non-confidential image and the confidential image into the steganalyzer to determine whether the image contains secret information, and obtaining a second determination result.
[0080] Specifically, a steganalyzer is used to distinguish the non-confidential image and the confidential image respectively, and two corresponding second distinction results are obtained.
[0081] In this embodiment, a discriminator D and a steganalyzer S are used to discriminate images simultaneously. The two and the generator G form a dual discriminator generative adversarial network. D can ensure the quality of the secret image, and S can reduce the difference between the synthesized secret / non-secret images. D mainly discriminates the image by extracting the low-frequency signal of the image, and S mainly distinguishes the input image by extracting the high-frequency signal of the image.
[0082] Specifically, this embodiment uses the discriminator in the original Style GAN-2 as D, and uses SR-net as the steganalyzer. The input of the discriminator D is the real image and the synthesized secret image (or non-secret image), and the input of S is the synthesized secret image and non-secret image.
[0083] S105, inputting the encrypted image into the information extractor to obtain hidden information.
[0084] The information extractor includes a plurality of information extraction modules, a plurality of point convolution and a binarization module connected in sequence.
[0085] The step of inputting the encrypted image into the information extractor to obtain the hidden information comprises:
[0086] Using the multiple information extraction modules to extract features from the encrypted image to obtain a feature map;
[0087] The feature map is convolved using the multiple point convolutions, and a binarization operation is performed using the binarization module to obtain the hidden information.
[0088] Specifically, Figure 4 As shown, the information extractor is a U-Net network, which is a U-shaped deep neural network consisting of an encoder, a decoder, a convolution operation block connecting the two parts, and channel-level connections between different levels as shortcuts.
[0089] Among them, the encoder part consists of eight encoding blocks, each block increases the number of channels by two times, including convolution layers, normalization layers, and activation units; the spatial size of the features is reduced by a pooling layer with a stride of 2; therefore, the output feature size of the encoding part is that of the original image; the decoder part also consists of eight decoding blocks, each block expands the spatial size of the input and reduces the number of channels by two times, including a transposed convolution layer with a stride of 2, a spectral normalization layer, and an exponential linear activation unit (ELU); features of the same level from the encoding part and the decoding part are concatenated together before passing the features to the next decoding block; after the last block of the decoder part, the final output is generated by applying a convolution layer and a Tanh activation function.
[0090] S106, calculating a total loss according to the non-confidential image, the confidential image, the first discrimination result, the second discrimination result, the hidden information and the corresponding label information.
[0091] The total loss includes generator loss, discriminator loss, analyzer loss and extractor loss;
[0092] The generator loss is calculated based on a first adversarial loss between the image generator and the discriminator, a second adversarial loss between the image generator and the steganalyzer, and a first regularization term;
[0093] The discriminator loss is calculated according to the first adversarial loss and the second regularization term;
[0094] The analyzer loss is calculated based on the encrypted image, its corresponding first label, the non-encrypted image and its corresponding second label;
[0095] The extractor loss is calculated based on the encrypted image, the secret information and the hidden information.
[0096] The generator loss is calculated by the generator loss formula:
[0097]
[0098] In the formula, Loss G is the generator loss, is the first counter-loss, is the second confrontation loss, is the first regularization term, λ 1 and λ 2 is a hyperparameter, p z 、p n and p x are the probability distributions of hidden variables, random noise, and training images, respectively. z is the hidden variable, x is the training image, n is the random noise, and x nis a randomly generated Gaussian noise image, w is an intermediate latent variable generated by the randomly sampled latent variable z through the mapping network, and J w T is x n The Jacobian matrix with respect to w; η is ‖J w T ‖ 2 The sliding average of 2 represents the L2 norm, d is the secret information, p d is the probability distribution of the secret information, Specifically, it is a regularization term used in Style GAN-2 to stabilize the network training λ 1 and λ 2 are two hyperparameters. G(z,d) is the secret image obtained by processing the hidden variable z and the secret information d using the image generator G, D(G(z,d)) is the first discrimination result obtained by using the discriminator D to discriminate the secret image, D(x) is the first discrimination result obtained by using the discriminator D to discriminate the real image, S(G(z,d)) is the second discrimination result obtained by using the steganalyzer S to discriminate the secret image, G(z,n) is the non-secret image obtained by using the image generator G to process the hidden variable z and the random noise n input, S(G(z,n)) is the second discrimination result obtained by using the steganalyzer S to discriminate the non-secret image.
[0099] The discriminator loss is calculated by the discriminant loss formula:
[0100]
[0101] In the formula, Loss D is the discriminator loss, α is a hyperparameter, is the second regularization term, which is used to stabilize the generative adversarial network training process and improve the quality of generated images.
[0102] The analyzer loss is calculated by the analyzer loss formula:
[0103]
[0104] In the formula, Loss S is the analyzer loss, y 1 and 2 They are the first label and the second label.
[0105] Specifically, if Figure 5As shown, the steganalyzer S includes two layers of type 1 modules, five layers of type 2 modules, four layers of type 3 modules, one layer of type 4 modules, a fully connected layer and an activation layer, wherein the type 1 module includes two Layertype 1s, the type 2 module includes five Layertype 2s, the type 3 module includes four Layertype 3s, and the type 4 module includes one Layertype 4. The structures of Layertype 1, Layertype 2, Layertype 3 and Layertype 4 are specifically described in Figure 5 .
[0106] Binary cross entropy is used in the steganalyzer S. Because S outputs a two-dimensional vector, unlike the one-dimensional output of the traditional GAN discriminator. S outputs the correct prediction by minimizing the binary cross entropy: when a secret image is input, the output prediction is close to the vector [0,1]; conversely, when a non-secret image is input, the output vector is close to [1,0]. However, in the confrontation between the generator and the steganalyzer, regardless of whether the input is a secret or non-secret image, the generator hopes that the vector output by the steganalyzer is close to [0.5,0.5]. Here, y 1 and 2 are the labels of the encrypted / non-encrypted images, and their values are [0,1] and [1,0] respectively.
[0107] The extractor loss is calculated by the extraction loss formula:
[0108]
[0109] In the formula, Loss data is the extractor loss, F is the feature map output by the information extractor, noise~N(0,0.01 2 ) is a noise that satisfies the Gaussian distribution. The purpose is to enhance the robustness of the secret image and improve the accuracy of extracting secret information. The Sigmoid(·) operation makes the result fall between (0,1).
[0110] S107, training and updating the generative steganography model based on the total loss to obtain a trained generative steganography model.
[0111] It should be noted that this embodiment adopts a multi-stage approach to the training of the generative steganographic network: the first stage trains the generation ability of the model, the second stage trains the embedding and extraction capabilities of the model, and finally the robustness of the model against public channel attacks is trained.
[0112] Specifically, we first use a large number of training set images, only train the generator and discriminator, and train for 300k cycles, so that the network can generate high-quality images. Then, inject secret information into the network, and train the discriminator, generator, and extractor step by step in each training iteration, so that the network can obtain the ability to penetrate and extract secrets through training while ensuring the quality of image generation. Finally, when the network has the ability to generate and steganography, we begin to introduce robust training. Specifically, before the generated images are sent to the extractor for training, they are subjected to noise attacks at the attack layer, including jpeg compression attacks, scaling attacks, etc.
[0113] Through a large number of sample training and iterative optimization, the resulting deep neural network can generate high-quality secret images, and the images can resist common JPEG compression and scaling attacks in public channels.
[0114] In the robustness training phase, a phased tuning training method was adopted. First, for the highly destructive lossy JPEG compression, the secret information was embedded in the early stage of the generator synthesizing the image, so that the secret information would be integrated into the low-frequency information of the image and would not be easily erased by the lossy JPEG compression attack. In addition, progressive training was adopted during the training. According to the size of the JPEG compression quality factor, the training started from a quality factor of 95. When the extraction rate reached 98 or above, the quality factor was reduced by 5 and continued to train until the model could resist the JPEG attack with a quality factor of 70. The model was randomly subjected to JPEG attacks between 95 and 70 according to probability during the training process. After a large number of iterations and tuning, the model was finally able to withstand the JPEG attacks under normal circumstances. Then the model was subjected to JPEG or scaling attacks according to probability, and finally the model could resist two common lossy attacks.
[0115] The generative steganography method based on deep neural network provided in the embodiment of the present application realizes the generation of secret images and the extraction of hidden information through image generator, information extractor, discriminator and steganalyzer, can efficiently embed and extract hidden information, and also performs well in extraction accuracy, ensuring accurate interpretation of information. At the same time, it also has excellent performance in image quality, and can generate high-quality and highly realistic images, so that hidden information and original images are integrated. It is also significantly excellent in ensuring security. After multiple rounds of attack training, the network can not only cope with common attack forms, such as JPEG compression attack and scaling attack, but also maintain a high degree of resistance to steganalysis during information transmission, providing a reliable solution for generative information hiding, with high application value and promotion potential.
[0116] In some embodiments of the present application, after obtaining the trained generative steganographic model, the method further includes:
[0117] The carrier image is input into the trained image generator to obtain the secret image.
[0118] The encrypted image is input into a trained information extractor to obtain the hidden information.
[0119] The trained image generator is used by a carrier image sender, and the trained information extractor is used by a carrier image receiver.
[0120] In this embodiment, after the generative steganography model reaches the preset training end condition, the image generator G is handed over to the information sender for use, and the information extractor E is handed over to the receiver for use. The sender uses G to synthesize a secret image to transmit secret information; after the receiver receives the secret image, E can be used to extract the hidden information in the image. Among them, the sender can use G to synthesize a secret image and a non-secret image, and send them to the receiver at the same time. The receiver can accurately extract the hidden information from the secret image through E, but cannot extract any useful information from the non-secret image. Among them, the training end condition can be a preset number of iterations, or it can be an end condition such as network parameter convergence.
[0121] The secret / non-secret images are difficult to distinguish visually and statistically, which can greatly confuse third parties. At the same time, if the adversary trains a new model and tries to use a new extractor E to extract hidden information, the information extraction accuracy is only about 50%, and the hidden information cannot be extracted at all, which helps to improve the security of secret communication.
[0122] In some embodiments of the present application, the preprocessing process of the training set specifically includes:
[0123] Use the cv2.imread method in the OpenCV library to read the image data, and convert the image color space to RGB format through cv2.cvtColor to ensure the accuracy and consistency of the image data in subsequent processing.
[0124] Next, check whether the height and width of the image meet the specified resolution requirements. If the image size does not match the required resolution, use the cv2.resize() method to resize the image to the preset resolution. This step uses the cv2.INTER_CUBIC interpolation method based on cubic spline interpolation to ensure smoothness and fidelity during image scaling.
[0125] Then, normalization is performed to scale the pixel values to a range between 0 and 1 by dividing the image data by 255 so that the neural network can better process the data.
[0126] Next, the transpose(2,0,1) method is used to rearrange the dimensions of the image data to match the input format expected by the neural network. This step adjusts the channel dimension of the image data to the first dimension to ensure that it matches the input requirements of the deep learning model.
[0127] Finally, the data type is converted to float32 to ensure that the numerical accuracy and adaptability of the data meet the needs of neural network training. This series of preprocessing steps is designed to provide image data that meets the standards and provide high-quality training samples for the neural network model.
[0128] In some embodiments of the present application, the mapping network in the image generator decouples the input latent variables through a fully connected layer group to generate intermediate variables, which are then sent to an adaptive instance normalization module to control the image style.
[0129] Among them, the purpose of disentanglement is to construct a linear latent variable space so that each dimension can control a variation factor, such as hair length, expression, and skin color. The feature distribution in the training data set is uneven. When the latent variables randomly sampled from the Gaussian distribution are mapped to various image features, the mapping results are often distorted. This means that changing any dimension of the latent variable will cause multiple image features to change simultaneously. This phenomenon is called entanglement. The mapping network maps the randomly sampled latent variable z to the latent variable space and obtains the intermediate variable w∈W. After being processed by the mapping network w=f(z), the generative model can control the properties of the generated image more linearly.
[0130] The mapping network specifically includes 8 fully connected layers connected in sequence. The input latent variables are decoupled through the mapping network to obtain a 512-dimensional intermediate variable. Then the affine transformation is performed through the affine module, and finally input to the adaptive instance normalization module for adaptive normalization to control the image style. The synthesis network consists of several layers, and the feature map resolution is gradually increased from 4×4 through upsampling to a maximum of 1024×1024. The feature map is finally output as an RGB image through a 1×1 convolution. The noise is scaled in each channel to increase the details of the image.
[0131] The synthesis network consists of several common modules and one secret module. The common module is the module used in Style GAN-2, which generates feature images step by step, while the secret module is uniquely designed to generate both confidential and non-confidential images at the same time.
[0132] The input of the synthesis network is a 512×4×4 trainable tensor, which is initially generated by a random distribution of Gaussian noise. Through the upsampling operation in each module, the resolution of the feature map is gradually increased from 8×8 to 16×16, until the length and width of the final output image are H×W. In this process, the N in the image represents standard Gaussian noise, which is added to the ordinary module to enhance the expression of details and make the generated image richer and more realistic. This complex and sophisticated structural design enables the generator to generate images with higher quality and richness.
[0133] In some embodiments of the present application, the information extractor E includes n information extraction modules, two 1×1 convolutions and a binarization module. Each information extraction module consists of two branches, and the feature map after convolution is first merged and sent to the next module. The feature map output by the information extraction model can be obtained after 1×1. Finally, after the binarization operation, the secret information can be obtained.
[0134] d′=Round(Sigmoid(F)),
[0135] Among them, Round(·) is a rounding operation, and the size of the feature map F and the prediction result d′ are both B×H×W.
[0136] In some embodiments of the present application, in order to verify the anti-steganalysis performance of the generative steganography method based on deep neural network provided by the present application, the scheme adopts two existing steganalysis algorithms Ye-net and SR-net to detect synthetic confidential / non-confidential images.
[0137] First, we train generative steganography models with different information load capacities on the CelebA dataset, and use the pre-trained model to randomly generate 5,000 synthetic network images containing secrets / non-secrets as training sets for steganalysis, and another 1,000 synthetic network images containing secrets / non-secrets as test sets. Then, we use the synthetic network Ye-net and the synthetic network SR-net to train on each image training set, and test on the corresponding test set to obtain the model's ability to resist steganalysis.
[0138] In addition, the generative steganography model proposed in this application can also be combined with other auxiliary technologies, such as a data enhancement module, which slightly perturbs the image to be trained through scaling, flipping, and image lossy JPEG compression operations to generate more training samples and expand the generalization ability of the network. Specifically, for image scaling, since the image needs to be predicted and interpolated during the scaling process, it will cause certain distortion, which will pose a potential threat and damage to the image detail information and immune information. The present invention uses a bilinear interpolation algorithm as an implementation method to randomly reduce or enlarge the original image, and then scale it back to its original size, and stipulates that the scaling ratio used falls within the range of [0.5, 2]. For JPEG compression, the algorithm divides the image into several small blocks, performs a two-bit cosine (DCT) transform, and quantizes the AC coefficients, which will also cause the loss and distortion of high-frequency signals in the image. The present invention stipulates that when performing JPEG compression, the quality factor (Quality Factor, referred to as QF) used falls within the range of [70, 100]. For the above enhancement, the present invention uses the corresponding functions in the python version library of opencv (opencv-python) to perform image enhancement, for example, using "cv2.resize" to perform interpolation operations, "cv2.imencode" to perform JPEG compression, and so on.
[0139] Another example is the attack layer module. The attack layer is a processing layer introduced after the image is generated. It aims to improve the robustness of the generated image and make it resistant to various common attack methods. This layer applies different forms of transformation and processing to the generated image, including but not limited to scaling, JPEG compression and other technical means. Its main purpose is to simulate the interference or changes that may occur in actual transmission or storage, so as to make the generated image more robust and resistant to adversarial forces. The design idea of the attack layer is to introduce noise or transformation to the generated image so that the model can adapt to different disturbances and environmental conditions during training, thereby improving the quality and robustness of the generated image. Scaling operations can change the size of the image, while JPEG compression compresses the image to simulate the compression loss that may be encountered in image transmission or storage. These transformations can not only verify the robustness of the generated model in different environments, but also enhance the model's resistance to various interferences and attacks. The role of the attack layer in the neural network model is to improve the robustness of the model and make it more usable and stable in real scenarios. This training method allows the model to better adapt to various interferences and attacks, thereby producing more robust generated images and providing more reliable support for the practical application of the model.
[0140] After the training is completed, the model is divided into two parts: the generator is delivered to the sender of the information, and the extractor is delivered to the receiver. The sender uses to synthesize a secret image to transmit secret information; after the receiver receives the secret image, the extractor can accurately extract the information hidden in the image. It is worth noting that the sender can use to generate secret images and non-secret images, and pass both to the receiver at the same time. The receiver can use the extractor to accurately extract hidden information from the secret image, but cannot extract any useful information from the non-secret image. This makes it difficult to distinguish between secret / non-secret images in visual and statistical analysis, which poses great difficulties for third parties.
[0141] At the same time, if the adversary tries to train a new model and tries to use a new extractor to extract hidden information, this will pose a challenge to the security of the model. The generative steganography model design provided in this application fully considers adversarial nature, so that if the adversary trains a new model, the resulting extractor cannot correctly extract the secret information. The adversarial training of this model makes it difficult for a third party to easily spy on or steal hidden information, ensuring the security and reliability of information transmission.
[0142] In order to evaluate the performance of generative steganography models, the applicable evaluation indicators are first introduced.
[0143] The capacity of hiding secret information in an image is usually measured using the indicator Bpp, which represents the number of bits of information hidden in each pixel (Bits Per Pixel). The calculation formula is as follows:
[0144]
[0145] Where len(d) represents the length of the hidden information (d is expanded into a one-dimensional vector). C, H and W represent the number of channels, height and width of the secret image respectively. The number of channels C of the generated color secret image is 3, and the number of channels of the grayscale secret image is 1.
[0146] Fid is a commonly used indicator for evaluating the quality of synthetic images, and its expression is:
[0147] Fid=||μ r -μ g || 2 +Tr(∑ r +∑ g -2(∑ r ∑ q ) 1 / 2 )
[0148] Among them, the feature extraction uses the neural network Inception-Net. r and μ g Represent the mean of the real image and synthetic image features respectively; ∑r and∑ g represents the covariance of the two image features; Tr represents the trace of the matrix, that is, the sum of the elements on the main diagonal. The smaller the Fid value, the better the effect of the generated model, that is, the clearer the image and the richer the diversity.
[0149] Acc is the extraction accuracy of hidden secret information, expressed as follows:
[0150]
[0151] In the above formula, d and d′ represent the hidden secret information and the extracted hidden information respectively, which are expanded into a one-dimensional vector. ⊙ is the pixel-level XOR operation. The larger the Acc value, the better the performance.
[0152] Pe is the most commonly used steganalysis security evaluation index. It measures the anti-steganalysis capability and security of the steganalysis algorithm by calculating the detection error rate of the steganalysis method. The definition is as follows:
[0153]
[0154] Among them, P FA and P MD It is the false alarm rate and missed detection rate in steganalysis detection. Pe is the average of the two values, and the false alarm rate is required to be as low as possible. The value range of Pe is between [0,1]. For the steganography algorithm, its optimal value is 0.5, at which point the steganalysis algorithm cannot distinguish the source of the image at all. In other words, the steganalysis method cannot distinguish whether the input is a confidential or non-confidential image.
[0155] The present invention uses the above-mentioned benchmark model (Table 1) and the robust model (Table 2) to train and test on various data sets, and the results are as follows:
[0156] Table 1
[0157] Dataset Bp Acc FID P Celeba256 1 99% 30 0.489 FFHQ512 1 99% 30 0.492
[0158] Table 2
[0159]
[0160] Among them, wo means no attack, JPEG means JPEG compression attack, JPEG70 to JPEG95 respectively refer to JPEG compression attacks with quality factors of 70, 75, 80, 85, 90, and 95. The smaller the quality factor, the greater the JPEG attack strength. Resize refers to a scaling attack, that is, first reducing the image by half and then enlarging it to the original size. It can be seen from Table 2 that when no attack is applied or the attack strength is low, the model maintains a high accuracy of about 99%. Even under the higher intensity attack of JPEG70, the accuracy of the two data sets can still be maintained above 98 and 96, which proves that the model provided by this application has considerable robustness.
[0161] Among the above indicators, Acc and Bpp indicators are generally the most important in generative steganographic models. The scheme proposed in the present invention has demonstrated leading performance in many aspects. Regardless of whether it is robust or not, the model maintains an acc of more than 98%, and can embed and accurately extract hidden information. In terms of Bpp, the model can achieve an embedding rate of 1Bpp, and the model with robustness can also reach 0.06Bpp, realizing a high-capacity steganographic method. At the same time, it also performs well in terms of image quality, with a low FID. In terms of security, after multiple rounds of attack training, the network can not only cope with common forms of attack, such as JPEG compression attacks and scaling attacks, but also maintain a high degree of resistance to steganalysis during information transmission, providing a reliable solution for generative information hiding, with high application value and promotion potential.
[0162] Based on any of the above embodiments, another embodiment of the present invention further provides an electronic device, which may include: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor may call logic instructions in the memory to execute the above method.
[0163] In addition, the logic instructions in the above-mentioned memory can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.
[0164] On the other hand, an embodiment of the present invention further provides a storage medium on which a plurality of instructions are stored, and the instructions are suitable for being loaded by a processor to execute the generative steganography method based on a deep neural network as provided in the above embodiments.
[0165] On the other hand, the technical solution of the present invention, in essence or in other words, the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.
[0166] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.
[0167] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.
[0168] In summary, although the present invention has been disclosed as above in terms of preferred embodiments, the above preferred embodiments are not intended to limit the present invention. A person skilled in the art may make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, the scope of protection of the present invention shall be based on the scope defined in the claims.
Claims
1. A generative steganography method based on deep neural network, characterized in that: include: Acquire a training image and a generative steganographic model, wherein the generative steganographic model includes an image generator, an information extractor, a discriminator, and a steganalyzer; Inputting hidden variables and random noise into the image generator to obtain a non-confidential image; at the same time, inputting the hidden variables and secret information into the image generator to obtain a confidential image; Inputting the non-confidential image and the training image, or the confidential image and the training image, into the discriminator to discriminate the authenticity of the non-confidential image and the confidential image, and obtaining a first discrimination result; Inputting the non-confidential image and the confidential image into the steganalyzer to determine whether the image contains secret information, and obtaining a second determination result; Inputting the encrypted image into the information extractor to obtain hidden information; Calculate the total loss according to the non-confidential image, the confidential image, the first discrimination result, the second discrimination result, the hidden information and the corresponding label information; The generative steganography model is trained and updated based on the total loss to obtain a trained generative steganography model.
2. The method according to claim 1, characterized in that After obtaining the trained generative steganographic model, the method further includes: Input the carrier image into the trained image generator to obtain the secret image; Inputting the secret image into a trained information extractor to obtain hidden information; The trained image generator is used by a carrier image sender, and the trained information extractor is used by a carrier image receiver.
3. The method according to claim 1, characterized in that The image generator includes a mapping network, a style controller and a synthesis network; The step of inputting hidden variables and random noise into the image generator to obtain a non-confidential image; and simultaneously inputting the hidden variables and secret information into the image generator to obtain a confidential image, comprises: Mapping the latent variables to a latent variable space using the mapping network to obtain intermediate variables; Inputting the intermediate variables into a plurality of style controllers to obtain a plurality of style information; The style information and the random noise are input into the synthesis network to obtain a non-confidential image; or the style information and the secret information are input into the synthesis network to obtain a confidential image.
4. The method according to claim 1, characterized in that: The information extractor comprises a plurality of information extraction modules, a plurality of point convolution and a binarization module connected in sequence; The step of inputting the encrypted image into the information extractor to obtain the hidden information comprises: Using the multiple information extraction modules to extract features from the encrypted image to obtain a feature map; The feature map is convolved using the multiple point convolutions, and a binarization operation is performed using the binarization module to obtain the hidden information.
5. The method according to claim 1, characterized in that The total loss includes generator loss, discriminator loss, analyzer loss and extractor loss; The generator loss is calculated based on a first adversarial loss between the image generator and the discriminator, a second adversarial loss between the image generator and the steganalyzer, and a first regularization term; The discriminator loss is calculated according to the first adversarial loss and the second regularization term; The analyzer loss is calculated based on the encrypted image, its corresponding first label, the non-encrypted image and its corresponding second label; The extractor loss is calculated based on the encrypted image, the secret information and the hidden information.
6. The method according to claim 5, characterized in that The generator loss is calculated by the generator loss formula: In the formula, Loss G is the generator loss, is the first counter-loss, is the second confrontation loss, is the first regularization term, λ1 and λ2 are hyperparameters, and p z 、p n and p x are the probability distributions of hidden variables, random noise, and training images, respectively. z is the hidden variable, x is the training image, n is the random noise, and x n is a randomly generated Gaussian noise image, w is an intermediate latent variable generated by the randomly sampled latent variable z through the mapping network, and J w T is x n The Jacobian matrix of w; η is ‖J w T ‖ 2 The sliding average of , d is the secret information, p d is the probability distribution of secret information, G(z,d) is the secret image obtained by processing the hidden variable z and the secret information d using the image generator G, D(G(z,d)) is the first discrimination result obtained by using the discriminator D to discriminate the secret image, D(x) is the first discrimination result obtained by using the discriminator D to discriminate the real image, S(G(z,d)) is the second discrimination result obtained by using the steganalyzer S to discriminate the secret image, G(z,n) is the non-secret image obtained by using the image generator G to process the hidden variable z and the random noise n input, S(G(z,n)) is the second discrimination result obtained by using the steganalyzer S to discriminate the non-secret image; The discriminator loss is calculated by the discriminant loss formula: In the formula, Loss D is the discriminator loss, α is a hyperparameter, is the second regularization term; The analyzer loss is calculated by the analyzer loss formula: In the formula, Loss S is the analyzer loss, y1 and y2 are the first label and the second label respectively; The extractor loss is calculated by the extraction loss formula: In the formula, Loss data is the extractor loss, F is the feature map output by the information extractor, noise~N(0,0.01 2 ) is the noise that satisfies Gaussian distribution.
7. The method according to claim 3, characterized in that The mapping network includes a plurality of fully connected layers connected in sequence.
8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: The processor runs the computer program to implement the method according to any one of claims 1 to 7.
9. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.
10. A storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.
