Generative steganography method based on extended latent space and contrastive learning

Abstract

The present invention discloses a generative steganography method based on extended latent space and contrastive learning, which belongs to the field of information security. The extended latent space and contrastive learning are used to achieve the hiding and extraction of secret information. The real reference image is converted into the latent space through a feature pyramid encoder to expand the steganographic capacity of the image carrier. The principal component analysis PCA is used to find the distribution interval of structural attributes in latent vectors at different levels to determine the best area for information embedding. Finally, the pre-trained StyleGAN2 generator is used to generate the secret image. At the same time, the variant contrastive learning mechanism of CLIP is introduced to align the image with the latent vector and freeze it into a loss module after training to improve the accuracy of information extraction. The present invention is applicable to the increasingly prominent multimedia communication and user privacy security issues in the big data era, and achieves the hiding and extraction of secret information.

Description

Technical Field

[0001] The present invention belongs to information security technology, and in particular relates to a generative steganography method based on extended latent space and contrastive learning. Background Art

[0002] With the rapid development of information technology, digital multimedia data (such as images, videos, audio, text, etc.) are increasingly used in daily life and work. However, the widespread dissemination and easy accessibility of digital information also bring new security risks. To address this problem, researchers have proposed a variety of effective means to ensure data security. Steganography, as a branch of information hiding, can achieve covert communication by embedding secret information in a carrier in an invisible form. Compared with data encryption technology, steganography can not only protect the secret information in the carrier from being cracked by a third party, but also prevent the secret data from being suspected and intercepted by a third party when it is transmitted in social networks, thereby providing additional protection at the communication behavior level.

[0003] Robust steganography is a hot topic in the field of steganography in recent years. Conventional traditional steganography is mainly aimed at transmitting steganographic data in point-to-point lossless channels, such as sharing steganographic images through emails and P2P software. However, this type of channel is easy to track the connection relationship and communication behavior of the two communicating parties. Especially when a large amount of media data is transmitted, it is easy for the detection party to perceive it as unusual behavior, thus arousing suspicion. In recent years, research in the field of steganography has gradually begun to consider using public data sharing platforms as steganographic transmission channels, that is, sharing steganographic images on social platforms, image sharing websites, etc. for secret communication, such as Facebook, Twitter, WeChat and other social platforms. Compared with traditional point-to-point channels, when this type of channel is used by steganography methods for secret communication, it has the characteristics of many ordinary users, large amount of media data, and complex data as natural cover, making it difficult for opponents to detect communication behavior. However, due to the limitations of network and device processing capabilities, the uploaded secret images are usually subjected to lossy operations such as compression and scaling, resulting in image degradation and information loss. Compared with digital watermarking, robust steganography not only focuses on the reliability of information transmission, but also on security. Therefore, it not only needs to inherit the existing adaptive steganography embedding technology to ensure security, but also needs to have an error correction decoding mechanism.

[0004] Compared with other forms of digital multimedia data, images have the characteristics of wide use, easy dissemination, and high data redundancy, making them an ideal carrier for steganography. At present, digital image steganography technology is mainly divided into two categories: embedded steganography and generative steganography. Early research mainly focused on embedded steganography schemes, and the most typical method is the least significant bit (LSB) steganography method. This method embeds secret information by modifying the least significant bit of each pixel in the image.

[0005] In recent years, researchers have been continuously improving steganography schemes based on the algorithmic ideas of LSB. Some methods further compress secret information by designing codes to reduce the number of pixels that need to be modified. Some methods select the modification position of the carrier by designing distortion functions to minimize the embedding distortion of the image. At first, researchers manually designed distortion functions. With the development of deep learning technology, researchers used neural networks to self-supervise learning of distortion functions, and the distortion of the secret image was further reduced. However, this embedding method inevitably changes the statistical characteristics of the image, making it easy to be detected by steganalyzers and suffer further attacks. This defect of the embedded steganography scheme prompted researchers to explore new steganography methods.

[0006] As an emerging steganographic technology, generative steganography directly generates a secret carrier based on secret information, and no longer relies on modifying the existing carrier image. Compared with traditional embedded steganography, generative steganography is theoretically more resistant to detection because it does not rely on modifying the statistical characteristics of existing images. In recent years, with the rapid development of deep learning technology, especially the widespread application of multimedia generative network models such as generative adversarial networks (GANs), the generated multimedia data has been able to achieve a level of authenticity. In the network, the proportion of such data is also growing rapidly. Therefore, using generated multimedia data as a secret carrier is less likely to arouse suspicion or attack, which provides favorable conditions for the rapid development of generative steganography technology.

[0007] According to the different ways of expressing secret information, the existing generative steganography techniques can be roughly divided into four types: pixel-based definition, low-level feature mapping, high-level feature association, and latent space mapping. The pixel-based method mainly encodes the secret information into pixel values ​​and forms the entire secret image. The research work based on low-level feature mapping is to map the secret information into the low-level features of the image (such as texture, contour, etc.), and then generate a secret image with this feature. Most researchers based on low-level feature mapping will encode the secret information into specific high-level features of the image (such as the semantic information of the image, the style features of the image, etc.), and generate a secret image that conforms to the feature. The method based on latent space mapping learns the mapping relationship between the natural image space and the latent space (such as high-dimensional Gaussian distribution, etc.), maps the image information to the low-dimensional latent space, and then generates the corresponding secret image, thereby realizing covert communication. While resisting detection, it can also find a more ideal balance between visual quality and information capacity.

[0008] Although the existing generative steganography technology has shown significant advantages over traditional steganography methods by converting secret information into suitable data types based on different mapping rules to generate secret images, it still has many shortcomings in terms of hiding capacity, extraction accuracy, and image generation quality. Specifically, the hiding capacity of existing generative steganography methods is low, which makes it impossible to effectively embed sufficient secret information. When the stego load is large, the image generation quality and the accuracy of secret information extraction will be significantly reduced. In addition, due to the poor quality of the generated secret image, it is easy to be detected by steganalysis tools, thus affecting the security of the steganography effect. Summary of the invention

[0009] Purpose of the invention: The purpose of the present invention is to solve the deficiencies in the prior art and provide a generative steganography method based on extended latent space and contrastive learning to improve the steganographic capacity, extraction accuracy and anti-detection performance. The core idea of ​​this technology is to use the extended latent space mapping to embed secret information into the generated image while maintaining the high quality and concealment of the image.

[0010] Technical solution: The generative steganography method based on extended latent space and contrastive learning of the present invention includes an information hiding stage and an information extraction stage;

[0011] The specific process of the hidden information stage is as follows:

[0012] Step 1: Input the reference image into the encoder based on the three-layer feature pyramid network FPN, and generate a series of style vectors through the encoder , where the style vector As the initial latent vector ;

[0013] Step 2: Initial latent vector PCA feature analysis is performed; at the same time, the binary secret information M is error-corrected and coded using Gray code to obtain coded information; then, according to the embedding position determined by PCA feature analysis, the coded information is embedded into the initial latent vector through the mapping function f. In the randomly selected layers, a random sampling vector is obtained;

[0014] Step 3: Initial hidden vector Mix it with the random sampling vector obtained in step 2, input the mixed vector layer by layer into the pre-trained StyleGAN2 generator, and use the pre-trained weights to generate a encrypted image; Here, the StyleGAN2 generator is equipped with a Mapping Network containing 8 layers of FC, which maps the mixed vector to space, and then the style vectors corresponding to the number of network layers The series is then sent to each layer of AdaIN for style processing. space;

[0015] The specific process of the information extraction stage is as follows:

[0016] The secret image transmitted through the lossy channel is input into the decoder. The decoder and encoder share weight parameters. After the style vector is obtained, it is decoded through the mapping function f to obtain the binary secret information.

[0017] Using contrastive learning loss function To align the reference image and the corresponding latent vector .

[0018] Furthermore, in the step 1, the encoder based on the three-layer feature pyramid network FPN first decomposes the reference image into feature maps of different scales step by step, and then divides all the feature maps into three groups, which are used to capture coarse image details, medium image details and fine image details respectively; the feature map of each scale is processed by the map2style module to generate a style vector, and the map2style module uses a convolution operation with a step size of 2 to gradually reduce the spatial size of the feature map, and performs a nonlinear transformation through a Leaky ReLU activation function; finally, each feature map generates a 512-dimensional style vector.

[0019] The encoder in this invention adopts a multi-level feature pyramid structure, divides the features into three levels: coarse, medium, and fine, and generates style vectors of different scales through the map2style module to match the hierarchical representation structure of the StyleGAN generator. This multi-scale feature extraction architecture effectively alleviates the bottleneck limitation of a single encoding architecture, and this improvement improves the encoder's ability to retain detail information.

[0020] Furthermore, the specific method of encoding secret information using Gray code is as follows:

[0021] First, split the binary secret information stream into Each fragment Converted to Gray code, the formula is:

[0022] ;

[0023] It refers to Gray code, It refers to Bit binary code;

[0024] Then, the encoded value is mapped to the noise vector , the calculation formula is:

[0025] ;

[0026] Finally, the encoded secret information is mapped into the principal component vector, thus achieving efficient and stable transmission of the secret information.

[0027] Furthermore, for the initial latent vector The specific method for PCA feature analysis is:

[0028] First, vector sampling is performed to sample a set of random latent vectors from the latent space , ,..., , N is the number of latent vectors in the latent vector group;

[0029] Then, the covariance matrix is ​​calculated, the sampled latent vector set is standardized, and the covariance matrix C of these latent vectors is calculated; the covariance matrix reflects the linear correlation between latent vectors and is used to reveal the main direction of change in the latent space;

[0030] ;

[0031] in, represents the average value of all latent vectors;

[0032] Next, perform eigendecomposition, that is, perform eigenvalue decomposition on the obtained covariance matrix C to obtain a set of eigenvectors (i.e., PCA principal components) and corresponding eigenvalues. The eigenvectors represent the main change directions in the latent space, and the eigenvalues ​​represent the variance size in each direction.

[0033] Finally, the principal components are selected. According to the size of the eigenvalues, the first K eigenvectors are selected as the principal component vectors. Captures the initial latent vector The main semantic changes in .

[0034] PCA shows that the implicit distribution structure of StyleGAN2 is relatively simple: the principal coordinates are almost independent variables with non-Gaussian unimodal distribution. The first 100 principal components are sufficient to describe the overall appearance of the image, and the remaining 412 dimensions control subtle changes in the appearance. Therefore, choosing PCA to locate the encoding position of secret information can not only reduce the impact on image quality during information transmission, but also improve the stability of secret information transmission.

[0035] Furthermore, the total loss function for training the encoder based on the three-layer feature pyramid network FPN is The expression is as follows:

[0036] ;

[0037] in, Pixel level loss, which is used to minimize the pixel difference between the input image and the generated image, represents the original input image, , , is the corresponding weight coefficient;

[0038] ;

[0039] LPIPS perceptual loss is used to maintain perceptual similarity and better preserve image quality than standard perceptual loss. refers to the image generated by the encoder;

[0040]

[0041] To regularize the loss function, the encoder is encouraged to output a style vector close to the average style vector, which is similar to the truncation technique in StyleGAN and helps improve image quality. F is the feature representation.

[0042] .

[0043] Furthermore, the contrastive learning loss function The details are as follows:

[0044] ;

[0045] ;

[0046] ;

[0047] Among them, ⟨·> represents cosine similarity, t is a learnable temperature parameter, and Represent the contrastive learning loss from image space to latent space and from latent space to image space respectively; is an image Through feature extractor The feature representation obtained after mapping with the projection head, is an image Through feature extractor The feature representation obtained after mapping with the projection head; λ is the corresponding weight parameter, S refers to the number of images and latent space vectors in the minibatch (the number of training at a time, the number of samples is large and cannot be trained at the same time); k is the index used to mark the kth sample in the minibatch data (minibatch).

[0048] Beneficial effects: The present invention generates high-quality encrypted images through multi-scale features while maintaining the visual authenticity of the image. Compared with the prior art, the present invention has the following advantages:

[0049] 1. The present invention converts the input real image into a series of style vectors through a three-layer feature pyramid structure encoder, and The steganographic operation is performed in the space, which effectively improves the steganographic capacity. And through random sampling and style vector mixing, a variety of secret images are generated.

[0050] 2. The present invention also analyzes the latent distribution of StyleGAN2 through PCA to determine the main semantic changes of the image in the form of latent vectors, thereby enhancing the robustness of the steganalytic information and improving the extraction accuracy.

[0051] 3. This invention optimizes the semantic consistency between the generated image and the latent space by introducing the CLIP contrastive learning loss function, further improving the accuracy of information extraction. Even in the face of complex generated images, the steganographic information can still be accurately restored, greatly improving the reliability of extraction.

[0052] 4. The present invention significantly enhances the anti-detection ability of steganographic images. By learning the statistical characteristics of natural images, the generated steganographic images do not destroy the statistical characteristics of the images and effectively resist the detection of steganalysis tools. Without relying on existing carrier images, the generated images are almost the same as natural images.

[0053] In summary, the present invention can adapt to more types of data while improving the steganographic capacity, extraction accuracy and anti-detection ability, and has strong practicality and technical advantages. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 It is the overall architecture diagram of the present invention;

[0055] Figure 2 Schematic diagram of feature pyramid encoder in the present invention;

[0056] Figure 3 It is a schematic diagram of the comparative learning process in the present invention;

[0057] Figure 4 Schematic diagram of the encrypted image generation effect in the embodiment. DETAILED DESCRIPTION

[0058] The technical solution of the present invention is described in detail below, but the protection scope of the present invention is not limited to the embodiments.

[0059] like Figure 1 As shown, the generative steganography method based on extended latent space and contrastive learning of the present invention includes an information hiding stage and an information extraction stage;

[0060] The specific process of the hidden information stage is as follows:

[0061] Step 1: Input the reference image into the encoder based on the three-layer feature pyramid network FPN, and generate a series of style vectors through the encoder , where the style vector As the initial latent vector ;

[0062] Step 2: Initial latent vector PCA feature analysis is performed; at the same time, the binary secret information M is error-corrected and coded using Gray code to obtain coded information; then, according to the embedding position determined by PCA feature analysis, the coded information is embedded into the initial latent vector through the mapping function f. In the randomly selected layers, a random sampling vector is obtained;

[0063] Step 3: Initial hidden vector Mix it with the randomly sampled vector obtained in step 2, input the mixed vector layer by layer into the pre-trained StyleGAN2 generator, and use the pre-trained weights to generate a encrypted image;

[0064] The specific process of the information extraction stage is as follows:

[0065] The secret image transmitted through the lossy channel is input into the decoder. The decoder and encoder share weight parameters. After the style vector is obtained, it is decoded through the mapping function f to obtain the binary secret information.

[0066] Using contrastive learning loss function To align the reference image and the corresponding latent vector .

[0067] In step 1 of the embodiment, the encoder based on the three-layer feature pyramid network FPN first gradually decomposes the reference image into feature maps of different scales, and then divides all feature maps into three groups, which are used to capture coarse image details, medium image details and fine image details respectively;

[0068] The feature map of each scale is processed by the map2style module to generate a style vector. The map2style module uses a convolution operation with a step size of 2 to gradually reduce the spatial size of the feature map, and performs a nonlinear transformation through the Leaky ReLU activation function; ultimately, each feature map generates a 512-dimensional style vector.

[0069] The specific method of encoding secret information using Gray code in this embodiment is as follows:

[0070] First, split the binary secret information stream into Each fragment Converted to Gray code, the formula is:

[0071] ;

[0072] It refers to Gray code, It refers to Bit binary code;

[0073] Then, the encoded value is mapped to the noise vector , the calculation formula is:

[0074]

[0075] Finally, the encoded secret information is mapped into the principal component vector, thus achieving efficient and stable transmission of the secret information.

[0076] In this embodiment, the initial latent vector The specific method for PCA feature analysis is:

[0077] First, vector sampling is performed to sample a set of random latent vectors from the latent space , ,..., , N is the number of latent vectors in the latent vector group;

[0078] Then, the covariance matrix is ​​calculated, the sampled latent vector set is standardized, and the covariance matrix C of these latent vectors is calculated;

[0079]

[0080] in, represents the average value of all latent vectors;

[0081] Next, perform eigendecomposition and eigenvalue decomposition on the obtained covariance matrix C to obtain a set of eigenvectors and corresponding eigenvalues. The eigenvectors represent the main change directions in the latent space, and the eigenvalues ​​represent the variance size in each direction.

[0082] Finally, the principal components are selected. According to the size of the eigenvalues, the first K eigenvectors are selected as the principal component vectors. Captures the initial latent vector The main semantic changes in .

[0083] The total loss function of this embodiment for training the encoder based on the three-layer feature pyramid network FPN is The expression is as follows:

[0084] ;

[0085] in, Pixel level loss, which is used to minimize the pixel difference between the input image and the generated image, represents the original input image, , , is the corresponding weight coefficient;

[0086]

[0087] is the LPIPS perception loss, refers to the image generated by the encoder;

[0088]

[0089] is the regularized loss function, F is the feature representation;

[0090] .

[0091] In this embodiment, the contrast learning loss function The details are as follows:

[0092] ;

[0093] ;

[0094] ;

[0095] Among them, ⟨·> represents cosine similarity, is a learnable temperature parameter, and Represent the contrastive learning loss from image space to latent space and from latent space to image space respectively; is an image Through feature extractor The feature representation obtained after mapping with the projection head, is an image Through feature extractor and the feature representation obtained after mapping with the projection head; λ is the corresponding weight parameter, S refers to the number of images and latent space vectors in the mini-batch; k is the index used to mark the kth sample in the mini-batch data.

[0096] To verify the effect of the present invention, a public dataset is used to train and compare the proposed generative steganography model based on extended latent space and contrastive learning. The animal face (AFHQ) dataset consists of 15,000 high-quality images with a resolution of 512×512 pixels. The dataset includes three fields: cats, dogs, and wild animals, each containing 5,000 images.

[0097] In the training process of the extended latent space encoder, the Ranger optimizer is used to train the encoder, and the optimization goal is to minimize the total loss function. When the loss decreases and remains stable, the training is considered to be over. In the contrastive learning process, the Adam optimizer is used to train the image and latent vector encoders following CLIP, and the batch size is set to 256 for training.

[0098] In terms of steganographic performance, the proposed method is evaluated in terms of extraction rate, anti-steganographic analysis ability, stegographic capacity, generated image quality and robustness, and compared with existing classical methods. In terms of security, the current excellent steganalyzer Xu'Net is used. Since the secret image is a generated image, the visual quality is measured by the no-reference image quality as sessment (NR-IQA) tool: hyperIQA. Bpp refers to bits per pixel, and Gaussian noise attack is used for lossy channels. The experimental results are shown in Table 1, which shows that the performance of the present invention in terms of image quality and stegographic security is better than that of the existing steganographic models as a whole, especially in terms of image visual fidelity and stegographic capacity.

[0099] Table 1 Performance comparison of different steganography models

[0100]

[0101] Table 2 Changes of acc and FID when λ is taken into account

[0102] λ 1 2 3 4 5 6 7 acc(%) 99.49 98.77 98.3 97.72 96.13 93.21 88.73 FID 38.99 39.27 38.92 39.81 39.41 38.49 38.89

[0103] This example also verifies the relationship between image quality and steganalysis capacity. The hyperparameter λ represents the number of layers of 18-layer style vectors used in the extended latent space. By adjusting the value of λ, the performance of the model in terms of information extraction accuracy (acc) and generated image quality (FID) is analyzed. In practical applications, the best balance between steganalysis capacity and information extraction accuracy can be found according to specific needs. The experimental results are shown in Table 2.

Claims

1. A generative steganography method based on extended latent space and contrastive learning, characterized in that: It includes the stage of hiding information and the stage of extracting information; The specific process of the hidden information stage is as follows: Step 1: Input the reference image into the encoder based on the three-layer feature pyramid network FPN, and generate a series of style vectors through the encoder. Here, the style vector is used as the initial latent vector ; Step 2: Initial hidden vector PCA feature analysis is performed; at the same time, the binary secret information M is error-corrected and coded using Gray code to obtain coded information; then, according to the embedding position determined by PCA feature analysis, the coded information is embedded into the initial latent vector through the mapping function f. In the randomly selected layers, a random sampling vector is obtained; Step 3: Initial hidden vector Mix it with the randomly sampled vector obtained in step 2, input the mixed vector layer by layer into the pre-trained StyleGAN2 generator, and use the pre-trained weights to generate a encrypted image; The specific process of the information extraction stage is as follows: The secret image transmitted through the lossy channel is input into the decoder. The decoder and encoder share weight parameters. After the style vector is obtained, it is decoded through the mapping function f to obtain the binary secret information. At the same time, a contrastive learning loss function is used to align the reference image and the corresponding latent vector .

2. The generative steganography method based on extended latent space and contrastive learning according to claim 1, characterized in that: In the step 1, the encoder based on the three-layer feature pyramid network FPN first gradually decomposes the reference image into feature maps of different scales, and then divides all the feature maps into three groups, which are used to capture coarse image details, medium image details and fine image details respectively; The feature map of each scale is processed by the map2style module to generate a style vector. The map2style module uses a convolution operation with a step size of 2 to gradually reduce the spatial size of the feature map, and performs a nonlinear transformation through the Leaky ReLU activation function; ultimately, each feature map generates a 512-dimensional style vector.

3. The generative steganography method based on extended latent space and contrastive learning according to claim 1, characterized in that: The specific method of encoding secret information using Gray code is as follows: First, split the binary secret information stream into Each fragment Converted to Gray code, the formula is: ； It refers to Gray code, It refers to Bit binary code; It refers to Bit binary code; Then, the encoded value is mapped to the noise vector , the calculation formula is: ； Each information segment The corresponding encoding value of Finally, the encoded secret information is mapped into the principal component vector.

4. The generative steganography method based on extended latent space and contrastive learning according to claim 1, characterized in that: For the initial latent vector The specific method for PCA feature analysis is: First, vector sampling is performed to sample a set of random latent vectors from the latent space , ,..., , N is the number of latent vectors in the latent vector group; Then, the covariance matrix is ​​calculated, the sampled latent vector set is standardized, and the covariance matrix C of these latent vectors is calculated; ； in, represents the average value of all latent vectors; Next, perform eigendecomposition and eigenvalue decomposition on the obtained covariance matrix C to obtain a set of eigenvectors and corresponding eigenvalues. The eigenvectors represent the main change directions in the latent space, and the eigenvalues ​​represent the variance size in each direction. Finally, the principal components are selected. According to the size of the eigenvalues, the first K eigenvectors are selected as the principal component vectors. , Captures the initial latent vector The main semantic changes in .

5. The generative steganography method based on extended latent space and contrastive learning according to claim 1 or 2, characterized in that: Total loss function for training an encoder based on a three-layer feature pyramid network FPN The expression is as follows: ； in, Pixel level loss, which is used to minimize the pixel difference between the input image and the generated image, represents the original input image, , , is the corresponding weight coefficient; ； is the LPIPS perception loss, refers to the image generated by the encoder; ； is the regularized loss function, F is the feature representation; ； represents the average value of all latent vectors.

6. The generative steganography method based on extended latent space and contrastive learning according to claim 1 or 2, characterized in that: Contrastive Learning Loss Function The details are as follows: ； ； ； in, represents the cosine similarity, is a learnable temperature parameter, and Represent the contrastive learning loss from image space to latent space and from latent space to image space respectively; The image passes through the feature extractor The feature representation obtained after mapping with the projection head, The image passes through the feature extractor and the feature representation obtained after mapping with the projection head; λ is the corresponding weight parameter, S refers to the number of images and latent space vectors in the mini-batch; k is the index used to mark the kth sample in the mini-batch data.

Images