An image content protection method and device based on semantic migration

Abstract

The application discloses a kind of image content protection method and device based on semantic migration, the method is first by training private diffusion model, secret image is encoded into high-dimensional latent semantic vector;Second, the public diffusion model is used to decode the vector, to ensure that the camouflage image has specified semantics;Finally, without destroying the premise of secret image reconstruction semantics, using reversible pixel embedding technology embeds ownership information in secret image.Experimental results show that: the application can effectively protect image content, the peak signal-to-noise ratio of authorized user reconstruction image is improved by 17.27% compared with the contrast method, and unauthorized users cannot completely realize the reconstruction of secret image.In addition, while realizing image content protection, the application can add authentication information, and can realize 100.00% accurate extraction of ownership identification.

Description

Technical Field

[0001] This invention relates to the field of information security technology, and in particular to a method and apparatus for protecting image content based on semantic transfer. Background Technology

[0002] With the rapid development of network technology, data security issues have become increasingly prominent, and the protection of image content is particularly critical. In key areas such as government approval archives, remote medical diagnostic images, and financial transaction vouchers, digital images have become the core carrier of information and interaction, and their security is directly related to the credibility of public services, the life and health rights of citizens, and the stability of the financial market.

[0003] From a security perspective, data storage must ensure that images are not illegally stolen or tampered with in databases or cloud spaces, maintaining long-term data integrity. Data usage urgently requires reliable anti-counterfeiting mechanisms and clear copyright authentication methods to distinguish image authenticity and define ownership. Data transmission requires that image content not be intercepted or deciphered during network transmission. Therefore, constructing an image protection system that balances content confidentiality, reliable anti-counterfeiting measures, and clear ownership has become a pressing research topic.

[0004] Considering the need for network image protection, existing image protection methods can be mainly divided into image privacy protection methods based on active interference (also known as availability attacks) and image information hiding methods represented by steganography. Image privacy protection methods based on active interference can be further divided into global active interference methods and local active interference methods. Image information hiding methods, represented by steganography, can typically hide secret binary sequences or secret images into the carrier image.

[0005] The core of global active perturbation methods is to reduce the availability of data for model training by injecting perturbations to make it difficult for the model to learn effective features from the global data. Existing research on global active perturbation methods is mostly used for image classification, which can be transferred to image content protection. Active perturbation essentially comes at the cost of compromising data availability, and it presents certain challenges in balancing perturbation of fake models with ensuring normal use by humans and legitimate systems. Furthermore, it exhibits certain vulnerabilities against specific defense strategies.

[0006] Image information hiding methods, as a key technology in the field of information security, utilize the visual and statistical redundancy characteristics of images to conceal secret information within the carrier image and achieve covert transmission via public channels, thereby hiding communication content and clandestine communication activities. Image information hiding methods, such as steganography, can provide security for the transmission and storage of images. When the secret information is an image, the corresponding information hiding technique is often referred to as "image hiding image." However, there is still room for improvement in the image restoration accuracy of these methods.

[0007] When the secret information is a binary sequence, these techniques can add authentication information to images or transmit secret information. Early methods, represented by Least Significant Bit (LSB) steganography, embedded secret information by directly replacing the least significant bit of image pixels. While simple to implement and with high embedding capacity, this was easily detected by steganalysis methods. To improve resistance to detection, adaptive steganography emerged. Its core lies in dynamically adjusting the embedding strength based on the local texture complexity of the carrier image. Typical algorithms such as HUGO and S-UNIWARD achieve adaptive embedding by constructing distortion functions, significantly improving steganography security. With the rise of deep learning technology, researchers have proposed many deep learning-based image steganography methods, which make pixel modifications more covert. In recent years, generative steganography has become a new research hotspot. Methods based on Generative Adversarial Networks (GANs) and diffusion models no longer rely on pre-existing carrier images but directly generate steganographic images containing secret information through generative models. Among them, GAN-based generative steganography methods achieve both naturalness and information concealment in generated images through adversarial training between the generator and discriminator; diffusion-based steganography methods encode secret information into the noise distribution of the image through a progressive denoising process, achieving a higher level of imperceptibility and resistance to detection, thus promoting the development of image steganography towards a more secure and intelligent direction. However, the embedding path of these methods with the "image-to-image" framework in terms of authentication information has not yet been fully established. Summary of the Invention

[0008] Existing forgery defense methods based on image camouflage and active interference often suffer from insufficient utilization of latent space, resulting in issues such as poor reconstructed image quality and unclear ownership. To address these problems, this invention proposes an image content protection framework based on semantic transfer. The core idea is to utilize the "semantic isomorphism" of the latent spaces of different diffusion models to first encode sensitive images or information into latent vectors using a user-customized private model, and then decode these vectors into publicly shareable "camouflaged images" using a public model. This process not only removes the traceable connection to the secret image but also eliminates the need to modify the parameters of the public model.

[0009] To achieve the above objectives, the present invention adopts the following technical solution:

[0010] The first aspect of this invention proposes an image content protection method based on semantic transfer, comprising:

[0011] Step 1: In the private model construction stage, the binary sequence is fused with the target image to form a fused image training sample set; the fused image training sample set is used to train the variational autoencoder (VAE) to learn the semantic compression representation of the image; the parameters of the trained variational autoencoder are obtained, and the U-Net network is trained based on the diffusion model architecture to optimize the mapping ability of noise perturbation and denoising process.

[0012] Step 2, in the semantic transfer stage, semantic transfer is achieved by combining the inversion characteristics of the diffusion model: the fused image is inverted and generated using a publicly available diffusion model, and converted into a disguised image that conforms to the public semantics, thereby achieving semantic stripping of the visual domain; an error correction mechanism is designed, and the disguised image is input into the visual language model to generate semantically relevant descriptive text; by selecting tokens with different probabilities in the process of generating descriptive text, the error correction information is hidden in the probability space of the visual language model.

[0013] Step 3: In the semantic recovery stage, the receiver uses the camouflaged image and corresponding descriptive text as input; by reverse parsing the probability distribution of text generation and combining it with an error correction mechanism, the target image and binary sequence are extracted from the cross-modal semantic space.

[0014] Furthermore, in step 1, the binary sequence is fused with the target image in the following manner:

[0015] First, obtain the width of the image. ,high Divide the binary sequence into rows Separate by characters;

[0016] Next, create two blank masks. and These are used to store the text center area and the text area including the boundary, respectively, and are masked in the text center. Draw the text content on top, and add a text boundary mask. The same character is drawn by offsetting it multiple times, forming a boundary area around the character;

[0017] The region containing only text boundaries is obtained through masking operations, and then converted into a Boolean mask.

[0018] Finally, each color channel of the image is traversed, and value increment operations are performed on the pixels covered by the boundary mask. Modulo operations are used to ensure that the pixel values ​​remain within the effective range, thereby achieving the brightness enhancement effect of the boundary region and completing the fusion operation.

[0019] Furthermore, in step 2, the camouflage image is generated in the following manner:

[0020] First, merge the images Preprocessing is performed to obtain a fused image of floating-point type. ;

[0021] Will The VAE of the private model is input for encoding to obtain the latent representation of the private model. ;

[0022] U-Net with proprietary model Using time-step embedding and conditional embedding as input, the recovered latent code is obtained. ;

[0023] U-Net receiver for public models Similarly, by combining time-step embedding and conditional embedding, feature transformation is performed to output the latent representation of the public side. This enables the transformation of semantics from a private latent space to a public latent space;

[0024] VAEs with public models As input, the latent spatial features are mapped back to the image pixel space to obtain a camouflaged image of floating-point type. And convert it into a real image to obtain a disguised image. .

[0025] Furthermore, in step 2, semantically relevant descriptive text is generated in the following manner:

[0026] Disguised image With error correction data A common input visual language model, which is based on error correction data, from candidate tokens Ranking of probability of cluster selection A token of bits Given the total number of tokens, a semantically consistent descriptive text sequence is ultimately generated. .

[0027] Furthermore, the error correction data It consists of the row index of the error data, the column index of the error data, and the error data itself.

[0028] Furthermore, step 2 also includes:

[0029] The fused image is pre-restored, and the difference between the restored image and the original fused image is verified. If information difference is detected, the error correction mechanism is activated.

[0030] Furthermore, the binary sequence includes ownership information.

[0031] A second aspect of the present invention provides an image content protection device based on semantic transfer, comprising:

[0032] A private model building module is used to fuse binary sequences with target images during the private model building stage to form a fused image training sample set; the fused image training sample set is used to train a variational autoencoder (VAE) to learn the semantic compression representation of the image; the parameters of the trained variational autoencoder are obtained, and a U-Net network is trained based on a diffusion model architecture to optimize the mapping capability between noise perturbation and denoising process.

[0033] The semantic transfer module is used to achieve semantic transfer in the semantic transfer stage by combining the inversion characteristics of the diffusion model: it uses the publicly available diffusion model to perform inversion generation on the fused image, converting it into a disguised image that conforms to the public semantics, thereby achieving semantic stripping in the visual domain; it designs an error correction mechanism, inputs the disguised image into the visual language model, and generates semantically relevant descriptive text; by selecting tokens with different probabilities in the process of generating descriptive text, the error correction information is hidden in the probability space of the visual language model.

[0034] The semantic recovery module is used in the semantic recovery stage. The receiver uses the camouflaged image and corresponding descriptive text as input; by reverse parsing the probability distribution of text generation and combining it with an error correction mechanism, the target image and binary sequence are extracted from the cross-modal semantic space.

[0035] A third aspect of the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements an image content protection method based on semantic transfer as described in any of the preceding claims.

[0036] A fourth aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements an image content protection method based on semantic transfer as described in any of the preceding claims.

[0037] Compared with the prior art, the present invention has the following beneficial effects:

[0038] 1) This invention proposes a semantic transfer paradigm that cascades private and public models, incorporating image content protection and copyright authentication mechanisms into the same framework;

[0039] 2) This invention proposes a pixel-reversible multimodal fusion algorithm. By creating a text boundary mask and performing pixel value increment processing on the boundary region, text is embedded in the original image, forming a fused image training dataset. This algorithm can minimize interference with the original image information while ensuring visual recognition, and the original semantics of the target image can be recovered losslessly through reverse operation.

[0040] 3) This invention utilizes a visual language model to construct a cross-modal hidden domain, correcting information loss during semantic transfer;

[0041] 4) This invention utilizes the semantic isomorphism of the latent spaces of different diffusion models to first encode the target image into a latent vector using a user's private model, and then decodes this vector into a publicly shareable camouflaged image using a public model. This invention can effectively protect image content, improving the peak signal-to-noise ratio comparison method for authorized users to reconstruct images by 17.27%, while unauthorized users are completely unable to reconstruct secret images. Attached Figure Description

[0042] Figure 1 A flowchart illustrating an image content protection method based on semantic transfer provided in an embodiment of the present invention;

[0043] Figure 2 This is a schematic diagram of information fusion provided for an embodiment of the present invention;

[0044] Figure 3 This is a schematic diagram of a pre-restored fused image provided in an embodiment of the present invention;

[0045] Figure 4 This is a schematic diagram of error correction data provided in an embodiment of the present invention;

[0046] Figure 5 This is an example of information pre-correction provided in the embodiments of the present invention;

[0047] Figure 6 A schematic diagram of an image content protection device based on semantic transfer provided in an embodiment of the present invention;

[0048] Figure 7 Examples of a fused image containing source tracing information, a corresponding camouflaged image, a restored fused image, and a restored target image provided in embodiments of the present invention;

[0049] Figure 8 This is a visual comparison example between the target image, the camouflaged image, and the restored target image provided in an embodiment of the present invention. Detailed Implementation

[0050] The present invention will be further explained below with reference to the accompanying drawings and specific embodiments:

[0051] like Figure 1 As shown, an image content protection method based on semantic transfer includes:

[0052] To extract semantic information from target images, this invention proposes a semantic transfer paradigm that cascades private and public models. Utilizing the "semantic isomorphism" of the latent spaces of different diffusion models, the target image is first encoded into a latent vector using a user-private customized model, and then decoded into a publicly shareable "camouflaged image" using a public model. The user image or secret image to be protected by the proposed method is referred to as the target image in this invention, and the information used for tracing or covert communication transmission is collectively referred to as a binary sequence. A brief flowchart is as follows:

[0053] In the private model construction phase, binary sequences are fused with target images to form a fused image training sample set. Then, model training is performed in two steps: training variational autoencoders (VAEs) to learn the semantic compression representation of images; after locking the VAE parameters, training a U-Net network based on a diffusion model architecture to optimize the mapping capability between noise perturbation and denoising.

[0054] In the semantic transfer stage, semantic transfer is achieved by combining the inversion characteristics of the diffusion model: the fused image is inverted using a publicly available diffusion model to generate a natural image that conforms to the public semantics, thus achieving semantic stripping from the visual domain. To ensure information integrity, the proposed error correction mechanism inputs the disguised image into the Vision Language Model (VLM) to generate semantically relevant descriptive text. By selecting tokens with different probabilities during text generation, the error correction information is hidden in the probability space of the language model.

[0055] In the semantic recovery phase, the receiver uses the camouflaged image and corresponding descriptive text as input; by reverse parsing the probability distribution of text generation and combining it with an error correction mechanism, the target image and binary sequence are extracted from the cross-modal semantic space.

[0056] Below, we will elaborate on these three stages.

[0057] 1. Construct a private model

[0058] 1.1 Multimodal Information Fusion

[0059] To avoid the interference of secret information with the semantic information of the target image, we propose a pixel-reversible multimodal fusion algorithm. By creating a text boundary mask and performing pixel value increment processing on the boundary region, the text is embedded in the original image, forming a training dataset of fused images. This algorithm can minimize the interference with the original image information while ensuring visual recognizability, and the original semantics of the target image can be recovered losslessly through reverse operation.

[0060] The algorithm first obtains the width of the image. ,high Ownership information Each line is... The characters are split to arrange them on the image; then two blank masks are created. and These are used to store the text center area and the text area including the boundary, respectively, and are masked in the text center. Accurately draw the text content. Use a text boundary mask. The same text is drawn by multiple offsets, forming a boundary region surrounding the text. The offset range is controlled by the boundary width parameter. This process is as follows: Figure 2 As shown. The region containing only the text boundaries is obtained through masking operations (boundary mask minus center mask), and then converted into a Boolean mask for subsequent pixel operations. This process can be formally expressed as:

[0061] (1)

[0062] Here, clip represents the mask subtraction operation. This indicates a pure boundary mask.

[0063] Boolean boundary mask The calculation process is as follows:

[0064] (2)

[0065] Finally, each color channel of the image is traversed, and value increment operations are performed on the pixels covered by the boundary mask. Modulo operations are used to ensure that the pixel values ​​remain within the valid range (0-255), thereby achieving the brightness enhancement effect of the boundary region. This process can be formally represented as:

[0066] (3)

[0067] in, Represents the target image. Indicates a fused image. "mod" indicates the pixel increment, and "mod" indicates the remainder operation.

[0068]

[0069] The above process can be described using Algorithm 1. In the algorithm... Indicates the line height of the font. , Represents the coordinates of the font.

[0070] 1.2 Training a Private VAE

[0071] To hide image content and authentication information, a private VAE was trained using a constructed fused image dataset. This indicates that the VAE contains an encoder. With decoder superscript Indicates private, subscript Indicates encoder, subscript The encoder converts the input image into a private latent representation, while the decoder reconstructs the image from the private latent representation. Through optimization objectives such as reconstruction loss, the VAE learns an effective latent feature representation of the image.

[0072] The goal of VAE is to reconstruct input data by learning the latent representation of the data; its training objective is... This can be formally represented as:

[0073] (4)

[0074] Among them, the first item It is the reconstruction error, representing the error from the latent variables. Reconstruct input The log-likelihood.

[0075] Second item It is the KL divergence, used to regularize the distribution of latent variables, making it closer to the prior distribution. .

[0076] 1.3 Training a Private U-Net

[0077] To enable U-Net to predict noise, a private U-Net needs to be trained. The private U-Net is trained based on the private latent representation generated by the private VAE. The private latent representation is input into the private U-Net, which outputs a noisy latent representation, which is then processed... Restore the image.

[0078] During U-Net training, the VAE is fixed. Its training process can be represented by the following formula:

[0079] (5)

[0080] in, It is the time step of the diffusion process, from 1 to... ; It is noise, sampled from a standard normal distribution; In time step Noisy images; This is the predicted noise of U-Net; the loss function is the mean square error between the predicted noise and the actual noise.

[0081] 2. Semantic transfer

[0082] To extract the semantics from the fused image, the semantic isomorphism of the latent spaces of different diffusion models is utilized. The fused image is first encoded into a latent vector using a user-private custom model, and then decoded into a publicly shareable "masked image" using a public model. The details of semantic transfer will be introduced next.

[0083] 2.1 Generating camouflage images

[0084] To convert a blended image into a camouflaged image, the blended image must first be... Preprocessing using Formula 6 yields a fused image of floating-point type. :

[0085] (6)

[0086] A private VAE undertakes the coded mapping from image to latent space, The VAE of the private model is input for encoding to obtain the latent representation of the private model. This process achieves preliminary encoding of image semantics in a private latent space, preserving core image features while compressing dimensionality. This process can be described as follows:

[0087] (7)

[0088] U-Net with proprietary model As input, combined with time step embedding Conditional embedding To obtain the latent encoding of the recovery This prepares for subsequent semantic conversion with public networks. At this point, the conversion variables between the private and public models are obtained:

[0089] (8)

[0090] The public U-Net receives the noisy latent representation from the output of the private U-Net. Similarly, combining time step embedding Conditional embedding It performs feature transformation. Continuing the classic U-Net encoder-decoder structure, in cross-scale feature fusion, it performs "denoising" feature processing (or semantic reconstruction transformation) on the noisy latent features of the input, further mapping the feature transformation from the private side to the public latent space, and outputting the latent representation of the public side. This process transforms semantics from a private latent space to a public latent space, allowing features to adapt to the subsequent decoding requirements of a public VAE. This process can be described as follows:

[0091] (9)

[0092] Public VAEs are represented by public potentials. As input, the latent spatial features are mapped back to the image pixel space, a process described by Equation 10:

[0093] (10)

[0094] The publicly available VAE model restores the semantic features compressed in the latent space into a visualized camouflage image. This completes the decoding from latent representation to image, allowing the semantically transformed result to be presented in image form, thus realizing a closed-loop process of generating a publicly perceptible image from a secret image semantics through multiple transformations.

[0095] Obtain floating-point image Then, it needs to be converted into a real image according to Formula 11:

[0096] (11)

[0097] This process completes the semantic transfer from the fused image to the camouflaged image.

[0098] 2.2 Pre-restored fused image

[0099] To ensure the accuracy of the secret information in the camouflaged image, the sender needs to pre-recover the fused image to prepare for the error correction mechanism in the next step, as illustrated in the diagram. Figure 3 As shown, the level of resource constraint, such as network and storage constraints, can lead to data loss, thereby affecting the restoration quality of the target image and the restoration accuracy of the binary sequence. Therefore, an appropriate resource constraint level can be selected based on the required restoration accuracy.

[0100] Restricted level It indicates that, from high to low, they are divided into , and As shown in Table 1. When restoring the fused image, the level uses the camouflaged image as input, making it suitable for scenarios with limited storage and network resources; The stage takes the latent representation of the public U-Net output as input, which is suitable for situations where resources such as storage and time are sufficient; The first level uses the latent representation output by the private U-Net as input, making it suitable for situations with ample storage resources and requiring less time. The higher the constraint level, the higher the accumulated error when recovering the fused image. Figure 3 Recovery of losses The loss is represented by a step index; for example, the loss in step 8 is represented as... .

[0101] Table 1 Resource Constraint Levels

[0102]

[0103] When the restricted level is At that time, the recovery path is The camouflage image used to restore the fused image is express. In order to be processed by the network, it needs to be transformed by Equation 12 before being input into the encoder network of VAE to obtain the public latent representation. This process can be described by the following formula:

[0104] (12)

[0105] (13)

[0106] Using the publicly available diffusion model, the U-Net network is used to represent the latent representation. Restore to Then, the private latent representation is recovered using the U-Net of the private model. The process is described by the following formula:

[0107] (14)

[0108] (15)

[0109] Similar to formulas 10 and 11, the recovered target image can be obtained through the VAE of the private model and the transformation formula. :

[0110] (16)

[0111] (17)

[0112] When the restricted level is At the same time, the fusion image restoration process and Similarly, the recovery path is When recovery begins, it is represented by a potential expression. As input, skip formulas 12 and 13, and use formula 14. Replace with As shown in the following formula:

[0113] (18)

[0114] Remaining processes and same.

[0115] When the restricted level is At the same time, the fusion image restoration process and Similarly, the recovery path is This will not be elaborated upon here.

[0116] 2.3 Hide error correction information

[0117] After the sender completes the pre-reconstruction of the fused image, it is necessary to verify whether there are any differences in the binary sequence before and after the conversion. This verification process can be aided by an ORC (Optical Recognition and Correction) model. If information differences are detected, a pre-correction mechanism needs to be activated.

[0118] The pre-correction mechanism needs to disguise the image With error correction data Common Input Visual Language Model The model will select candidate tokens based on error correction data. ( Ranking based on probability of selecting tokens from the total number of tokens. The bit token is ultimately used to generate a semantically consistent descriptive text sequence. The generated text can be publicly published on social networks along with the public images; this process can be formally described using Formula 19.

[0119] (19)

[0120] Error correction data for each erroneous data Indexed by row of erroneous data Column index The error correction data consists of three items: error data, and error data. A schematic diagram is shown below. Figure 4 As shown.

[0121] function Give the number of bits required for each data item. and These represent the maximum values ​​for the row index and column index, respectively. The row index and column index each require... and It is represented by bits of binary data. and This can be formalized as Equations 20 and 21 respectively:

[0122] (20)

[0123] (twenty one)

[0124] For bit-level errors in hexadecimal data, it is necessary to Bit data is used for representation. For character-level data, the determination depends on the specific application scenario; if it only contains letters, then it needs to be represented using... Bits are used to represent this; if numbers are also needed, then... The number of bits is used to represent the error; even if some common punctuation marks are needed, the number of bits required for the error data will not exceed 6 bits. The total number of bits required to indicate the error character is... This is represented by Formula 22.

[0125] (twenty two)

[0126] in, A value of 0 indicates bit-level secret information hiding; a value of 1 indicates character-level secret information hiding.

[0127] Figure 5 Provide an example of hiding ownership information. Suppose the information to be hidden is... = , to disguise images Input visual language model The text description is generated by cyclically selecting the token index corresponding to the secret bit based on the image encoding.

[0128] Figure 5 In the middle, step 0 Based on the input image, two tokens are provided. Since the hidden bit in this step is "0", the 0th token, "Every", is selected. Step 1: Based on the current state... Given two candidate tokens, with the hidden bit set to "1", select the first token "moment". The generated description text will be "Every moment". In the second step, with the hidden bit set to "1", select the first token again. The generated description text will be "Every moment of". Similarly, the complete description text can be generated using this method.

[0129] 3. Semantic Recovery

[0130] Then, following the same steps as the pre-extraction of secret information during transmission, the recovered fused image can be obtained. and the target image.

[0131] Extract error correction information from the description text and camouflage image (if necessary). When extracting error correction information, first initialize an empty error correction information sequence. Next, tokens are extracted sequentially from the description text and compared with the tokens predicted by next. Tokens with the same index are added to the sequence. The error correction information extraction process is shown in Algorithm 2.

[0132]

[0133] in This indicates the number of bits of hidden information for each token. (Next) The function refers to the list of predicted tokens obtained based on the current state.

[0134] Based on the above embodiments, such as Figure 6 As shown, the present invention also proposes an image content protection device based on semantic transfer, comprising:

[0135] A private model building module is used to fuse binary sequences with target images during the private model building stage to form a fused image training sample set; the fused image training sample set is used to train a variational autoencoder (VAE) to learn the semantic compression representation of the image; the parameters of the trained variational autoencoder are obtained, and a U-Net network is trained based on a diffusion model architecture to optimize the mapping capability between noise perturbation and denoising process.

[0136] The semantic transfer module is used to achieve semantic transfer in the semantic transfer stage by combining the inversion characteristics of the diffusion model: it uses the publicly available diffusion model to perform inversion generation on the fused image, converting it into a disguised image that conforms to the public semantics, thereby achieving semantic stripping in the visual domain; it designs an error correction mechanism, inputs the disguised image into the visual language model, and generates semantically relevant descriptive text; by selecting tokens with different probabilities in the process of generating descriptive text, the error correction information is hidden in the probability space of the visual language model.

[0137] The semantic recovery module is used in the semantic recovery stage. The receiver uses the camouflaged image and corresponding descriptive text as input; by reverse parsing the probability distribution of text generation and combining it with an error correction mechanism, the target image and binary sequence are extracted from the cross-modal semantic space.

[0138] The present invention also proposes an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the above-described semantic transfer-based image content protection method.

[0139] The present invention also proposes a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the above-described semantic transfer-based image content protection method.

[0140] To verify the effectiveness of this invention, the following experimental design was conducted:

[0141] 4. Experimental Setup

[0142] 4.1 Experimental Environment

[0143] In the experiments, the public model used was insaneRealistic (https: / / huggingface.co / Yntec / insaneRealistic_v2). There were two types of private models: the first type was models obtained from the community, which required no training; the private models used in the experiments were dreamshaper (https: / / huggingface.co / Lykon / dreamshaper-8). The second type required training, which was conducted using PyTorch 1.11.0 on Ubuntu 20.04, utilizing an NVIDIA A6000 (48GB VRAM) on the FFHQ dataset. FFHQ is a high-quality face image dataset containing 70,000 images at a resolution of 1024. Images at 1024. In the experiment, all images were adjusted to 512. 512. During the training phase, 10,000 images are randomly sampled from the dataset.

[0144] 4.1.2 Evaluation Indicators

[0145] 1) PSNR (Peak Signal to Noise Ratio): It is an objective evaluation metric used to measure the similarity between two images (or signals), and the higher the value, the better.

[0146] 2) SSIM (Structural Similarity Index): This is an index that measures the structural similarity between two images. It evaluates image quality by comparing brightness, contrast, and structural information. The closer the value is to 1, the better.

[0147] 3) MEA (Mean Absolute Error): This metric represents the average absolute error between the fused image and the restored fused image. The smaller the value, the better.

[0148] 4) RMSE (Root Mean Square Error): This metric represents the sample standard deviation of the difference (called residual) between the fused image and the restored fused image. RMSE indicates the dispersion of the sample; a smaller value is better.

[0149] 5) Computational cost: This metric is characterized by the time consumed during the execution process, and the closer the value is to 0, the better.

[0150] 6) Embedding capacity: This metric describes the number of binary sequences that can be hidden in an image; the higher the value, the better.

[0151] 7) Embedding rate: This metric describes the number of bits that can be embedded in 1 pixel; the higher the value, the better.

[0152] 8) Extraction accuracy: This metric describes the ratio of accurately extracted binary sequences to embedded binary sequences; the closer the value is to 1, the better.

[0153] 4.2 Method Performance

[0154] This section presents experiments demonstrating the proposed method's performance in terms of image restoration quality, accuracy, and computational cost under different parameters.

[0155] 4.2.1 Experiment on the quality of fused image restoration

[0156] To verify the proposed method's ability to hide images, this experiment focuses on the quality of image restoration through fusion, exploring different U-Net training steps (15000 and 25000 steps) and DDIM sampling steps. (50, 100, 500, 1000) and restricted levels ( , , The impact of parameters on the restoration results was investigated. The embedding rate in this experiment was 24 bpp. The restoration quality of the fused image under different parameter combinations was measured using four metrics: PSNR, SSIM, MEA, and RMSE. The best results are indicated in bold, and the second best results are indicated in underline.

[0157] Table 2 shows the restoration quality of the fused image when U-Net is 15000.

[0158]

[0159] From the perspective of restricted levels, Regardless of whether the number of training steps in -UNet is 15,000 or 25,000, the overall quality metrics of the recovered data (such as low PSNR and high MEA) are worse than before. , ,illustrate The data loss is even greater. along with Increased, and all quality indicators continued to be optimized.

[0160] Table 3 shows the restoration quality of the fused image when U-Net is 25000.

[0161]

[0162] Comparing different numbers of iterations during U-Net training, with the same level of constraint, As the number of training steps for U-Net increased from 15,000 to 25,000, the metrics improved accordingly (e.g., , At that time, PSNR increased from 18.89 to 19.06, indicating that more training steps are beneficial for improving recovery quality. Overall, the restricted level selection... , Superior , Increasing the number of steps typically improves recovery performance, and increasing the number of U-Net training steps positively promotes recovery quality.

[0163] The experimental results in Tables 2 and 3 show that the image reconstruction quality improves with increasing U-Net training steps. At the same number of sampling steps, the PSNR and SSIM values ​​are generally higher when the training steps are 25,000 than when they are 15,000, while the MEA and RMSE values ​​are generally lower. This indicates that more training steps contribute to improving the model's reconstruction ability. Furthermore, with increasing sampling steps... With the increase in sampling frequency, the image reconstruction quality also improves. This is reflected in the increase in PSNR and SSIM values, and the decrease in MEA and RMSE values. This is likely because more sampling times provide more information, thus helping the model to reconstruct the image better.

[0164] Figure 7 The image presents visual images of the fused image, the corresponding camouflaged image, the restored fused image, and the restored target image. As can be seen from the figure, with... The reduction ( The highest level of image restoration results in higher image quality and improved restoration accuracy.

[0165] In summary, increasing the number of training steps for U-Net and the number of sampling times for DDIM both contribute to improving the reconstruction quality of the fused image. , This approach yields the best results. Unless otherwise specified, subsequent experiments will use the parameters set described above.

[0166] 4.2.3 Cost Calculation Experiment

[0167] To investigate the computational cost of the proposed method, this experiment explores different parameters. The changes in camouflage image conversion time, target image recovery time, and overall migration time were observed under values ​​of 50, 100, 500, and 1000. The experiment observed the changes in these three time indicators as... The numerical fluctuations were used to measure the computational cost of the proposed framework when performing related operations. The experimental results are shown in the table below.

[0168] Table 4. Cost Calculation Experiment (seconds)

[0169]

[0170] From the results, with As the length increases, the conversion time of the camouflage image, the recovery time of the target image, and the overall migration time all show an upward trend. For example... As the number of vectors increased from 50 to 1000, the time to convert the camouflage image increased from 13.33 to 262.30, the target image recovery time increased from 13.30 to 262.26, and the overall migration time increased from 26.63 to 524.56. This indicates that... The larger the value, the greater the computational load and the longer the time required for the proposed method to perform transformation and restoration operations, reflecting the limitations of the parameter. The significant impact on computational cost, with computational resource consumption increasing accordingly. Increased as a result of improvement.

[0171] Further analysis reveals that the conversion time of the camouflage image and the recovery time of the target image are quite similar, indicating that the computational complexity of these two operations is comparable in the proposed method. The overall migration time is the sum of the two, and its growth trend is consistent with the former two, further verifying that the computational cost increases with... The pattern of increasing and accumulating is evident. This provides a reference for the proposed methods in practical applications. However, if high computational efficiency is required, reasonable control is necessary. The value of is chosen to balance functional requirements and computational cost.

[0172] 4.3 Comparative Experiment

[0173] This section presents comparative experiments between the proposed method and existing methods. In image preservation, we compare the quality of the recovered image and the embedding rate with methods where the secret information is the image, such as HiDDeN, Baluja, and DiffStega. Regarding ownership information, we compared embedding capacity and load rate using methods such as TransStego, WGAN-Steg [Jun Li, Ke Niu, Liwei Liao, Lijie Wang, Jia Liu, Yu Lei, and Minqing Zhang. A Generative Steganography Method Based on WGAN-GP. In Proceedings of International Conference on Artificial Intelligence and Security (ICAIS), pp. 386–397, 2020], IDEAS, PARIS, and GDA-Steg [Fei Peng, Guanfu Chen, and Min Long. A Robust Coverless Steganography Based on Generative Adversarial Networks and Gradient Descent Approximation. IEEE Transactions on Circuits and Systems for Video Technology, 2022, 32(9): 5817-5829], where the secret information is a binary sequence.

[0174] 4.3.1 Comparison Experiment of Restored Image Quality

[0175] To verify the restoration quality of the fused images, we compared them with methods such as HiDDeN, Baluja, and DiffStega. The experimental results are shown in Table 5.

[0176] Experimental data show that as As the number of cases increases, the advantages of the proposed method become increasingly apparent. When the peak signal-to-noise ratio (PSNR) reaches 1000, its PSNR is as high as 37.82, which is much higher than HiDDeN's 31.80 and Baluja's 32.25. This means that the restored fused image retains a quality closer to the original image in terms of peak signal-to-noise ratio. The SSIM reaches 0.9731, which is not only better than HiDDeN's 0.8957 and Baluja's 0.9448, but also close to 1. This indicates that the distortion caused by image structure, texture and other information is minimal, and the similarity between the image and the original state is maintained to a great extent.

[0177] Table 5. Restoration quality of fused images

[0178]

[0179] Meanwhile, the MEA is as low as 1.86 and the RMSE is only 3.58. Compared with the comparison method, these two indicators reflecting pixel error are significantly lower, demonstrating that the proposed method effectively suppresses interference at the image pixel level.

[0180] also, Figure 8 This paper presents a comparison of the proposed framework's performance in generating camouflaged images from target images with baseline methods, as well as its ability to recover target images from camouflaged images. Experiments compare it with various methods including HiDDeN, HiNet, Baluja, CRoSS, and DiffStega. Using the target image as the ideal recovery standard, the visual similarity and distortion level of the recovered target image are compared to determine the algorithm's performance: the closer the recovered image is to the target image, the better the recovery effect of the corresponding method.

[0181] Observations reveal that the target images restored by Hidden, HiNet, and BalujaNet exhibit significant texture distortion compared to the original target images (e.g., the Hidden restored image shows a mottled appearance, while the HiNet restored image displays abnormal textures such as freckles). The expression in the CRoSS method is altered. Although the DiffStega restored image shows differences, it is closer to the target than the aforementioned algorithms. Our method, however, better preserves the details, colors, and textures of the target image when restoring the secret image, demonstrating superior accuracy in image hiding and extraction tasks.

[0182] 4.3.2 Embedding Capacity Comparison Experiment

[0183] To verify the performance of the proposed method in terms of embedding capacity, we tested the embedding capacity performance of different methods when processing different types of messages (binary sequences, secret images), with embedding capacity and load as the metrics. The experiments covered methods such as TransStego, WGAN-Steg, IDEAS, PARIS, and GDA-Steg with binary sequences as secret information, and methods such as HiDDeN, Baluja, and DiffStega with images as secret information. The experimental results are shown in Table 6.

[0184] Table 6. Experimental Results of Embedding Capacity Comparison

[0185]

[0186] The results show that among the methods where the secret information is a binary sequence, the load rate varies significantly, with TransStego exhibiting the lowest load rate. PARIS and GDA-Steg have relatively high load factors (reaching 100%). This reflects the differences in binary sequence embedding load among different algorithms. Methods where the secret information is an image typically have a load rate of 24 bpp, indicating that these methods perform similarly in image embedding capacity, perhaps limited by the characteristics of the image itself and general embedding frameworks.

[0187] The method proposed in this invention can hide binary streams and images simultaneously. The embedding capacity of binary streams increases with the number of characters (e.g., 6×11, 9×16), and the load rate is greater than 24bpp. This demonstrates that the method can break through the capacity limit of pure image methods under mixed message types. Furthermore, parameter adjustment can optimize the embedding performance, indicating that its design has advantages in improving embedding capacity. It provides a more promising solution for anti-forgery and steganography in multi-type message embedding scenarios, and also reflects the role of algorithm innovation in expanding embedding capabilities.

[0188] 4.3.3 Comparison Experiment of Extraction Accuracy

[0189] To verify the accuracy and performance of the proposed method in extracting binary sequences, our method was compared with TransStego, ERGI, IDEAS, IDEAS and PARIS, and the experimental results are shown in Table 7.

[0190] Table 7 Comparison of Extraction Accuracy

[0191]

[0192] As shown in Table 7, the different methods exhibit significant differences in extraction accuracy. The TransStego method performs best at 400... At an image size of 400, it achieved an accuracy of 96.26%, which is relatively good. The ERGI method at 512... Achieving 98.05% accuracy at an image size of 512 demonstrates high precision. The IDEAS method achieves similar accuracy at 256... The PARIS method achieved 100% accuracy at an image size of 256, while at 64... Both methods achieved 100% accuracy at an image size of 64, demonstrating excellent performance and perfect accuracy at their respective image sizes. The method proposed in this invention achieves this accuracy regardless of the image size. 11 or 9 With 16 characters, the total is 512. Achieving 100% accuracy at an image size of 512 demonstrates the method's high precision and stability when handling large images. Overall, while different methods exhibit varying performance across different image sizes, the proposed method demonstrates superior performance at larger image sizes, showcasing its potential and advantages in binary sequence hiding tasks.

[0193] In summary, to protect the security of image content and assign verifiable identities to masquerading images, this invention proposes an image content protection framework based on semantic transfer. This framework utilizes the "semantic isomorphism" of the latent spaces of different diffusion models. The target image is first encoded into a latent vector using a user-private customized model, and then decoded into a publicly shareable "masquerading image" using a public model. Experimental results show that the proposed method reconstructs images with 17.27% higher quality than existing methods and accurately recovers authentication information.

[0194] The above description is only a preferred embodiment of the present invention. It should be noted that those skilled in the art can make several improvements and modifications without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for image content protection based on semantic transfer, characterized in that, include: Step 1: In the private model construction stage, the binary sequence is fused with the target image to form a fused image training sample set; the fused image training sample set is used to train the variational autoencoder (VAE) to learn the semantic compression representation of the image; the parameters of the trained variational autoencoder are obtained, and the U-Net network is trained based on the diffusion model architecture to optimize the mapping ability of noise perturbation and denoising process. Step 2, in the semantic transfer stage, semantic transfer is achieved by combining the inversion characteristics of the diffusion model: the fused image is inverted and generated using a publicly available diffusion model, and converted into a disguised image that conforms to the public semantics, thereby achieving semantic stripping of the visual domain; an error correction mechanism is designed, and the disguised image is input into the visual language model to generate semantically relevant descriptive text; by selecting tokens with different probabilities in the process of generating descriptive text, the error correction information is hidden in the probability space of the visual language model. Step 3: In the semantic recovery stage, the receiver uses the camouflaged image and corresponding descriptive text as input; by reverse parsing the probability distribution of text generation and combining it with an error correction mechanism, the target image and binary sequence are extracted from the cross-modal semantic space.

2. The image content protection method based on semantic transfer according to claim 1, characterized in that, In step 1, the binary sequence is fused with the target image in the following manner: First, obtain the width of the image. ,high Divide the binary sequence into rows Separate by characters; Next, create two blank masks. and These are used to store the text center area and the text area including the boundary, respectively, and are masked in the text center. Draw the text content on top, and add a text boundary mask. The same character is drawn by offsetting it multiple times, forming a boundary area around the character; The region containing only text boundaries is obtained through masking operations, and then converted into a Boolean mask. Finally, each color channel of the image is traversed, and value increment operations are performed on the pixels covered by the boundary mask. Modulo operations are used to ensure that the pixel values ​​remain within the effective range, thereby achieving the brightness enhancement effect of the boundary region and completing the fusion operation.

3. The image content protection method based on semantic transfer according to claim 1, characterized in that, In step 2, the camouflage image is generated in the following manner: First, merge the images Preprocessing is performed to obtain a fused image of floating-point type. ; Will The VAE of the private model is input for encoding to obtain the latent representation of the private model. ; U-Net with proprietary model Using time-step embedding and conditional embedding as input, the recovered latent code is obtained. ; U-Net receiver for public models Similarly, by combining time-step embedding and conditional embedding, feature transformation is performed to output the latent representation of the public side. This enables the transformation of semantics from a private latent space to a public latent space; VAEs with public models As input, the latent spatial features are mapped back to the image pixel space to obtain a camouflaged image of floating-point type. And convert it into a real image to obtain a disguised image. .

4. The image content protection method based on semantic transfer according to claim 1, characterized in that, In step 2, semantically relevant descriptive text is generated in the following manner: Disguised image With error correction data A common input visual language model, which is based on error correction data, from candidate tokens Ranking of probability of cluster selection A token of bits Given the total number of tokens, a semantically consistent descriptive text sequence is ultimately generated. .

5. The image content protection method based on semantic transfer according to claim 4, characterized in that, The error correction data It consists of the row index of the error data, the column index, and the error data itself.

6. The image content protection method based on semantic transfer according to claim 1, characterized in that, Step 2 also includes: The fused image is pre-restored, and the difference between the restored image and the original fused image is verified. If information difference is detected, the error correction mechanism is activated.

7. The image content protection method based on semantic transfer according to claim 1, characterized in that, The binary sequence includes ownership information.

8. An image content protection device based on semantic transfer, characterized in that, include: A private model building module is used to fuse binary sequences with target images during the private model building stage to form a fused image training sample set; the fused image training sample set is used to train a variational autoencoder (VAE) to learn the semantic compression representation of the image; the parameters of the trained variational autoencoder are obtained, and a U-Net network is trained based on a diffusion model architecture to optimize the mapping capability between noise perturbation and denoising process. The semantic transfer module is used to achieve semantic transfer in the semantic transfer stage by combining the inversion characteristics of the diffusion model: it uses the publicly available diffusion model to perform inversion generation on the fused image, converting it into a disguised image that conforms to the public semantics, thereby achieving semantic stripping in the visual domain; it designs an error correction mechanism, inputs the disguised image into the visual language model, and generates semantically relevant descriptive text; by selecting tokens with different probabilities in the process of generating descriptive text, the error correction information is hidden in the probability space of the visual language model. The semantic recovery module is used in the semantic recovery stage. The receiver uses the camouflaged image and corresponding descriptive text as input; by reverse parsing the probability distribution of text generation and combining it with an error correction mechanism, the target image and binary sequence are extracted from the cross-modal semantic space.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements an image content protection method based on semantic transfer as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements an image content protection method based on semantic transfer as described in any one of claims 1 to 7.

Images