Image watermark embedding method and device, electronic equipment and storage medium

Abstract

The application provides an image watermark embedding method and device, electronic equipment and storage medium, and relates to the technical field of image processing. Wherein, the original image, the text semantic information of the original image and the watermark of the original image are acquired; the original image, the text semantic information and the watermark are input into an image watermark embedding model to acquire a watermark image output by the image watermark embedding model. Through fusing the local feature and the second global feature enhanced by the text, and combining the cross attention mechanism, the application solves the problem of low watermark invisibility and poor robustness caused by ignoring global semantic information in the existing image watermark embedding, and improves the coordination of the embedded watermark and the original image.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to an image watermark embedding method, apparatus, electronic device, and storage medium. Background Technology

[0002] Image watermarking, as a core technology for copyright protection and information traceability, embeds copyright information imperceptibly into images. Even if an image has been maliciously edited, its authenticity and ownership can still be verified by extracting the watermark. Traditional image watermarking methods are mainly based on spatial or frequency domain features and rely on manually designed embedding strategies, making it difficult to comprehensively capture complex features, thus rendering the watermark vulnerable to diverse attacks.

[0003] In recent years, given the automatic feature extraction and end-to-end training capabilities of deep neural networks, image watermarking methods based on deep learning have rapidly become a research hotspot. These methods are primarily built upon Convolutional Neural Networks (CNNs). Due to the characteristics of their local receptive fields, they can effectively extract detailed and texture features of images for watermark embedding, demonstrating outstanding performance in improving visual concealment and robustness against local attacks. However, CNNs struggle to effectively model long-range dependencies and cannot utilize the overall spatial relationships and semantic information of the image. This often results in watermarks being embedded in semantically irrelevant non-subject regions, making them vulnerable to attacks. Furthermore, during watermark embedding, the lack of understanding of the overall structure may disrupt the visual harmony of the image, affecting the watermark's invisibility. Summary of the Invention

[0004] This invention provides a method, apparatus, electronic device, and storage medium for embedding image watermarks, in order to solve the defects of inconsistency between embedded watermarks and original images in the prior art, and to improve the consistency between embedded watermarks and original images.

[0005] This invention provides an image watermark embedding method, comprising: acquiring an original image, textual semantic information of the original image, and a watermark of the original image; inputting the original image, textual semantic information, and watermark into an image watermark embedding model to acquire a watermark image output by the image watermark embedding model; wherein, the image watermark embedding model extracts a first global feature and local features of the original image, performs cross-attention fusion on the first global feature and textual semantic information to obtain a second global feature; performs cross-attention fusion on the second global feature and local features to obtain a fused feature; and generates a watermark image based on the fused feature, the watermark feature, and the original image; the image watermark embedding model is obtained through end-to-end training and adversarial training based on sample original images, sample textual semantic information, and sample watermarks, and the watermark feature is obtained after preprocessing the watermark.

[0006] According to the image watermark embedding method provided by the present invention, the first global feature is obtained based on the following steps: extracting features from the original image to obtain the original image features; generating a multi-head query vector, a multi-head key vector, and a multi-head value vector based on the original image features; performing multi-head attention fusion on the multi-head query vector, the multi-head key vector, and the multi-head value vector; and adding the multi-head attention fusion result to the original image features to obtain the first global feature.

[0007] According to the image watermark embedding method provided by the present invention, the second global feature is obtained based on the following steps: generating a semantic key vector and a semantic value vector based on text semantic information; generating a query vector of the first global feature; performing a first cross-attention fusion on the query vector of the first global feature, the generated semantic key vector and the semantic value vector, adding the result of the first cross-attention fusion to the first global feature to obtain a global semantic feature; and performing feature enhancement on the global semantic feature to obtain the second global feature.

[0008] According to the image watermark embedding method provided by the present invention, the fusion feature is obtained based on the following steps: using local features as query vectors and second global features as key vectors and value vectors, a second cross-attention fusion is performed on the local features and the second global features; the result of the second cross-attention fusion is added to the local features to obtain local enhanced features; using the second global features as query vectors and local features as key vectors and value vectors, a third cross-attention fusion is performed on the local features and the second global features; the result of the third cross-attention fusion is added to the second global features to obtain global enhanced features; the local enhanced features and global enhanced features are fused to obtain the fusion feature.

[0009] According to the image watermark embedding method provided by the present invention, the watermarked image is generated based on the following steps: concatenating the fused features and the preprocessed watermark features in the channel dimension, performing multiple convolutions and skip connections on the concatenation result to obtain the watermarked image features; fusing and convolving the watermarked image features with the original image to obtain the watermarked image.

[0010] According to the image watermark embedding method provided by the present invention, the image watermark embedding model is trained based on the following steps: obtaining training samples based on the original sample image, sample text semantic information, and sample watermark; inputting the training samples into a preset model to obtain the predicted watermark image output by the encoder of the preset model; the preset model includes an encoder, a discriminator, and a decoder; the discriminator is used to distinguish between the predicted watermark image and the original sample image, and the encoder is optimized based on the discriminant loss of the discriminator; obtaining the discriminant error of the encoder based on the predicted watermark image and the original sample image; obtaining the first mean square error between the predicted watermark image and the original sample image; subjecting the predicted watermark image to noise attack to obtain a noisy image; decoding the noisy image based on the decoder to obtain a decoded watermark; obtaining the second mean square error between the decoded watermark and the sample watermark; weightedly fusing the discriminant error, the first mean square error, and the second mean square error of the encoder to obtain the loss function of the preset model; training the preset model based on the training samples until the loss value of the loss function of the preset model is less than the loss threshold, and ending the training to obtain the image watermark embedding model.

[0011] According to the image watermark embedding method provided by the present invention, the text semantic information is obtained based on the following steps: inputting the text description of the original image into the contrastive language-image pre-training model to obtain the initial text semantic information output by the contrastive language-image pre-training model; wherein, the contrastive language-image pre-training model performs fixed-dimensional text feature extraction on the text description to obtain the initial text semantic information; and performs multi-level linear transformation on the initial text semantic information to obtain the text semantic information.

[0012] This invention also provides an image watermark embedding device, comprising: an acquisition module for acquiring an original image, textual semantic information of the original image, and a watermark of the original image; and a watermark embedding module for inputting the original image, textual semantic information, and watermark into an image watermark embedding model to acquire a watermark image output by the image watermark embedding model; wherein the image watermark embedding model extracts a first global feature and a local feature of the original image, performs cross-attention fusion on the first global feature and the textual semantic information to obtain a second global feature; performs cross-attention fusion on the second global feature and the local feature to obtain a fused feature; and generates a watermark image based on the fused feature, the watermark feature, and the original image; the image watermark embedding model is obtained through end-to-end training and adversarial training based on sample original images, sample textual semantic information, and sample watermarks, and the watermark feature is obtained after preprocessing the watermark.

[0013] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement any of the above-described image watermark embedding methods.

[0014] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-described image watermark embedding methods.

[0015] This invention provides an image watermark embedding method, apparatus, electronic device, and storage medium. The method involves acquiring an original image, its textual semantic information, and a watermark. The original image, textual semantic information, and watermark are input into an image watermark embedding model to obtain a watermarked image output by the model. The image watermark embedding model extracts first global features and local features from the original image. It then performs cross-attention fusion on the first global features and textual semantic information to obtain a second global feature. Cross-attention fusion is performed on the second global feature and local features to obtain a fused feature. Based on the fused feature, the watermark feature, and the original image, a watermarked image is generated. The image watermark embedding model is obtained through end-to-end training and adversarial training based on sample original images, sample textual semantic information, and sample watermarks. The watermark feature is obtained after preprocessing the watermark. This invention solves the problems of low watermark invisibility and poor robustness caused by neglecting global semantic information in existing image watermark embedding methods by fusing local features with text-enhanced second global features and combining a cross-attention mechanism, thus improving the coordination between the embedded watermark and the original image. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0017] Figure 1 This is one of the flowcharts illustrating the image watermark embedding method provided by the present invention.

[0018] Figure 2 This is the second flowchart illustrating the image watermark embedding method provided by the present invention.

[0019] Figure 3 This is a schematic diagram of the process for generating the second global feature provided by the present invention.

[0020] Figure 4 This is a schematic diagram of the process for generating fusion features provided by the present invention.

[0021] Figure 5 This is a schematic diagram of the image watermark embedding device provided by the present invention.

[0022] Figure 6 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0024] The following is combined with Figures 1-6 The present invention describes an image watermarking embedding method, apparatus, and electronic device.

[0025] Figure 1 This is one of the flowcharts illustrating the image watermark embedding method provided by the present invention, such as... Figure 1 and Figure 2 As shown, the image watermark embedding method includes steps S100 and S200, and the specific steps are as follows.

[0026] S100: Obtain the original image, the textual semantic information of the original image, and the watermark of the original image.

[0027] Obtain the original image. Extract features from the text description of the original image to obtain semantic information. Obtain the watermark that the user wants to add to the original image.

[0028] S200: Input the original image, text semantic information, and watermark into the image watermark embedding model, and obtain the watermarked image output by the image watermark embedding model.

[0029] The image watermark embedding model extracts the first global feature and local feature of the original image, and performs cross-attention fusion on the first global feature and text semantic information to obtain the second global feature; it then performs cross-attention fusion on the second global feature and local feature to obtain the fused feature; based on the fused feature, watermark feature and original image, a watermark image is generated; the image watermark embedding model is obtained by end-to-end training and adversarial training based on the original sample image, sample text semantic information and sample watermark, and the watermark feature is obtained after preprocessing the watermark.

[0030] Based on the original sample image, sample text semantic information, and sample watermark, an image watermark embedding model is obtained through end-to-end training and adversarial training on a pre-defined model. The pre-defined model includes an encoder, a discriminator, and a decoder.

[0031] The trained image watermarking embedding model includes an encoder. The encoder comprises a local feature extraction module, a cross-modal global feature extraction module, a multi-scale feature fusion (MSF) module, a watermark processing module, and a watermark image generation module. The image watermarking embedding model uses the Semantic Enhanced Dual Feature Robust Image Watermarking Algorithm (SEDF) to embed the watermark into the original image.

[0032] The original image, textual semantic information, and watermark are input into the image watermark embedding model. The encoder of the image watermark embedding model extracts the original image features through a Convolutional-Batch Normalization-Activation (ConvBNReLU) unit. The original image features are then input into the local feature extraction module and the cross-modal global feature extraction module, respectively. The ConvBNReLU unit consists of a 3×3 convolutional layer, a batch normalization layer, and a ReLU activation function.

[0033] The local feature extraction module focuses on extracting spatial details and texture features from the original image, obtaining local features of the original image. The local feature extraction module includes four cascaded Squeeze-and-Excitation Networks (SENet).

[0034] Textual semantic information and original image features are input together into the cross-modal global feature extraction module. The cross-modal global feature extraction module performs global feature extraction on the original image features to obtain the first global feature. The cross-modal global feature extraction module then performs cross-attention fusion on the textual semantic information and the first global feature to obtain the second global feature.

[0035] The watermarking module reshapes the watermark into a matrix with the same size as the original image, and encodes the reshaped watermark into shallow watermark features through ConvBNReLU units to obtain the preprocessed watermark features.

[0036] For example, given a length of binary watermark and the original image First of all Remodeling into A matrix of uniform size is encoded into shallow watermark features using a ConvBNReLU unit, several transposed convolutional-normalized-activation units (ConvTBNReLU), and SENet. The ConvBNReLU unit consists of a convolutional layer, a batch normalization layer, and a ReLU activation function. The ConvTBNReLU unit consists of a two-dimensional transposed convolutional layer, a batch normalization layer, and a ReLU activation function.

[0037] The watermark image generation module stitches and merges the preprocessed watermark features, fusion features, and the original image to obtain the watermark image.

[0038] The image watermark embedding method provided in this invention involves acquiring an original image, its textual semantic information, and its watermark; inputting the original image, textual semantic information, and watermark into an image watermark embedding model to obtain a watermarked image output by the model; wherein the image watermark embedding model extracts a first global feature and local features from the original image, performs cross-attention fusion on the first global feature and textual semantic information to obtain a second global feature; performs cross-attention fusion on the second global feature and local features to obtain a fused feature; and generates a watermarked image based on the fused feature, the watermark feature, and the original image; the image watermark embedding model is obtained through end-to-end training and adversarial training based on sample original images, sample textual semantic information, and sample watermarks, and the watermark feature is obtained after preprocessing the watermark. This invention, by fusing local features with the text-enhanced second global feature and combining a cross-attention mechanism, solves the problems of low watermark invisibility and poor robustness caused by neglecting global semantic information in existing image watermark embedding methods, thus improving the coordination between the embedded watermark and the original image.

[0039] Based on the above embodiments, text semantic information is obtained by the following steps: inputting the text description of the original image into the contrastive language-image pre-training model to obtain the initial text semantic information output by the contrastive language-image pre-training model; wherein, the contrastive language-image pre-training model performs fixed-dimensional text feature extraction on the text description to obtain the initial text semantic information; and performs multi-level linear transformation on the initial text semantic information to obtain the text semantic information.

[0040] Contrastive Language-Image Pre-Training (CLIP) models include pre-trained and frozen CLIP models.

[0041] Obtain the text description corresponding to the original image. Input the text description into the contrastive language-image pre-trained model. The contrastive language-image pre-trained model extracts fixed-dimensional text features from the text description to obtain initial text semantic information. Input the initial text semantic information into a multi-layer linear layer for multi-layer linear transformation to obtain the final text semantic information.

[0042] This invention obtains initial text semantic information by comparing language-image pre-trained models, and achieves semantic alignment between the original image and the text, which is beneficial for providing higher-level guidance for watermark embedding of the original image.

[0043] Based on the above embodiments, the first global feature is obtained based on the following steps:

[0044] Feature extraction is performed on the original image to obtain its features;

[0045] Generate multi-head query vectors, multi-head key vectors, and multi-head value vectors based on the original image features;

[0046] Multi-head attention fusion is performed on the multi-head query vector, multi-head key vector, and multi-head value vector. The multi-head attention fusion result is added to the original image features to obtain the first global feature.

[0047] The original image features are extracted using ConvBNReLU units. For example... Figure 2 As shown, the cross-modal global feature extraction module includes a Transformer unit. The original image features are input to the Transformer unit after passing through a normalization layer (Norm) and a linear mapping (Linear). The Transformer unit performs layer normalization on the input original image features, and then maps them to multi-head query vectors, multi-head key vectors, and multi-head value vectors through three learnable linear transformations (including query linear transformation, key linear transformation, and value linear transformation). Attention fusion is performed on each head to obtain the fusion result for each head. The fusion results of all heads are concatenated, and a linear mapping is performed on the concatenated result (…). This process yields the multi-head attention fusion result (overall attention value). The multi-head attention fusion result is then added to the original image features to obtain the first global feature (global visual feature).

[0048] ;

[0049] in, For the first The result of the fusion of sizes For linear mapping, Original image features As the first global feature, For multi-head query vectors, For multi-head key vectors, A multi-head value vector, To query linear transformations, For the bond linear transformation, For linear transformation of values, For the first A query vector, For the first A key vector, For the first A vector of values For the number of heads, This is the result of multi-head attention fusion. This is the normalized layer.

[0050] This invention enhances the ability of image watermark embedding models to capture semantic patterns from different subspaces by performing feature extraction and multi-head self-attention fusion on the original image features.

[0051] Based on the above embodiments, the second global feature is obtained based on the following steps:

[0052] Based on textual semantic information, generate semantic key vectors and semantic value vectors;

[0053] Generate the query vector of the first global feature;

[0054] The query vector, generated semantic key vector, and semantic value vector of the first global feature are subjected to the first cross-attention fusion. The result of the first cross-attention fusion is added to the first global feature to obtain the global semantic feature.

[0055] Feature enhancement is performed on the global semantic features to obtain the second global features.

[0056] like Figure 2 As shown, in order to keep the text semantic information and the original image features consistent in terms of spatial resolution and number of channels, the dimension of the text semantic information is adjusted by convolutional projection, and the text semantic information is reshaped into a multi-channel feature map.

[0057] ;

[0058] in, For textual semantic information, For multi-channel feature maps, For text description, To contrast language-image pre-trained models, This is a convolutional projection.

[0059] like Figure 4 As shown, the multi-channel feature map Through linear layer mapping, semantic key vectors and semantic value vectors are obtained. The first global feature is normalized and linearly mapped to obtain its query vector. The query vector, semantic key vector, and semantic value vector of the first global feature are input into the Transformer unit of the cross-modal global feature extraction module for first cross-attention fusion, yielding the first cross-attention fusion result (cross-attention value between text and image). This first cross-attention fusion result is then element-wise added to the first global feature to obtain the global semantic feature. The global semantic feature is input into a feedforward neural network (FNN) and fused with its own global semantic feature through residual connections to obtain the second global feature (global semantic enhancement feature representation). The calculation formulas for the global semantic feature and the second global feature are as follows.

[0060] ;

[0061] in, As the second global feature, For global semantic features, It is a feedforward neural network. The query vector for the first global feature. For semantic key vectors, It is a semantic value vector. As the first global feature, For the number of channels, This is the result of the first cross-attention fusion.

[0062] This invention obtains global semantic features by performing cross-attention fusion on semantic key vectors, semantic value vectors, and query vectors of the first global feature, which can selectively highlight visual regions related to textual semantic information. The second global feature of this invention not only includes global contextual information but also integrates high-level semantic information. It is a key input in the multi-scale feature fusion module, used to supplement local features, which helps to improve the robustness and invisibility of the watermark.

[0063] Based on the above embodiments, the fusion features are obtained based on the following steps:

[0064] The local features are used as the query vector, and the second global features are used as the key vector and value vector. The local features and the second global features are then fused using a second cross-attention method. The result of the second cross-attention fusion is added to the local features to obtain the local enhanced features.

[0065] The second global feature is used as the query vector, and the local features are used as the key vector and value vector. The third cross-attention fusion is performed on the local features and the second global features. The result of the third cross-attention fusion is added to the second global feature to obtain the global enhanced feature.

[0066] The local enhancement features and the global enhancement features are fused to obtain the fused features.

[0067] The local features and the second global features are then input into the multi-scale feature fusion module. The multi-scale feature fusion module uses the local features as the query vector and the second global features as the key and value vectors. It performs a second cross-attention fusion on the query vector, key vector, and value vector, and adds the result of the second cross-attention fusion to the local features to obtain the local enhanced features. The calculation formula for the local enhanced features is as follows.

[0068] ;

[0069] in, For local enhancement features, For local features (query vector). This is the second global feature (key vector and value vector). The number of channels is, This is the result of the second cross-attention fusion.

[0070] The multi-scale feature fusion module uses the second global feature as the query vector and local features as the key and value vectors. It then performs a third cross-attention fusion on the query vector, key vector, and value vector, adding the result of this third cross-attention fusion to the second global feature to obtain the global enhanced feature. The calculation formula for the global enhanced feature is as follows.

[0071] ;

[0072] in, For global feature enhancement, These are local features (key vectors and value vectors). This is the second global feature (query vector). The number of channels is, This is the result of the third cross-attention fusion.

[0073] The local and global enhancement features are fused to obtain the fused feature. The formula for calculating the fused feature is as follows.

[0074] ;

[0075] in, For global feature enhancement, For local enhancement features, This is a feature of fusion.

[0076] This invention obtains fused features by performing cross-attention fusion on local features and second global features, which significantly improves the anti-attack capability of watermarked images while ensuring the visual concealment of the watermark.

[0077] Based on the above embodiments, the watermark image is generated according to the following steps:

[0078] The fusion features and watermark features are concatenated along the channel dimension, and the concatenation result is subjected to multiple convolutions and skip connections to obtain the watermarked image features.

[0079] The watermarked image features are fused and convolved with the original image to obtain the watermarked image.

[0080] The watermarking module reshapes the watermark into a matrix with the same size as the original image, and encodes the reshaped watermark into shallow watermark features through ConvBNReLU units to obtain the preprocessed watermark features.

[0081] The watermarking module concatenates the fused features and the preprocessed watermark features along the channel dimension, and generates watermarked features through several residual blocks (each residual block includes a convolutional layer and a skip connection layer). To preserve the low-dimensional texture and color information of the original image, the watermark features and the original image are fused again through skip connections, and finally, the final watermarked image is generated through a 1×1 convolutional layer.

[0082] This invention generates a watermarked image by fusing fusion features, preprocessed watermark features, and the original image, thereby achieving accurate watermark embedding.

[0083] Based on the above embodiments, the image watermark embedding model is trained using the following steps:

[0084] Training samples are obtained based on the original sample images, sample text semantic information, and sample watermarks.

[0085] The training samples are input into the preset model to obtain the predicted watermark image output by the encoder of the preset model; the preset model includes an encoder, a discriminator and a decoder; the discriminator is used to distinguish between the predicted watermark image and the original sample image, and the encoder is optimized based on the discriminant loss of the discriminator;

[0086] Based on the predicted watermark image and the original sample image, the discrimination error of the encoder is obtained;

[0087] Obtain the first mean square error between the predicted watermark image and the original sample image;

[0088] A noise attack is performed on the predicted watermark image to obtain a noisy image;

[0089] The noisy image is decoded using a decoder to obtain the decoded watermark;

[0090] Obtain the second mean square error between the decoded watermark and the sample watermark;

[0091] The discrimination error, first mean square error and second mean square error of the encoder are weighted and fused to obtain the loss function of the preset model;

[0092] The training continues until the loss value of the loss function of the preset model is less than the loss threshold, at which point the training ends and the image watermark embedding model is obtained.

[0093] like Figure 2 As shown, training samples are obtained based on the original image, semantic information of the sample text, and the sample watermark. Adversarial training is then performed on the preset model using these training samples. During adversarial training, the encoder outputs the predicted watermark image. To achieve better watermark imperceptibility, this invention employs an adversarial network consisting of multiple convolutional layers and a global average pooling layer. During adversarial training, the discriminator attempts to distinguish between the original image and the watermark image, while the encoder continuously improves the watermark's concealment by minimizing the adversarial loss. The formulas for calculating the discriminator's loss and the encoder's discrimination error are as follows.

[0094] ;

[0095] in, For the loss of the discriminator, This refers to the encoder's discrimination error. These are the learnable parameters of the discriminator. For encoder parameters, The discrimination process of the discriminator, For the embedding process, The original image of the sample. To predict the watermark image, For sample watermarks.

[0096] To ensure the quality and imperceptibility of the predicted watermark image of this invention, the encoder uses mean squared error (MSE) to measure the difference between the predicted watermark image and the original sample image. The formula for calculating the first mean squared error is as follows.

[0097] ;

[0098] in, For the embedding process, The original image of the sample. To predict the watermark image, For sample watermarking, The first mean square error, This represents the mean square error.

[0099] The predicted watermark image is passed through a noise layer containing various image attacks to obtain a noisy image. Then, the sample watermark is accurately extracted from the potentially corrupted image. First, the noisy image is feature-enhanced using a ConvBNReLU unit. Subsequently, several SENets are introduced for further processing, resulting in a multi-channel tensor. Finally, the decoder uses a ConvBNReLU unit to convert the multi-channel feature tensor into a single-channel decoded watermark. To ensure the accuracy of watermark extraction, this invention uses mean squared error (MSE) to minimize the difference between the decoded watermark and the sample watermark. The formula for calculating the second mean squared error is as follows.

[0100] ;

[0101] in, This is the second mean square error. For the decoding process of the decoder, For sample watermarking, To decode the watermark, These are the parameters for the decoder. This is the original image of the sample.

[0102] The discrimination error, first mean square error, and second mean square error of the encoder are weighted and fused to obtain the loss function of the preset model. The calculation formula of the loss function of the preset model is as follows.

[0103] ;

[0104] in, The loss function of the preset model, The weight of the first mean square error. The weight of the second mean square error. The weights of the encoder's discrimination error. The first mean square error, This is the second mean square error. This represents the encoder's discrimination error.

[0105] The preset model is trained based on the training samples. The loss value of the loss function of the preset model is calculated. When the loss value is less than the loss threshold, the training ends and the image watermark embedding model is obtained.

[0106] The present invention constructs a loss function of a preset model based on the first mean square error, the second mean square error, and the discrimination error of the encoder, which helps to minimize the error between the predicted watermark image and the original sample image, and helps to improve the invisibility of the watermark image generated by the image watermark embedding model.

[0107] Furthermore, to verify the advantage of the SEDF algorithm of the image watermarking embedding model of the present invention in terms of watermark invisibility, the present invention quantitatively compares the following models: Hiding Data With Deep Networks (HiDDeN), Robust Data Hiding Using Inverse Gradient Attention (IGA), Enhancing Robustness of DNN-based Watermarking by Mini-Batch of Real and Simulated JPEG Compression (MBRS), Attention Guided Robust Image Watermarking Model Based on GAN (ARWGAN), Using LatentSpace for In-Generation Image Watermarking (LaWa), and the image watermarking embedding model of the present invention.

[0108] In the robustness testing of the watermarked images, six types of noise were selected: Gaussian noise, salt-and-pepper noise, Gaussian filtering, median filtering, JPEG compression, and random discarding. All methods trained a specific watermarking network on each type of noise using the same dataset.

[0109] This invention uses Peak Signal-to-Noise Ratio (PSNR) and Structure Similarity Index Measure (SSIM) to evaluate the invisibility of watermarks. A higher PSNR and a SSIM closer to 1 indicate better invisibility. Accuracy (ACC) is used to measure the robustness of the watermark under different attack conditions; a higher ACC value reflects stronger robustness. Table 1 shows the watermark invisibility of different models under various distortion conditions.

[0110] Table 1

[0111]

[0112] As shown in Table 1, the PSNR, SSIM, and ACC of the image watermarking embedding model of the present invention are significantly higher than those of the other models in Table 1.

[0113] Furthermore, the watermark robustness (ACC) results of different models under various distortions are compared, as shown in Table 2.

[0114] Table 2

[0115]

[0116] As shown in Table 2, the image watermark embedding model of this invention maintains an extraction accuracy of over 99.99% under various attacks, demonstrating high robustness and visual quality. This is due to the deep fusion of global semantics and local details in the image watermark embedding model of this invention, enabling the watermark to be embedded in a more concealed and structurally stable manner, effectively resisting various image perturbations.

[0117] To verify the importance of the Cross-modal Global Feature Extraction Module (CGEF) of this invention, a control model (No-CGFE) was designed, which retains only the local feature extraction module. The experimental results are shown in Table 3.

[0118] To verify the beneficial effect of textual semantic information on global feature extraction, this invention designed a control experiment (No-text) that removes textual semantic information. The experimental results are shown in Table 3.

[0119] Table 3

[0120]

[0121] As shown in Table 3, although the watermark still maintains high extraction accuracy under various noise conditions after removing CGEF, the average accuracy decreases by approximately 2.46%. This performance gap indicates that CGEF, by capturing the semantic structure of the image, enables the network to adaptively select a more reasonable embedding region during the embedding process, allowing the watermark to naturally integrate with the semantic content of the image. This not only improves the invisibility of the watermark but also enhances its resistance to attacks, thus effectively verifying the value of CGEF in improving the overall watermark performance.

[0122] As shown in Table 3, removing the textual semantic information enhancement generally reduces the watermark extraction accuracy under various noise conditions. This indicates that the textual semantic information extracted by CLIP effectively improves the image-text alignment capability through multi-head cross-attention, enhancing the semantic expression of the image and thus contributing to the robustness of the watermark. Notably, even without textual guidance, the image watermark embedding model maintains a high extraction accuracy, demonstrating that on datasets containing only images and lacking textual information, this model can still stably extract watermarks while ensuring their invisibility.

[0123] To further verify the effectiveness of the Multi-Scale Feature Fusion (MSF) module, this invention was compared with a fusion method based on Direct Weighted Summation (DWS), and the comparison results are shown in Table 4. The DWS fusion method assigns learnable weight parameters to local and global features respectively and then sums them, as shown in the following formula.

[0124] ;

[0125] in, The fusion feature is obtained by weighted summation. For local features, As the second global feature, and For weights.

[0126] Table 4

[0127]

[0128] As shown in Table 4, under various noise attacks, the average PSNR of the images generated by the DWS method is 35.73 dB, and the SSIM is 0.9183, both lower than that of SEDF, and the extraction accuracy is 0.88% lower on average. This gap indicates that the MSF module proposed in this paper can adaptively adjust the fusion strategy of local and global features through a bidirectional cross-attention mechanism, guiding the watermark to be embedded in the semantically key region, thereby significantly improving robustness while ensuring the invisibility of the watermark.

[0129] The image watermark embedding device provided by the present invention will be described below. The image watermark embedding device described below can be referred to in correspondence with the image watermark embedding method described above.

[0130] like Figure 5 As shown, an image watermark embedding device includes an acquisition module and a watermark embedding module.

[0131] The acquisition module 501 is used to acquire the original image, the textual semantic information of the original image, and the watermark of the original image.

[0132] The watermark embedding module 502 is used to input the original image, text semantic information, and watermark into the image watermark embedding model to obtain the watermark image output by the image watermark embedding model. Specifically, the image watermark embedding model extracts the first global feature and local features of the original image, performs cross-attention fusion on the first global feature and text semantic information to obtain the second global feature, performs cross-attention fusion on the second global feature and local features to obtain the fused feature, and generates the watermark image based on the fused feature, the watermark feature, and the original image. The image watermark embedding model is obtained through end-to-end training and adversarial training based on sample original images, sample text semantic information, and sample watermarks. The watermark feature is obtained after preprocessing the watermark.

[0133] The image watermark embedding device provided in this invention acquires an original image, the textual semantic information of the original image, and the watermark of the original image; inputs the original image, the textual semantic information, and the watermark into an image watermark embedding model to obtain a watermark image output by the image watermark embedding model; wherein, the image watermark embedding model extracts a first global feature and local features of the original image, performs cross-attention fusion on the first global feature and the textual semantic information to obtain a second global feature; performs cross-attention fusion on the second global feature and the local features to obtain a fused feature; and generates a watermark image based on the fused feature, the watermark feature, and the original image; the image watermark embedding model is obtained through end-to-end training and adversarial training based on sample original images, sample textual semantic information, and sample watermarks, and the watermark feature is obtained after preprocessing the watermark. This invention solves the problems of low watermark invisibility and poor robustness caused by neglecting global semantic information in existing image watermark embedding by fusing local features with the text-enhanced second global feature and combining a cross-attention mechanism, thereby improving the coordination between the embedded watermark and the original image.

[0134] In one embodiment, the watermark embedding module 502 is used to: extract features from the original image to obtain original image features; generate a multi-head query vector, a multi-head key vector, and a multi-head value vector based on the original image features; perform multi-head attention fusion on the multi-head query vector, the multi-head key vector, and the multi-head value vector; and add the multi-head attention fusion result to the original image features to obtain the first global feature.

[0135] In one embodiment, the watermark embedding module 502 is used to: generate a semantic key vector and a semantic value vector based on text semantic information; generate a query vector of a first global feature; perform a first cross-attention fusion on the query vector of the first global feature, the generated semantic key vector and the semantic value vector, add the result of the first cross-attention fusion to the first global feature to obtain a global semantic feature; and perform feature enhancement on the global semantic feature to obtain a second global feature.

[0136] In one embodiment, the watermark embedding module 502 is configured to: use local features as a query vector and second global features as key and value vectors to perform a second cross-attention fusion on the local features and second global features; add the result of the second cross-attention fusion to the local features to obtain local enhanced features; use the second global features as a query vector and local features as key and value vectors to perform a third cross-attention fusion on the local features and second global features; add the result of the third cross-attention fusion to the second global features to obtain global enhanced features; and fuse the local enhanced features and global enhanced features to obtain fused features.

[0137] In one embodiment, the watermark embedding module 502 is used to: concatenate the fused features and the watermark features in the channel dimension, perform multiple convolutions and skip connections on the concatenation result to obtain a watermarked image feature map; and fuse and convolve the watermarked image features and the original image to obtain a watermarked image.

[0138] In one embodiment, the image watermark embedding model is trained based on the following steps: obtaining training samples based on the original sample image, sample text semantic information, and sample watermark; inputting the training samples into a preset model to obtain the predicted watermark image output by the encoder of the preset model; the preset model includes an encoder, a discriminator, and a decoder; the discriminator is used to distinguish between the predicted watermark image and the original sample image, and the encoder is optimized based on the discriminator's discriminant loss; obtaining the discriminant error of the encoder based on the predicted watermark image and the original sample image; obtaining the first mean square error between the predicted watermark image and the original sample image; performing a noise attack on the predicted watermark image to obtain a noisy image; decoding the noisy image based on the decoder to obtain a decoded watermark; obtaining the second mean square error between the decoded watermark and the sample watermark; weightedly fusing the discriminant error, the first mean square error, and the second mean square error of the encoder to obtain the loss function of the preset model; training the preset model based on the training samples until the loss value of the loss function of the preset model is less than the loss threshold, ending the training, and obtaining the image watermark embedding model.

[0139] In one embodiment, the acquisition module 501 is used to: input the text description of the original image into the contrastive language-image pre-training model, and acquire the initial text semantic information output by the contrastive language-image pre-training model; wherein, the contrastive language-image pre-training model performs fixed-dimensional text feature extraction on the text description to obtain the initial text semantic information; and performs multi-level linear transformation on the initial text semantic information to obtain the text semantic information.

[0140] Figure 6 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 6As shown, the electronic device may include: a processor 610, a communications interface 620, a memory 630, and a communications bus 640, wherein the processor 610, the communications interface 620, and the memory 630 communicate with each other through the communications bus 640. The processor 610 can call logical instructions in the memory 630 to execute an image watermark embedding method. This method includes: acquiring an original image, textual semantic information of the original image, and a watermark of the original image; inputting the original image, textual semantic information, and watermark into an image watermark embedding model to acquire a watermark image output by the image watermark embedding model; wherein the image watermark embedding model extracts a first global feature and local features of the original image, performs cross-attention fusion on the first global feature and textual semantic information to obtain a second global feature; performs cross-attention fusion on the second global feature and local features to obtain a fused feature; and generates a watermark image based on the fused feature, the watermark feature, and the original image; the image watermark embedding model is obtained through end-to-end training and adversarial training based on sample original images, sample textual semantic information, and sample watermarks, and the watermark feature is obtained after preprocessing the watermark.

[0141] Furthermore, the logical instructions in the aforementioned memory 630 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0142] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements the image watermark embedding method provided by the above methods. The method includes: acquiring an original image, textual semantic information of the original image, and a watermark of the original image; inputting the original image, textual semantic information, and watermark into an image watermark embedding model to acquire a watermark image output by the image watermark embedding model; wherein the image watermark embedding model extracts a first global feature and a local feature of the original image, performs cross-attention fusion on the first global feature and the textual semantic information to obtain a second global feature; performs cross-attention fusion on the second global feature and the local feature to obtain a fused feature; and generates a watermark image based on the fused feature, the watermark feature, and the original image; the image watermark embedding model is obtained by end-to-end training and adversarial training based on sample original images, sample textual semantic information, and sample watermarks, and the watermark feature is obtained after preprocessing the watermark.

[0143] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0144] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0145] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for embedding an image watermark, characterized in that, include: Obtain the original image, the textual semantic information of the original image, and the watermark of the original image; The original image, the text semantic information, and the watermark input image watermark embedding model are combined to obtain the watermark image output by the image watermark embedding model. The image watermark embedding model extracts the first global features and local features of the original image. Based on the text semantic information, a semantic key vector and a semantic value vector are generated; a query vector of a first global feature is generated; a first cross-attention fusion is performed on the query vector of the first global feature, the generated semantic key vector, and the semantic value vector; the result of the first cross-attention fusion is added to the first global feature to obtain a global semantic feature; the global semantic feature is enhanced to obtain a second global feature. The local features are used as query vectors, and the second global features are used as key vectors and value vectors. A second cross-attention fusion is performed on the local features and the second global features. The result of the second cross-attention fusion is added to the local features to obtain the local enhanced features. The second global feature is used as the query vector, and the local feature is used as the key vector and value vector. A third cross-attention fusion is performed on the local feature and the second global feature. The result of the third cross-attention fusion is added to the second global feature to obtain the global enhanced feature. The local enhancement features and the global enhancement features are fused to obtain the fused features; The watermarked image is generated based on the fusion features, watermark features, and the original image; the image watermark embedding model is obtained by end-to-end training and adversarial training based on the original sample image, sample text semantic information, and sample watermark; the watermark features are obtained by preprocessing the watermark.

2. The image watermark embedding method according to claim 1, characterized in that, The first global feature was obtained based on the following steps: Feature extraction is performed on the original image to obtain the original image features; Generate a multi-head query vector, a multi-head key vector, and a multi-head value vector based on the original image features; Multi-head attention fusion is performed on the multi-head query vector, the multi-head key vector, and the multi-head value vector. The multi-head attention fusion result is added to the original image features to obtain the first global feature.

3. The image watermark embedding method according to claim 1, characterized in that, The watermarked image is generated based on the following steps: The fusion feature and the watermark feature are concatenated along the channel dimension, and the concatenation result is subjected to multiple convolutions and skip connections to obtain the watermarked image features. The watermarked image features are fused and convolved with the original image to obtain the watermarked image.

4. The image watermark embedding method according to claim 1, characterized in that, The image watermark embedding model is trained based on the following steps: Training samples are obtained based on the original image of the sample, the semantic information of the sample text, and the sample watermark; The training samples are input into a preset model to obtain the predicted watermark image output by the encoder of the preset model; the preset model includes an encoder, a discriminator, and a decoder. The discriminator is used to distinguish between the predicted watermark image and the original sample image, and the encoder is optimized based on the discriminator's discrimination loss. Based on the predicted watermark image and the original sample image, the discrimination error of the encoder is obtained; Obtain the first mean square error between the predicted watermark image and the original sample image; The predicted watermark image is subjected to a noise attack to obtain a noisy image; The noisy image is decoded using the decoder to obtain a decoded watermark; Obtain the second mean square error between the decoded watermark and the sample watermark; The discrimination error of the encoder, the first mean square error, and the second mean square error are weighted and fused to obtain the loss function of the preset model; The preset model is trained based on the training samples until the loss value of the loss function of the preset model is less than the loss threshold, at which point the training ends and the image watermark embedding model is obtained.

5. The image watermark embedding method according to claim 1, characterized in that, The semantic information of the text is obtained based on the following steps: The text description of the original image is input into the contrastive language-image pre-training model to obtain the initial text semantic information output by the contrastive language-image pre-training model; wherein, the contrastive language-image pre-training model performs fixed-dimensional text feature extraction on the text description to obtain the initial text semantic information. The initial text semantic information is subjected to multi-level linear transformation to obtain the text semantic information.

6. An image watermark embedding device, characterized in that, include: The acquisition module is used to acquire the original image, the textual semantic information of the original image, and the watermark of the original image; The watermark embedding module is used to embed the original image, the text semantic information, and the watermark input image into the watermark embedding model to obtain the watermark image output by the image watermark embedding model. The image watermark embedding model extracts the first global features and local features of the original image. Based on the text semantic information, a semantic key vector and a semantic value vector are generated; a query vector of a first global feature is generated; a first cross-attention fusion is performed on the query vector of the first global feature, the generated semantic key vector, and the semantic value vector; the result of the first cross-attention fusion is added to the first global feature to obtain a global semantic feature; the global semantic feature is enhanced to obtain a second global feature. The local features are used as query vectors, and the second global features are used as key vectors and value vectors. A second cross-attention fusion is performed on the local features and the second global features. The result of the second cross-attention fusion is added to the local features to obtain the local enhanced features. The second global feature is used as the query vector, and the local feature is used as the key vector and value vector. A third cross-attention fusion is performed on the local feature and the second global feature. The result of the third cross-attention fusion is added to the second global feature to obtain the global enhanced feature. The local enhancement features and the global enhancement features are fused to obtain the fused features; The watermarked image is generated based on the fusion features, watermark features, and the original image; the image watermark embedding model is obtained by end-to-end training and adversarial training based on the original sample image, sample text semantic information, and sample watermark; the watermark features are obtained by preprocessing the watermark.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the image watermark embedding method as described in any one of claims 1 to 5.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the image watermark embedding method as described in any one of claims 1 to 5.

Images