An active defense method based on face identity watermark and hybrid attention module

Abstract

This invention discloses an active defense method based on a facial identity watermark and a hybrid attention module, relating to the field of image processing technology. The method comprises the following steps: S1: extraction of a facial identity code; S2: generation of a facial identity code; S3: a hybrid attention encoder module; S4: loss function design; and S5: facial feature correlation analysis. The technical problem to be solved by this invention is to provide an active defense method based on a facial identity watermark and a hybrid attention module, enabling detection and traceability of facial tampering, significantly improving the practicality and functionality of the model.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an active defense method based on face identity watermark and hybrid attention module. Background Art

[0002] With the continuous advancement of deepfake technology, the level of fakery in forged images continues to surpass previous limits, posing unprecedented challenges to traditional image detection methods. Deepfake combines deep learning and fake technology. This cutting-edge technology, based on deep learning models such as generative adversarial networks (GANs), creates a sophisticated facial forgery system. By leveraging deep learning from massive amounts of image data, the model accurately captures biometric features such as facial bone structure, muscle movement patterns, and skin texture, and can even reproduce the effects of light and shadow on the face under varying lighting conditions. During the forgery phase, the target face's characteristic parameters are deeply embedded into the source image. Through intelligent pixel-level adjustments, the system generates fake image content that is indistinguishable to the naked eye. Facial forgery technology leverages artificial intelligence algorithms to precisely target facial regions within an image and manipulate them. The resulting fake content is visually highly similar to the real image, posing a significant threat to social order, information security, and privacy protection.

[0003] To overcome the technical bottlenecks of passive detection, academia and industry are turning their attention to active defense methods. The core of active defense lies in pre-embedding invisible digital watermarks, signatures, and other signals into the original image, enhancing the image's resistance to tampering at the source. This defense strategy is particularly effective in tracing the source, effectively recording the original image information and providing key clues for identifying the source of forgeries. Compared to the "post-verification" of passive detection, active defense achieves "pre-emptive prevention," significantly improving the system's anti-attack and robustness.

[0004] Deficiencies of existing technology:

[0005] Limited detection scope: Traditional image tampering detection methods focus on extracting overall image features and have a certain ability to detect global tampering. However, when faced with local facial tampering common in Deepfake technology, detection efficiency is greatly reduced. It is difficult to accurately identify forgery traces that only modify the facial area while other areas such as the background remain unchanged, resulting in reduced detection accuracy and robustness.

[0006] Poor adaptability to complex environments: The invisible watermarks embedded in existing active defense methods are too sensitive to common image processing operations such as compression, blurring, and noise. Once the Deepfake image undergoes such conventional processing, the detection performance becomes difficult to stabilize, which seriously limits its application effect in actual scenarios. Summary of the Invention

[0007] The technical problem to be solved by the present invention is to provide an active defense method based on facial identity watermark and hybrid attention module, which realizes the detection and tracing functions of facial tampering, greatly improving the practicality and functionality of the model.

[0008] The present invention adopts the following technical solutions to achieve the invention objectives:

[0009] An active defense method based on face identity watermark and hybrid attention module, characterized by comprising the following steps:

[0010] S1: Extraction of facial identification code;

[0011] S2: Generation of facial identification code;

[0012] S3: hybrid attention encoder module;

[0013] S4: loss function design;

[0014] S5: Correlation analysis of facial features.

[0015] As a further limitation of this technical solution, the specific steps of S1 are:

[0016] The first step in generating a facial identity watermark is to accurately extract stable facial features from the face area of ​​the input image. The retinal face detection algorithm in the deep learning-based face detection and alignment algorithm is used to detect and locate the face in the input image. The main loss of locating the face area is the face classification loss:

[0017] (1);

[0018] in: is the true label; It is the probability that the model predicts that the prior box contains a face; is the weight of the positive sample; It is the focusing parameter that adjusts the weight of positive and negative samples;

[0019] Construct a 2D Mel face feature extraction structure for image texture feature extraction. The 2D Mel face feature extraction structure mainly consists of three parts: multi-resolution stacked atlas, multi-scale 2D Mel filter and logarithmic energy spectrum feature extraction;

[0020] Multi-resolution stacking atlas is to downsample the image according to different resolutions to form a series of images of different scales. This process mainly includes Gaussian smoothing and downsampling operations;

[0021] The core of Gaussian smoothing is to use the Gaussian kernel to perform convolution operations with the image. The expression of the Gaussian kernel is:

[0022] (2);

[0023] in: are the coordinates of the pixel in the Gaussian kernel; is the standard deviation of the Gaussian function;

[0024] The multi-scale two-dimensional Mel filter construction formula is as follows:

[0025] (3);

[0026] in: ; ; is the wavelength of the sine function; is the direction of the filter; Indicates phase shift; It is a variable parameter that controls the degree of spatial localization of the filter; Represents the spatial aspect ratio, which controls the shape of the filter in different directions;

[0027] For images of different resolutions after multi-resolution stacking processing, two-dimensional Mel filters of different scales are used to perform convolution operations with the layer image to obtain response images at different scales. Finally, the frequency distribution and energy information of the image are extracted using the logarithmic energy spectrum extraction method. The extraction formula is as follows:

[0028] Energy spectrum calculation:

[0029] (4);

[0030] in: Represents a multi-scale response image; represents the frequency domain coordinates of the corresponding image, Indicates taking the absolute value;

[0031] Logarithmic extraction:

[0032] (5);

[0033] Where: Energy represents the coordinates in the image frequency domain The facial information extracted from is a small constant.

[0034] As a further limitation of this technical solution, the specific steps of S2 are:

[0035] The average hash algorithm is used to convert the extracted facial feature vector into a binary hash value of a specified length. The generation formula is as follows:

[0036] The facial feature vector array is:

[0037] (6);

[0038] The calculation formula of its average value E is:

[0039] (7);

[0040] For each element in the facial feature array ,if , then the corresponding binary value is 1, otherwise it is 0. The mathematical expression is as follows:

[0041] (8).

[0042] As a further limitation of this technical solution, the specific steps of S3 are:

[0043] S31: multi-scale identity watermark preprocessing module;

[0044] In the identity code watermark transformation module, the binary watermark information of length L is processed and converted into a format consistent with the image tensor dimension. The watermark is first reshaped into a two-dimensional array of shape (H / 2, W / 2) , and then select multiple scale factors right Upsampling is performed, and the upsampling formula is:

[0045] (9);

[0046] in: Represents an upsampling operation; Indicates different scale factors;

[0047] After multi-scale upsampling, the feature maps sampled at different scales are weighted fused. The formula for weighted fusion is:

[0048] (10);

[0049] in: Indicates the The weight corresponding to each scale factor;

[0050] S32: Hybrid Attention Embedding Module;

[0051] This module receives the original image as input and the watermark message after multi-scale preprocessing , a visual Mamba-like linear attention U-type network combining a Mamba-like linear attention mechanism with a U-type network is constructed to specifically handle watermark embedding. The model mainly consists of three parts: a feature initial processing module based on channel attention (SE-Stem) for initial feature extraction, a linear attention module, and a multi-scale dilated downsampling convolution block;

[0052] S321: feature initial processing module SE-Stem based on channel attention;

[0053] Construct a two-branch structure to process the input image and watermark separately. Each branch processes the image and watermark separately through convolution operation and channel attention mechanism, gradually reducing their spatial dimensions and increasing the channel dimensions while retaining important information. Finally, the two feature information are spliced ​​together to prepare for subsequent operations. The formula is as follows:

[0054] (11);

[0055] in: represents the channel attention operation; and Represents the convolution operations performed sequentially; Represents feature splicing; Represents the input source image; represents the pre-processed identity watermark;

[0056] S322: linear attention module;

[0057] The fusion features of the source image and the identity watermark after preliminary processing by the channel attention-based feature initial processing module are input into the Mamba-like linear attention block for further processing;

[0058] The two linear blocks in the original Mamba-like linear attention module are replaced by a row-wise feature collaboration module and a column-wise feature collaboration module to form a new MLLA-structured linear attention module.

[0059] The row-wise feature collaboration module consists of a feature extraction convolutional layer, a horizontal position encoding layer, and a linear layer. The input feature vector first passes through a convolutional layer to extract row local features:

[0060] (12);

[0061] in: is the extracted row local feature, is the feature map preliminarily processed by SE-Stem, Conv () represents the convolution operation;

[0062] Then enter the horizontal position encoding layer to perform matrix operations:

[0063] (13);

[0064] in: for The calculation process of the horizontal position encoding is as follows:

[0065] Input feature map , then the horizontal position encoding parameter , It is the preset maximum number of horizontal positions, which indicates the maximum position range in the horizontal direction that the model can handle. Select the width of the current feature map W Corresponding part ;

[0066] for No. Row eigenvector , the calculation of the linear layer is:

[0067] (14);

[0068] in: 、 and is the projection matrix, is the processed row feature; 、 、 Respectively represent Row eigenvector The weights obtained by multiplying the three different projection matrices, Represents the linear activation of the key matrix, softmax represents a linear activation function;

[0069] The column-wise feature collaboration module consists of a feature extraction convolution layer, a vertical position encoding layer, and a linear layer. The input feature vector first passes through a convolution layer to extract column local features:

[0070] (15);

[0071] in: is the extracted column local feature, are the features that have been preliminarily processed by SE-Stem;

[0072] Then enter the vertical position encoding layer to perform matrix operations:

[0073] (16);

[0074] in: for The calculation process of the vertical position encoding is as follows:

[0075] Input feature map , then the vertical position encoding parameter , It is the preset maximum vertical position number, which indicates the maximum position range in the vertical direction that the model can handle. Select the part corresponding to the current feature map height H ;

[0076] for No. Column feature vector , the calculation of the linear layer is:

[0077] (17);

[0078] in: is the processed column feature; Respectively represent Column feature vector The weights obtained by multiplying the three different projection matrices, Represents linear activation of the key matrix;

[0079] S323: multi-scale dilated downsampling convolution block;

[0080] By combining dilated convolution and depthwise separable convolution, the receptive field can be expanded without significantly reducing the resolution. Dilated convolution uses different dilation rates in the convolution kernel to capture features at three different scales: local, medium, and global, while retaining image details. The idea of ​​dynamically adjusting the dilation rate is adopted, and learnable parameters are introduced to dynamically determine the optimal dilation rate of each convolution layer. The parameters Dynamically adjust according to the features of the input image to capture more contextual information at different scales. The feature analysis layer analyzes the statistical information of the input features;

[0081] First calculate the gradient of the input feature , then according to the gradient With parameters Calculating dynamic expansion rate :

[0082] (18);

[0083] Finally, dilated convolutions with different expansion rates are dynamically selected according to d for operation; int() indicates integerization;

[0084] The entire downsampling process can be expressed as:

[0085] (19);

[0086] in: Represents the feature map of the final output; Indicates the flattened layer; represents convolutional layer 2; Represents multi-scale dilated convolution; represents depth-wise separable convolution; represents convolutional layer 1; Indicates the initial reshaping of the input features; Represents the input features.

[0087] As a further limitation of this technical solution, the specific steps of S4 are:

[0088] The loss function consists of three parts: weighted cross entropy loss, multi-scale image pixel loss and adaptive message loss;

[0089] S41: weighted cross entropy loss;

[0090] The weighted cross entropy loss is used to solve the class imbalance problem by assigning different weights to different classes. The formula is as follows:

[0091] (20);

[0092] in: It is i The true category label of the face area detected by the individual face sample; is the probability of the positive class predicted by the model; is the weighted weight;

[0093] S42: multi-scale image pixel loss;

[0094] By introducing the idea of ​​multi-scale into the mean square error and calculating the mean square error at different scales, the details and structural information of the image can be better captured. The formula is as follows:

[0095] (twenty one);

[0096] in: is the number of scales; It is The number of pixels at each scale; and The watermarked image and the original image are Representation on a scale;

[0097] S43: Adaptive texture feature message loss;

[0098] Construct innovative adaptive loss functions;

[0099] First, we need to extract the local texture features of the image. Local texture features help us understand the complexity of the image and thus decide in which areas to embed a stronger watermark. The formula is as follows:

[0100] (twenty two);

[0101] in: and The input images are gradient; is the image position Local texture features;

[0102] Then, according to the local texture features, the adaptive weight is calculated to increase the watermark embedding strength in the texture-rich area and reduce the watermark embedding strength in the smooth area.

[0103] (twenty three);

[0104] in: is the image position Adaptive weights of and are the minimum and maximum values ​​of the texture features, respectively;

[0105] Finally, the definition of the adaptive message loss function is:

[0106] (twenty four);

[0107] in: is the original watermark message; is the extracted watermark message; is a classification label, is the decoding threshold, is the decoding confidence; is the adaptive weight of position i, is the total number of pixels of the watermark;

[0108] S44: The overall loss function is the weighted sum of weighted cross entropy loss, multi-scale image loss and adaptive message loss:

[0109] (25);

[0110] in: 、 and The weight of each loss term.

[0111] As a further limitation of this technical solution, the specific steps of S5 are:

[0112] Because two decoding methods are used in this model: using a decoder to decode the watermark embedded in the face image and using the same face recognition algorithm and two-dimensional Mel face extraction structure to regenerate the watermark in the face area of ​​the face image, traditional correlation comparison includes comparing the Hamming distance, but if there is a small amount of bit difference between the two vectors, the Hamming distance may increase significantly. Even if the two watermarks are visually very similar, due to the difference in individual bits, the Hamming distance may cause them to be misjudged as dissimilar. In order to accurately evaluate the similarity between the watermarks decoded under the two decoding methods, the regularized Pearson correlation coefficient is used to quantify the linear correlation between them. The formula is as follows:

[0113] (26);

[0114] in: is a regularization term; and Represents the eigenvectors and No. Dimensional elements; is the dimension of the feature vector; and is the eigenvector and The average value of When it is 1, it means that the two vectors are completely positively correlated, and when it is 0, it means that there is no linear relationship between the two eigenvectors.

[0115] Compared with the prior art, the advantages and positive effects of the present invention are:

[0116] 1. Compared with traditional methods, the present invention can simultaneously realize the detection and tracing functions of face tampering through a different decoding structure, which greatly improves the practicality and functionality of the model.

[0117] 2. This paper proposes a new method for extracting facial identity watermarks, which has been experimentally verified to be robust to conventional noise operations.

[0118] 3. This paper proposes VM-UNet (Linear Attention UNet similar to the state-space model), which is a new architecture that specifically handles watermark embedding tasks by innovatively combining linear attention, channel attention, Mamba model and Unet architecture. The new architecture has achieved significant improvements in the natural visual and robustness of watermarked images. BRIEF DESCRIPTION OF THE DRAWINGS

[0119] Figure 1 It is the overall structural diagram of the present invention.

[0120] Figure 2 This is the VM-Unet structure diagram of the present invention.

[0121] Figure 3 Schematic diagram of the two-dimensional Mel face feature extraction structure of the present invention.

[0122] Figure 4 Schematic diagram of the SE-Stem module structure of the present invention.

[0123] Figure 5 Schematic diagram of the structure of the multi-scale dilated downsampling convolution block (MSDC) of the present invention.

[0124] Figure 6 It is a schematic diagram of the HLM LLA module structure of the present invention. DETAILED DESCRIPTION

[0125] A specific embodiment of the present invention is described in detail below with reference to the accompanying drawings, but it should be understood that the protection scope of the present invention is not limited by the specific embodiment.

[0126] In this paper, an active defense method for face forgery based on face identity watermark and hybrid attention module is proposed. By embedding a unique face identity watermark in the face image, this method can simultaneously detect and trace the forgery behavior. Specifically, the method first inputs a source target face image, uses face recognition technology to divide the face area of ​​the face image, and uses the rich texture features of the face area to generate a face identity code. The extracted identity code is then processed by shape adjustment and average hashing as a robust watermark. A Embedded into the entire image. In the embedding process, a new framework VM-Unet that combines a linear attention mechanism (MLLA) similar to a state-space model with a Unet network is proposed to specifically handle watermark embedding. This architecture combines the advantages of linear attention, channel attention and state-space model (SSM), supplemented by the efficient symmetric sampling structure of Unet to enhance feature processing and watermark embedding accuracy. Afterwards, the image is processed through a series of model robustness enhancement processes such as the noise layer (compression, blurring, cropping), and an image containing a face identity code watermark is generated. In the construction of the watermark decoder, this method adopts two different decoding methods for tampering detection and traceability analysis respectively: on the one hand, the identity code watermark I is extracted from the face area of ​​the tampered image through the same face recognition algorithm as the previously generated face identity code. B On the other hand, the decoder is used to extract the embedded identity watermark I from the image A’， By comparing I B with I A’ The correlation between the two can be used to determine whether the image has been tampered with. On the other hand, by comparing I A with I A’The source of the image can be further traced to achieve traceability analysis.

[0127] The present invention comprises the following steps:

[0128] S1: Extraction of facial identity code.

[0129] The specific steps of S1 are:

[0130] The first step in generating a facial identity watermark is to accurately extract stable facial features from the face area of ​​the input image. The RetinaFace algorithm, a deep learning-based face detection and alignment algorithm, is used to detect and locate the face in the input image. The main loss in locating the face area is the face classification loss:

[0131] (1);

[0132] in: is the true label; It is the probability that the model predicts that the prior box contains a face; is the weight of the positive sample; It is the focusing parameter that adjusts the weight of positive and negative samples;

[0133] Inspired by the application of Mel spectrum coefficients in audio anti-counterfeiting, after the input image is detected by the RetinaFace algorithm to locate the face area, a two-dimensional Mel face feature extraction structure is constructed for image texture feature extraction. Figure 3 As shown in the figure. This 2D Mel-scale facial feature extraction structure uses a multi-scale feature extraction method to extract facial features at different scales, capturing the multi-scale texture information in the image. This makes the extracted facial features more stable and robust under various distortion conditions such as JPEG compression, noise, and scaling. This 2D Mel-scale facial feature extraction structure mainly consists of three parts: a multi-resolution stacked atlas, a multi-scale 2D Mel-scale filter, and logarithmic energy spectrum feature extraction.

[0134] The multi-resolution stacked atlas downsamples an image to different resolutions, creating a series of images at different scales. This process primarily involves Gaussian smoothing and downsampling. Gaussian smoothing involves applying a Gaussian filter to the face region, removing high-frequency noise and smoothing the image. Downsampling involves downsampling the smoothed image, selecting pixels in every other row and column to create images of varying resolutions.

[0135] The core of Gaussian smoothing is to use the Gaussian kernel to perform convolution operations with the image. The expression of the Gaussian kernel is:

[0136] (2);

[0137] in: are the coordinates of the pixel in the Gaussian kernel; is the standard deviation of the Gaussian function;

[0138] The multi-scale two-dimensional Mel filter construction formula is as follows:

[0139] (3);

[0140] in: ; ; is the wavelength of the sine function; is the direction of the filter; Indicates phase shift; It is a variable parameter that controls the degree of spatial localization of the filter; Represents the spatial aspect ratio, which controls the shape of the filter in different directions;

[0141] For images of different resolutions after multi-resolution stacking processing, two-dimensional Mel filters of different scales are used to perform convolution operations with the layer image to obtain response images at different scales. These response images reflect the texture features and energy information of the image at different scales and directions. Finally, the frequency distribution and energy information of the image are extracted using the logarithmic energy spectrum extraction method. The extraction formula is as follows:

[0142] Energy spectrum calculation:

[0143] (4);

[0144] in: Represents a multi-scale response image; represents the frequency domain coordinates of the corresponding image, Indicates taking the absolute value;

[0145] Logarithmic extraction:

[0146] (5);

[0147] Where: Energy represents the coordinates in the image frequency domain The facial information extracted from is a small constant used to prevent numerical problems in logarithmic operations.

[0148] S2: Generation of facial identification code.

[0149] The specific steps of S2 are:

[0150] The facial feature vector generated by the above steps is too long and directly embedding it into the image will affect the quality of watermark embedding. Therefore, the average hash algorithm is used to convert the generated facial feature vector into a binary hash value of a specified length. The generation formula is as follows:

[0151] The facial feature vector array is:

[0152] (6);

[0153] The calculation formula of its average value E is:

[0154] (7);

[0155] For each element in the facial feature array ,if , then the corresponding binary value is 1, otherwise it is 0. The mathematical expression is as follows:

[0156] (8).

[0157] S3: Hybrid Attention Encoder Module.

[0158] The specific steps of S3 are:

[0159] In the image identity code watermark embedding process, in order to ensure the high integrity of the image in terms of visual quality and maintain the original details and clarity of the image as much as possible, while ensuring the strong robustness of the embedded watermark so that it can still exist stably and completely when facing common image processing attacks such as compression, cropping, filtering, or when suffering from noise interference, the encoder module is composed of two parts: a multi-scale identity watermark preprocessing module and a hybrid attention embedding module.

[0160] S31: multi-scale identity watermark preprocessing module;

[0161] In order to make the binary hash watermark generated by the facial feature code better integrated with the image features, in the identity code watermark transformation module, the binary watermark information of length L is processed and converted into a format consistent with the image tensor dimension. This conversion process ensures that the watermark information can be effectively integrated with the image features in the same dimension. Specifically, the watermark is first reshaped into a two-dimensional array with a shape of (H / 2, W / 2). , and then select multiple scale factors right Upsampling is performed, and the upsampling formula is:

[0162] (9);

[0163] in: Represents an upsampling operation; Indicates different scale factors;

[0164] After multi-scale upsampling, the feature maps sampled at different scales are weighted fused. The formula for weighted fusion is:

[0165] (10);

[0166] in: Indicates the The weight corresponding to each scale factor;

[0167] By preprocessing the watermark at multiple scales, the watermark has a certain representation at different scales, which can effectively enhance the redundancy and robustness of the watermark while ensuring dimensional consistency with image features.

[0168] S32: Hybrid Attention Embedding Module;

[0169] This module receives the original image as input (R represents the image dataset, 3 represents the image is input in RGB three-channel format, H and W represent the height and width of the image respectively) and the watermark message after multi-scale preprocessing In order to more comprehensively represent the content of the image, a Mamba-like linear attention mechanism (MLLA) combined with the U-net (Unet) is constructed to specifically handle watermark embedding. Figure 2 As shown in the figure, the model mainly consists of three parts: a channel-attention-based feature initial processing (Squeeze-and-Excitation Network Stem, SE-Stem) module (SE-Stem) for initial feature extraction, a linear attention module (HLMLLA) and a multi-scale dilated downsampling convolution block (MSDC);

[0170] S321: feature initial processing module SE-Stem based on channel attention;

[0171] This module is designed as the initial processing module. In order to make our proposed architecture specifically for watermark embedding, we improve the Stem module. Specifically, we construct a two-branch structure to process the input image and watermark separately, such as Figure 4 As shown in the figure, each branch processes the image and watermark respectively through convolution operation and channel attention mechanism (SEnet), gradually reducing their spatial dimensions and increasing the channel dimensions while retaining important information. Finally, the two feature information are spliced ​​together to prepare for subsequent operations. The formula is as follows:

[0172] (11);

[0173] in: represents the channel attention operation; and Represents the convolution operations performed sequentially; Represents feature splicing; Represents the input source image; represents the pre-processed identity watermark;

[0174] S322: Linear Attention Module (HLMLLA);

[0175] The fusion features of the source image and the identity watermark after preliminary processing by the channel attention-based feature initial processing module are input into the Mamba-Like Linear Attention block (MLLA) for further processing;

[0176] Mamba-Like Linear Attention (MLLA) is an improved method that combines the Mamba model with the linear attention mechanism, aiming to improve model performance in visual tasks. MLLA primarily integrates two key elements from Mamba: the forget gate and block design. However, the forget gate requires recursive computation, which may not be well-suited for modeling non-causal data. For example, images are two-dimensional spatial data whose information primarily resides in the spatial relationships and local features between pixels. Unlike sequence data, which has a clear temporal order or causal relationships, images do not inherently require recursion. To effectively model the spatial features in facial images, we need to avoid the unnecessary time consumption and complexity of recursive computation when processing non-causal data.

[0177] The two linear blocks in the original Mamba-like linear attention module are replaced by the row-wise feature coordination module and the column-wise feature coordination module to form a new MLLA structure linear attention module; Figure 6 As shown in the figure, these two components combine horizontal position encoding and vertical position encoding through a linear attention mechanism to enhance the model's understanding of pixel position relationships, without relying on a recurrent structure to capture image sequence information.

[0178] The row-wise feature collaboration module consists of a feature extraction convolutional layer, a horizontal position encoding layer, and a linear layer. The input feature vector first passes through a convolutional layer to extract row local features:

[0179] (12);

[0180] in: is the extracted row local feature, is the feature map preliminarily processed by SE-Stem, Conv () represents the convolution operation;

[0181] Then enter the horizontal position encoding layer to perform matrix operations:

[0182] (13);

[0183] in: for The calculation process of the horizontal position encoding is as follows:

[0184] Input feature map , 、 、 Represent the channel, height and width of the feature map respectively, then the horizontal position encoding parameter , It is the preset maximum number of horizontal positions, which indicates the maximum position range in the horizontal direction that the model can handle. Select the width of the current feature map W Corresponding part ;

[0185] for No. Row eigenvector , the calculation of the linear layer is:

[0186] (14);

[0187] in: 、 and is the projection matrix, is the processed row feature; 、 、 Respectively represent Row eigenvector The weights obtained by multiplying the three different projection matrices, Represents the linear activation of the key matrix, softmax represents a linear activation function;

[0188] The column-wise feature collaboration module consists of a feature extraction convolution layer, a vertical position encoding layer, and a linear layer. The input feature vector first passes through a convolution layer to extract column local features:

[0189] (15);

[0190] in: is the extracted column local feature, are the features that have been preliminarily processed by SE-Stem;

[0191] Then enter the vertical position encoding layer to perform matrix operations:

[0192] (16);

[0193] in: for The calculation process of the vertical position encoding is as follows:

[0194] Input feature map , then the vertical position encoding parameter , It is the preset maximum vertical position number, which indicates the maximum position range in the vertical direction that the model can handle. Select the part corresponding to the current feature map height H ;

[0195] for No. Column feature vector , the calculation of the linear layer is:

[0196] (17);

[0197] in: is the processed column feature; Respectively represent Column feature vector The weights obtained by multiplying the three different projection matrices, Represents linear activation of the key matrix;

[0198] S323: multi-scale dilated downsampling convolution block;

[0199] Common downsampling methods use maximum pooling and average pooling to achieve downsampling. Although these methods can reduce the resolution, they often lose some feature information. By combining dilated convolution and depthwise separable convolution, the receptive field can be expanded without significantly reducing the resolution. The specific structure is as follows Figure 5 As shown in the figure, the dilated convolution uses different dilation rates in the convolution kernel to capture features at three different scales: local, medium, and global, while retaining the details of the image. In order to improve the adaptability and performance of the dilated convolution, the idea of ​​dynamically adjusting the dilation rate is adopted, and a learnable parameter is introduced to dynamically determine the optimal dilation rate of each convolution layer. The parameter Dynamically adjust according to the features of the input image to capture more contextual information at different scales. The feature analysis layer analyzes the statistical information of the input features;

[0200] First calculate the gradient of the input feature , then according to the gradient With parameters Calculating dynamic expansion rate :

[0201] (18);

[0202] Finally, dilated convolutions with different expansion rates are dynamically selected according to d for operation; int() indicates integerization;

[0203] The entire downsampling process can be expressed as:

[0204] (19);

[0205] in: Represents the feature map of the final output; Indicates the flattened layer; represents convolutional layer 2; Represents multi-scale dilated convolution; represents depth-wise separable convolution; represents convolutional layer 1; Indicates the initial reshaping of the input features; Represents the input features.

[0206] S4: Loss function design.

[0207] The specific steps of S4 are:

[0208] The loss function consists of three parts: weighted cross entropy loss, multi-scale image pixel loss, and adaptive message loss; ensuring that watermark messages can be effectively embedded and extracted while maintaining high image fidelity;

[0209] S41: weighted cross entropy loss;

[0210] In the face detection stage, in order to reduce the category imbalance problem caused by different face categories and positions, weighted cross entropy loss is used to solve the category imbalance problem by assigning different weights to different categories. The formula is as follows:

[0211] (20);

[0212] in: It is i The true category label of the face area detected by the individual face sample; is the probability of the positive class predicted by the model; is a weighted weight so that the loss contribution of each category can be adjusted according to its importance;

[0213] S42: multi-scale image pixel loss;

[0214] Image loss is used to measure the difference between the original image and the watermarked image to ensure that the watermark embedding process does not significantly degrade the image quality. Image loss is usually calculated using mean square error (MSE), but MSE usually only focuses on the difference in pixel values ​​and ignores the structure and texture information of the image. By introducing the idea of ​​multi-scale into the mean square error and calculating the mean square error at different scales, the details and structural information of the image can be better captured. The formula is as follows:

[0215] (twenty one);

[0216] in: is the number of scales, usually set to 1, 2, or 4; It is The number of pixels at each scale; and The watermarked image and the original image are Representation on a scale;

[0217] S43: Adaptive texture feature message loss;

[0218] To ensure that the watermark message can be accurately extracted, the message loss function plays a vital role in the watermark embedding process. However, the traditional message loss function may cause the watermark to be embedded too strongly in some cases, which has a negative impact on the image quality. To solve this problem, an innovative adaptive loss function is constructed, which can flexibly adjust the strength of the watermark embedding according to the local texture features of the image. This adaptive mechanism not only helps to maintain the overall quality of the image, but also significantly improves the robustness of the watermark, ensuring that the watermark information can be effectively extracted under various conditions;

[0219] First, we need to extract the local texture features of the image. Local texture features can help us understand the complexity of the image and thus determine in which areas a stronger watermark can be embedded. The formula is as follows:

[0220] (twenty two);

[0221] in: and The input images are gradient; is the image position Local texture features;

[0222] Then, according to the local texture features, the adaptive weight is calculated to increase the watermark embedding strength in the texture-rich area and reduce the watermark embedding strength in the smooth area.

[0223] (twenty three);

[0224] in: is the image position Adaptive weights of and are the minimum and maximum values ​​of the texture features, respectively;

[0225] Finally, the definition of the adaptive message loss function is:

[0226] (twenty four);

[0227] in: is the original watermark message; is the extracted watermark message; is a classification label used to separate watermark messages into Convert to a binary classification problem (1 or -1), is the decoding threshold, is the decoding confidence, which represents the extracted watermark message bits With threshold the gap between; is the adaptive weight of position i, is the total number of pixels of the watermark;

[0228] S44: The overall loss function is the weighted sum of weighted cross entropy loss, multi-scale image loss and adaptive message loss:

[0229] (25);

[0230] in: 、 and The weight of each loss term.

[0231] S5: Correlation analysis of facial features.

[0232] The specific steps of S5 are:

[0233] Because two decoding methods are used in this model: using a decoder to decode the watermark embedded in the face image and using the same face recognition algorithm and two-dimensional Mel face extraction structure to regenerate the watermark in the face area of ​​the face image, traditional correlation comparison includes comparing the Hamming distance, but if there is a small amount of bit difference between the two vectors, the Hamming distance may increase significantly. Even if the two watermarks are very similar visually, the Hamming distance may cause them to be misjudged as dissimilar due to the difference in individual bits. In order to accurately evaluate the similarity between the watermarks decoded under the two decoding methods, the regularized Pearson correlation coefficient is used to quantify the linear correlation between them. The formula is as follows:

[0234] (26);

[0235] in: It is a regularization term that can avoid the situation where the denominator is zero and improve numerical stability; and Represents the eigenvectors and No. Dimensional elements; is the dimension of the feature vector; and is the eigenvector and The average value of When it is 1, it means that the two vectors are completely positively correlated, and when it is 0, it means that there is no linear relationship between the two eigenvectors.

[0236] Experimental setup

[0237] Training and testing are performed on the CelebA face dataset. All images are cropped to 128×128 pixels. The training set and test set are split in a ratio of 0.8:0.2.

[0238] The training parameters were configured over 100 epochs. A high initial learning rate was used in the first 50 epochs. This strategy helps the model quickly explore the parameter space early in training, accelerating convergence. Subsequently, the learning rate was lowered for the remaining 50 epochs to allow for more refined model adjustments in the later stages of training.

[0239] The batch size is set to 64. This ensures sufficient sample diversity during each gradient update, effectively improving the model's generalization ability. The Adam optimizer is used as the optimizer.

[0240] In the construction of the loss function, we set the weight coefficients in the loss function, and the initial weights are 、 、 .

[0241] Experimental results:

[0242] Table 1 Comparison of visual quality of watermarked images

[0243]

[0244] Table 2 Conventional noise robustness test

[0245]

[0246] Table 3 Generalization test: trained on the CelebA dataset and tested on the FFHQ dataset

[0247]

[0248] The above disclosure is only a specific embodiment of the present invention, but the present invention is not limited thereto. Any changes that can be conceived by those skilled in the art should fall within the scope of protection of the present invention.

Claims

1. An active defense method based on face identity watermark and hybrid attention module, characterized in that: The following steps are involved: S1: Extraction of facial identity code; The first step in generating facial identity watermark is to accurately extract stable facial features from the face area of ​​the input image. The retinal face detection algorithm in the deep learning-based face detection and alignment algorithm is used to detect and locate the face in the input image. S2: Generation of facial identity code; using the average hashing algorithm to convert the extracted facial feature vector into a binary hash value of a specified length; S3: Hybrid attention encoder module; the specific steps of S3 are: S31: multi-scale identity watermark preprocessing module; In the identity code watermark conversion module, the binary watermark information of length L is processed and converted into a format consistent with the image tensor dimension; after multi-scale upsampling, the feature maps sampled at different scales are weighted fused; S32: Hybrid Attention Embedding Module; This module receives the original image as input and the watermark message after multi-scale preprocessing , a visual Mamba-like linear attention U-type network combining a Mamba-like linear attention mechanism with a U-type network is constructed to specifically handle watermark embedding. The model mainly consists of three parts: a feature initial processing module based on channel attention for initial feature extraction, a linear attention module, and a multi-scale dilated downsampling convolution block; S321: feature initial processing module SE-Stem based on channel attention; Construct a two-branch structure to process the input image and watermark separately. Each branch processes the image and watermark separately through convolution operation and channel attention mechanism, gradually reducing their spatial dimensions and increasing the channel dimensions while retaining important information. Finally, the two feature information are spliced ​​together to prepare for subsequent operations. S322: linear attention module; The fusion features of the source image and the identity watermark after preliminary processing by the channel attention-based feature initial processing module are input into the Mamba-like linear attention block for further processing; The two linear blocks in the original Mamba-like linear attention module are replaced by a row-wise feature collaboration module and a column-wise feature collaboration module to form a new MLLA-structured linear attention module. The row-wise feature collaboration module consists of a feature extraction convolutional layer, a horizontal position encoding layer, and a linear layer. The input feature vector first passes through a convolutional layer to extract row local features: S323: multi-scale dilated downsampling convolution block; By combining dilated convolution and depthwise separable convolution, the receptive field can be expanded without significantly reducing the resolution. Dilated convolution uses different dilation rates in the convolution kernel to capture features at three different scales: local, medium, and global, while retaining image details. The idea of ​​dynamically adjusting the dilation rate is adopted, and learnable parameters are introduced to dynamically determine the optimal dilation rate of each convolution layer. The parameters Dynamically adjust according to the features of the input image to capture more contextual information at different scales. The feature analysis layer analyzes the statistical information of the input features; S4: Loss function design; the specific steps of S4 are: The loss function consists of three parts: weighted cross entropy loss, multi-scale image pixel loss and adaptive message loss; S5: Correlation analysis of facial features.

2. The active defense method based on face identity watermark and hybrid attention module according to claim 1 is characterized by: The specific steps of S1 are: The main loss for locating the face area is the face classification loss: （1）； in: is the true label; It is the probability that the model predicts that the prior box contains a face; is the weight of the positive sample; It is the focusing parameter that adjusts the weight of positive and negative samples; Construct a 2D Mel face feature extraction structure for image texture feature extraction. The 2D Mel face feature extraction structure mainly consists of three parts: multi-resolution stacked atlas, multi-scale 2D Mel filter and logarithmic energy spectrum feature extraction; Multi-resolution stacking atlas is to downsample the image according to different resolutions to form a series of images of different scales. This process mainly includes Gaussian smoothing and downsampling operations; The core of Gaussian smoothing is to use the Gaussian kernel to perform convolution operations with the image. The expression of the Gaussian kernel is: （2）； in: are the coordinates of the pixel in the Gaussian kernel; is the standard deviation of the Gaussian function; The multi-scale two-dimensional Mel filter construction formula is as follows: （3）； in: ; ; is the wavelength of the sine function; is the direction of the filter; Indicates phase shift; It is a variable parameter that controls the degree of spatial localization of the filter; Represents the spatial aspect ratio, which controls the shape of the filter in different directions; For images of different resolutions after multi-resolution stacking processing, two-dimensional Mel filters of different scales are used to perform convolution operations with the layer image to obtain response images at different scales. Finally, the frequency distribution and energy information of the image are extracted using the logarithmic energy spectrum extraction method. The extraction formula is as follows: Energy spectrum calculation: （4）； in: Represents a multi-scale response image; represents the frequency domain coordinates of the corresponding image, Indicates taking the absolute value; Logarithmic extraction: （5）； Where: Energy represents the coordinates in the image frequency domain The facial information extracted from is a small constant.

3. The active defense method based on face identity watermark and hybrid attention module according to claim 2 is characterized by: The specific steps of S2 are: The average hash algorithm is used to convert the extracted facial feature vector into a binary hash value of a specified length. The generation formula is as follows: The facial feature vector array is: （6）； The calculation formula of its average value E is: （7）； For each element in the facial feature array ,if , then the corresponding binary value is 1, otherwise it is 0. The mathematical expression is as follows: （8）。 4. The active defense method based on face identity watermark and hybrid attention module according to claim 3 is characterized by: The specific steps of S3 are: S31: multi-scale identity watermark preprocessing module; In the identity code watermark transformation module, the binary watermark information of length L is processed and converted into a format consistent with the image tensor dimension. The watermark is first reshaped into a two-dimensional array of shape (H / 2, W / 2) , and then select multiple scale factors right Upsampling is performed, and the upsampling formula is: （9）； in: Represents an upsampling operation; Indicates different scale factors; After multi-scale upsampling, the feature maps sampled at different scales are weighted fused. The formula for weighted fusion is: （10）； in: Indicates the The weight corresponding to each scale factor; S32: Hybrid Attention Embedding Module; This module receives the original image as input and the watermark message after multi-scale preprocessing , a visual Mamba-like linear attention U-type network combining a Mamba-like linear attention mechanism with a U-type network is constructed to specifically handle watermark embedding. The model mainly consists of three parts: a feature initial processing module based on channel attention (SE-Stem) for initial feature extraction, a linear attention module, and a multi-scale dilated downsampling convolution block; S321: feature initial processing module SE-Stem based on channel attention; Construct a two-branch structure to process the input image and watermark separately. Each branch processes the image and watermark separately through convolution operation and channel attention mechanism, gradually reducing their spatial dimensions and increasing the channel dimensions while retaining important information. Finally, the two feature information are spliced ​​together to prepare for subsequent operations. The formula is as follows: （11）； in: represents the channel attention operation; and Represents the convolution operations performed sequentially; Represents feature splicing; Represents the input source image; represents the pre-processed identity watermark; S322: linear attention module; The row-wise feature collaboration module consists of a feature extraction convolutional layer, a horizontal position encoding layer, and a linear layer. The input feature vector first passes through a convolutional layer to extract row local features: （12）； in: is the extracted row local feature, is the feature map preliminarily processed by SE-Stem, Conv () represents the convolution operation; Then enter the horizontal position encoding layer to perform matrix operations: （13）； in: for The calculation process of the horizontal position encoding is as follows: Input feature map , then the horizontal position encoding parameter , It is the preset maximum number of horizontal positions, which indicates the maximum position range in the horizontal direction that the model can handle. Select the width of the current feature map W Corresponding part ; for No. Row eigenvector , the calculation of the linear layer is: （14）； in: 、 and is the projection matrix, is the processed row feature; 、 、 Respectively represent Row eigenvector The weights obtained by multiplying the three different projection matrices, Represents the linear activation of the key matrix, softmax represents a linear activation function; The column-wise feature collaboration module consists of a feature extraction convolution layer, a vertical position encoding layer, and a linear layer. The input feature vector first passes through a convolution layer to extract column local features: （15）； in: is the extracted column local feature, are the features that have been preliminarily processed by SE-Stem; Then enter the vertical position encoding layer to perform matrix operations: （16）； in: for The calculation process of the vertical position encoding is as follows: Input feature map , then the vertical position encoding parameter , It is the preset maximum vertical position number, which indicates the maximum position range in the vertical direction that the model can handle. Select the part corresponding to the current feature map height H ; for No. Column feature vector , the calculation of the linear layer is: （17）； in: is the processed column feature; Respectively represent Column feature vector The weights obtained by multiplying the three different projection matrices, Represents linear activation of the key matrix; S323: multi-scale dilated downsampling convolution block; First calculate the gradient of the input feature , then according to the gradient With parameters Calculating dynamic expansion rate : （18）； Finally, dilated convolutions with different expansion rates are dynamically selected according to d for operation; int() indicates integerization; The entire downsampling process can be expressed as: （19）； in: Represents the feature map of the final output; Indicates the flattened layer; represents convolutional layer 2; Represents multi-scale dilated convolution; represents depth-wise separable convolution; represents convolutional layer 1; Indicates the initial reshaping of the input features; Represents the input features.

5. The active defense method based on face identity watermark and hybrid attention module according to claim 4 is characterized in that: The specific steps of S4 are: S41: weighted cross entropy loss; The weighted cross entropy loss is used to solve the class imbalance problem by assigning different weights to different classes. The formula is as follows: （20）； in: It is i The true category label of the face area detected by the individual face sample; is the probability of the positive class predicted by the model; is the weighted weight; S42: multi-scale image pixel loss; By introducing the idea of ​​multi-scale into the mean square error and calculating the mean square error at different scales, the details and structural information of the image can be better captured. The formula is as follows: （21）； in: is the number of scales; It is The number of pixels at each scale; and The watermarked image and the original image are Representation on a scale; S43: Adaptive texture feature message loss; Construct innovative adaptive loss functions; First, we need to extract the local texture features of the image. Local texture features help us understand the complexity of the image and thus decide in which areas to embed a stronger watermark. The formula is as follows: （22）； in: and The input images are gradient; is the image position Local texture features; Then, according to the local texture features, the adaptive weight is calculated to increase the watermark embedding strength in the texture-rich area and reduce the watermark embedding strength in the smooth area. （23）； in: is the image position Adaptive weights of and are the minimum and maximum values ​​of the texture features, respectively; Finally, the definition of the adaptive message loss function is: （24）； in: is the original watermark message; is the extracted watermark message; is a classification label, is the decoding threshold, is the decoding confidence; is the adaptive weight of position i, is the total number of pixels of the watermark; S44: The overall loss function is the weighted sum of weighted cross entropy loss, multi-scale image loss and adaptive message loss: （25）； in: 、 and The weight of each loss term.

6. The active defense method based on face identity watermark and hybrid attention module according to claim 5 is characterized by: The specific steps of S5 are: In order to accurately evaluate the similarity between the watermarks decoded under the two decoding methods, the regularized Pearson correlation coefficient is used to quantify the linear correlation between them. The regularized Pearson correlation coefficient is used to quantify the linear correlation between them. The formula is as follows: （26）； in: is a regularization term; and Represents the eigenvectors and No. Dimensional elements; is the dimension of the feature vector; and is the eigenvector and The average value of When it is 1, it means that the two vectors are completely positively correlated, and when it is 0, it means that there is no linear relationship between the two eigenvectors.

Images