Local image watermarking method, device and electronic equipment based on mask guidance and geometric constraint

Abstract

The application provides a local image watermarking method, device and electronic equipment based on mask guidance and geometric constraint. The method comprises the following steps: inputting a carrier image, an ROI mask and a traceable identifier into a watermark model, and outputting a watermark image through an encoder of the watermark model; the encoder samples the foreground contour of the ROI mask to generate a geometric landmark, splices the traceable identifier and the geometric landmark into a joint load, generates a watermark load feature map based on the joint load, and splices the carrier image and the ROI mask in a channel to generate a watermark image through a U-Net coding network; inputting a to-be-verified image into a decoder of the watermark model, wherein the decoder comprises a mask branch and a load branch; the mask branch predicts a watermark area of the to-be-verified image and outputs a predicted mask; the load branch performs ROI gating processing on the to-be-verified image based on the predicted mask, and recovers the traceable identifier and the geometric landmark based on an ROI gating feature map.

Description

Technical Field

[0001] This invention relates to the field of image security technology, specifically to a local image watermarking method, apparatus, and electronic device based on mask guidance and geometric constraints. Background Technology

[0002] Digital watermarking refers to embedding a set of identifying information into digital media such as images, audio, and video that is difficult to perceive and does not disrupt the normal use of the media. This allows the watermark information to be detected or extracted with high reliability even after conventional processing such as compression, filtering, cropping, and transcoding. The core difference between watermarking and cryptographic encryption is that watermarking typically requires that the content of the media be visually or audibly preserved, and that the media remain identifiable to third parties who know the key or the secret embedding rules.

[0003] Based on robustness, digital watermarks can be divided into robust watermarks, fragile watermarks, and semi-fragile watermarks: robust watermarks can resist conventional signal processing such as compression, noise, filtering, and geometric transformations, and are mostly used for copyright identification and traceability; fragile watermarks are extremely sensitive to modification, and are difficult to extract or fail to verify if slightly tampered with, and are mostly used for integrity authentication; semi-fragile watermarks are in between, and can still be maintained under conventional post-processing operations, but can be effectively detected when the carrier is maliciously tampered with.

[0004] Watermarking can be categorized into global watermarking and local watermarking based on its embedding scope. Global watermarking distributes information across the energy distribution of the entire image, while local watermarking concentrates information primarily in a designated Region of Interest (ROI), preserving the original image's statistical characteristics in non-ROI areas as much as possible. Compared to global watermarking, local watermarking offers several advantages: First, it provides a more visually appealing experience, embedding the watermark only in critical areas while completely preserving the original image's appearance and pixel information in non-protected areas, meeting the stringent requirements for background integrity and visual aesthetics in documents, tickets, and portraits. Second, it supports independent protection of multiple regions within a single image, allowing for the embedding of independent watermark information for different ROIs within the same image. This enables zoned tracing, detection, and localization, adapting to the refined security management needs of complex scenarios.

[0005] In practical business scenarios, the core requirements of digital watermarking are to achieve three major functions: source tracing, tamper detection, and location. Source tracing is used to determine the origin and ownership of an image; tamper detection is used to determine whether an image has been tampered with; and tamper location is used to accurately pinpoint the specific area where tampering occurred. Currently, global watermarking can simultaneously perform these three functions within a single watermark system. Because global watermarking embeds watermark information into the entire image domain, during integrity verification, the integrity of the watermark can be directly determined based on its global distribution characteristics, and the damaged area can be accurately located, thereby achieving tamper detection and location. However, local watermarking is fundamentally different: the watermark is only embedded in a specified local protected area, and the embedding position and range vary for different images. This leads to a key defect in integrity detection: even if the watermark appears to be ineffective across the entire image, it is impossible to distinguish whether the watermark-free area is the original unprotected area or has been maliciously tampered with, thus making it impossible to reliably complete tamper detection and location.

[0006] Deep learning has propelled digital watermarking into an end-to-end learnable paradigm, with a typical architecture consisting of an encoder, a distortion layer, and a decoder. The encoder maps the carrier image to the watermark message, resulting in a watermarked image. The distortion layer simulates real-world image perturbations such as JPEG compression, noise, blurring, and geometric transformations. The decoder recovers the watermark information from the perturbed image and outputs spatial location information such as the watermark region mask. Model training employs a multi-task loss function for joint optimization, automatically balancing watermark capacity, imperceptibility, and robustness.

[0007] Existing mask-guided local watermarking methods achieve local embedding and extraction of watermarks through masks. Among them, the WAM method uses mask supervision to locate the watermark region at the decoding end; while the MaskWM-ED method (local watermarking method) introduces mask prior at the encoding end, making the watermark information more concentrated in the region of interest (ROI).

[0008] However, the relevant local watermarking schemes either can only achieve a single function or have defects such as the need to rely on the mask used in the embedding stage as prior information in the watermark verification stage, multiple model concatenation, and weak interpretability. At present, there is no local image watermarking method that can simultaneously achieve the three core functions of source tracing, tamper detection and tamper location within an end-to-end deep learning framework. Summary of the Invention

[0009] In view of this, this application proposes a local image watermarking method, device and electronic device based on mask guidance and geometric constraints, which solves the technical problem that local watermarking cannot simultaneously achieve the three core functions of source tracing, tamper detection and localization, realizes the integrated synergy of the three functions, and improves the interpretability and practical applicability of local watermarking.

[0010] Specifically, this application is implemented through the following technical solution:

[0011] According to a first aspect of the embodiments of this specification, a local image watermarking method based on mask guidance and geometric constraints is provided, comprising the following steps:

[0012] Step S1: Input the carrier image to be embedded with watermark, the ROI mask, and the source identifier into a watermarking model based on an encoding / decoding architecture. Output the watermarked image through the encoder of the watermarking model. The encoder is configured to sample the foreground contour of the ROI mask to generate geometric landmarks, and concatenate the source identifier and the geometric landmarks into a joint payload. Generate a watermark payload feature map based on the joint payload. After channel concatenation of the watermark payload feature map, the carrier image, and the ROI mask, the image is processed by a U-Net encoding network to generate the watermarked image.

[0013] Step S2: The image to be verified is input into the decoder of the watermark model. The decoder includes a mask branch and a payload branch. The mask branch is configured to predict the watermark region of the image to be verified and output a predicted mask. The payload branch is configured to perform ROI gating processing on the image to be verified based on the predicted mask, that is, to retain only the image features of the region corresponding to the predicted mask, suppress the interference of non-protected region features, and recover the source identification and geometric landmarks based on the ROI gating feature map.

[0014] According to a second aspect of the embodiments of this specification, a local image watermarking device based on mask guidance and geometric constraints is provided, the device comprising:

[0015] A watermark embedding unit is used to input the carrier image to be embedded with watermark, the ROI mask, and the source identifier into a watermarking model based on an encoding / decoding architecture, and output a watermarked image through the encoder of the watermarking model; wherein, the encoder is configured to sample the foreground contour of the ROI mask to generate geometric landmarks, and concatenate the source identifier and the geometric landmarks into a joint payload, and generate a watermark payload feature map based on the joint payload; after channel concatenation of the watermark payload feature map, the carrier image, and the ROI mask, the watermarked image is generated through a U-Net encoding network.

[0016] A watermark detection unit is used to input the image to be verified into the decoder of the watermark model. The decoder includes a mask branch and a payload branch. The mask branch is configured to predict the watermark region of the image to be verified and output a predicted mask. The payload branch is configured to perform ROI gating processing on the image to be verified based on the predicted mask and recover the source identification and geometric landmarks based on the ROI-gated feature map.

[0017] According to a third aspect of the embodiments of this specification, an electronic device is provided, comprising:

[0018] processor;

[0019] A computer-readable storage medium storing computer program instructions that, when executed by the processor, cause the processor to perform the method as described in the first aspect.

[0020] The embodiments of this application have at least the following technical effects:

[0021] First, the embodiments of this application integrate the three core functions of source tracing, tamper detection and localization in local watermarking. Through joint payload design and multi-task joint optimization, source tracing identifier extraction, tamper detection and localization are simultaneously realized in a single end-to-end model, which solves the problem that the existing technology cannot take into account the three functions at the same time.

[0022] Second, the architecture of the payload branch consists of a shared backbone network and two decoding heads, which decouples the two different subtasks of geometric landmark regression and source identification classification. This allows for joint optimization of the two tasks while respecting their unique mathematical properties, reducing system fragmentation, reducing interface costs and latency for multi-model joint debugging, and ensuring the coordination and consistency of the three major functions.

[0023] Third, the geometric landmarks and predicted masks in the embodiments of this application support cross-validation to intuitively present the tampered area and degree of tampering, which facilitates auditing and visualization report generation, helps to improve the interpretability of evidence collection, and can enhance the practicality of detection and location.

[0024] Fourth, while keeping the core topology of the watermark model unchanged, the embodiments of this application support flexible adjustment of the number of traceability identifiers, the number of landmark sampling points, and the number of watermark payload feature map channels, achieving a flexible trade-off between payload capacity and computing power consumption, and adapting to the differentiated needs of the three major functions in different scenarios.

[0025] Fifth, in this application embodiment, the watermark is embedded only within the ROI, while the non-ROI area retains the original state of the carrier, meeting the strict requirements for the appearance of the image background in scenarios such as certificates and signatures, while ensuring that the watermark information is concentrated and improving the accuracy of the three functions. Attached Figure Description

[0026] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the accompanying drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Some specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings in an exemplary and non-limiting manner. The same reference numerals in the drawings indicate the same or similar parts or components. Those skilled in the art should understand that these drawings are not necessarily drawn to scale. In the drawings:

[0027] Figure 1This is a schematic diagram illustrating an exemplary embodiment of the present application of a process for watermark embedding and watermark detection based on a watermark model;

[0028] Figure 2 This is a flowchart illustrating a local image watermarking method based on mask guidance and geometric constraints, as shown in an exemplary embodiment of this application.

[0029] Figure 3 This is a schematic diagram illustrating the payload image and its ROI mask according to an exemplary embodiment of this application;

[0030] Figure 4 This is a schematic diagram illustrating the effect of partial watermark embedding and restoration in an exemplary embodiment of this application;

[0031] Figure 5 This is a schematic diagram of experimental data shown in an exemplary embodiment of this application;

[0032] Figure 6 This is a structural block diagram of an electronic device illustrated in an exemplary embodiment of this application;

[0033] Figure 7 This is a structural block diagram of a local image watermarking device based on mask guidance and geometric constraints, as illustrated in an exemplary embodiment of this application. Detailed Implementation

[0034] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0035] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0036] This application provides a local image watermarking scheme based on mask guidance and geometric constraints. The scheme embeds and detects local watermarks based on a pre-constructed watermark model. This watermark model can be an end-to-end model based on a U-Net encoding / decoding network, such as... Figure 1As shown, the watermarking model in this embodiment includes an encoder and a decoder. The encoder is used for local watermark embedding, using a U-Net coding structure to transform the joint payload into a watermark payload feature map, and, guided by the ROI mask, concentrates the watermark information into the ROI region. Simultaneously, the JND module (Just-Aware Distortion Module, used to calculate the maximum tolerable distortion threshold for local image based on human visual characteristics) controls invisibility. The decoder receives potentially vulnerable images to be verified, and uses a dual-branch architecture to predict the mask and recover the source identification and geometric landmarks.

[0037] Ideally, the prediction mask output by the decoder should match the ROI mask input by the encoder, the recovered tracing identifier should match the tracing identifier embedded by the encoder, and the recovered geometric landmark should match the geometric landmark sampled by the encoder.

[0038] In this embodiment, both the encoder and decoder are deep networks, but their structures are not symmetrical. For example... Figure 1 As shown, the encoder's input consists of a carrier image, a ROI mask, and a source identifier. The backbone network employs a U-Net structure and is equipped with a payload preprocessing component and a JND modulation component. The payload preprocessing component maps, reassembles, upsamples, and convolves the input to obtain input data suitable for the backbone network. The decoder's input is a watermarked image. It uses a dual-branch architecture consisting of a mask branch and a payload branch. The payload branch also uses a U-Net structure as a shared backbone network and is equipped with a dual-head output component.

[0039] In this embodiment, the encoder and decoder of the watermark model are trained through a joint task and robustness enhancement processing is performed. During training, a distortion layer is inserted after the encoder output and before the decoder input to simulate attacks such as JPEG, noise, and filtering. The decoder must recover information from the attacked image, thereby prompting the encoder to learn a robust embedding method. The encoder and decoder form a joint optimization around distortion resistance, constituting a training mechanism of attack and defense game.

[0040] For example, during the training phase, a combined distortion enhancement strategy is employed, randomly applying various image perturbations, including: identity transformation, JPEG compression simulation (JPEG Mask, quality factor 50, chroma subsampling 0), JPEG compression (JPEG Compression, quality factor 50–80 random, chroma subsampling 2), Gaussian filter (standard deviation 1, kernel size 5×5), Gaussian noise (mean 0, standard deviation 0.1), median filter (median filter, kernel size 3×3), and salt and pepper noise (perturbation probability 0.1).

[0041] Each training step randomly selects a sublayer from the attack list to apply the attack, ensuring that each attack type is fully sampled during long-term training, rather than using a fixed-order cascade pipeline. This allows for a more comprehensive simulation of image degradation in the real propagation chain, improving the model's robustness and generalization ability, and ensuring that the source tracing, detection, and localization functions remain stable under complex attacks.

[0042] The noise preheating iteration step number is set to 6000. Before reaching this number of iteration steps, only the identity transformation is applied, that is, no image distortion is introduced. After reaching the preset number of iteration steps, the full combination of distortion enhancement is switched to avoid the model being difficult to converge due to excessive perturbation intensity in the early stage of training.

[0043] It is worth noting that the above-mentioned distortion types and noise preheating iteration steps are only preferred parameters and can be flexibly adjusted according to actual training needs, as long as the robustness of the three major functions can be guaranteed.

[0044] The joint loss function used in this embodiment is as follows:

[0045] (1)

[0046] In formula (1), The decoder weights decay with training, for example from 2.0 to 0.05, with a decay rate of 0.05; The encoder loss is used to constrain the visual distortion of the watermark embedding, ensuring that the watermark image remains consistent with the original carrier. The original carrier image, The watermark image output by the encoder. The per-pixel mean square error function; For the decoder loss, where:

[0047] To optimize the accuracy of source identification extraction, a binary cross-entropy loss with sigmoid activation (BCEWithLogits) is used.

[0048] (2)

[0049] In formula (2), m represents the truth source identifier bit sequence; z represents the inactive prediction logits output by the decoder identifier header, that is, the original prediction value that has not been activated by Sigmoid.

[0050] To constrain the accuracy of geometric landmark coordinate prediction, the pixel-plane average Euclidean distance is used.

[0051] (3)

[0052] In formula (3), B is the batch size, and K is the number of landmarks. Let (x, y) represent the ground truth pixel coordinates of the k-th landmark in the b-th image. Let (x, y) represent the predicted pixel coordinates (x, y) of the k-th landmark in the b-th image.

[0053] To improve the accuracy of protected area prediction, pixel-wise mean square error is used:

[0054] (4)

[0055] In formula (4), For the truth mask, The mask is used to predict the decoder.

[0056] The optimizer used in this embodiment is AdamW, with a learning rate (lr) of 1×10−4 and a cosine learning rate scheduling strategy. Training parameters: the number of noise warm-up iterations is 200,000, and the batch size (batch_size, i.e., the number of training samples input to the network in a single iteration) is 16; the JND module is set to start at 0.05 iterations, meaning that the JND module is turned off within the corresponding number of iterations in the early stage of training, and then turned on again after the corresponding number of iterations is reached to control the watermark invisibility.

[0057] During the inference phase of the watermarking model, the encoder and decoder can be used independently. That is, when a watermark needs to be embedded in an image, only the encoder needs to be run, and the decoder does not need to be run. When it is necessary to verify the image source or detect tampering, only the decoder needs to be run, and the encoder does not need to be run.

[0058] The embodiments described in this specification will now be described in detail.

[0059] This application provides a local image watermarking method based on mask guidance and geometric constraints, which is applicable to scenarios where the original image appearance needs to be maintained as much as possible in non-ROI areas, such as ID card portraits, invoice signatures, product labels, and privacy-sensitive occluded areas. This embodiment supports the use of the predicted mask, recovered traceability identifier and geometric landmarks output by the decoder at the verification end to realize copyright traceability, tamper detection, tamper area location and visualization analysis, providing reliable technical support for the security management of digital images.

[0060] Figure 2 This is a flowchart illustrating an exemplary embodiment of a local image watermarking method based on mask guidance and geometric constraints, as shown in this application. Figure 2 As shown, the local image watermarking method includes the following steps:

[0061] Step S1: Input the carrier image to be embedded with watermark, the ROI mask, and the source identifier into a watermarking model based on an encoding / decoding architecture. Output the watermarked image through the encoder of the watermarking model. The encoder is configured to sample the foreground contour of the ROI mask to generate geometric landmarks, and to concatenate the source identifier and the geometric landmarks into a joint payload. A watermark payload feature map is generated based on the joint payload. After channel concatenation of the watermark payload feature map, the carrier image, and the ROI mask, the watermarked image is generated through a U-Net encoding network.

[0062] In this embodiment, geometric landmarks are generated by uniformly sampling the closed foreground contour of the ROI mask according to the arc length.

[0063] For example, several points are obtained by uniformly sampling the outer contour of the ROI mask according to the arc length. The coordinates of each point are normalized and flattened into a one-dimensional vector, which is used to perform joint verification with the prediction mask at the decoding end to realize tamper location.

[0064] The watermark payload feature map is a multi-channel feature map aligned with the carrier image space, used to carry the watermark information to be embedded.

[0065] Step S2: Input the image to be verified into the decoder of the watermark model. The decoder includes a mask branch and a payload branch. The mask branch is configured to predict the watermark region of the image to be verified and output a predicted mask. The payload branch is configured to perform ROI gating processing on the image to be verified based on the predicted mask and recover the source identification and geometric landmarks based on the ROI gating feature map.

[0066] Based on the predicted mask, the image to be verified is subjected to ROI gating processing to obtain an ROI gating feature map. The predicted ROI gating feature map is the predicted watermark region. That is, the gating processing retains only the image features of the region corresponding to the predicted mask and suppresses the interference of non-protected region features.

[0067] In some embodiments, step S1, generating a watermarked load feature map based on the joint load, includes:

[0068] The joint payload is linearly mapped to high-dimensional features and shaped into a two-dimensional feature map; the two-dimensional feature map is upsampled to the resolution of the carrier image; the upsampled two-dimensional feature map is convolved to generate the watermark payload feature map.

[0069] In some embodiments, step S1, which processes and outputs the watermarked image via a U-Net encoding network, includes:

[0070] The multi-channel watermark feature map generated by the U-Net encoding network is concatenated with the carrier image and then processed by convolution to obtain the whole image encoding result. The whole image encoding result and the carrier image are then weighted and fused pixel by pixel using the ROI mask to output the watermarked image.

[0071] In this embodiment, the carrier image resolution is 256×256 pixels. During inference, a higher resolution image can be downsampled to 256×256 for encoding, and the watermark residual can be magnified back to the original resolution. (Continue to refer to...) Figure 1 The encoding process in this embodiment is as follows:

[0072] (1) Generation of geometric landmarks and joint loads.

[0073] Using OpenCV's findContours function, configured as follows

[0074] RETR_EXTERNAL+CHAIN_APPROX_NONE extracts the closed polyline of the foreground contour from the ROI mask. The carrier image and its corresponding ROI mask can be found in [reference needed]. Figure 3 As shown, in practical applications, if the ROI mask contains multiple connected components, such as holes, then the landmark sampling points are allocated according to the perimeter ratio of each contour to ensure a reasonable distribution of landmarks.

[0075] The number of landmark sampling points is set to K=12. Sampling is performed uniformly along the contour according to the arc length. After flattening the coordinates of each point, a 24-dimensional coordinate vector is obtained. Coordinate normalization uses an independent mode (x / image width, y / image height) to make the coordinates independent of the image size. The traceability identifier can be set to 8 bits. The 8-bit traceability identifier is concatenated with the 24-dimensional geometric landmark vector to obtain a 32-dimensional joint payload.

[0076] (2) Generation of watermark load feature map.

[0077] The 32-dimensional joint payload is mapped to 32×32×32-dimensional features through a linear layer and shaped into a 32-channel feature map with a spatial size of 32×32. After being refined by two layers of watermark branch convolutional blocks at this resolution, it is upsampled to the carrier image resolution of 256×256 using bilinear interpolation. The upsampled feature map is then passed through three layers of 3×3 convolution, normalization and rectification units (ConvNormReLU) in sequence, with the number of channels being 32→32→32→32. Group normalization is used to obtain the final 32-channel watermark payload feature map.

[0078] (3) Encoding network processing.

[0079] The three-channel carrier image features, the 32-channel watermark payload feature map, and the 1-channel ROI mask are concatenated in the channel dimension to obtain 36-channel input features.

[0080] The feature is input into the U-Net encoding network, which contains symmetrical downsampling and upsampling paths: the first convolutional block maps 36 channels to 64 channels, the number of channels on the downsampling side is successively expanded to 64, 128, 256, and 512, the bottleneck layer is expanded to 1024 dimensions, and then the spatial resolution is restored through step-by-step upsampling and skip connections, outputting a multi-channel watermark encoded feature map, such as 64 channels.

[0081] The multi-channel watermark encoding feature map is then concatenated with the carrier image in the channel dimension and fused through a 1×1 convolutional layer to obtain a 3-channel whole image encoding result with the same resolution as the carrier.

[0082] (4) JND visual modulation.

[0083] After inversely normalizing the above image encoding results to the [0,1] domain, visual weighted modulation is performed using the Just Noticeable Distortion (JND) module. The JND module converts the RGB image into a luminance channel according to weights [0.299, 0.587, 0.114], calculates luminance masking (LA) and contrast masking (CM) respectively, and fuses them to obtain an imperceptible heatmap. The watermark residual is modulated in the [0,1] domain, and then the result is mapped back to the original normalized domain.

[0084] It is worth noting that JND is disabled proportionally in the early stages of model training to stabilize convergence. Those skilled in the art can refer to relevant technologies for the specific processing procedures of the JND module; this embodiment will not elaborate further.

[0085] (5) Mask fusion yields a watermarked image.

[0086] The modulated whole-image encoding result is weighted and fused pixel-by-pixel with the original carrier image using the ROI mask, and the formula is as follows:

[0087] (5)

[0088] in, The encoded result of the entire image output by the encoder. The original carrier image, This is the ROI mask.

[0089] During the fusion process, the watermark is used to encode the image only in the ROI region where M=1, while the original image remains unchanged in the non-ROI region where M=0, resulting in the following image. Figure 4 The watermarked image shown ensures the visual integrity of the background area while concentrating the watermark information on the ROI, improving the accuracy of tracing, detection, and localization, and avoiding background interference.

[0090] In some embodiments, step S2, which involves recovering the source identification and geometric landmarks based on the ROI-gated feature map, includes:

[0091] The ROI gated feature map is downsampled to a two-dimensional feature map after feature extraction by the U-Net decoding network, and then flattened into a feature vector. The feature vector is then input into the identifier header and the landmark header respectively. The source identification is recovered through the identifier header, and the geometric landmark is recovered through the landmark header.

[0092] Watermarked images may be subjected to disturbances such as noise, compression, and filtering during transmission or storage. The disturbed images are then input into the decoder.

[0093] The decoder in this embodiment adopts a dual-branch architecture, including a mask branch and a payload branch, which are the core components for realizing the three major functions of source tracing, detection, and localization. During inference, the watermarked area is first obtained through the mask branch, and then the payload is extracted in that area to avoid interference from watermark-free areas.

[0094] For the mask branch:

[0095] The mask branch is used to predict the watermark region to support tamper detection and localization functions. The mask branch adopts the U²-Net small mask prediction branch, which includes 6 levels of RSU coding, 5 downsampling and multi-scale side output fusion, and outputs a single-channel predicted mask with the same resolution as the carrier image.

[0096] During training, the loss function of the mask branch is the mean square error (MSE) between the predicted mask and the reference mask, and the weights can be adjusted flexibly, with an optimal value of 0.5.

[0097] For the load branch:

[0098] The payload branch first multiplies the image to be validated pixel-by-pixel with the predicted mask output by the mask branch to obtain the ROI-gated feature map, and then performs the following operations on the ROI-gated feature map:

[0099] Feature extraction: The feature extraction branch, composed of convolution and U-Net, extracts features from the ROI-gated feature map.

[0100] Feature compression and dimension normalization mapping: The extracted features are compressed into a single channel through 3 layers of convolution, then downsampled to the same size as the square of the joint payload length (e.g., 32×32) by nearest neighbor, and then flattened into a feature vector of fixed dimensions (e.g., 1024-dimensional).

[0101] Dual decoding head output: The feature vector is fed into the identifier head and the landmark head respectively.

[0102] The identifier header is used to recover the source identifier, supporting the source tracing function. The source of the image can be determined by the recovered source identifier. The identifier header adopts a structure of fully connected, non-linear activation and fully connected layers in sequence, such as Linear(1024, 512), ReLU, Linear(512, 8), and outputs the logits of the source identifier, which are then processed by sigmoid and other operations to obtain an 8-bit source identifier.

[0103] The loss function during training is BCEWithLogitsLoss, and the weights can be adjusted flexibly, with an optimal value of 0.4.

[0104] The landmark header is used to recover geometric landmarks and support the positioning function. The landmark header adopts a structure of fully connected, non-linear activation and fully connected layers in sequence, such as Linear(1024, 512), ReLU, Linear(512, 24), and outputs 24-dimensional geometric landmark coordinates.

[0105] During training, the loss function is the contour average Euclidean distance loss. It is necessary to map the normalized coordinates back to the pixel plane, calculate the point-by-point Euclidean distance, and then take the average value. The weight can be flexibly adjusted, with an optimal value of 0.6.

[0106] In some embodiments, Figure 2 The method shown also includes:

[0107] Step S3: Use the recovered source identifier to trace the image source, and perform tamper sensitivity analysis based on the predicted mask and the recovered geometric landmarks without needing to obtain the ROI mask used when encoding the image to be verified.

[0108] If it is determined that the image to be verified has been tampered with, this embodiment visualizes the positional relationship between the restored geometric landmarks and the outline.

[0109] Without needing to obtain the original ROI mask used when encoding the image to be verified, the decoder performs tamper sensitivity analysis based on the predicted mask and the recovered geometric landmarks. The specific process is as follows:

[0110] First, the predicted mask is binarized to obtain a clear outline of the watermark region. Then, using this outline as a reference, the decoded geometric landmark coordinates are mapped to pixel coordinates, and the distribution consistency between the recovered landmarks and the predicted mask outline is checked.

[0111] Since the ground truth landmarks generated during encoding are obtained by uniformly sampling along the complete edge of the mask by arc length, the recovered landmarks should normally be evenly distributed near the predicted mask contour. For example... Figure 4 The red dots in the image are located on the outline of the ROI mask. The image is considered tampered with if any of the following conditions are met:

[0112] The positions of the restored landmarks and the predicted mask contours are clearly mismatched (e.g., landmarks are off-center or unevenly distributed).

[0113] The predicted mask outline shows voids or breaks without corresponding landmark support.

[0114] In practical applications, the specific threshold for determining tampering, such as the value at which the offset distance between the landmark and the predicted mask outline is greater than a certain value, is customized by the deployer based on the actual business scenario and is not fixed to the model itself, thereby ensuring the flexibility and adaptability of the solution.

[0115] In addition, to verify the effectiveness of the local image watermarking method based on mask guidance and geometric constraints in this application, a systematic comparative experiment was conducted on the COCO dataset in this embodiment.

[0116] The experiment selected image samples from the COCO dataset that contained clear foreground targets and were suitable for local protected area annotation. All images were uniformly adjusted to 512×512 resolution to ensure the consistency and comparability of the experimental inputs.

[0117] The local protection region masks used in the experiment include three typical forms: random rectangular masks (simulating regular local protection regions, such as ID photo frames and fixed areas for signatures), COCO instance masks (based on the original target masks of the dataset to simulate naturally irregular protection regions), and irregular stroke masks (simulating irregular flexible protection regions such as manual annotation and privacy occlusion). During the mask generation process, the mask is randomly horizontally flipped with a 50% probability to enhance the generalization and robustness verification of the experiment and to meet the diverse needs of protection regions in real business.

[0118] The noise used specifically includes:

[0119] Identity() (no attack control), JpegMask(50) (50 quality factor JPEG compression simulation attack), JpegTest(50, 80) (JPEG compression attack with quality factor randomly selected between 50 and 80), GF(1,5) (Gaussian blur attack, radius 1, kernel size 5), GN(0,0.1) (Gaussian noise attack, mean 0, variance 0.1), MF(3) (median filtering attack, kernel size 3×3), SP(0.1) (salt and pepper noise attack, noise density 0.1).

[0120] The technical solution of this application was compared with existing source tracing local image watermarking methods WAM and MassWM-ED. All comparison methods adopted the optimal JND parameter configuration and training strategy, and the test environment, hardware platform and evaluation process were unified to ensure the fairness and scientific nature of the comparison experiment.

[0121] The experiment uses the following quantitative indicators to comprehensively evaluate the performance of each method. The definitions and criteria for judging the merits of each indicator are as follows:

[0122] SSIM (Structural Similarity): Used to measure the structural similarity between a watermarked image and the original image. The higher the value, the better the imperceptibility of the watermark and the better the visual quality of the image.

[0123] PSNR (Peak Signal-to-Noise Ratio): Used to evaluate the objective fidelity of watermarked images. The higher the value, the smaller the distortion effect of the watermark on the image and the stronger its visual imperceptibility.

[0124] Identity bit error rate: This measures the accuracy of extracting watermark identity information. The smaller the value, the higher the accuracy of watermark extraction and the stronger the traceability capability.

[0125] Mask IoU (Intersection over Union): Used to evaluate the prediction accuracy of the watermark area mask. The larger the value, the more accurate the mask prediction and the stronger the local area positioning capability.

[0126] Average Euclidean distance between contour points: This measures the accuracy of contour landmark point restoration. The smaller the value, the smaller the landmark offset and the better the geometric constraint effect.

[0127] Among them, the experimental results of various watermarking methods are as follows: Figure 5 As shown, the SSIM and PSNR of the method in this application are only slightly lower than those of MassWM_ED. Under noise-free conditions, the identity bit error rate is slightly higher than that of MassWM_ED. This difference is due to the fact that the method in this application needs to optimize multiple tasks such as watermark embedding, mask prediction and contour landmark restoration within a single framework, and cannot perform extreme optimization on a single indicator as the comparative method. However, the mask IoU of the method in this application reaches 0.99, and it can achieve accurate contour landmark restoration. The average Euclidean distance between contour point pairs is only 4.74, the landmark offset is small, and the geometric consistency is excellent.

[0128] Under noisy conditions, the mask IoU of the comparative methods dropped significantly to 0.79, while the mask IoU of the method in this application remained at 0.91, which is significantly better than the comparative methods, demonstrating stronger mask prediction robustness to experimental noise. At the same time, the average Euclidean distance between contour point pairs of the method in this application is only 9.00, which still maintains extremely low landmark offset under noise interference, and outstanding geometric constraint stability.

[0129] Overall, the method of this application can effectively maintain indicators such as imperceptibility and identity bit error rate, and significantly improves landmark geometric accuracy, comprehensive positioning capability and functional integrity, realizing the synergistic effect of tracing, detection and positioning in local watermarking scenarios.

[0130] Based on the above embodiments of this application, the speech enhancement solution of this application has at least the following advantages:

[0131] First, it breaks through the industry's technical bottlenecks and achieves the integration of three core functions in local watermarking for the first time: through joint payload design, dual-branch decoding architecture and multi-task joint optimization, it simultaneously realizes source identification extraction, tamper detection and pixel-level positioning in a single end-to-end model, solving the problem that existing technologies cannot take all three functions into account.

[0132] Second, single-model end-to-end integration: source identification and contour landmark sharing network backbone optimization path realize the integration of the three functions of source tracing, landmark restoration and tamper location, reduce system fragmentation, reduce interface cost and latency of multi-model joint debugging, and ensure the coordination and consistency of the three functions;

[0133] Third, interpretable evidence collection assistance: the outline landmark and the predicted mask can be cross-validated, which can intuitively present the tampered area and the degree of tampering, facilitate the generation of audit and visualization reports, improve the interpretability of evidence collection, and further enhance the practicality of detection and location.

[0134] Fourth, robustness adapts to real-world scenarios: Combined random distortion training covers common signal processing and attacks, improving the model's adaptability to real propagation paths and ensuring the reliability of source tracing, detection, and localization functions in practical applications.

[0135] Fifth, the capacity is flexible and scalable: while keeping the core topology of joint load-square grid-TP mapping-dual-head split unchanged, the number of traceability identifier bits, the number of landmark sampling points and the number of watermark feature map channels can be flexibly adjusted, achieving a flexible trade-off between load capacity and computing power consumption, and adapting to the differentiated needs of the three major functions in different scenarios.

[0136] Sixth, high local fidelity: the watermark is embedded only within the ROI, while the non-ROI area retains the original state of the carrier, meeting the strict requirements for image background appearance in scenarios such as certificates and signatures, while ensuring the concentration of watermark information and improving the accuracy of the three functions.

[0137] Figure 6 This is a schematic diagram of an electronic device illustrated in this specification according to an exemplary embodiment. Please refer to... Figure 6At the hardware level, the device includes a processor 602, an internal bus 604, a network interface 606, memory 608, a hardware acceleration device 610, and non-volatile memory 612, and may also include other hardware required for its functions. One or more embodiments of this application can be implemented in software, for example, the processor 602 reads the corresponding computer program from the non-volatile memory 612 into memory 608 and then runs it. Of course, in addition to software implementation, one or more embodiments of this application do not exclude other implementation methods, such as logic devices or a combination of hardware and software, etc. That is to say, the execution subject of the above processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0138] Figure 7 This is a structural block diagram illustrating a local image watermarking device based on mask guidance and geometric constraints, as shown in an exemplary embodiment of this application. The local image watermarking device can be applied to, for example... Figure 6 The electronic device shown implements the technical solution of this application. The local image watermarking device includes: a watermark embedding unit 710 and a watermark detection unit 720, wherein:

[0139] The watermark embedding unit 710 is used to input the carrier image to be embedded with watermark, the ROI mask, and the traceability identifier into a watermark model based on an encoding and decoding architecture, and output a watermarked image through the encoder of the watermark model; wherein, the encoder is configured to sample the foreground contour of the ROI mask to generate geometric landmarks, and concatenate the traceability identifier and the geometric landmarks into a joint payload, and generate a watermark payload feature map based on the joint payload; after channel concatenation of the watermark payload feature map, the carrier image, and the ROI mask, the watermarked image is generated through a U-Net encoding network.

[0140] The watermark detection unit 720 is used to input the image to be verified into the decoder of the watermark model. The decoder includes a mask branch and a payload branch. The mask branch is configured to predict the watermark region of the image to be verified and output a predicted mask. The payload branch is configured to perform ROI gating processing on the image to be verified based on the predicted mask and recover the source identification and geometric landmarks based on the ROI gating feature map.

[0141] In some embodiments, the local image watermarking device includes;

[0142] The horizontal evaluation unit 730 is used to perform image tracing using the recovered tracing identifiers, and to perform tamper sensitivity analysis based on the predicted mask and the recovered geometric landmarks without needing to obtain the ROI mask used when the image to be verified was encoded.

[0143] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this application according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0144] Accordingly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the methods described in any of the above embodiments.

[0145] Accordingly, embodiments of this application also provide a computer program product configured to perform the methods described in any of the above embodiments.

[0146] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer, which can take the form of a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email sending and receiving device, game console, tablet computer, wearable device, or any combination of these devices.

[0147] In a typical configuration, a computer includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0148] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0149] Computer-readable media, including both permanent and non-permanent, removable and non-removable media, can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random-access memory (SRAM), dynamic random-access memory (DRAM), other types of random-access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, disk storage, quantum memory, graphene-based storage media or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0150] While this specification contains numerous specific implementation details, these should not be construed as limiting the scope of any invention or the scope of the claims, but rather are primarily intended to describe features of specific embodiments of a particular invention. Certain features described in the various embodiments herein may also be implemented in combination in a single embodiment. Conversely, various features described in a single embodiment may also be implemented separately in various embodiments or in any suitable sub-combination. Furthermore, while features may function in certain combinations as described above and even initially claimed in this way, one or more features from a claimed combination may be removed from that combination in some cases, and a claimed combination may refer to a sub-combination or a variation thereof.

[0151] Similarly, although the operations are depicted in a specific order in the accompanying drawings, this should not be construed as requiring these operations to be performed in the specific order shown or sequentially, or requiring all illustrated operations to be performed to achieve the desired result. In some cases, multitasking and parallel processing may be advantageous. Furthermore, the separation of various system modules and components in the above embodiments should not be construed as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

[0152] Thus, specific embodiments of the subject matter have been described. Other embodiments are within the scope of the appended claims. In some cases, the actions recited in the claims may be performed in a different order and still achieve the desired result. Furthermore, the processes depicted in the drawings are not necessarily shown in a specific order or sequence to achieve the desired result. In some implementations, multitasking and parallel processing may be advantageous.

[0153] It should be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0154] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A local image watermarking method based on mask guidance and geometric constraints, characterized in that, The method includes at least the following steps: Step S1: Input the carrier image to be embedded with watermark, the ROI mask, and the source identifier into a watermarking model based on an encoding / decoding architecture. Output the watermarked image through the encoder of the watermarking model. The encoder is configured to sample the foreground contour of the ROI mask to generate geometric landmarks, and concatenate the source identifier and the geometric landmarks into a joint payload. Generate a watermark payload feature map based on the joint payload. After channel concatenation of the watermark payload feature map, the carrier image, and the ROI mask, the image is processed by a U-Net encoding network to generate the watermarked image. Step S2: Input the image to be verified into the decoder of the watermark model. The decoder includes a mask branch and a payload branch. The mask branch is configured to predict the watermark region of the image to be verified and output a predicted mask. The payload branch is configured to perform ROI gating processing on the image to be verified based on the predicted mask and recover the source identification and geometric landmarks based on the ROI gating feature map.

2. The method according to claim 1, characterized in that, Step S1, which generates a watermarked payload feature map based on the joint payload, includes: The joint load is linearly mapped to high-dimensional features and then shaped into a two-dimensional feature map; The two-dimensional feature map is upsampled to the resolution of the carrier image; The upsampled two-dimensional feature map is convolved to generate the watermark payload feature map.

3. The method according to claim 1, characterized in that, The step S1, which processes and outputs the watermarked image via the U-Net encoding network, includes: The multi-channel watermark feature map generated by the U-Net encoding network is concatenated with the carrier image and then subjected to convolution processing to obtain the whole image encoding result; The ROI mask is used to perform pixel-by-pixel weighted fusion of the whole image encoding result and the carrier image to output a watermarked image.

4. The method according to claim 1, characterized in that, Step S2, which involves recovering the source identification and geometric landmarks based on the ROI-gated feature map, includes: The ROI gated feature map is downsampled to a two-dimensional feature map after feature extraction by the U-Net decoding network, and then flattened into a feature vector. The feature vectors are input into the identifier header and the landmark header respectively. The source identifier is recovered through the identifier header, and the geometric landmark is recovered through the landmark header.

5. The method according to any one of claims 1 to 4, characterized in that, It also includes the following steps: Step S3: Use the recovered source identifier to trace the image source, and perform tamper sensitivity analysis based on the predicted mask and the recovered geometric landmarks without needing to obtain the ROI mask used when encoding the image to be verified.

6. The method according to claim 5, characterized in that, Step S3 involves performing tamper sensitivity analysis based on the predicted mask and the recovered geometric landmarks, including: Based on the outline of the predicted mask, verify the consistency between the distribution of the recovered geometric landmarks and the outline. If the recovered geometric landmarks do not match the position of the outline, it is determined that the image to be verified has been tampered with.

7. The method according to claim 6, characterized in that, If it is determined that the image to be verified has been tampered with, the method further includes: The visualization shows the positional relationship between the restored geometric landmarks and the outline.

8. A local image watermarking device based on mask guidance and geometric constraints, characterized in that, The device includes: A watermark embedding unit is used to input the carrier image to be embedded with watermark, the ROI mask, and the source identifier into a watermarking model based on an encoding / decoding architecture, and output a watermarked image through the encoder of the watermarking model; wherein, the encoder is configured to sample the foreground contour of the ROI mask to generate geometric landmarks, and concatenate the source identifier and the geometric landmarks into a joint payload, and generate a watermark payload feature map based on the joint payload; after channel concatenation of the watermark payload feature map, the carrier image, and the ROI mask, the watermarked image is generated through a U-Net encoding network. A watermark detection unit is used to input the image to be verified into the decoder of the watermark model. The decoder includes a mask branch and a payload branch. The mask branch is configured to predict the watermark region of the image to be verified and output a predicted mask. The payload branch is configured to perform ROI gating processing on the image to be verified based on the predicted mask and recover the source identification and geometric landmarks based on the ROI-gated feature map.

9. The apparatus according to claim 8, characterized in that, The device further includes: The watermark evaluation unit is used to perform image source tracing using the recovered source identifier, and to perform tamper sensitivity analysis based on the predicted mask and the recovered geometric landmarks without needing to obtain the ROI mask used when the image to be verified was encoded.

10. An electronic device, characterized in that, include: processor; A computer-readable storage medium storing computer program instructions that, when executed by the processor, cause the processor to perform the method as described in any one of claims 1 to 8.

Images