Three-dimensional model-based digital watermarking method, apparatus, device, medium and product

Abstract

The application discloses a kind of based on three-dimensional model digital watermarking method, device, equipment, medium and product.Based on three-dimensional model digital watermarking method, including: in response to image rendering request, based on target virtual camera view angle in image rendering request, using three-dimensional Gaussian sputtering 3DGS watermarking model rendering watermark image;Wherein, 3DGS watermarking model is on the basis of pre-training 3DGS model, freeze geometric structure parameter, and obtain by joint training to pre-training 3DGS model, classifier and decoder;Classifier is used to determine the probability of containing watermark in input image, and decoder is used to extract watermark information in input image;In response to receiving image decoding request, using decoder to the watermark extraction of to-be-decoded image in the image decoding request is carried out, and watermark information is obtained.The technical scheme of the embodiment of the application can solve the problem that 3DGS model parameter disturbance is caused by embedding watermark, which affects rendering quality.

Description

Technical Field

[0001] This invention relates to the field of three-dimensional steganography technology, and in particular to a digital watermarking method, apparatus, device, medium and product based on a three-dimensional model. Background Technology

[0002] 3D Gaussian Splatting (3DGS), as an emerging 3D scene representation technology, has garnered widespread attention due to its high-quality real-time rendering effects. With the increasing prevalence of 3DGS models, the issue of copyright protection for their digital assets has become increasingly prominent. Unauthorized copying, distribution, and use can cause significant losses to the original creators. Digital watermarking is an effective technical means to address this problem.

[0003] In existing technologies, embedding watermarks may disturb the parameters of the 3DGS model, thereby reducing the visual quality of the final rendered image. Furthermore, if watermarking methods designed for other 3D representations or general 2D image watermarking methods are directly applied to the 3DGS model, they cannot adapt to the model's own feature representation, easily leading to a decrease in rendering quality and bit accuracy. Summary of the Invention

[0004] This invention provides a digital watermarking method, apparatus, device, medium, and product based on a 3D model to solve the problem of 3DGS model parameter disturbance caused by embedded watermarks, which affects rendering quality.

[0005] According to one aspect of the present invention, a digital watermarking method based on a three-dimensional model is provided, comprising:

[0006] In response to an image rendering request, a watermark image is rendered using a 3D Gaussian sputtering 3DGS watermark model based on the target virtual camera viewpoint in the image rendering request.

[0007] The 3DGS watermarking model is obtained by freezing the geometric structure parameters on the basis of a pre-trained 3DGS model, and jointly training the pre-trained 3DGS model, classifier, and decoder; the classifier is used to determine the probability of watermark in the input image, and the decoder is used to extract watermark information from the input image.

[0008] In response to receiving an image decoding request, the decoder is used to extract the watermark from the image to be decoded in the image decoding request to obtain watermark information.

[0009] According to another aspect of the present invention, a digital watermarking device based on a three-dimensional model is provided, comprising:

[0010] The watermark image rendering module is used to respond to an image rendering request and render a watermark image using a three-dimensional Gaussian sputtering 3DGS watermark model based on the target virtual camera viewpoint in the image rendering request.

[0011] The 3DGS watermarking model is obtained by freezing the geometric structure parameters on the basis of a pre-trained 3DGS model, and jointly training the pre-trained 3DGS model, classifier, and decoder; the classifier is used to determine the probability of watermark in the input image, and the decoder is used to extract watermark information from the input image.

[0012] The watermark information extraction module is used to extract watermark information from the image to be decoded in the image decoding request by using the decoder in response to receiving the image decoding request.

[0013] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0014] At least one processor; and

[0015] A memory communicatively connected to the at least one processor; wherein,

[0016] The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the digital watermarking method based on a three-dimensional model as described in any embodiment of the present invention.

[0017] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the digital watermarking method based on a three-dimensional model as described in any embodiment of the present invention.

[0018] According to another aspect of the present invention, a computer program product is provided, including a computer program that, when executed by a processor, implements a digital watermarking method based on a three-dimensional model according to any embodiment of the present disclosure.

[0019] The technical solution of this invention, in response to an image rendering request, renders a watermarked image using a 3D Gaussian sputtering 3DGS watermarking model based on the target virtual camera viewpoint in the image rendering request. The 3DGS watermarking model is obtained by freezing geometric parameters on a pre-trained 3DGS model and jointly training the pre-trained 3DGS model, classifier, and decoder. In response to receiving an image decoding request, the decoder extracts the watermark from the image to be decoded in the image decoding request to obtain watermark information. By adding a watermark to the 3DGS watermarking model obtained through joint training of the pre-trained 3DGS model, classifier, and decoder, a watermark can be embedded without affecting the rendering quality.

[0020] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a digital watermarking method based on a three-dimensional model provided in Embodiment 1 of the present invention;

[0023] Figure 2a This is a digital watermarking method based on a three-dimensional model provided in Embodiment 2 of the present invention;

[0024] Figure 2b This is a schematic diagram of 3DGS watermarking model training according to Embodiment 2 of the present invention;

[0025] Figure 3 This is a schematic diagram of the structure of a digital watermarking device based on a three-dimensional model according to Embodiment 3 of the present invention;

[0026] Figure 4 This is a schematic diagram of the structure of an electronic device that implements the digital watermarking method based on a three-dimensional model according to an embodiment of the present invention. Detailed Implementation

[0027] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0028] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0029] Example 1

[0030] Figure 1 This document provides a flowchart of a digital watermarking method based on a three-dimensional model, as described in Embodiment 1 of the present invention. This embodiment is applicable to adding watermarks using a 3DGS watermarking model obtained through joint training of a pre-trained 3DGS model, a classifier, and a decoder. The method can be executed by a digital watermarking device based on a three-dimensional model, which can be implemented in hardware and / or software and can be configured in various general-purpose computing devices. Figure 1 As shown, the method includes:

[0031] S110, In response to the image rendering request, render the watermark image using a 3D Gaussian sputtering 3DGS watermark model based on the target virtual camera viewpoint in the image rendering request.

[0032] The 3DGS watermarking model is obtained by freezing the geometric structure parameters on the basis of the pre-trained 3DGS model and jointly training the pre-trained 3DGS model, classifier and decoder; the classifier is used to determine the probability of watermark in the input image, and the decoder is used to extract watermark information from the input image.

[0033] When fine-tuning the pre-trained 3DGS model, the impact of watermark embedding on rendering quality is reduced by freezing geometric parameters. Furthermore, a decoder-classifier joint training framework is employed, eliminating the need for an encoder and making it more lightweight and efficient than traditional encoder-decoder architectures. End-to-end co-optimization between the decoder and the pre-trained 3DGS model allows the watermark embedding and extraction processes to better adapt to the model's own feature representation, resulting in better overall performance.

[0034] In this embodiment of the invention, after receiving an image rendering request, the watermark image is rendered using a 3DGS watermark model trained based on a pre-trained 3DGS model, based on the target virtual camera viewpoint in the image rendering request.

[0035] S120. In response to receiving an image decoding request, a decoder is used to extract the watermark from the image to be decoded in the image decoding request to obtain watermark information.

[0036] In this embodiment of the invention, when watermark extraction and verification are required, the user can initiate an image decoding request. Upon receiving the request, the system can directly input the image to be decoded from the request into the decoder to extract the watermark information. Alternatively, upon receiving the request, the system can first input the image to be decoded into a trained classifier to obtain the probability of the image containing a watermark. If the probability is equal to or lower than a probability threshold, feature coefficients of a selected frequency band are extracted from the image to be decoded, directly determining that the image does not contain watermark information, eliminating the need for subsequent decoding operations and reducing computation. If the probability is higher than the probability threshold, after extracting the feature coefficients of the selected frequency band from the image to be decoded, the feature coefficients and the probability are concatenated and input into the decoder to obtain the watermark information output by the decoder.

[0037] The technical solution of this invention, in response to an image rendering request, renders a watermarked image using a 3D Gaussian sputtering 3DGS watermarking model based on the target virtual camera viewpoint in the image rendering request. The 3DGS watermarking model is obtained by freezing geometric parameters on a pre-trained 3DGS model and jointly training the pre-trained 3DGS model, classifier, and decoder. In response to receiving an image decoding request, the decoder extracts the watermark from the image to be decoded in the image decoding request to obtain watermark information. By adding a watermark to the 3DGS watermarking model obtained through joint training of the pre-trained 3DGS model, classifier, and decoder, a watermark can be embedded without affecting the rendering quality.

[0038] Example 2

[0039] Figure 2aThis is a flowchart of a digital watermarking method based on a three-dimensional model provided in Embodiment 2 of the present invention. This embodiment further refines the above embodiment, providing specific steps for extracting watermark information from the image to be decoded in response to receiving an image decoding request, using a decoder. Figure 2a As shown, the method includes:

[0040] S210. In response to the image rendering request, based on the target virtual camera viewpoint in the image rendering request, render the watermark image using a three-dimensional Gaussian sputtering 3DGS watermark model.

[0041] The 3DGS watermarking model is obtained by freezing the geometric structure parameters on the basis of a pre-trained 3DGS model, and jointly training the pre-trained 3DGS model, classifier, and decoder. The classifier is used to determine the probability of watermarks in the input image, and the decoder is used to extract watermark information from the input image.

[0042] Optionally, the 3DGS watermarking model is trained in the following way:

[0043] Freeze the geometric structure parameters in the pre-trained 3DGS model to obtain the 3DGS model to be fine-tuned. Using the 3DGS model to be fine-tuned, render the watermark image from the perspective of the target virtual camera. Using the pre-trained 3DGS model, render the reference image from the perspective of the target virtual camera.

[0044] The watermarked image and the reference image are respectively input into the classifier to obtain the watermark probability of the watermarked image and the reference image output by the classifier;

[0045] Feature coefficients of selected frequency bands are extracted from the watermarked image and the reference image respectively. Based on the feature coefficients and the watermark probability, the decoder outputs the decoded watermark information of the watermarked image and the reference image.

[0046] The model loss value is calculated based on the watermarked image, the reference image, the decoded watermark information of the watermarked image and the probability of watermarking.

[0047] Based on the model loss value, update the parameters in the 3DGS model, decoder, and classifier to be fine-tuned to obtain a new 3DGS model, decoder, and classifier to be fine-tuned. Then, return to execute the operation of rendering the watermark image from the perspective of the target virtual camera using the 3DGS model to be fine-tuned until the end of training conditions are met, and obtain the 3DGS watermark model, decoder, and classifier.

[0048] In this optional embodiment, a training method for a 3DGS watermarking model is provided: such as Figure 2bAs shown, first load a pre-trained 3DGS model that has been pre-trained and has not embedded a watermark. Furthermore, by freezing the geometric parameters in the pre-trained 3DGS model and disabling point cloud densification and pruning functions, the spatial distribution structure of Gaussian points is preserved during subsequent model fine-tuning, reducing the negative impact of embedded watermarks on the rendering quality of the 3DGS model. The pre-trained 3DGS model with frozen geometric parameters is then used as the 3DGS model to be fine-tuned.

[0049] In addition, the decoder also needs to be... and classifier The weights are randomly initialized to facilitate the joint training of the 3DGS model, decoder, and classifier to be fine-tuned later.

[0050] Furthermore, using the 3DGS model to be fine-tuned, the watermark image is rendered from the perspective P of the target virtual camera. Simultaneously, a pre-trained 3DGS model is used to render a baseline image from the perspective P of the aforementioned target virtual camera. , used to calculate visual fidelity loss.

[0051] Watermark image The input is fed into a classifier to obtain the watermark probability of the watermarked image output by the classifier. Simultaneously, the baseline image is... The input is fed into the classifier to obtain the watermark probability of the baseline image output by the classifier.

[0052] Furthermore, feature coefficients for selected frequency bands are extracted from both the watermark image and the reference image. Specifically, a two-dimensional discrete cosine transform (DTC) is performed on both the watermark image and the reference image to obtain the frequency domain feature representation Y, as shown in the following formula: ,in, and These are the DTC transformation matrices corresponding to the image height and width, respectively. yes The transpose of the vector is then used. A mask is then selected based on a preset region to filter out the low-frequency characteristic coefficients from the feature representation in the frequency domain.

[0053] Before feeding the watermarked image and the reference image into the decoder, a two-dimensional discrete cosine transform is performed on them to convert them to the frequency domain. Information embedding and decoding are performed in low-frequency regions that are less sensitive to human vision and can effectively resist attacks. Compared with traditional high-frequency embedding, this achieves better visual quality and robustness. Furthermore, a masking mechanism is applied to selectively update only some Gaussian point attributes, such as color, scaling, rotation, and opacity, thereby maintaining the geometric and appearance integrity of the original scene while embedding the watermark.

[0054] The feature coefficients and watermark probability of the watermarked image are then concatenated and input into the decoder. The decoded watermark information output by the decoder is obtained. Similarly, the feature coefficients and watermark probability of the reference image are concatenated and then input into the decoder. The decoded watermark information output by the decoder is obtained. .

[0055] Furthermore, the model loss value is calculated based on the watermarked image, the reference image, the decoded watermark information of the watermarked image and the watermark probability. Specifically, the difference between the watermarked image and the reference image can be used as the image quality loss value; the difference between the decoded watermark information of the watermarked image and the preset decoded information, and the difference between the decoded watermark information of the reference image and the all-zero bitstream can be used as the watermark decoding loss value; the watermark probability of the watermarked image and the watermark probability of the reference image can be used to determine the image classification loss value. Finally, the model loss value is determined by the combined image quality loss value, watermark decoding loss value, and image classification loss value.

[0056] Finally, backpropagation is performed based on the model loss value to update the parameters in the 3DGS model, decoder, and classifier to be fine-tuned, resulting in new 3DGS models, decoders, and classifiers to be fine-tuned. The process of rendering the watermark image from the perspective of the target virtual camera using the 3DGS model to be fine-tuned is then repeated until the loss function converges or the required number of iterations is met, resulting in the 3DGS watermark model, decoder, and classifier.

[0057] Watermark information is embedded by fine-tuning a pre-trained 3DGS model. A dedicated decoder network and a classifier network are designed and trained end-to-end in conjunction with the pre-trained 3DGS model. The decoder is responsible for extracting watermark information from the rendered image, while the classifier determines whether the image contains a watermark. Its output guides the decoder, preventing erroneous extraction results from unwatermarked images. This results in a decoder-classifier watermarking framework closely linked to the 3DGS watermarking model, achieving superior watermark image rendering and watermark extraction effects.

[0058] The framework employs a decoder-classifier joint training approach, eliminating the need for an encoder and offering greater lightweight and efficiency compared to traditional encoder-decoder architectures. End-to-end co-optimization between the decoder and the 3DGS model allows the watermark embedding and extraction processes to better adapt to the model's own feature representations, resulting in superior overall performance.

[0059] Optionally, before inputting the watermarked image and the reference image into the classifier respectively to obtain the watermark probability of the watermarked image and the reference image output by the classifier, the method further includes:

[0060] The watermark image and the reference image are respectively subjected to distortion transformation to obtain the watermark image and the reference image after distortion processing;

[0061] The distortion processing includes at least one of Gaussian noise, cropping, rotation, and scaling.

[0062] In this optional embodiment, an execution step is provided before inputting the watermark image and the reference image into the classifier respectively to obtain the watermark probability of the watermark image and the reference image output by the classifier: the watermark image and the reference image can be distorted to obtain a distorted watermark image. After comparing the base image with the distorted watermark image, classification and watermark decoding operations are performed on both. Distortion processing can include one or more of the following: Gaussian noise, cropping, rotation, and scaling. A distortion layer is introduced into the training process. This layer applies various random distortions to the rendered image in real time, forcing the decoder to learn to stably extract the watermark even with damaged image information. This allows the decoder to effectively resist common image processing attacks such as Gaussian noise, cropping, rotation, and scaling, and even maintain extraction accuracy under combined attacks, thereby enhancing the robustness of the watermark.

[0063] Optionally, based on the watermarked image, the reference image, the decoded watermark information of the watermarked image and the watermark probability, the model loss value is calculated, including:

[0064] Based on the watermarked image and the reference image, determine the image quality loss value;

[0065] Based on the decoded watermark information of the watermark image and the preset standard watermark information, the first watermark decoding loss value is determined;

[0066] Based on the decoded watermark information and all-zero bitstream output from the reference image, the decoding loss value of the second watermark is determined.

[0067] The image classification loss value is determined based on the watermark probability of the watermarked image and the watermark probability of the baseline image.

[0068] The model loss value is determined based on the image quality loss value, the first watermark decoding loss value, the second watermark decoding loss value, and the image classification loss value.

[0069] In this optional embodiment, a specific method is provided for calculating the model loss value based on the watermark image, the reference image, the decoded watermark information of the watermark image and the probability of watermark presence: First, the image quality loss value is determined by comparing the difference between the watermark image and the reference image. For example, the mean absolute error between the watermark image and the reference image can be calculated as the image quality loss value. The specific calculation formula is as follows:

[0070] .

[0071] Furthermore, the difference between the decoded watermark information of the watermarked image and the preset standard watermark information is calculated. For example, the binary cross-entropy between the two is calculated as the first watermark decoding loss. The specific calculation formula is as follows:

[0072]

[0073] in, It is the preset standard watermark information.

[0074] Furthermore, the difference between the decoded watermark information output from the reference image and the all-zero bitstream is calculated. For example, the binary cross-entropy between the two is calculated as the second watermark decoding loss value. The specific calculation formula is as follows:

[0075]

[0076] in, It is a zero-bit stream.

[0077] Furthermore, based on the watermark probability of the watermarked image and the watermark probability of the baseline image, the image classification loss value is determined, and the specific calculation formula is as follows:

[0078]

[0079] Finally, the model loss value is determined based on the image quality loss value, the first watermark decoding loss value, the second watermark decoding loss value, and the image classification loss value. Specifically, the model loss value can be obtained by weighted summing of the image quality loss value, the first watermark decoding loss value, the second watermark decoding loss value, and the image classification loss value. The specific calculation formula is as follows:

[0080]

[0081] in, , , and These are the pre-set weighting coefficients.

[0082] Optionally, feature coefficients of selected frequency bands are extracted from the watermark image and the reference image, respectively, including:

[0083] The watermark image and the reference image are respectively subjected to frequency domain transformation to obtain the feature representations of the watermark image and the reference image in the frequency domain dimension;

[0084] Based on the preset mask, feature coefficients of the selected frequency band are extracted from the feature representations of the watermark image and the reference image in the frequency domain dimension.

[0085] In this optional embodiment, a specific method is provided for extracting feature coefficients of a selected frequency band from the watermark image and the reference image respectively: Frequency domain transformation is performed on the watermark image and the reference image respectively to obtain feature representations of the watermark image and the reference image in the frequency domain dimension. Further, based on a preset mask, feature coefficients of the selected frequency band are extracted from the feature representations of the watermark image and the reference image in the frequency domain dimension respectively.

[0086] Optionally, the frequency domain transformation is a two-dimensional discrete cosine transform; the mask is used to determine at least one subset of parameters in the 3DGS model, including spherical harmonic coefficients, scaling parameters, rotation parameters, and opacity parameters.

[0087] In this optional embodiment, the frequency domain transformation is defined as a two-dimensional discrete cosine transform. A preset mask is used to determine at least one subset of parameters among the spherical harmonic coefficients, scaling parameters, rotation parameters, and opacity parameters in the 3DGS model. Through frequency domain transformation and mask feature extraction, information embedding and decoding are performed in low-frequency regions that are not visually sensitive to the human eye and can effectively resist attacks, thereby improving rendering quality and watermark extraction robustness.

[0088] S220. In response to receiving an image decoding request, input the image to be decoded in the image decoding request into the classifier to obtain the probability that the image to be decoded contains a watermark.

[0089] In this embodiment of the invention, upon receiving an image decoding request, the image to be decoded in the image decoding request is input into a trained classifier to obtain the probability of watermarks in the image to be decoded output by the classifier.

[0090] S230. When the probability of watermarking is higher than the probability threshold, extract the feature coefficients of the selected frequency band from the image to be decoded.

[0091] In this embodiment of the invention, when the probability of containing a watermark is higher than a probability threshold, feature coefficients of a selected frequency band are extracted from the image to be decoded to facilitate subsequent watermark information extraction by the decoder. When the probability of containing a watermark is equal to or lower than the probability threshold, feature coefficients of the selected frequency band are extracted from the image to be decoded, directly determining that the image to be decoded does not contain watermark information, eliminating the need for subsequent decoding operations and reducing computational load.

[0092] S240. The feature coefficients and the watermark probability are concatenated and input into the decoder to obtain the watermark information output by the decoder.

[0093] In this embodiment of the invention, when the probability of watermarking is higher than the probability threshold, after extracting the feature coefficients of the selected frequency band in the image to be decoded, the feature coefficients and the probability of watermarking are concatenated and input into the decoder to obtain the watermark information output by the decoder.

[0094] The technical solution of this invention adopts a decoder-classifier joint training framework, which eliminates the need for an encoder and is more lightweight and efficient than the traditional encoder-decoder architecture. The decoder and the 3DGS model are optimized end-to-end, enabling the watermark embedding and extraction process to better adapt to the model's own feature representation, thereby achieving better overall performance. At the same time, the geometric structure parameters are frozen during training, and information is embedded in the low-frequency domain to minimize the impact of watermark embedding on rendering quality. Furthermore, a distortion layer containing multiple attack methods is introduced to improve the robustness of watermark processing.

[0095] Example 3

[0096] Figure 3 This is a schematic diagram of a digital watermarking device based on a three-dimensional model, provided in Embodiment 3 of the present invention. Figure 3 As shown, the device includes:

[0097] The watermark image rendering module 310 is used to render a watermark image based on the target virtual camera view in the image rendering request using a three-dimensional Gaussian sputtering 3DGS watermark model in response to the image rendering request.

[0098] The 3DGS watermarking model is obtained by freezing the geometric structure parameters on the basis of a pre-trained 3DGS model, and jointly training the pre-trained 3DGS model, classifier, and decoder; the classifier is used to determine the probability of watermark in the input image, and the decoder is used to extract watermark information from the input image.

[0099] The watermark information extraction module 320 is used to extract watermark information from the image to be decoded in the image decoding request by using the decoder in response to receiving the image decoding request.

[0100] The technical solution of this invention, in response to an image rendering request, renders a watermarked image using a 3D Gaussian sputtering 3DGS watermarking model based on the target virtual camera viewpoint in the image rendering request. The 3DGS watermarking model is obtained by freezing geometric parameters on a pre-trained 3DGS model and jointly training the pre-trained 3DGS model, classifier, and decoder. In response to receiving an image decoding request, the decoder extracts the watermark from the image to be decoded in the image decoding request to obtain watermark information. By adding a watermark to the 3DGS watermarking model obtained through joint training of the pre-trained 3DGS model, classifier, and decoder, a watermark can be embedded without affecting the rendering quality.

[0101] Optionally, the digital watermarking device based on a 3D model also includes a model training module for training a 3DGS watermarking model, including:

[0102] The image rendering unit is used to freeze the geometric structure parameters in the pre-trained 3DGS model to obtain the 3DGS model to be fine-tuned. Using the 3DGS model to be fine-tuned, the watermark image is rendered from the perspective of the target virtual camera. The pre-trained 3DGS model is also used to render the reference image from the perspective of the target virtual camera.

[0103] The watermark classification unit is used to input the watermark image and the reference image into the classifier respectively, and obtain the watermark probability of the watermark image and the reference image output by the classifier.

[0104] The watermark information determination unit is used to extract feature coefficients of a selected frequency band from the watermark image and the reference image respectively, and based on the feature coefficients and the watermark probability, use a decoder to output the decoded watermark information of the watermark image and the reference image.

[0105] The model loss calculation unit is used to calculate the model loss value based on the watermarked image, the reference image, the decoded watermark information of the watermarked image and the probability of watermarking;

[0106] The model training unit is used to update the parameters in the 3DGS model, decoder, and classifier to be fine-tuned based on the model loss value, to obtain a new 3DGS model, decoder, and classifier to be fine-tuned, and then return to execute the operation of rendering the watermark image from the perspective of the target virtual camera using the 3DGS model to be fine-tuned, until the end of training conditions are met, and the 3DGS watermark model, decoder, and classifier are obtained.

[0107] Optionally, the model training module also includes:

[0108] The distortion unit is used to perform distortion transformation on the watermark image and the reference image respectively before inputting the watermark image and the reference image into the classifier to obtain the watermark probability of the watermark image and the reference image output by the classifier, so as to obtain the watermark image and the reference image after distortion processing.

[0109] The distortion processing includes at least one of Gaussian noise, cropping, rotation, and scaling.

[0110] Optional, the model loss calculation unit is specifically used for:

[0111] Based on the watermarked image and the reference image, determine the image quality loss value;

[0112] Based on the decoded watermark information of the watermark image and the preset standard watermark information, the first watermark decoding loss value is determined;

[0113] Based on the decoded watermark information and all-zero bitstream output from the reference image, the decoding loss value of the second watermark is determined.

[0114] The image classification loss value is determined based on the watermark probability of the watermarked image and the watermark probability of the baseline image.

[0115] The model loss value is determined based on the image quality loss value, the first watermark decoding loss value, the second watermark decoding loss value, and the image classification loss value.

[0116] Optional, the watermark information determination unit is specifically used for:

[0117] The watermark image and the reference image are respectively subjected to frequency domain transformation to obtain the feature representations of the watermark image and the reference image in the frequency domain dimension;

[0118] Based on the preset mask, feature coefficients of the selected frequency band are extracted from the feature representations of the watermark image and the reference image in the frequency domain dimension.

[0119] Optionally, the frequency domain transformation is a two-dimensional discrete cosine transform; the mask is used to determine at least one subset of parameters among the spherical harmonic coefficients, scaling parameters, rotation parameters, and opacity parameters in the 3DGS model.

[0120] Optional, the watermark information extraction module 320 is specifically used for:

[0121] In response to receiving an image decoding request, the image to be decoded in the image decoding request is input into the classifier to obtain the probability that the image to be decoded contains a watermark;

[0122] If the probability of watermarking is higher than the probability threshold, feature coefficients of a selected frequency band are extracted from the image to be decoded.

[0123] The feature coefficients and the watermark probability are concatenated and input into the decoder to obtain the watermark information output by the decoder.

[0124] The digital watermarking device based on a three-dimensional model provided in the embodiments of the present invention can execute the digital watermarking method based on a three-dimensional model provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the method execution.

[0125] In the technical solution of this invention, the information collected is information and data authorized by the user or fully authorized by all parties. The collection, storage, use, processing, transmission, provision, disclosure and application of related data all comply with the relevant laws, regulations and standards of relevant countries and regions, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entry points for users to choose to authorize or refuse.

[0126] Example 4

[0127] According to embodiments of the present invention, the present invention also provides an electronic device, a readable storage medium, and a computer program product.

[0128] Figure 4 A schematic diagram of an electronic device 10 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, application processors, blade application processors, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (such as helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0129] like Figure 4 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 can also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0130] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0131] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as a digital watermarking method based on a 3D model.

[0132] In some embodiments, the three-dimensional model-based digital watermarking method can be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the three-dimensional model-based digital watermarking method described above can be performed. Alternatively, in other embodiments, processor 11 can be configured to perform the three-dimensional model-based digital watermarking method by any other suitable means (e.g., by means of firmware).

[0133] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include: implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0134] Computer programs used to implement the methods of the present invention can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs can be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or application.

[0135] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0136] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0137] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data application processors), or computing systems that include middleware components (e.g., application application processors), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0138] A computing system can include clients and applications. Clients and applications are generally geographically separated and typically interact via a communication network. The client-application relationship is established by computer programs running on the respective computers and having a client-application relationship with each other. An application can be a cloud application, also known as a cloud computing application or cloud server, which is a hosting product within the cloud computing application ecosystem. It addresses the shortcomings of traditional physical servers and VPS applications, such as high management difficulty and weak business scalability.

[0139] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0140] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A digital watermarking method based on a three-dimensional model, characterized in that, include: In response to an image rendering request, a watermark image is rendered using a 3D Gaussian sputtering 3DGS watermark model based on the target virtual camera viewpoint in the image rendering request. The 3DGS watermarking model is obtained by freezing the geometric parameters of the pre-trained 3DGS model, disabling the point cloud densification and pruning functions of the pre-trained 3DGS model, and jointly training the pre-trained 3DGS model, classifier, and decoder. The classifier is used to determine the probability of watermarks in the input image, and the decoder is used to extract watermark information from the input image. In response to receiving an image decoding request, the decoder is used to extract the watermark from the image to be decoded in the image decoding request to obtain watermark information; The 3DGS watermarking model is trained in the following way: Freeze the geometric structure parameters in the pre-trained 3DGS model to obtain the 3DGS model to be fine-tuned. Using the 3DGS model to be fine-tuned, render the watermark image from the perspective of the target virtual camera. Using the pre-trained 3DGS model, render the reference image from the perspective of the target virtual camera. The watermarked image and the reference image are respectively input into the classifier to obtain the watermark probability of the watermarked image and the reference image output by the classifier; Feature coefficients of selected frequency bands are extracted from the watermarked image and the reference image respectively. The feature coefficients and the watermark probability are concatenated and input into the decoder. The decoder outputs the decoded watermark information of the watermarked image and the reference image. The model loss value is calculated based on the watermarked image, the reference image, the decoded watermark information of the watermarked image and the probability of watermarking. Based on the model loss value, update the parameters in the 3DGS model, decoder, and classifier to be fine-tuned to obtain a new 3DGS model, decoder, and classifier to be fine-tuned. Then, return to execute the operation of rendering the watermark image from the perspective of the target virtual camera using the 3DGS model to be fine-tuned until the end of training conditions are met, and obtain the 3DGS watermark model, decoder, and classifier.

2. The method according to claim 1, characterized in that, Before inputting the watermarked image and the reference image into the classifier respectively to obtain the watermark probability of the watermarked image and the reference image output by the classifier, the method further includes: The watermark image and the reference image are respectively subjected to distortion transformation to obtain the watermark image and the reference image after distortion processing; The distortion processing includes at least one of Gaussian noise, cropping, rotation, and scaling.

3. The method according to claim 1, characterized in that, Based on the watermarked image, the reference image, the decoded watermark information of the reference image, and the probability of watermark inclusion, the model loss value is calculated, including: Based on the watermarked image and the reference image, determine the image quality loss value; Based on the decoded watermark information of the watermark image and the preset standard watermark information, the first watermark decoding loss value is determined; Based on the decoded watermark information and all-zero bitstream output from the reference image, the decoding loss value of the second watermark is determined. The image classification loss value is determined based on the watermark probability of the watermarked image and the watermark probability of the baseline image. The model loss value is determined based on the image quality loss value, the first watermark decoding loss value, the second watermark decoding loss value, and the image classification loss value.

4. The method according to claim 1, characterized in that, Extracting feature coefficients for selected frequency bands from the watermark image and the reference image respectively, including: The watermark image and the reference image are respectively subjected to frequency domain transformation to obtain the feature representations of the watermark image and the reference image in the frequency domain dimension; Based on the preset mask, feature coefficients of the selected frequency band are extracted from the feature representations of the watermark image and the reference image in the frequency domain dimension.

5. The method according to claim 4, characterized in that, The frequency domain transform is a two-dimensional discrete cosine transform; the mask is used to determine at least one subset of parameters in the 3DGS model, including spherical harmonic coefficients, scaling parameters, rotation parameters, and opacity parameters.

6. The method according to claim 1, characterized in that, In response to receiving an image decoding request, the decoder is used to extract the watermark from the image to be decoded in the image decoding request, obtaining watermark information, including: In response to receiving an image decoding request, the image to be decoded in the image decoding request is input into the classifier to obtain the probability that the image to be decoded contains a watermark; If the probability of watermarking is higher than the probability threshold, feature coefficients of a selected frequency band are extracted from the image to be decoded. The feature coefficients and the watermark probability are concatenated and input into the decoder to obtain the watermark information output by the decoder.

7. A digital watermarking device based on a three-dimensional model, characterized in that, include: The watermark image rendering module is used to respond to an image rendering request and render a watermark image using a three-dimensional Gaussian sputtering 3DGS watermark model based on the target virtual camera viewpoint in the image rendering request. The 3DGS watermarking model is obtained by freezing the geometric parameters of the pre-trained 3DGS model, disabling the point cloud densification and pruning functions of the pre-trained 3DGS model, and jointly training the pre-trained 3DGS model, classifier, and decoder. The classifier is used to determine the probability of watermarks in the input image, and the decoder is used to extract watermark information from the input image. The watermark information extraction module is used to extract watermark information from the image to be decoded in the image decoding request by using the decoder in response to receiving an image decoding request. The model training module, used to train the 3DGS watermark model, includes: The image rendering unit is used to freeze the geometric structure parameters in the pre-trained 3DGS model to obtain the 3DGS model to be fine-tuned. Using the 3DGS model to be fine-tuned, the watermark image is rendered from the perspective of the target virtual camera. The pre-trained 3DGS model is also used to render the reference image from the perspective of the target virtual camera. The watermark classification unit is used to input the watermark image and the reference image into the classifier respectively, and obtain the watermark probability of the watermark image and the reference image output by the classifier. The watermark information determination unit is used to extract feature coefficients of a selected frequency band from the watermark image and the reference image respectively, and to concatenate the feature coefficients and the watermark probability and input them into the decoder, and to output the decoded watermark information of the watermark image and the reference image using the decoder. The model loss calculation unit is used to calculate the model loss value based on the watermarked image, the reference image, the decoded watermark information of the watermarked image and the probability of watermarking; The model training unit is used to update the parameters in the 3DGS model, decoder, and classifier to be fine-tuned based on the model loss value, to obtain a new 3DGS model, decoder, and classifier to be fine-tuned, and then return to execute the operation of rendering the watermark image from the perspective of the target virtual camera using the 3DGS model to be fine-tuned, until the end of training conditions are met, and the 3DGS watermark model, decoder, and classifier are obtained.

8. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the digital watermarking method based on a three-dimensional model as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the digital watermarking method based on a three-dimensional model as described in any one of claims 1-6.

10. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the digital watermarking method based on a three-dimensional model according to any one of claims 1-6.

Images