Watermark embedding method and device, storage medium and computer program product

Abstract

The embodiment of the invention provides a watermark embedding method and device, a storage medium and a computer program product. The method comprises the steps that first information is acquired; wherein the first information comprises any one or more view angle direction information for observing the three-dimensional space object; generating a first picture corresponding to the first information based on the first information and a first model; wherein the first model at least comprises a first network and a second network, the first picture comprises a picture embedded with first watermark information, the first network is at least used for generating color values corresponding to N sampling points, and the N sampling points are used for representing N space coordinate points corresponding to the first information; the second network is at least used for generating a watermark residual value corresponding to the color value and the first watermark information, that is, the second network is more focused on learning the watermark residual, so that a prediction target of the first model is simplified to the watermark residual value from the color value embedded with the watermark information, the complexity of network calculation is greatly reduced, and the prediction efficiency is improved. And the training convergence efficiency is improved.

Description

Technical Field

[0001] The present application relates to the field of computer vision technology, and in particular to a watermark embedding method, device, storage medium and computer program product. Background Art

[0002] Three-dimensional vision technology based on implicit neural representation is a hot research direction in the field of computer vision in recent years. Implicit three-dimensional models represented by neural radiance fields (NeRF) can generate high-quality images of three-dimensional scenes under free observation perspective. As the application scale of NeRF models expands, the importance of copyright protection of NeRF models has become increasingly prominent. Currently, the direct encryption protection of NeRF model files can only ensure that the network structure and weights of the model will not be illegally copied or tampered with during the transmission and sharing stage, but it cannot protect the image assets obtained by NeRF model reasoning and rendering. Currently, the rendered images can be watermarked one by one based on the general two-dimensional (2D) image watermarking technology.

[0003] However, it is difficult to ensure cross-view consistency by embedding watermarks one by one in rendered images based on general 2D image watermarking technology. That is, some areas in the rendered image with relatively sparse geometric information may produce easily detectable visual defects. Therefore, it is urgent to propose a NeRF network framework that can ensure that the watermark embedded in the image has cross-view consistency. Summary of the invention

[0004] The embodiments of the present application provide a watermark embedding method, device, storage medium and computer program product, which can ensure that the watermark embedded in the image has cross-viewing consistency while reducing the complexity of network calculation.

[0005] The technical solution of the embodiment of the present application is implemented as follows:

[0006] In a first aspect, an embodiment of the present application provides a watermark embedding method, the method comprising:

[0007] Acquire first information; wherein the first information includes any one or more viewing direction information of observing a three-dimensional space object;

[0008] A first picture corresponding to the first information is generated based on the first information and a first model; wherein the first model includes at least a first network and a second network, the first picture includes a picture embedded with first watermark information, the first network is at least used to generate color values ​​corresponding to N sampling points, the N sampling points are used to represent N spatial coordinate points corresponding to the first information, and the second network is at least used to generate a watermark residual value corresponding to the color value and the first watermark information, and N is a positive integer.

[0009] In a second aspect, an embodiment of the present application provides a watermark embedding device, the watermark embedding device comprising: an acquisition unit and a generation unit; wherein:

[0010] The acquisition unit is used to acquire first information; wherein the first information includes any one or more viewing direction information of observing a three-dimensional space object;

[0011] The generating unit is used to generate a first picture corresponding to the first information based on the first information and a first model; wherein the first model includes at least a first network and a second network, the first picture includes a picture embedded with first watermark information, the first network is at least used to generate color values ​​corresponding to N sampling points, the N sampling points are used to represent N spatial coordinate points corresponding to the first information, and the second network is at least used to generate a watermark residual value corresponding to the color value and the first watermark information, and N is a positive integer.

[0012] In a third aspect, an embodiment of the present application provides a watermark embedding device, the watermark embedding device comprising: a processor and a memory; wherein:

[0013] The memory is used to store a computer program that can be run on the processor;

[0014] The processor is used to execute the watermark embedding method as described above when running the computer program.

[0015] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a computer program code is stored. When the computer program code is executed by a computer, the watermark embedding method as described above is implemented.

[0016] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the watermark embedding method as described above.

[0017] The embodiments of the present application provide a watermark embedding method, device, storage medium and computer program product, the method comprising: obtaining first information; wherein the first information comprises any one or more viewing direction information for observing a three-dimensional space object; generating a first picture corresponding to the first information based on the first information and a first model; wherein the first model comprises at least a first network and a second network, the first picture comprises a picture embedded with the first watermark information, the first network is at least used to generate color values ​​corresponding to N sampling points, the N sampling points are used to represent N spatial coordinate points corresponding to the first information, the second network is at least used to generate a watermark residual value corresponding to the color value and the first watermark information, and N is a positive integer. It can be seen that the embodiment of the present application can generate a first image corresponding to the first information based on any one or more viewing direction information of a three-dimensional object, the first network and the second network, and the first image includes an image embedded with the first watermark information, that is, the embodiment of the present application can generate an image embedded with the first watermark information corresponding to any viewing direction information based on the first model, and has consistent robustness and invisibility across viewing angles; and the second network is at least used to generate a watermark residual value corresponding to the color value and the first watermark information, that is, the second network is more focused on learning the watermark residual, so that the prediction target of the first model is simplified from the color value embedded with the watermark information to the watermark residual value, thereby greatly reducing the complexity of network calculation and improving the efficiency of training convergence. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 The watermark embedding method proposed in this embodiment is shown in FIG. Figure 1 ;

[0019] Figure 2 The NeRF network framework proposed in this application embodiment is shown in FIG. Figure 1 ;

[0020] Figure 3 The NeRF network framework proposed in this application embodiment is shown in FIG. Figure 2 ;

[0021] Figure 4 A schematic diagram of the first model structure proposed in the embodiment of the present application;

[0022] Figure 5 The watermark embedding method proposed in this embodiment is shown in FIG. Figure 2 ;

[0023] Figure 6 A schematic diagram of a NeRF network framework capable of embedding watermarks proposed in an embodiment of the present application;

[0024] Figure 7 The NeRF network framework proposed in this application embodiment is shown in FIG. Figure 3 ;

[0025] Figure 8 The structure diagram of the watermark embedding device proposed in the embodiment of the present application is as follows: Figure 1 ;

[0026] Fig. 9 The structure diagram of the watermark embedding device proposed in the embodiment of the present application is as follows: Figure 2 . DETAILED DESCRIPTION

[0027] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. It is understood that the specific embodiments described herein are only used to explain the related applications, rather than to limit the applications. It should also be noted that, for ease of description, only the parts related to the related applications are shown in the drawings.

[0028] Three-dimensional vision technology based on implicit neural representation is a hot research direction in the field of computer vision in recent years. Implicit three-dimensional models represented by NeRF can generate high-quality images of three-dimensional scenes under free observation perspectives and support efficient learning at any resolution. It has become one of the basic research paradigms in the field of three-dimensional vision and has promoted the development of three-dimensional vision reconstruction, rendering, generation and other tasks. As the application scale of NeRF technology expands, the importance of its copyright protection continues to highlight. Direct encryption protection of model files can only ensure that the network structure and weights of the model will not be illegally copied or tampered with during the transmission and sharing stage, but it cannot protect the image assets obtained by NeRF model reasoning and rendering. At present, the rendered images can be watermarked one by one based on the general two-dimensional 2D image watermarking technology.

[0029] However, it is difficult to ensure cross-view consistency by embedding watermarks one by one in rendered images based on general 2D image watermarking technology. That is, some areas in the rendered image with relatively sparse geometric information may produce easily detectable visual defects. Therefore, it is urgent to propose a NeRF network framework that can ensure that the watermark embedded in the image has cross-view consistency.

[0030] In order to solve the problem that it is difficult to ensure cross-view consistency when watermarking rendered images one by one, the embodiments of the present application provide a watermark embedding method, device, storage medium and computer program product, the method comprising: obtaining first information; wherein the first information includes any one or more viewing direction information of observing three-dimensional space objects; generating a first picture corresponding to the first information based on the first information and a first model; wherein the first model includes at least a first network and a second network, the first picture includes a picture embedded with the first watermark information, the first network is at least used to generate color values ​​corresponding to N sampling points, the N sampling points are used to represent N spatial coordinate points corresponding to the first information, and the second network is at least used to generate a watermark residual value corresponding to the color value and the first watermark information, and N is a positive integer. It can be seen that the embodiment of the present application can generate a first image corresponding to the first information based on any one or more viewing direction information of a three-dimensional object, the first network and the second network, and the first image includes an image embedded with the first watermark information, that is, the embodiment of the present application can generate an image embedded with the first watermark information corresponding to any viewing direction information based on the first model, and has consistent robustness and invisibility across viewing angles; and the second network is at least used to generate a watermark residual value corresponding to the color value and the first watermark information, that is, the second network is more focused on learning the watermark residual, so that the prediction target of the first model is simplified from the color value embedded with the watermark information to the watermark residual value, thereby greatly reducing the complexity of network calculation and improving the efficiency of training convergence.

[0031] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0032] The present application embodiment provides a watermark embedding method. Figure 1 The watermark embedding method proposed in this embodiment is shown in FIG. Figure 1 ,like Figure 1 As shown, the watermark embedding method may include the following steps:

[0033] Step 101: Acquire first information; wherein the first information includes any one or more viewing direction information for observing a three-dimensional space object.

[0034] In an embodiment of the present application, the watermark embedding device can obtain the first information.

[0035] It should be noted that, in an embodiment of the present application, the first information may include any one or more viewing direction information for observing three-dimensional space objects; wherein the viewing direction information may include the light direction of the camera, and the present application does not specifically limit the number and type of parameters included in the viewing direction information.

[0036] It should be noted that, in an embodiment of the present application, the watermark embedding device may first train the initial first model to obtain a trained first model, so that the first picture corresponding to the first information can be predicted based on the trained first model; wherein, the first model may be obtained by training for a certain three-dimensional space object, and the present application does not specifically limit the type of three-dimensional space object.

[0037] It should be noted that, in the embodiments of the present application, Figure 2 The NeRF network framework proposed in this application embodiment is shown in FIG. Figure 1 ,like Figure 2 As shown, when the watermark embedding device trains the initial first model, it can be training the initial NeRF network framework, and the initial NeRF network framework can include the initial first model and the initial fifth network; wherein, the initial first model includes not only the initial first network and the initial second network, but also the initial third network and the initial fourth network. The present application does not specifically limit the number and type of networks included in the initial NeRF network framework.

[0038] It should be noted that, in an embodiment of the present application, the initial first network may include an initial color feature extraction network, and the initial color feature extraction network may be a multi-layer perceptron (MLP) network. The present application does not specifically limit the network type of the initial first network.

[0039] It should be noted that, in the embodiments of the present application, the initial second network may include an initial watermark residual coding network, and the initial watermark residual coding network may be an MLP network. The present application does not specifically limit the network type of the initial second network.

[0040] It should be noted that, in the embodiments of the present application, the initial third network may be an initial color MLP network, and the present application does not specifically limit the network type of the initial third network.

[0041] It should be noted that, in the embodiments of the present application, the initial fourth network may be an initial density MLP network, and the present application does not specifically limit the network type of the initial fourth network.

[0042] It should be noted that, in an embodiment of the present application, the initial fifth network may be an initial watermark extraction network, which may include a convolutional neural network (CNN), an adaptive global average pooling layer, and an MLP network. The present application does not specifically limit the network type of the initial fifth network.

[0043] That is to say, in the embodiments of the present application, Figure 2 As shown, the watermark embedding device can add an initial first network and an initial second network on the basis of the original NeRF model (initial third network and initial fourth network), and then construct an initial first model capable of embedding watermark information.

[0044] Furthermore, in an embodiment of the present application, when training the initial NeRF network framework, the watermark embedding device can obtain a set of snapshots of a three-dimensional object captured from different angles and corresponding camera parameters as training data; each training sample can be composed of a picture and corresponding camera parameters, for example, the training sample can include N sampling points (N three-dimensional space coordinate points) corresponding to the picture and the corresponding camera parameters.

[0045] It should be noted that, in the embodiments of the present application, the camera parameters may include information such as the direction of the camera and the position of the camera, and the present application does not specifically limit the type and number of parameters included in the camera parameters.

[0046] It should be noted that, in the embodiment of the present application, the watermark embedding device can input the training samples into the initial third network and the initial fourth network, that is, it can map N sampling points and corresponding light directions to their corresponding color values ​​and volume density values. For example, the density values ​​corresponding to the N sampling points can be calculated by the initial fourth network, and the calculation formula is shown in the following formula (1).

[0047] [σ,z]=F σ (γ x (x)) (1)

[0048] Among them, σ represents the density value, z represents the intermediate feature of the initial fourth network output, and F σ represents the initial fourth network, γ x A fixed encoding function representing a spatial coordinate point x.

[0049] It should be noted that, in the embodiment of the present application, the watermark embedding device can input the intermediate feature z and the light direction d into the initial third network to obtain the color values ​​corresponding to N sampling points, and the calculation formula is shown in the following formula (2).

[0050] c=F c (z,γ d (d)) (2)

[0051] Among them, c represents the color value, F c represents the initial third network, γ d Represents the fixed encoding function of direction d.

[0052] That is, in the embodiment of the present application, for the N sampling points on the light r emitted from the camera, their color values ​​(first color values) and volume density values ​​(first density values) calculated by the initial third network and the initial fourth network can be expressed as

[0053] Furthermore, in an embodiment of the present application, the watermark embedding device can train the initial third network and the initial fourth network based on the first loss function to obtain the third network and the fourth network; wherein the first loss function is shown in the following formula (3).

[0054]

[0055] Among them, L photo represents the first loss function, R refers to the set of camera rays passing through the image pixels, represents a first prediction result, the first prediction result includes a picture after volume rendering according to the first color value and the first density value, and C(r) represents an actual observation result.

[0056] It should be noted that, in an embodiment of the present application, the first prediction result includes a picture after volume rendering according to the above-mentioned first color value and first density value, and the calculation formula for volume rendering according to the first color value and the first density value is shown in the following formula (4).

[0057]

[0058] in, represents the first prediction result, represents the first density value, Represents the first color value, represents the distance between the two adjacent sampling points i+1 and i, As shown in the following formula (5).

[0059]

[0060] It should be noted that, in the embodiment of the present application, the watermark embedding device can continuously adjust the network parameters involved in the initial third network and the initial fourth network by minimizing the first loss function, so as to obtain the trained third network and fourth network.

[0061] Furthermore, in an embodiment of the present application, the watermark embedding device can fix the network parameters of the trained third network and the fourth network, and then train the initial first network, the initial second network and the initial fifth network. The initial first network, the initial second network and the initial fifth network can be trained based on the second loss function to obtain the first network, the second network and the fifth network.

[0062] It should be noted that, in an embodiment of the present application, the second loss function may include a third loss function, a fourth loss function and a fifth loss function; wherein the second loss function is as shown in the following formula (6).

[0063] L=λ MAE L MAE +λ message L message +λ percept L percept +λ reg L reg (6)

[0064] Among them, L represents the second loss function, L MAE represents the third loss function, L message represents the fourth loss function, L percept represents the fifth loss function, λ MAE , message , percept , reg is the weight coefficient of each loss function, L reg is the network regularization loss, which is the L2 sum of all network weights.

[0065] It should be noted that, in the embodiment of the present application, the third loss function is as shown in the following formula (7), and the third loss function is associated with the first prediction result and the second prediction result.

[0066]

[0067] Among them, H and W represent the height and width of the rendered image. represents the first prediction result, Represents the second prediction result, which includes the picture predicted by the first model.

[0068] Exemplarily, in an embodiment of the present application, when determining the second prediction result, the watermark embedding device can learn N hidden features for watermark expression from the first color values ​​corresponding to the N sampling points obtained above through the initial first network. The hidden features can be expressed by the following formula (8).

[0069] f c =E c (c,γ x (x),γ d (d)) (8)

[0070] Among them, f c represents hidden features, E c represents the initial first network, c represents the first color value, γ x and γ dRepresent the fixed encoding functions of spatial coordinate x and direction d respectively.

[0071] It should be noted that, in the embodiments of the present application, Figure 2 As shown, after obtaining the hidden features, the watermark embedding device can splice the N hidden features with the watermark information according to the channel dimension, and then input the N spliced ​​features into the initial second network to obtain N watermark residual values. A skip connection is introduced between the output end of the initial third network and the output end of the initial second network, and the N first color values ​​predicted by the third network can be superimposed with the N watermark residual values, so as to obtain the color value of the embedded watermark. In addition, as shown in the following formula (9), the present application can use an adjustable parameter to multiply the watermark residual value and then add it to the first color value, so that the watermark embedding strength can be adjusted according to different actual application requirements, thereby balancing the relative levels of watermark invisibility and robustness, and improving the adaptability and flexibility of the algorithm to different application scenarios.

[0072]

[0073] in, represents the color value of the embedded watermark, c represents the first color value, α represents the adjustment parameter, R represents the watermark residual value, E R represents the initial second network, f c represents hidden features, and M represents watermark information.

[0074] It should be noted that in the embodiment of the present application, the watermark information may be a length of N M Binary watermark sequence This application does not impose any specific limitation on the length of the watermark information.

[0075] That is to say, in the embodiment of the present application, when the watermark embedding device is embedding the watermark, it can superimpose the color value predicted by the original NeRF model to the output end of the initial second network through a long jump connection. This structure simplifies the prediction target of the newly added initial first network and the initial second network from the color value embedded with the watermark information to the watermark residual, thereby greatly reducing the complexity of network calculation and improving the efficiency of training convergence; in addition, the training of the watermark algorithm is a multi-objective optimization process (multi-objective optimization), and the invisibility and robustness of the watermark are two key factors that restrict each other (trade-off). In order to balance the relative levels of the two according to different needs in practical applications, an adjustable parameter is added in the embodiment of the present application to control the intensity of the watermark residual value superimposed on the original color value, thereby improving the adaptability and flexibility of the algorithm to different application scenarios.

[0076] Further, in the embodiment of the present application, after obtaining the color value of the embedded watermark, the watermark embedding device can perform volume rendering calculation based on the color value of the embedded watermark and the first density value calculated by the above formula (1) to obtain a rendered image (second prediction result). The calculation formula is shown in the following formula (10); wherein the color value of the embedded watermark and the first density value corresponding to the N sampling points on the light ray r emitted from the camera can be expressed as

[0077]

[0078] in, represents the first density value, represents the distance between the two adjacent sampling points i+1 and i, Represents the color value of the embedded watermark.

[0079] It should be noted that, in the embodiments of the present application, Figure 3 The NeRF network framework proposed in this application embodiment is shown in FIG. Figure 2 ,like Figure 3 As shown, after the watermark embedding device performs volume rendering calculation based on the color value of the embedded watermark and the first density value to obtain a rendered image, it can apply the following distortion operations to the rendered image with the embedded watermark: (1) random noise: adding Gaussian noise with a mean of μ and a variance of v, or salt and pepper noise with a ratio of r to the image; (2) random rotation: performing a random rotation of the image at an angle ω; (3) random scale transformation: a scale transformation with a ratio of s; (4) blurring: introducing a Gaussian blur with a variance of v' or a median blur with a blur window size of k×k. The present application does not specifically limit the type and number of distortion operations applied.

[0080] Furthermore, in an embodiment of the present application, the fourth loss function is as shown in the following formula (11), and the fourth loss function is associated with the first watermark information and the second watermark information. The first watermark information represents the watermark information embedded in the color value of the watermark, and the second watermark information represents the watermark information obtained by the initial fifth network performing watermark extraction on the rendered image. The fourth loss function can be used to measure the difference between the first watermark information and the second watermark information.

[0081]

[0082] Among them, L message represents the fourth loss function, M represents the first watermark information, Represents the second watermark information, N M Indicates the length of the watermark information.

[0083] It should be noted that, in an embodiment of the present application, the watermark embedding device can extract the watermark of the rendered image through the initial fifth network to obtain the second watermark information. The initial fifth network may include a convolutional neural network, an adaptive global average pooling layer and an MLP network. The watermark embedding device can learn the features of the attacked rendered image through the convolutional neural network, and then input the features into the adaptive global average pooling layer and then send them into an MLP network to obtain the second watermark information.

[0084] It should be noted that, in the embodiment of the present application, the second watermark information extracted by the watermark embedding device from the rendered image can be binarized by the following formula (12).

[0085]

[0086] in, represents the second watermark information. The clamp(·,0,1) function limits the input value to between 0 and 1. Through training, It should be consistent with the embedded first watermark information, so as to realize the functions of the first model's title confirmation and authentication and infringement tracing.

[0087] Furthermore, in an embodiment of the present application, the fifth loss function is as shown in the following formula (13), and the fifth loss function is associated with the sixth network, the first prediction result, and the second prediction result; wherein the sixth network may include a pre-trained VGG-19 network, and the pre-trained VGG-19 network may be used in the fifth loss function to extract texture and structural information of two images (the image corresponding to the second prediction result and the image corresponding to the first prediction result) and compare them. The present application does not specifically limit the network type of the sixth network.

[0088]

[0089] Among them, L percept represents the fifth loss function, They respectively represent the feature representation of the two images at the i-th layer in the pre-trained VGG-19 network, and l is the number of feature layers.

[0090] It should be noted that, in the embodiment of the present application, as shown in the above formula (6), the second loss function includes a third loss function, a fourth loss function and a fifth loss function. The watermark embedding device can adjust the network parameters involved in the initial first network, the initial second network and the initial fifth network by minimizing the second loss function, so as to obtain the trained first network, the second network and the fifth network.

[0091] It should be noted that, in the embodiment of the present application, after obtaining the trained first network, second network, third network, fourth network and fifth network, the watermark embedding device can predict the picture corresponding to the first information based on each network.

[0092] Step 102: Generate a first picture corresponding to the first information based on the first information and the first model; wherein the first model includes at least a first network and a second network, the first picture includes a picture embedded with first watermark information, the first network is at least used to generate color values ​​corresponding to N sampling points, the N sampling points are used to represent N spatial coordinate points corresponding to the first information, and the second network is at least used to generate a watermark residual value corresponding to the color value and the first watermark information, where N is a positive integer.

[0093] In an embodiment of the present application, after acquiring the first information, the watermark embedding device may generate a first picture corresponding to the first information based on the first information and the first model.

[0094] It should be noted that, in the embodiment of the present application, the first watermark information may be a length of N M The binary watermark sequence M = {0,1} NM , this application does not specifically limit the length of the first watermark information.

[0095] It should be noted that, in the embodiments of the present application, the first network may include a color feature extraction network, and the color feature extraction network may be an MLP network. The present application does not specifically limit the network type of the first network.

[0096] It should be noted that, in the embodiments of the present application, the second network may include a watermark residual coding network, and the watermark residual coding network may be an MLP network. The present application does not specifically limit the network type of the second network.

[0097] It should be noted that, in the embodiments of the present application, Figure 4 This is a schematic diagram of the first model structure proposed in the embodiment of the present application, such as Figure 4 As shown, the first model may include a third network and a fourth network in addition to the first network and the second network. The present application does not specifically limit the number and type of networks included in the first model.

[0098] It should be noted that, in the embodiments of the present application, the third network may be a color MLP network, and the present application does not specifically limit the network type of the third network.

[0099] It should be noted that, in the embodiments of the present application, the fourth network may be a density MLP network, and the present application does not specifically limit the network type of the fourth network.

[0100] It should be noted that, in an embodiment of the present application, when the watermark embedding device generates a first image corresponding to the first information based on the first information and the first model, the N sampling points can be input into the fourth network to obtain the first feature and N first density values; then the first information and the first feature can be input into the third network to obtain N first color values ​​corresponding to the N sampling points; and then the first image corresponding to the first information can be generated based on the N first color values, the N first density values, the first network and the second network.

[0101] It should be noted that, in the embodiment of the present application, the first feature may be an intermediate feature output after the watermark embedding device inputs N sampling points into the fourth network.

[0102] Exemplarily, in an embodiment of the present application, when the watermark embedding device inputs N sampling points into the fourth network to obtain the first feature and N first density values, they can be calculated using the above formula (1).

[0103] Exemplarily, in an embodiment of the present application, when the watermark embedding device inputs the first information and the first feature into the third network and obtains N first color values ​​corresponding to N sampling points, it can be calculated by the above formula (2).

[0104] It should be noted that, in an embodiment of the present application, when the watermark embedding device generates a first image corresponding to the first information based on N first color values, N first density values, a first network and a second network, it can first determine N first hidden features based on the N first color values, the first information, N sampling points and the first network; wherein the first hidden features are used to characterize the first watermark information; then the N first hidden features can be fused with the second features respectively to obtain N third features; wherein the second features contain feature information corresponding to the first watermark information; and then N second color values ​​can be determined based on the N third features and the N first color values; finally, the first image can be generated based on the N second color values ​​and the N first density values.

[0105] Exemplarily, in an embodiment of the present application, when the watermark embedding device determines N first hidden features based on N first color values, the first information, N sampling points and the first network, it can be calculated by the above formula (8).

[0106] Exemplarily, in an embodiment of the present application, when the watermark embedding device fuses N first hidden features with the second features respectively to obtain N third features, it may be to splice the N first hidden features with the second features respectively to obtain N third features. The present application does not specifically limit the method of fusion processing.

[0107] Furthermore, in an embodiment of the present application, when the watermark embedding device determines N second color values ​​based on N third features and N first color values, the N third features can be input into the second network to obtain N watermark residual values; and then the N second color values ​​can be determined based on the N watermark residual values ​​and the N first color values.

[0108] Furthermore, in an embodiment of the present application, when the watermark embedding device determines N second color values ​​based on N watermark residual values ​​and N first color values, it can determine N second color values ​​based on N watermark residual values, N first color values ​​and adjustment parameters; wherein the second color value includes the color value embedded in the first watermark information.

[0109] Exemplarily, in an embodiment of the present application, when the watermark embedding device determines N second color values ​​based on N watermark residual values, N first color values ​​and adjustment parameters, as shown in the above formula (9), the present application can introduce a long jump connection between the output end of the third network and the output end of the second network, and can multiply the adjustment parameter with the N watermark residual values ​​and then add the N first color values ​​to obtain N second color values. The addition of the long jump connection allows the first network and the second network to focus on learning the watermark residual, thereby greatly reducing the computational complexity and improving the efficiency of training convergence; and the present application can adjust the watermark embedding strength according to different actual application requirements by adjusting the parameters, thereby balancing the relative levels of watermark invisibility and robustness, and improving the adaptability and flexibility of the algorithm to different application scenarios.

[0110] It should be noted that, in the embodiment of the present application, after the watermark embedding device determines N second color values ​​based on N third features and N first color values, it can generate a first picture based on the N second color values ​​and N first density values.

[0111] Furthermore, in an embodiment of the present application, when the watermark embedding device generates the first image based on N second color values ​​and N first density values, the first image can be generated based on voxel rendering technology, N first density values ​​and N second color values. As shown in the above formula (10), the watermark embedding device can perform voxel rendering calculation based on N first density values ​​and N second color values ​​to obtain the rendered image, i.e., the first image.

[0112] It should be noted that, in the embodiments of the present application, Figure 5 The watermark embedding method proposed in this embodiment is shown in FIG. Figure 2 ,like Figure 5 As shown, after the watermark embedding device generates the first picture corresponding to the first information based on the first information and the first model, that is, after step 102, it can also include the following steps:

[0113] Step 103: input the first image into the fifth network to obtain second watermark information; wherein the fifth network is at least used to extract the watermark information of the image.

[0114] It should be noted that, in an embodiment of the present application, the fifth network may be a watermark extraction network, which may include a convolutional neural network, an adaptive global average pooling layer, and an MLP network. The present application does not specifically limit the network type of the fifth network.

[0115] Exemplarily, in an embodiment of the present application, the watermark embedding device can learn the features of the first image through a convolutional neural network, and then input the features into an adaptive global average pooling layer and then send them into an MLP network, so as to obtain the second watermark information.

[0116] In summary, the embodiment of the present application can add a first network and a second network on the basis of the original NeRF model (the third network and the fourth network), so as to construct a first model capable of embedding watermark information. The newly added first network and the second network effectively enhance the representation and embedding capabilities of the watermark information, so that the pictures of any viewing direction obtained by rendering by the first model contain embedded watermark information and have consistent robustness and invisibility across viewing angles; in addition, the present application can introduce a long jump connection between the output end of the third network and the output end of the second network, and the adjustment parameter can be multiplied by N watermark residual values ​​and then added to N first color values ​​to obtain N second color values. The addition of the long jump connection allows the first network and the second network to focus on learning the watermark residual, so that the prediction targets of the first network and the second network are simplified from the color values ​​embedded with the watermark information to the watermark residual values, thereby greatly reducing the computational complexity and improving the efficiency of training convergence; and the present application can adjust the watermark embedding strength according to different actual application needs by adjusting the parameters, thereby balancing the relative levels of watermark invisibility and robustness, and improving the adaptability and flexibility of the algorithm to different application scenarios.

[0117] An embodiment of the present application provides a watermark embedding method, the method comprising: obtaining first information; wherein the first information comprises any one or more viewing direction information for observing a three-dimensional space object; generating a first picture corresponding to the first information based on the first information and a first model; wherein the first model comprises at least a first network and a second network, the first picture comprises a picture embedded with the first watermark information, the first network is at least used to generate color values ​​corresponding to N sampling points, the N sampling points are used to represent N spatial coordinate points corresponding to the first information, the second network is at least used to generate a watermark residual value corresponding to the color value and the first watermark information, and N is a positive integer. It can be seen that the embodiment of the present application can generate a first image corresponding to the first information based on any one or more viewing direction information of a three-dimensional object, the first network and the second network, and the first image includes an image embedded with the first watermark information, that is, the embodiment of the present application can generate an image embedded with the first watermark information corresponding to any viewing direction information based on the first model, and has consistent robustness and invisibility across viewing angles; and the second network is at least used to generate a watermark residual value corresponding to the color value and the first watermark information, that is, the second network is more focused on learning the watermark residual, so that the prediction target of the first model is simplified from the color value embedded with the watermark information to the watermark residual value, thereby greatly reducing the complexity of network calculation and improving the efficiency of training convergence.

[0118] Based on the above embodiments, another embodiment of the present application provides a watermark embedding method, and proposes a NeRF network framework schematic diagram capable of embedding watermarks based on the original NeRF model of a static three-dimensional scene. Figure 6 A schematic diagram of a NeRF network framework that can embed watermarks proposed in an embodiment of the present application is shown in FIG. Figure 6 As shown in the figure, images of any viewpoint rendered by the network framework are all watermarked, and have consistent robustness and invisibility across viewpoints. Figure 7 The NeRF network framework proposed in this application embodiment is shown in FIG. Figure 3 ,like Figure 7 As shown, the embodiment of the present application can construct a watermark NeRF model (i.e., the first model) capable of embedding watermark information based on the original NeRF model, so that any viewing angle image rendered by the latter contains embedded watermark information (i.e., the first watermark information); wherein, the first model includes a color feature extraction network (i.e., the first network), a watermark residual coding network (i.e., the second network), a density MLP network (i.e., the fourth network) and a color MLP network (i.e., the third network), and the subsequent simulated attack layer is used to simulate various attacks that the rendered image may be subjected to during actual use and transmission during the training process, impose multiple data interferences on the rendered image, and enhance the robustness of the watermark against multiple attacks, and finally a watermark extraction network (i.e., the fifth network) is used to extract the watermark information (i.e., the second watermark information) from the attacked rendered image.

[0119] It should be noted that in the embodiments of the present application, in reasoning tests or actual applications, the model copyright owner only needs to use the watermark NeRF model (i.e., the first model) and the watermark extraction network (i.e., the fifth network). Even if the watermark NeRF model is accidentally leaked, the copyright owner inputs the suspected infringing image into the watermark extraction network. The extracted watermark should be consistent with the watermark originally embedded in the model, indicating the copyright ownership of the model and providing evidence for infringement identification.

[0120] It should be noted that, in the embodiments of the present application, the watermark embedding device needs to prepare a set of three-dimensional scene snapshots captured from different angles and corresponding camera parameters as training data for training the NeRF model (initial NeRF network framework). Each training sample consists of a picture and corresponding camera parameters. For example, the training sample may include N sampling points (N three-dimensional space coordinate points) corresponding to the picture and the corresponding camera parameters.

[0121] It should be noted that, in the embodiments of the present application, the camera parameters may include information such as the direction of the camera and the position of the camera, and the present application does not specifically limit the type and number of parameters included in the camera parameters.

[0122] It should be noted that, in the embodiment of the present application, in order to identify the NeRF model copyright information, a length of N is also required. M Binary watermark sequence

[0123] It should be noted that, in the embodiment of the present application, the goal of the original NeRF model is to construct a continuous volume representation F, which converts the three-dimensional space coordinate point x∈R 3 and the light direction d∈R 2 Mapped to its corresponding color value c∈R 3 and the volume density value σ∈R + , that is, F:(x,d)→(c,σ), two MLP networks F can be used σ (i.e. the fourth network), F c (i.e., the third network) is used to learn the representation, as shown in the above formula (1) and the above formula (2); wherein the original NeRF model may include the third network and the fourth network.

[0124] It should be noted that, in the embodiment of the present application, for the N sampling points on the light r emitted from the camera, they are connected through the network F σ 、F c The calculated color value (i.e., the first color value) and volume density value (i.e., the first density value) are expressed as Then the color value of the light can be calculated by the above formula (4).

[0125] It should be noted that, in the embodiment of the present application, the watermark embedding device is connected via the network F σ (i.e. the fourth network), F c When calculating the color value (i.e., the first color value) and the volume density value (i.e., the first density value) by the third network, the N sampling points (the three-dimensional space coordinate point x) can be input into the fourth network to obtain the first feature and the N first density values; the first information (the light direction d) and the first feature can also be input into the third network to obtain the N first color values ​​corresponding to the N sampling points.

[0126] It should be noted that, in the embodiment of the present application, the first feature may be an intermediate feature output after the watermark embedding device inputs N sampling points into the fourth network.

[0127] Exemplarily, in an embodiment of the present application, when the watermark embedding device inputs N sampling points into the fourth network to obtain the first feature and N first density values, they can be calculated using the above formula (1).

[0128] Exemplarily, in an embodiment of the present application, when the watermark embedding device inputs the first information and the first feature into the third network and obtains N first color values ​​corresponding to N sampling points, it can be calculated by the above formula (2).

[0129] It should be noted that in the embodiment of the present application, since modifying the three-dimensional geometric information may have a significant impact on the rendering quality, the proposed watermark NeRF model (i.e., the first model) embeds the watermark information M (the first watermark information) into the color value c (i.e., the first color value) predicted by the original NeRF based on the residual learning principle. First, a color feature extraction network E with an MLP structure is used. c (i.e., the first network) learns a hidden feature f for watermark expression from the color value c before embedding the watermark information c , and E c Taking spatial information (3D position x and viewing angle d) as input, we can promote the c The embedded watermark has the robustness and invisibility that are consistent across viewing angles. The calculation formula for the hidden feature is shown in the above formula (8).

[0130] It should be noted that, in the embodiment of the present application, after the watermark embedding device obtains the hidden feature, it can c The third feature is obtained by concatenating the watermark information M (the second feature) according to the channel dimension. In practice, M can be first mapped to f c The third feature can then be input into a watermark residual encoding network E of an MLP structure. R(i.e. the second network) to obtain the watermark residual. The output of the original NeRF color MLP network (i.e. the third network) is connected to E R A skip connection is introduced between the output of the second network to connect the color value c (i.e., the first color value) predicted by the original NeRF with E R The output (watermark residual) is superimposed to obtain the color value of the embedded watermark (i.e., the second color value). The addition of long-hop connections makes the network structure of the watermark embedding part (E c and E R ) focuses on learning the watermark residual R, which greatly reduces the computational complexity and improves the efficiency of training convergence. In addition, an adjustable parameter α is multiplied by R and then added to c, as shown in the above formula (9). When applying the inference, the watermark embedding strength can be adjusted according to different actual application requirements, thereby balancing the relative levels of watermark invisibility and robustness, and improving the adaptability and flexibility of the algorithm to different application scenarios.

[0131] It should be noted that, in the embodiment of the present application, the color value based on the embedded watermark (i.e., the second color value) and the original NeRF predicted volume density value (i.e., the first density value) can be expressed as

[0132] It should be noted that, in an embodiment of the present application, the watermark embedding device can perform volume rendering calculation based on the color value of the embedded watermark (i.e., the second color value) and the first density value to obtain a rendered image (first image), and the calculation formula is as shown in the above formula (10).

[0133] It should be noted that, in the embodiments of the present application, in order to train the watermark algorithm to resist various 2D attacks, the following distortion operations (including but not limited to) may be applied to the rendered image (first image) embedded with the watermark: (1) random noise: adding Gaussian noise with a mean of μ and a variance of v, or salt and pepper noise with a ratio of r to the image; (2) random rotation: performing a random rotation of the image at an angle ω; (3) random scaling: scaling with a ratio of s; (4) blurring: introducing a Gaussian blur with a variance of v' or a median blur with a blur window size of k×k.

[0134] That is to say, in the embodiment of the present application, when the watermark embedding device is embedding the watermark, it can superimpose the color value (first color value) predicted by the original NeRF model to the output end of the second network through a long jump connection. This structure simplifies the prediction target of the newly added first network and the second network from the color value embedded with the watermark information to the watermark residual, thereby greatly reducing the complexity of network calculation and improving the efficiency of training convergence. In addition, the training of the watermark algorithm is a multi-objective optimization process. The invisibility and robustness of the watermark are two key factors that restrict each other. In order to balance the relative levels of the two according to different needs in practical applications, an adjustable parameter is added in the embodiment of the present application to control the intensity of the watermark residual value superimposed on the original color value, thereby improving the adaptability and flexibility of the algorithm to different application scenarios.

[0135] It should be noted that in the embodiment of the present application, the watermark extraction network (fifth network) is mainly built based on a standard convolutional neural network to learn the features of the attacked rendered image (first image), which is fed into an output layer of an MLP structure after adaptive global average pooling to obtain watermark information. (Second watermark information).

[0136] It should be noted that in the embodiments of the present application, with respect to the training and reasoning process, the training process can be mainly divided into two stages; wherein, (1) the first stage of training: the normal training process of the original NeRF model, the watermark embedding device can extract training samples one by one from the training data set, the original NeRF model predicts the volume density value and color value according to the camera parameters (viewpoint and viewing angle) of the sample for volume rendering, and compares the rendering result (the first prediction result) with the actual observation result, as shown in the above formula (3), by minimizing the photometric loss (photometric loss) L photo To adjust F σ (i.e. the initial third network), F c (i.e., the initial fourth network) network parameters; (2) Second stage of training: fix the trained original NeRF model F σ (i.e. the third network), F c (i.e. the fourth network) network parameters, and then train the E of the watermark NeRF model c (i.e. the initial first network), E R (i.e. the initial second network) and the watermark extraction network D M(i.e. the initial fifth network), the training goal is to make the watermark invisible and robust against various attacks, so the training loss of the second stage needs to include at least an invisibility loss (i.e. the third loss function) that measures the difference between the rendered image before and after the watermark is embedded, and a robustness loss (i.e. the fourth loss function) that measures the difference between the embedded and extracted watermarks. In practice, as shown in the above formula (7), the calculation of the invisibility loss (i.e. the third loss function) can be done by using the original NeRF rendered image at the same viewing angle (i.e. the first prediction result) and watermark NeRF (i.e. the first model) rendered image (i.e., the second prediction result) is the mean absolute error (robustness of the Mean Absolute difference, MAE); as shown in the above formula (11), the robustness loss (i.e., the fourth loss function) can be calculated by using the mean square error (MSE) to measure the difference between the embedded watermark (i.e., the first watermark information) and the extracted watermark (i.e., the second watermark information). This application does not specifically limit the function types of the first loss function and the second loss function.

[0137] Furthermore, in an embodiment of the present application, the watermark embedding device can introduce perceptual loss (Perceptual Loss), i.e., the fifth loss function, on this basis. The fifth loss function is shown in the above formula (13). A pre-trained VGG-19 network (sixth network) is used to extract the texture and structural information of two images (the image corresponding to the second prediction result and the image corresponding to the first prediction result) and compare them. The present application does not specifically limit the network type of the sixth network.

[0138] That is to say, in the embodiment of the present application, the loss function of the second stage (i.e., the second loss function) is as shown in the above formula (6), the second loss function may include a third loss function, a fourth loss function, and a fifth loss function, and the watermark embedding device can adjust the network parameters involved in the initial first network, the initial second network, and the initial fifth network by minimizing the second loss function, so as to obtain the trained first network, the second network, and the fifth network.

[0139] It should be noted that, in the embodiment of the present application, in the inference or practical application stage, the watermark embedding device extracts the watermark information from the rendered image (ie, the first image). (ie, the second watermark information) can be binarized using the above formula (12).

[0140] In summary, the embodiment of the present application can add a first network and a second network on the basis of the original NeRF model (the third network and the fourth network), so as to construct a first model capable of embedding watermark information. The newly added first network and the second network effectively enhance the representation and embedding capabilities of the watermark information, so that the pictures of any viewing direction obtained by rendering by the first model contain embedded watermark information and have consistent robustness and invisibility across viewing angles; in addition, the present application can introduce a long jump connection between the output end of the third network and the output end of the second network, and the adjustment parameter can be multiplied by N watermark residual values ​​and then added to N first color values ​​to obtain N second color values. The addition of the long jump connection allows the first network and the second network to focus on learning the watermark residual, so that the prediction targets of the first network and the second network are simplified from the color values ​​embedded with the watermark information to the watermark residual values, thereby greatly reducing the computational complexity and improving the efficiency of training convergence; and the present application can adjust the watermark embedding strength according to different actual application needs by adjusting the parameters, thereby balancing the relative levels of watermark invisibility and robustness, and improving the adaptability and flexibility of the algorithm to different application scenarios.

[0141] An embodiment of the present application provides a watermark embedding method, the method comprising: obtaining first information; wherein the first information comprises any one or more viewing direction information for observing a three-dimensional space object; generating a first picture corresponding to the first information based on the first information and a first model; wherein the first model comprises at least a first network and a second network, the first picture comprises a picture embedded with the first watermark information, the first network is at least used to generate color values ​​corresponding to N sampling points, the N sampling points are used to represent N spatial coordinate points corresponding to the first information, the second network is at least used to generate a watermark residual value corresponding to the color value and the first watermark information, and N is a positive integer. It can be seen that the embodiment of the present application can generate a first image corresponding to the first information based on any one or more viewing direction information of a three-dimensional object, the first network and the second network, and the first image includes an image embedded with the first watermark information, that is, the embodiment of the present application can generate an image embedded with the first watermark information corresponding to any viewing direction information based on the first model, and has consistent robustness and invisibility across viewing angles; and the second network is at least used to generate a watermark residual value corresponding to the color value and the first watermark information, that is, the second network is more focused on learning the watermark residual, so that the prediction target of the first model is simplified from the color value embedded with the watermark information to the watermark residual value, thereby greatly reducing the complexity of network calculation and improving the efficiency of training convergence.

[0142] Based on the above embodiments, the present application provides a watermark embedding device. Figure 8 Schematic diagram of the watermark embedding device Figure 1 ,like Figure 8As shown, the device 10 includes: an acquisition unit 11 and a generation unit 12; wherein,

[0143] The acquisition unit 11 is used to acquire first information; wherein the first information includes any one or more viewing direction information of observing a three-dimensional space object;

[0144] The generating unit 12 is used to generate a first picture corresponding to the first information based on the first information and the first model; wherein the first model includes at least a first network and a second network, the first picture includes a picture embedded with first watermark information, the first network is at least used to generate color values ​​corresponding to N sampling points, the N sampling points are used to represent N spatial coordinate points corresponding to the first information, and the second network is at least used to generate a watermark residual value corresponding to the color value and the first watermark information, and N is a positive integer.

[0145] In the embodiments of the present application, further, Fig. 9 Schematic diagram of the watermark embedding device Figure 2 ,like Fig. 9 As shown, the watermark embedding device 10 proposed in the embodiment of the present application may also include a processor 13, a memory 14 storing executable instructions of the processor 13, and further, the device 10 may also include a communication interface 15, and a bus 16 for connecting the processor 13, the memory 14 and the communication interface 15.

[0146] In the embodiment of the present application, the processor 13 can be at least one of an application-specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field programmable gate array (FPGA), a central processing unit (CPU), a controller, a microcontroller, and a microprocessor. It can be understood that for different devices, the electronic device used to implement the function of the processor can also be other, and the embodiment of the present application is not specifically limited. The device 10 can also include a memory 14, which can be connected to the processor 13, wherein the memory 14 is used to store executable program code, the program code includes computer operation instructions, and the memory 14 may include a high-speed RAM memory, and may also include a non-volatile memory, for example, at least two disk memories.

[0147] In the embodiment of the present application, the bus 16 is used to connect the communication interface 15, the processor 13 and the memory 14, and the mutual communication between these devices.

[0148] In the embodiment of the present application, the memory 14 is used to store instructions and data.

[0149] Further, in an embodiment of the present application, the above-mentioned processor 13 is used to obtain first information; wherein, the first information includes any one or more viewing direction information for observing a three-dimensional space object; and a first picture corresponding to the first information is generated based on the first information and a first model; wherein, the first model includes at least a first network and a second network, the first picture includes a picture embedded with first watermark information, the first network is at least used to generate color values ​​corresponding to N sampling points, the N sampling points are used to represent N spatial coordinate points corresponding to the first information, and the second network is at least used to generate a watermark residual value corresponding to the color value and the first watermark information, and N is a positive integer.

[0150] In practical applications, the memory 14 may be a volatile memory, such as a random access memory (RAM); or a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk (HDD) or a solid-state drive (SSD); or a combination of the above types of memory, and provide instructions and data to the processor 13.

[0151] An embodiment of the present application provides a watermark embedding device, which obtains first information; wherein the first information includes any one or more viewing direction information of observing a three-dimensional space object; a first picture corresponding to the first information is generated based on the first information and a first model; wherein the first model includes at least a first network and a second network, the first picture includes a picture embedded with the first watermark information, the first network is at least used to generate color values ​​corresponding to N sampling points, the N sampling points are used to represent N spatial coordinate points corresponding to the first information, and the second network is at least used to generate a watermark residual value corresponding to the color value and the first watermark information, and N is a positive integer. It can be seen that the embodiment of the present application can generate a first image corresponding to the first information based on any one or more viewing direction information of a three-dimensional object, the first network and the second network, and the first image includes an image embedded with the first watermark information, that is, the embodiment of the present application can generate an image embedded with the first watermark information corresponding to any viewing direction information based on the first model, and has consistent robustness and invisibility across viewing angles; and the second network is at least used to generate a watermark residual value corresponding to the color value and the first watermark information, that is, the second network is more focused on learning the watermark residual, so that the prediction target of the first model is simplified from the color value embedded with the watermark information to the watermark residual value, thereby greatly reducing the complexity of network calculation and improving the efficiency of training convergence.

[0152] An embodiment of the present application provides a computer-readable storage medium on which a program is stored. When the program is executed by a processor, the watermark embedding method as described above is implemented.

[0153] Specifically, the program instructions corresponding to a watermark embedding method in this embodiment can be stored on a storage medium such as a CD, a hard disk, a USB flash drive, etc. When the program instructions corresponding to a watermark embedding method in the storage medium are read or executed by an electronic device, the following steps are included:

[0154] Acquire first information; wherein the first information includes any one or more viewing direction information of observing a three-dimensional space object;

[0155] A first picture corresponding to the first information is generated based on the first information and a first model; wherein the first model includes at least a first network and a second network, the first picture includes a picture embedded with first watermark information, the first network is at least used to generate color values ​​corresponding to N sampling points, the N sampling points are used to represent N spatial coordinate points corresponding to the first information, and the second network is at least used to generate a watermark residual value corresponding to the color value and the first watermark information, and N is a positive integer.

[0156] The embodiment of the present application further provides a computer program product, including a computer program, and the computer program can be executed by the processor 13 of the watermark embedding device 10 to complete the steps of any of the above methods.

[0157] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of hardware embodiments, software embodiments, or embodiments in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) that contain computer-usable program code.

[0158] The present application is described with reference to implementation flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the process in the flowchart. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0159] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which is implemented in the implementation flow diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0160] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide for implementing the process described in the flowchart. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0161] The above description is only a preferred embodiment of the present application and is not intended to limit the protection scope of the present application.

Claims

1. A watermark embedding method, characterized in that: The method comprises: Acquire first information; wherein the first information includes any one or more viewing direction information of observing a three-dimensional space object; A first picture corresponding to the first information is generated based on the first information and a first model; wherein the first model includes at least a first network and a second network, the first picture includes a picture embedded with first watermark information, the first network is at least used to generate color values ​​corresponding to N sampling points, the N sampling points are used to represent N spatial coordinate points corresponding to the first information, and the second network is at least used to generate a watermark residual value corresponding to the color value and the first watermark information, and N is a positive integer.

2. The method according to claim 1, characterized in that The first model further includes a third network and a fourth network, and the generating a first picture corresponding to the first information based on the first information and the first model includes: Inputting the N sampling points into the fourth network to obtain a first feature and N first density values; Inputting the first information and the first feature into the third network to obtain N first color values ​​corresponding to the N sampling points; A first picture corresponding to the first information is generated based on the N first color values, the N first density values, the first network, and the second network.

3. The method according to claim 2, characterized in that The generating a first picture corresponding to the first information based on the N first color values, the N first density values, the first network, and the second network includes: Determine N first hidden features based on the N first color values, the first information, the N sampling points, and the first network; wherein the first hidden features are used to characterize the first watermark information; The N first hidden features are respectively fused with the second features to obtain N third features; wherein the second features include feature information corresponding to the first watermark information; Determining N second color values ​​based on the N third features and the N first color values; The first picture is generated based on the N second color values ​​and the N first density values.

4. The method according to claim 3, characterized in that The determining the N second color values ​​based on the N third features and the N first color values ​​comprises: Inputting the N third features into the second network to obtain N watermark residual values; The N second color values ​​are determined based on the N watermark residual values ​​and the N first color values.

5. The method according to claim 4, characterized in that The determining the N second color values ​​based on the N watermark residual values ​​and the N first color values ​​comprises: The N second color values ​​are determined based on the N watermark residual values, the N first color values ​​and an adjustment parameter; wherein the second color values ​​include color values ​​embedded in the first watermark information.

6. The method according to claim 3, characterized in that The generating the first picture based on the N second color values ​​and the N first density values ​​includes: The first picture is generated based on voxel rendering technology, the N first density values, and the N second color values.

7. The method according to claim 1, characterized in that After generating the first picture corresponding to the first information based on the first information and the first model, the method further includes: The first image is input into a fifth network to obtain second watermark information; wherein the fifth network is at least used to extract the watermark information of the image.

8. The method according to claim 1, characterized in that The third network and the fourth network are obtained by training the initial third network and the initial fourth network based on the first loss function; wherein the first loss function is associated with the first prediction result and the actual observation result, and the first prediction result includes a picture after volume rendering according to the first color value and the first density value.

9. The method according to claim 1, characterized in that: The first network, the second network and the fifth network are obtained by training the initial first network, the initial second network and the initial fifth network based on the second loss function; Among them, the second loss function includes a third loss function, a fourth loss function and a fifth loss function, the third loss function is associated with the first prediction result and the second prediction result, the second prediction result includes the first picture, the fourth loss function is associated with the first watermark information and the second watermark information, and the fifth loss function is associated with the sixth network, the first prediction result and the second prediction result.

10. A watermark embedding device, characterized in that: The watermark embedding device comprises: an acquisition unit and a generation unit; wherein, The acquisition unit is used to acquire first information; wherein the first information includes any one or more viewing direction information of observing a three-dimensional space object; The generating unit is used to generate a first picture corresponding to the first information based on the first information and a first model; wherein the first model includes at least a first network and a second network, the first picture includes a picture embedded with first watermark information, the first network is at least used to generate color values ​​corresponding to N sampling points, the N sampling points are used to represent N spatial coordinate points corresponding to the first information, and the second network is at least used to generate a watermark residual value corresponding to the color value and the first watermark information, and N is a positive integer.

11. A watermark embedding device, characterized in that: The watermark embedding device comprises: a processor and a memory; wherein, The memory is used to store a computer program that can be run on the processor; The processor is configured to execute the method according to any one of claims 1 to 9 when running the computer program.

12. A computer-readable storage medium, characterized in that: The storage medium stores computer program codes, and when the computer program codes are executed by a computer, the method according to any one of claims 1 to 9 is executed.

13. A computer program product, comprising a computer program, characterized in that The computer program implements the method according to any one of claims 1 to 9 when executed by a processor.

Images