Non-planar zero watermark method based on composite energy seam cutting

Abstract

The invention provides a non-planar zero watermark method based on composite energy seam cutting, and proposes a non-planar watermark method for adaptively mapping a two-dimensional image to a three-dimensional grid model surface and embedding a watermark; the method comprises: reconstructing an object in three dimensions to generate a three-dimensional grid model; defining an energy value according to a FT saliency map value, an MB saliency map value, a gradient value and a structure tensor of a pixel, in combination with a pixel edge cost, and proposing a composite energy seam cutting method to achieve a better image scaling effect; dividing the two-dimensional image into blocks, scaling the blocks according to a grid shape of a three-dimensional grid model, and mapping the blocks to the three-dimensional grid model; watermark generation: generating a zero watermark for the mapped three-dimensional grid model, encrypting the zero watermark and uploading it to a blockchain, making full use of the non-forgeable, open and transparent properties of the blockchain, and ensuring the security and authority of the generated zero watermark; watermark detection: determining whether a watermark exists in the three-dimensional grid model to be detected according to the zero watermark stored in the blockchain.

Description

Technical Field

[0001] The invention relates to the field of information hiding, and in particular to a non-planar zero watermark method based on composite energy seam shearing. Background Art

[0002] Digital watermarking technology has been widely studied in the fields of copyright protection, traceability, anti-counterfeiting, and authentication of two-dimensional images and three-dimensional models. Many researchers have proposed various methods and strategies that take into account both the invisibility and robustness of watermarks. However, embedding watermarks only in digital carriers is far from enough. There are still very few studies on embedding watermarks in real non-planar objects. Existing digital watermarking technology is difficult to apply to the anti-counterfeiting and traceability of non-planar objects such as ceramic utensils and bronze tripods.

[0003] Seam cutting image adjustment technology, or seam carving method, is simple and has good image redirection performance. This type of method generally obtains an energy map representing the importance of pixels through the original image heuristic, and repeatedly cuts the smallest energy seams in the horizontal or vertical direction according to the energy map. However, since the energy map only detects the edge information of the image, the cut seams inevitably pass through important main objects. Nowadays, there is an urgent need for a highly robust and imperceptible non-planar watermarking method to embed watermarks in physical carriers. Summary of the invention

[0004] Objective: To overcome the shortcomings of the prior art, the present invention provides a non-planar zero watermark method based on composite energy seam cutting, which can extract zero watermarks from objects and accurately determine whether there are zero watermarks on the objects.

[0005] To achieve the above object, the present invention provides the following technical solution: a non-planar zero watermark method based on composite energy seam shearing, comprising the following steps:

[0006] Step 1: Reconstruct the object in three dimensions and generate a three-dimensional mesh model;

[0007] Step 2: Determine the image mapping area, and according to the grid division of the three-dimensional grid model, use the composite energy seam cutting method to scale the two-dimensional image so that the size of the M×N two-dimensional image and the size of the area to be mapped on the U×V three-dimensional grid model are kept within a certain range difference, where M and N represent the length and width of the two-dimensional image, respectively, and U and V represent the length and width of the area to be mapped on the three-dimensional grid model, respectively, and divide the two-dimensional image into blocks to obtain two-dimensional image sub-blocks;

[0008] Among them, the specific process of the composite energy seam shearing method is:

[0009] For vertical joints, select the cumulative energy E from top to bottom. Y(i,j) low pixel connection generation, the generation rule is:

[0010]

[0011] Among them, α, β, θ and μ represent different weights, F(i, j) represents the saliency value of the pixel in row i and column j in the FT saliency map generated by the full-resolution frequency modulation method of IMG, B(i, j) represents the saliency value of the pixel in row i and column j in the MB saliency map generated by the fast raster scanning saliency detection method of IMG, G(i, j) represents the gradient value of the pixel in row i and column j in IMG, T(i, j) represents the amplitude of the pixel in row i and column j in IMG determined by the regional pattern obtained by the structure tensor, C L (i,j),C D (i,j),C R (i, j) represents the pixel edge cost of the pixel in row i and column j belonging to the seam in IMG in the main diagonal, vertical and sub-diagonal directions respectively;

[0012] For horizontal seams, select the cumulative energy E from left to right. X (i,j) low pixel connection generation, the generation rule is:

[0013]

[0014] Among them, C H (i, j) represents the pixel edge cost in the horizontal direction;

[0015] The composite energy seam cutting method reduces the size of a two-dimensional image by deleting p columns of seams in the horizontal direction of the M×N two-dimensional image to adjust the size of the two-dimensional image to M×(Np), and then interpolating and adding q rows of seams in the vertical direction to adjust the size of the two-dimensional image to (M+q)×(Np), performing a similar transformation, scaling the two-dimensional image from (M+q)×(Np) to U×((Np)×M / (M+q)), and directly cutting the side with the smallest cumulative energy of (Np)×M / (M+q)-V columns on both sides of the image to obtain a U×V two-dimensional image;

[0016] The process of the composite energy seam cutting method for magnifying a two-dimensional image is to delete the seam found and insert two new pixels generated by the most similar pixel among the eight adjacent pixels of each pixel of the seam and the left and right adjacent pixels at the position of each deleted pixel;

[0017] Divide the two-dimensional image into blocks according to the average values ​​of the length and width of the quadrilateral grid of the three-dimensional grid model to obtain two-dimensional image sub-blocks;

[0018] Step 3: Each 2D image sub-block is compared with the length and width of the quadrilateral grid of the corresponding 3D grid model according to the grid shape of the corresponding 3D grid model, and then scaled accordingly;

[0019] Step 4: Map the scaled 2D image sub-block onto the 3D grid model;

[0020] Step 5: Generate a zero watermark for the mapped three-dimensional mesh model, encrypt the zero watermark and upload it to the blockchain, and save the hash value returned by the blockchain;

[0021] Step 6: For the three-dimensional mesh model to be detected, a binary watermark is generated according to the zero watermark stored in the blockchain, and compared with the original binary watermark. The correlation coefficient Corr is used to represent the similarity between the binary watermark and the original binary watermark to determine whether there is a watermark in the three-dimensional mesh model to be detected.

[0022] As a preferred implementation manner of the present invention: the step 1 specifically comprises: using a 3D laser scanner to scan the object to obtain 3D point cloud data of the object, generating a 3D mesh model of the object after preprocessing through certain meshing rules, wherein the mesh shape is a triangle, using the Morse-Smale complex method to convert the 3D mesh model with a triangular mesh shape into a 3D mesh model with a quadrilateral mesh shape, using a camera to capture the image of the object, and mapping the texture color to the 3D mesh model through collinear equation calculation, thereby generating a 3D mesh model with texture color.

[0023] As a preferred embodiment of the present invention: in the vertical seam generation rule in step 2, C L (i,j),C D (i,j),C R (i, j) and T(i, j) are determined by the following formulas:

[0024]

[0025]

[0026] Where IMG(i,j) represents the grayscale value of the pixel in row i and column j in the two-dimensional image IMG, t represents the set amplitude, and (i,j) represents the pixel in row i and column j in IMG;

[0027] For the horizontal seam generation rule, the pixel edge cost C in the horizontal direction is H (i,j) is determined as:

[0028] C H (i,j)=|IMG(i-1,j-1)-IMG(i+1,j-1)|+|IMG(i-1,j)-IMG(i+1,j)|.

[0029] As a preferred embodiment of the present invention: in step 3, the specific method for performing the corresponding scaling is:

[0030]

[0031]

[0032] Among them, m and n represent the length and width of the two-dimensional image sub-block, u and v represent the length and width of the quadrilateral mesh of the three-dimensional mesh model, and the reduction and enlargement are to redirect the image of the two-dimensional image sub-block using the composite energy seam cutting method.

[0033] As a preferred implementation manner of the present invention: Step 4 is specifically the specific process of mapping the two-dimensional image sub-block onto the three-dimensional mesh model, which is to correspond the four vertices of the quadrilateral mesh of the three-dimensional mesh model to the four vertices of the scaled two-dimensional image sub-block, and perform texture mapping on the quadrilateral mesh of the three-dimensional mesh model, and after processes such as edge correction and color adjustment, high-quality texture mapping of the two-dimensional image to the three-dimensional mesh model is achieved.

[0034] As a preferred implementation manner of the present invention: step 5 specifically generates a watermark for the mapped three-dimensional mesh model, which is a three-dimensional zero watermark method based on small line transformation that resists rotation attacks; for the original three-dimensional mesh model, determine the candidate vertex set, generate a cutting plane, create a small line dictionary based on the binary watermark, generate a zero watermark image, encrypt it with a private key, upload it to the blockchain, and retain the hash value returned by the blockchain.

[0035] As a preferred implementation manner of the present invention: step 6 specifically extracts a watermark for the three-dimensional mesh model to be detected, which is a three-dimensional zero watermark method for the three-dimensional mesh model to be detected that resists rotation attacks based on small line transformation; for the three-dimensional mesh model to be detected, a candidate vertex set is determined, a cutting plane is generated, and the small line coefficient is calculated according to the zero watermark stored in the blockchain to obtain a binary watermark; the correlation coefficient Corr is used to represent the similarity between the binary watermark extracted from the three-dimensional mesh model to be detected and the original binary watermark, and a threshold Δ is set. When Corr>Δ, it is considered that the three-dimensional mesh model has a watermark, and when Corr≤Δ, it is considered that the three-dimensional mesh model does not have a watermark.

[0036] Compared with the prior art, the present invention has the following beneficial effects:

[0037] 1. A non-planar watermarking method is proposed to adaptively map a two-dimensional image to the surface of a three-dimensional mesh model and embed the watermark, so as to embed watermarks on real non-planar objects;

[0038] 2. Upload the generated zero watermark to the blockchain, making full use of the blockchain's unforgeable, open and transparent nature to ensure the security and authority of the generated zero watermark;

[0039] 3. According to the pixel's FT saliency map value, MB saliency map value, gradient value and structure tensor, combined with the pixel edge cost, the energy value is determined, and a composite energy seam cutting method is proposed, which can more accurately represent the pixel's energy value, thereby making the image scaling effect better. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 It is the overall flow chart of non-planar zero watermark based on composite energy seam cutting;

[0041] Figure 2 Schematic diagram of pixel edge cost;

[0042] Figure 3 The flowchart of the method for generating three-dimensional zero watermark based on small line transformation and resistant to rotation attack;

[0043] Figure 4 This is a flow chart of a three-dimensional zero watermark detection method based on small line transformation that resists rotation attacks. DETAILED DESCRIPTION

[0044] The present invention is further explained below in conjunction with the accompanying drawings and specific embodiments. It should be understood that these examples are only used to illustrate the present invention and are not used to limit the scope of the present invention. After reading the present invention, various equivalent forms of modifications to the present invention by those skilled in the art all fall within the scope defined by the claims attached to this application.

[0045] The process of the non-planar zero watermark method based on composite energy seam cutting is as follows:

[0046] Step 1 specifically uses a 3D laser scanner to scan the object to obtain the 3D point cloud data of the object, and after preprocessing, generates a 3D mesh model of the object through certain mesh construction rules. The mesh shape is triangular, and the Morse-Smale complex method is used to convert it into a 3D mesh model with a quadrilateral shape. The camera is used to capture the object image, and the texture color is mapped to the 3D mesh model through collinear equation calculation, thereby generating a 3D mesh model with texture color.

[0047] The specific process of step 2 is as follows: in order to map the two-dimensional image IMG to the surface of the three-dimensional grid model, it is necessary to keep the size of the two-dimensional image of M×N and the size of the area to be mapped on the three-dimensional grid model of U×V within a certain range, otherwise it is necessary to scale the two-dimensional image using a composite energy seam cutting method, where M and N represent the length and width of the two-dimensional image, respectively, and U and V represent the length and width of the area to be mapped on the three-dimensional grid model, respectively;

[0048] The specific process of the composite energy seam shearing method is:

[0049] For vertical joints, select the cumulative energy E from top to bottom. Y (i,j) low pixel connection generation, the generation rule is:

[0050]

[0051] Among them, α, β, θ and μ represent different weights, F(i, j) represents the saliency value of the pixel in row i and column j in the FT saliency map generated by the full-resolution frequency modulation method of IMG, B(i, j) represents the saliency value of the pixel in row i and column j in the MB saliency map generated by the fast raster scanning saliency detection method of IMG, G(i, j) represents the gradient value of the pixel in row i and column j in IMG, T(i, j) represents the amplitude of the pixel in row i and column j in IMG determined by the regional pattern obtained by the structure tensor, C L (i,j),C D (i,j),C R (i, j) represents the pixel edge cost of the pixel in row i and column j belonging to the seam in IMG in the main diagonal, vertical and sub-diagonal directions, respectively. L (i,j),C D (i,j),C R (i, j) and T(i, j) are determined by the following formulas:

[0052]

[0053]

[0054] Where IMG(i,j) represents the grayscale value of the pixel in row i and column j in IMG, t represents the set amplitude, and (i,j) represents the pixel in row i and column j in IMG;

[0055] For horizontal seams, select the same cumulative energy E from left to right. X (i,j) low pixel connection generation, the generation rule is:

[0056]

[0057] Among them, C H (i,j) represents the pixel edge cost in the horizontal direction, which is determined as:

[0058] C H (i,j)=|IMG(i-1,j-1)-IMG(i+1,j-1)|+|IMG(i-1,j)-IMG(i+1,j)|

[0059] The composite energy seam cutting method reduces the size of a two-dimensional image by deleting p columns of seams in the horizontal direction of the M×N two-dimensional image to adjust the size of the two-dimensional image to M×(Np), and then interpolating and adding q rows of seams in the vertical direction to adjust the size of the two-dimensional image to (M+q)×(Np), performing a similar transformation, scaling the two-dimensional image from (M+q)×(Np) to U×((Np)×M / (M+q)), and directly cutting the side with the smallest cumulative energy of (Np)×M / (M+q)-V columns on both sides of the image to obtain a U×V two-dimensional image;

[0060] The process of the composite energy seam cutting method for magnifying a two-dimensional image is to delete the seams found and insert two new pixels generated by the pixel with the smallest difference in pixel value among the eight adjacent pixels of each pixel of the seam and the left and right adjacent pixels at the position of each deleted pixel. The generation rule is:

[0061]

[0062]

[0063] Among them, A represents 9 pixels in the local pixel area of ​​the original image, a0, a1, a2, a3, a4, a5, a6, a7, and a8 represent 9 pixels in A, a0 is the pixel in the seam, and b1 and b2 are the pixels added after the insertion operation of a0 is completed. Indicates the pixels a4, a0, a k The average value is rounded to an integer. Indicates the pixels a5, a0, a k The average value of a is rounded to an integer. k It is the pixel with the smallest difference from the pixel value of a0 among a1 to a8. After the insertion operation is completed, the pixels in the area [a4 a0 a5] in A become [a4 b1 b2 a5];

[0064] The two-dimensional image is divided into blocks according to the length and width of the quadrilateral grid of the three-dimensional grid model.

[0065] The specific process of step 3 is: for each 2D image sub-block, the length and width of the corresponding quadrilateral mesh of the 3D mesh model are compared respectively, and then the corresponding scaling is performed. The specific method is:

[0066]

[0067]

[0068] Among them, m and n represent the length and width of the two-dimensional image sub-block, u and v represent the length and width of the quadrilateral mesh of the three-dimensional mesh model, and the reduction and enlargement are to redirect the image of the two-dimensional image sub-block using the composite energy seam cutting method.

[0069] Step 4 is specifically the specific process of mapping the two-dimensional image sub-block onto the three-dimensional mesh model, which is to correspond the four vertices of the quadrilateral mesh of the three-dimensional mesh model to the four vertices of the scaled two-dimensional image sub-block, and perform texture mapping on the quadrilateral mesh of the three-dimensional mesh model. After edge correction, color adjustment and other processes, high-quality texture mapping of the two-dimensional image to the three-dimensional mesh model is achieved.

[0070] Step 5 specifically generates a watermark for the mapped three-dimensional mesh model, which uses a three-dimensional zero watermark method based on small line transformation to resist rotation attacks; for the original three-dimensional mesh model, determine the candidate vertex set, generate a cutting plane, create a small line dictionary based on the binary watermark, generate a zero watermark image, encrypt it with a private key, upload it to the blockchain, and retain the hash value returned by the blockchain.

[0071] Step 6 specifically extracts a watermark from the three-dimensional mesh model to be detected, which is a three-dimensional zero watermark method based on small line transformation to resist rotation attacks for the three-dimensional mesh model to be detected; for the three-dimensional mesh model to be detected, determine the candidate vertex set, generate a cutting plane, calculate the small line coefficient according to the zero watermark stored in the blockchain, and obtain a binary watermark; use the correlation coefficient Corr to represent the similarity between the binary watermark extracted from the three-dimensional mesh model to be detected and the original binary watermark, set a threshold Δ, when Corr>Δ, it is considered that the three-dimensional mesh model has a watermark, when Corr≤Δ, it is considered that the three-dimensional mesh model does not have a watermark.

[0072] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A non-planar zero watermark method based on composite energy seam cutting, characterized in that: The steps include: Step 1: Reconstruct the object in three dimensions and generate a three-dimensional mesh model; Step 2: Determine the image mapping area, and according to the grid division of the three-dimensional grid model, use the composite energy seam cutting method to scale the two-dimensional image so that the size of the M×N two-dimensional image and the size of the area to be mapped on the U×V three-dimensional grid model are kept within a certain range difference, where M and N represent the length and width of the two-dimensional image, respectively, and U and V represent the length and width of the area to be mapped on the three-dimensional grid model, respectively, and divide the two-dimensional image into blocks to obtain two-dimensional image sub-blocks; Among them, the specific process of the composite energy seam shearing method is: For vertical joints, select the cumulative energy E from top to bottom. Y (i,j) low pixel connection generation, the generation rule is: Among them, α, β, θ and μ represent different weights, F(i, j) represents the saliency value of the pixel in row i and column j in the FT saliency map generated by the full-resolution frequency modulation method of IMG, B(i, j) represents the saliency value of the pixel in row i and column j in the MB saliency map generated by the fast raster scanning saliency detection method of IMG, G(i, j) represents the gradient value of the pixel in row i and column j in IMG, T(i, j) represents the amplitude of the pixel in row i and column j in IMG determined by the regional pattern obtained by the structure tensor, C L (i,j),C D (i,j),C R (i, j) represents the pixel edge cost of the pixel in row i and column j belonging to the seam in IMG in the main diagonal, vertical and sub-diagonal directions respectively; The C L (i,j),C D (i,j),C R (i, j) and T(i, j) are determined by the following formulas: Where IMG(i,j) represents the grayscale value of the pixel in row i and column j in the two-dimensional image IMG, t represents the set amplitude, and (i,j) represents the pixel in row i and column j in IMG; For the horizontal seam generation rule, the pixel edge cost C in the horizontal direction is H (i,j) is determined as: C H (i,j)= | IMG(i-1,j-1)-IMG(i+1,j-1) | + | IMG(i-1,j)-IMG(i+1,j) | ; For horizontal seams, select the cumulative energy E from left to right. X (i,j) low pixel connection generation, the generation rule is: Among them, C H (i, j) represents the pixel edge cost in the horizontal direction; The composite energy seam cutting method reduces the size of a two-dimensional image by deleting p columns of seams in the horizontal direction of the M×N two-dimensional image to adjust the size of the two-dimensional image to M×(Np), and then interpolating and adding q rows of seams in the vertical direction to adjust the size of the two-dimensional image to (M+q)×(Np), performing a similar transformation, scaling the two-dimensional image from (M+q)×(Np) to U×((Np)×M / (M+q)), and directly cutting the side with the smallest cumulative energy of (Np)×M / (M+q)-V columns on both sides of the image to obtain a U×V two-dimensional image; The process of the composite energy seam cutting method for magnifying a two-dimensional image is to delete the seam found and insert two new pixels generated by the most similar pixel among the eight adjacent pixels of each pixel of the seam and the left and right adjacent pixels at the position of each deleted pixel; Divide the two-dimensional image into blocks according to the average values ​​of the length and width of the quadrilateral grid of the three-dimensional grid model to obtain two-dimensional image sub-blocks; Step 3: Each 2D image sub-block is compared with the length and width of the quadrilateral grid of the corresponding 3D grid model according to the grid shape of the corresponding 3D grid model, and then scaled accordingly; Step 4: Map the scaled 2D image sub-block onto the 3D grid model; Step 5: Generate a zero watermark for the mapped three-dimensional mesh model, encrypt the zero watermark and upload it to the blockchain, and save the hash value returned by the blockchain; Step 6: For the three-dimensional mesh model to be detected, a binary watermark is generated according to the zero watermark stored in the blockchain, and compared with the original binary watermark. The correlation coefficient Corr is used to represent the similarity between the binary watermark and the original binary watermark to determine whether there is a watermark in the three-dimensional mesh model to be detected.

2. According to claim 1, a non-planar zero watermark method based on composite energy seam cutting is characterized in that: Specifically, step 1 includes scanning an object with a 3D laser scanner to obtain 3D point cloud data of the object, generating a 3D mesh model of the object after preprocessing using certain meshing rules, wherein the mesh shape is a triangle, converting the 3D mesh model with a triangular mesh shape into a 3D mesh model with a quadrilateral mesh shape using the Morse-Smale complex method, capturing an image of the object using a camera, and mapping texture color to the 3D mesh model through collinear equation calculation, thereby generating a 3D mesh model with texture color.

3. According to claim 2, a non-planar zero watermark method based on composite energy seam cutting is characterized in that: In step 3, the specific method for performing the corresponding scaling is as follows: Among them, m and n represent the length and width of the two-dimensional image sub-block, u and v represent the length and width of the quadrilateral mesh of the three-dimensional mesh model, and the reduction and enlargement are to redirect the image of the two-dimensional image sub-block using the composite energy seam cutting method.

4. The non-planar zero watermark method based on composite energy seam cutting according to claim 3, characterized in that: Step 4 is specifically the specific process of mapping the two-dimensional image sub-block onto the three-dimensional mesh model, which is to correspond the four vertices of the quadrilateral mesh of the three-dimensional mesh model to the four vertices of the scaled two-dimensional image sub-block, and perform texture mapping on the quadrilateral mesh of the three-dimensional mesh model. After edge correction, color adjustment and other processes, high-quality texture mapping of the two-dimensional image to the three-dimensional mesh model is achieved.

5. A non-planar zero watermark method based on composite energy seam cutting according to claim 4, characterized in that: Step 5 specifically generates a watermark for the mapped three-dimensional mesh model, which uses a three-dimensional zero watermark method based on small line transformation to resist rotation attacks; for the original three-dimensional mesh model, determine the candidate vertex set, generate a cutting plane, create a small line dictionary based on the binary watermark, generate a zero watermark image, encrypt it with a private key, upload it to the blockchain, and retain the hash value returned by the blockchain.

6. A non-planar zero watermark method based on composite energy seam cutting according to claim 5, characterized in that: Step 6 specifically extracts a watermark from the three-dimensional mesh model to be detected, which is a three-dimensional zero watermark method based on small line transformation to resist rotation attack for the three-dimensional mesh model to be detected; for the three-dimensional mesh model to be detected, a candidate vertex set is determined, a cutting plane is generated, and the small line coefficient is calculated according to the zero watermark stored in the blockchain to obtain a binary watermark; The correlation coefficient Corr is used to represent the similarity between the binary watermark extracted from the 3D mesh model to be detected and the original binary watermark. A threshold Δ is set. When Corr>Δ, it is considered that the 3D mesh model has a watermark. When Corr≤Δ, it is considered that the 3D mesh model does not have a watermark.

Images