Robust Watermarking Method for Screen Photography under Lighting Conditions

Abstract

The present invention discloses a robust watermarking method for screen photographing under illumination conditions, belonging to the technical field of information hiding. (1) Calculate the illumination component and illumination threshold of the image; (2) Select the embedding area of the watermark, where the area exceeding the illumination threshold is not used as the embedding area; (3) Embed the watermark sequence into the image in the DCT domain to obtain the watermarked image; (4) Perform perspective correction and illumination correction on the captured watermarked image; (5) Use bilateral filtering to perform edge-preserving denoising on the illumination-corrected image; (6) Select the extraction area of the watermark; (7) Extract the watermark sequence in the DCT domain by means of cross-validation. The present invention reduces the influence of illumination on the invisibility of the watermarked image, reduces the illumination component in the overexposed area, and can enhance the features of the image; bilateral filtering is performed after illumination processing to achieve the function of edge-preserving denoising.

Description

Technical Field

[0001] The present invention belongs to the technical field of information hiding, and particularly relates to a robust watermarking method for screen photography in a lighting environment. Background Technique

[0002] Digital watermarking is an information hiding technology that can hide digital signal information such as images and texts in original data such as images, videos, and audios, so as to achieve the purpose of digital product copyright protection and integrity authentication. Due to the convenience of obtaining information by taking pictures, some important information is easily leaked through taking pictures, causing certain losses. Therefore, a watermark that can resist screen photography is needed to protect the copyright of important information.

[0003] At present, some scholars have studied screen photography watermarks and achieved certain results. Existing methods include embedding watermark information in the DCT domain, using scale-invariant feature transform to select the watermark feature area. In order to retain the watermark information as much as possible, it is proposed to embed the watermark in a small template and repeat the embedding multiple times to achieve a certain robustness. There are also methods of embedding watermarks in the DFT domain, using the Harris-Laplace detector and accelerated robust features to construct the feature area, and embedding the watermark in a non-rotating embedding manner. With the success of the application of deep learning in the traditional watermarking field, some scholars have also begun to study screen photography watermarks based on deep learning. Among them, some scholars have proposed a template watermark embedding method, which embeds a message template and a positioning template in the image. When extracting the watermark, a two-stage network of an enhancer network and a classification network is used to extract the watermark. Some scholars have also proposed an end-to-end neural network for hiding information and extracting information, and a distortion network is established between the encoder and the decoder to maintain the resilience of the hidden information to the camera.

[0004] However, the existing screen photography watermarking schemes proposed above rarely consider the influence of the lighting factor on the robustness of the watermark. Some scholars have proposed to use histogram equalization to process the uneven lighting areas, but there will be problems such as serious color distortion and over-enhancement. Moreover, embedding the watermark in areas with too strong lighting will reduce the invisibility of the watermark. When extracting, if there are overexposed areas caused by the influence of external light sources, the features of the image will be unclear, thus affecting watermark positioning and extraction. Summary of the Invention

[0005] Object of the Invention: To solve the influence of lighting on screen photography watermarks, the present invention proposes a robust watermarking method for screen photography in a lighting environment.

[0006] Technical Solution: To achieve the object of the present invention, the technical solution adopted by the present invention is: a robust watermarking method for screen photography in a lighting environment, and the steps are as follows:

[0007] (1) Calculate the illumination component and illumination threshold of the image;

[0008] (2) Select the embedding area of the watermark, where the area exceeding the illumination threshold is not used as the embedding area;

[0009] (3) Embed the watermark sequence into the image in the DCT domain to obtain the watermarked image;

[0010] (4) Perform perspective correction and illumination correction on the captured watermarked image;

[0011] (5) Use bilateral filtering to remove noise while preserving edges for the illumination-corrected image;

[0012] (6) Select the extraction area of the watermark;

[0013] (7) Extract the watermark sequence in the DCT domain by means of cross-validation.

[0014] Further, the specific process of the step (1) of calculating the illumination component and illumination threshold of the image is as follows:

[0015] Obtain the original image, convert the image from the RGB space to the HSV space, select the brightness space V, and use multi-scale Gaussian convolution to obtain the illumination component F(x, y):

[0016]

[0017] where N represents the number of scales, i represents the counter, ω i represents the weight coefficient of the illumination component, I(x, y) represents the input image, G(x, y) represents the Gaussian function, (x, y) represents the pixel coordinates, λ represents the normalization constant, and m represents the scale factor;

[0018] The calculation method of the illumination threshold e is as follows:

[0019] Arbitrarily select an illumination component value, which divides the illumination component into two sets A and B. The illumination component values in set A are greater than or equal to this value, and the illumination component values in set B are less than this value;

[0020] The average values of the illumination components in sets A and B are denoted as f A and f B , and the proportions of the number of pixels in sets A and B to the total number of pixels are denoted as P A and P B , and the between-class variance ICV is defined as:

[0021] ICV = P A α (f A - f)2 +P B α (f B -f) 2

[0022] Among them, α represents a parameter between 0 and 1 for adjusting the segmentation effect, and f represents the average value of all illumination components; the illumination component that makes the ICV value the largest is selected as the illumination threshold e;

[0023] The illumination threshold e is set for the selection of the feature region. If there is an illumination component in the feature region that exceeds the illumination threshold e, then this feature region is not selected for watermark embedding.

[0024] Furthermore, the specific process of selecting the watermark embedding region in step (2) is as follows:

[0025] The watermark sequence is subjected to BCH coding to obtain the binary watermark sequence to be embedded. A watermark matrix is constructed according to the watermark sequence to be embedded. The size of the matrix is s*t, where s and t are the number of rows and columns of the matrix respectively;

[0026] Using the intensity-based SIFT algorithm, the top h feature points with the largest intensity are selected as candidate feature points according to the intensity of the feature points. The absolute value in the difference-of-Gaussians domain in the SIFT algorithm is the intensity of the feature points;

[0027] Corresponding candidate feature regions are obtained according to the candidate feature points. The region of s*t*8*8 centered on the candidate feature points is used as the candidate feature region; the candidate feature regions are screened to obtain the final watermark embedding region.

[0028] Furthermore, the B-channel image of the original RGB image is selected for the selection of the watermark feature region and the embedding of the watermark.

[0029] Furthermore, the screening method for the candidate feature regions is as follows:

[0030] First, the regions where the feature regions exceed the image boundary are removed; second, the feature regions with illumination components greater than the illumination threshold are removed; then, the feature regions where the density of the feature points in the feature regions reaches the set value are selected;

[0031] Finally, if there are overlapping regions in the obtained feature regions, then the region with the larger intensity of the feature points is selected as the final feature region.

[0032] Furthermore, the watermark embedding process in step (3) is as follows:

[0033] The watermark is embedded in the DCT domain. The image is transformed from the spatial domain to the DCT domain, and the embedding of the watermark is achieved by comparing the magnitudes of a pair of coefficient values in the frequency of the DCT domain. Specifically:

[0034] Select the discrete cosine transform coefficients C1 and C2 at two positions in the middle frequency of the DCT domain. If the watermark information to be embedded is 1, adjust the magnitudes of C1 and C2 through the watermark embedding strength, and keep C1 < C2. If the watermark information to be embedded is 0, adjust the magnitudes of C1 and C2 through the watermark embedding strength, and keep C1 > C2.

[0035] Furthermore, the watermark embedding strength is obtained from the attention mask and the JPEG compression value. Specifically:

[0036] The image passes through the attention model G A , generates the attention mask AM, normalizes each value in AM to the range of 0 to 1, determines the watermark embedding strength according to this value, and simultaneously considers the JPEG compression value to ensure that the watermark embedding strength still maintains the corresponding relationship after compression. The watermark embedding formula is as follows:

[0037]

[0038] where AM represents the attention mask generated by the image passing through the attention model, G A represents the attention model, I(x, y) represents the input image, d represents the watermark embedding strength determined by the JPEG compression factor, q1 and q2 respectively represent the JPEG compression factors of C1 and C2, C1 and C2 represent the discrete cosine transform coefficient values at two positions in the middle frequency of the DCT domain, β represents the watermark embedding strength control factor, and wm represents the watermark.

[0039] Furthermore, the specific steps of step (6) are as follows:

[0040] Use the intensity-based SIFT algorithm to select candidate feature points and their corresponding candidate feature regions; then screen the candidate feature regions, and the screening conditions are as follows:

[0041] 1) Remove the feature regions that exceed the image boundary;

[0042] 2) Select the feature regions where the density of feature points within the feature region reaches the set value;

[0043] Finally, obtain the screened feature points and feature regions according to the two screening conditions, and use the screened feature regions for watermark extraction.

[0044] Furthermore, the specific steps of step (7) are as follows:

[0045] The watermark extraction process is to perform a DCT transform on the feature region to convert it to the DCT domain, select a pair of coefficient values C1 and C2 in the middle frequency of the DCT domain, and determine whether the embedded watermark is 0 or 1 by comparing the magnitudes of C1 and C2; if C1 > C2, the embedded watermark is 0, otherwise it is 1;

[0046] Perform offset compensation on the filtered feature points and feature regions, and select 9 feature points centered on the filtered feature points as a group of feature points;

[0047] Extract a watermark information from the feature region centered on each feature point. A group of feature points extracts 9 watermark information as a watermark group; Obtain the binary watermark sequence through cross-verification of each watermark group, and use BCH decoding to obtain the final watermark sequence.

[0048] Beneficial effects: Compared with the prior art, the technical solution of the present invention has the following beneficial technical effects:

[0049] The present invention can accurately extract the watermark information from the screen photographed image with light distortion. When selecting the watermark embedding area, the area with too strong light in the original image is not embedded with information to reduce the impact of light on the invisibility of the watermark image; In the watermark extraction stage, it is proposed to perform light processing on the image obtained after shooting to reduce the light component in the overexposed area, which can enhance the features of the image, so as to more accurately locate the watermark embedding area and improve the accuracy of watermark extraction; Since light processing of the image will bring problems such as edge blurring and increased noise, it is proposed to perform bilateral filtering after light processing to achieve the function of edge-preserving denoising. Brief Description of the Drawings

[0050] Figure 1 is the flowchart of the method of the present invention;

[0051] Figure 2 is the flowchart of watermark generation and embedding;

[0052] Figure 3 is the flowchart of watermark extraction and authentication. Detailed Embodiments

[0053] The technical solution of the present invention will be further described below in conjunction with the drawings and embodiments.

[0054] The robust watermark method for screen photography in the light environment described in the present invention has a process as Figure 1 , and the steps are as follows:

[0055] (1) Calculate the light component and light threshold of the image;

[0056] (2) Select the watermark embedding area, and the area exceeding the light threshold is not used as the embedding area;

[0057] (3) Embed the watermark sequence into the image in the DCT domain to obtain the watermark image;

[0058] (4) Perform perspective correction and light correction on the photographed watermark image;

[0059] (5) Perform edge-preserving denoising on the illumination-corrected image using bilateral filtering;

[0060] (6) Select the extraction region of the watermark;

[0061] (7) Extract the watermark sequence in the DCT domain by means of cross-validation.

[0062] The specific process of calculating the illumination component and illumination threshold in step (1) is as follows: First, obtain the original image, convert the image from the RGB space to the HSV space, select the brightness space V, and use multi-scale Gaussian convolution to obtain the illumination component F(x, y):

[0063]

[0064] where N represents the number of scales, i represents the counter, ω i represents the weight coefficient of the illumination component, I(x, y) represents the input image, G(x, y) represents the Gaussian function, (x, y) represents the pixel coordinates, λ represents the normalization constant, and m represents the scale factor.

[0065] Calculate the average value f and variance of the illumination component F(x, y). If the variance is less than a certain value, the illumination of this image is uniform, and there is no need to consider the influence of illumination on watermark embedding. Otherwise, it is considered that there is a region with overly strong local illumination in this image, and this region with overly strong illumination is not suitable for watermark embedding. At this time, an illumination threshold e needs to be set for the selection of the feature region. If there is an illumination component exceeding the illumination threshold e in the feature region, this feature region is not selected for watermark embedding. The calculation method of the illumination threshold e is as follows:

[0066] Arbitrarily select an illumination component value, which divides the illumination components into two sets A and B. The illumination component values in set A are greater than or equal to this value, and the illumination component values in set B are less than this value; the average values of the illumination components in sets A and B are denoted as f A 、f B , and the proportions of the number of pixels in sets A and B to the total number of pixels are denoted as P A 、P B , and the between-class variance ICV is defined as:

[0067] ICV = P A α (f A -f) 2 +P B α (f B -f) 2

[0068] Among them, α represents a parameter between 0 and 1 for adjusting the segmentation effect, and f represents the average value of all light components; the light component that makes the ICV value the largest is selected as the light threshold e; the light threshold e is set for the selection of the feature region, and if there is a light component in the feature region that exceeds the light threshold e, then this feature region is not selected for watermark embedding.

[0069] Since image modification is performed on the B channel of the original RGB image and is less perceptible to the human eye, which is better for the invisibility of the watermark. Therefore, in this embodiment, the B channel image of the original RGB image is selected for the selection of the watermark feature region and the embedding of the watermark. For the flowchart of watermark generation and embedding, see Figure 2 。

[0070] The specific process of selecting the watermark embedding region in step (2) is as follows:

[0071] The watermark sequence is BCH-encoded (Bose Chaudhuri-Hocqunghem) to obtain the binary watermark sequence to be embedded. A watermark matrix is constructed according to the watermark sequence to be embedded. The size of the matrix is s*t, where s and t are the number of rows and columns of the matrix respectively; the Scale-invariant feature transform (SIFT) algorithm based on intensity is used. According to the intensity of the feature points, the top h feature points with the largest intensity are selected as candidate feature points. The absolute value in the difference Gaussian domain in the SIFT algorithm is the intensity of the feature points; the corresponding candidate feature regions are obtained according to the candidate feature points, and the region of s*t*8*8 centered on the candidate feature points is used as the candidate feature region; the candidate feature regions are screened to obtain the final watermark embedding region. Among them, the screening method for the candidate feature regions is as follows:

[0072] First, remove the regions where the feature regions exceed the image boundary; secondly, remove the feature regions where the light components are greater than the light threshold e; then, select the feature regions where the density of the feature points in the feature regions reaches the set value; finally, if there are overlapping regions in the obtained feature regions, then select the region with the greater intensity of the feature points as the final feature region.

[0073] The watermark embedding process in step (3) is: embedding the watermark in the DCT domain. The image (B channel image) is transformed from the spatial domain to the DCT domain, and the embedding of the watermark is realized by comparing the magnitudes of a pair of coefficient values in the frequency of the DCT domain. Specifically: select the discrete cosine transform coefficients C1 and C2 at two positions in the frequency of the DCT domain. If the watermark information to be embedded is 1, then adjust the magnitudes of C1 and C2 through the watermark embedding intensity, keeping C1 < C2. If the watermark information to be embedded is 0, then adjust the magnitudes of C1 and C2 through the watermark embedding intensity, keeping C1 > C2.

[0074] The embedding strength of the watermark is obtained from the attention mask and the JPEG compression value, specifically as follows: The image passes through the attention model G A , generating an attention mask AM. Each value in AM is normalized to the range of 0 to 1. The watermark embedding strength is determined according to this value, and the JPEG compression value is also considered to ensure that the watermark embedding strength still maintains the corresponding relationship after compression. The watermark embedding formula is as follows:

[0075]

[0076] where AM represents the attention mask generated by the image passing through the attention model, G A represents the attention model, I(x, y) represents the input image, d represents the watermark embedding strength determined by the JPEG compression factor, q1 and q2 respectively represent the JPEG compression factors of C1 and C2, C1 and C2 represent the discrete cosine transform coefficient values at two positions in the middle frequency of the DCT domain, β represents the watermark embedding strength control factor, and wm represents the watermark.

[0077] The specific process of step (4) is as follows:

[0078] The watermarked image is photographed to obtain a photographed image. Perspective transformation and cropping operations are performed on this image to obtain an image to be extracted. For the image to be extracted, illumination correction is performed. First, the illumination component F(x, y) is obtained using the formula in step (1). Then, the two-dimensional gamma function is used to correct the image brightness to make the image brightness uniform, facilitating subsequent image feature acquisition and watermark extraction. The formula of the two-dimensional gamma function is:

[0079]

[0080] where o(x, y) represents the image after correction by the two-dimensional gamma function, I(x, y) represents the input image, γ represents the exponential value for brightness enhancement, F(x, y) represents the illumination component, and f represents the average value of all illumination components.

[0081] The specific process of using bilateral filtering in step (5) is as follows:

[0082] Since performing illumination correction on the image is equivalent to adding one more attack to the image, which will cause problems such as edge blurring and noise increase. In the present invention, bilateral filtering is used for the corrected image to achieve the function of edge-preserving denoising, and then the image is converted to the RGB space. The specific formula of bilateral filtering is:

[0083]

[0084]

[0085] Among them, z(x, y) represents the output pixel point, r(k, l) represents the input point, w(x, y, k, l) represents the value calculated through two Gaussian functions, (x, y) represents the central coordinates of the template window, (k, l) represents the coordinates of other pixels in the template window, σ d and σ r respectively represent the domain variance and the range variance, and r(x, y) represents the central input point of the template window.

[0086] The specific steps of step (6) include:

[0087] Consistent with the method of step (2), use the intensity-based SIFT algorithm to select candidate feature points and their corresponding candidate feature regions; then screen the candidate feature regions, and the screening conditions are as follows: 1) Remove the feature regions that exceed the image boundary; 2) Select the feature regions where the density of feature points within the feature region reaches the set value; finally, obtain the screened feature points and feature regions according to the two screening conditions, and use the screened feature regions to extract the watermark.

[0088] The specific steps of step (7) include:

[0089] The watermark extraction process is to perform DCT transformation on the feature region to convert it into the DCT domain, select a pair of coefficient values C1 and C2 in the middle frequency of the DCT domain, and determine whether the embedded watermark is 0 or 1 by comparing the sizes of C1 and C2; if C1 > C2, the embedded watermark is 0, otherwise it is 1; since the feature points of the image obtained after shooting may be slightly offset, therefore, perform offset compensation on the screened feature points and feature regions, and select 9 feature points centered on the screened feature points as a group of feature points; a watermark information is extracted from the feature region centered on each feature point, and 9 watermark information are extracted from a group of feature points as a watermark group; by cross-verifying each watermark group, a binary watermark sequence is obtained, and finally, the final watermark sequence is obtained using BCH decoding.

[0090] The above is the preferred embodiment of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the technical principle of the present invention, several improvements and deformations can be made, and these improvements and deformations should also be regarded as the protection scope of the present invention.

Claims

1. A robust watermarking method for photographing a screen under a lighting environment, characterized in that: The method steps are as follows: (1) Calculate the illumination component and illumination threshold of the image; (2) Select the embedding area of the watermark, where the area exceeding the illumination threshold is not used as the embedding area; (3) Embed the watermark sequence into the image in the DCT domain to obtain the watermarked image; (4) Perform perspective correction and illumination correction on the captured watermarked image; (5) Use bilateral filtering to perform edge-preserving denoising on the illumination-corrected image; (6) Select the extraction area of the watermark; (7) Extract the watermark sequence in the DCT domain by means of cross-validation.

2. The robust watermarking method for screen photographing in the illumination environment according to claim 1, wherein: The specific process of calculating the illumination component and illumination threshold in step (1) is as follows: Obtain the original image, convert the image from the RGB space to the HSV space, select the brightness space V, and use multi-scale Gaussian convolution to obtain the illumination component F(x, y): where N represents the number of scales, i represents the counter, ω i represents the weight coefficient of the illumination component, I(x, y) represents the input image, G(x, y) represents the Gaussian function, (x, y) represents the pixel coordinates, λ represents the normalization constant, and m represents the scale factor; The calculation method of the illumination threshold e is as follows: Arbitrarily select an illumination component value, which divides the illumination component into two sets A and B. The illumination component values in set A are greater than or equal to this value, and the illumination component values in set B are less than this value; The average values of the light components in sets A and B are denoted as f A and f B , and the proportions of the number of pixels in sets A and B to the total number of pixels are denoted as P A and P B . The inter-class variance ICV is defined as: ICV = P A α (f A - f) 2 + P B α (f B - f) 2 Among them, α represents a parameter between 0 and 1 for adjusting the segmentation effect, and f represents the average value of all illumination components; select the illumination component that makes the ICV value the largest as the illumination threshold e; Set the illumination threshold e for the selection of the feature area. If there is an illumination component in the feature area that exceeds the illumination threshold e, then this feature area is not selected for watermark embedding.

3. The robust watermarking method for screen photography in a lighting environment according to claim 1 or 2, characterized in that: The specific process of selecting the watermark embedding area in step (2) is as follows: Perform BCH coding on the watermark sequence to obtain the binary watermark sequence to be embedded, construct a watermark matrix according to the watermark sequence to be embedded, and the size of the matrix is s*t, where s and t are the number of rows and columns of the matrix respectively; Use the intensity-based SIFT algorithm, and select the top h feature points with the largest intensity as candidate feature points according to the intensity of the feature points. The absolute value in the difference-of-Gaussians domain in the SIFT algorithm is the intensity of the feature points; Obtain the corresponding candidate feature areas according to the candidate feature points, and use the area of s*t*8*8 centered on the candidate feature points as the candidate feature areas; perform certain screening on the candidate feature areas to obtain the final watermark embedding area.

4. The robust watermarking method for screen photographing in a lighting environment according to claim 3, wherein: Select the B-channel image of the original RGB image for the selection of the watermark feature area and the embedding of the watermark.

5. The robust watermarking method for screen photographing in a lighting environment according to claim 3, wherein: The screening method of the candidate feature areas is as follows: First, remove the areas where the feature areas exceed the image boundary; second, remove the feature areas where the illumination component is greater than the illumination threshold; then, select the feature areas where the density of the feature points in the feature areas reaches the set value; Finally, if there are overlapping areas in the obtained feature areas, then select the area with the larger feature point intensity as the final feature area.

6. The robust watermarking method for screen photographing in a lighting environment according to claim 1 or 2, characterized in that: The watermark embedding process in step (3) is as follows: Embed the watermark in the DCT domain, convert the image from the spatial domain to the DCT domain, and realize the embedding of the watermark by comparing the magnitudes of a pair of coefficient values in the frequency of the DCT domain. Specifically: Select the discrete cosine transform coefficients C1 and C2 at two positions in the middle frequency of the DCT domain. If the watermark information to be embedded is 1, adjust the magnitudes of C1 and C2 through the watermark embedding strength, keeping C1 < C2. If the watermark information to be embedded is 0, adjust the magnitudes of C1 and C2 through the watermark embedding strength, keeping C1 > C2.

7. The robust watermarking method for screen photographing under illumination environment according to claim 6, wherein: The embedding strength of the watermark is obtained from the attention mask and the JPEG compression value, specifically as follows: The image passes through the attention model G A , generating an attention mask AM. Each value in AM is normalized to the range of 0 to 1. The watermark embedding strength is determined according to this value, and the JPEG compression value is also considered to ensure that the watermark embedding strength still maintains the corresponding relationship after compression. The watermark embedding formula is as follows: Among them, AM represents the attention mask generated by the image through the attention model, G A represents the attention model, I(x, y) represents the input image, d represents the watermark embedding strength determined by the JPEG compression factor, q1 and q2 respectively represent the JPEG compression factors of C1 and C2, C1 and C2 represent the discrete cosine transform coefficient values at two positions in the middle frequency of the DCT domain, β represents the watermark embedding strength control factor, and wm represents the watermark.

8. The robust watermarking method for screen photographing in a lighting environment according to claim 1 or 2, characterized in that: The specific steps of step (6) include: Use the intensity-based SIFT algorithm to select candidate feature points and their corresponding candidate feature regions; then screen the candidate feature regions, and the screening conditions are as follows: 1) Remove the feature regions that exceed the image boundary; 2) Select the feature regions where the density of feature points within the feature regions reaches a set value; Finally, obtain the screened feature points and feature regions according to the two screening conditions, and use the screened feature regions for watermark extraction.

9. The robust watermarking method for screen photographing in a lighting environment according to claim 8, wherein: The specific steps of step (7) include: The watermark extraction process is to perform DCT transformation on the feature regions to convert them into the DCT domain, select a pair of coefficient values C1 and C2 in the middle frequency of the DCT domain, and determine whether the embedded watermark is 0 or 1 by comparing the magnitudes of C1 and C2; if C1 > C2, the embedded watermark is 0, otherwise it is 1; Perform offset compensation on the screened feature points and feature regions, and select 9 feature points centered on the screened feature points as a group of feature points; Extract one watermark information from the feature region centered on each feature point. A group of feature points extracts 9 watermark information, which is used as a watermark group; obtain the binary watermark sequence through cross-verification of each watermark group, and use BCH decoding to obtain the final watermark sequence.

Images