A method for detecting and removing image watermarks

Abstract

The present invention is a method for detecting and removing image watermarks, including using a normalized correlation coefficient to detect the position where a watermark appears in an image, and then restoring the original image for the image pixels missing at the watermark position by utilizing the structural representation learned from general natural images and an algorithm for maximizing the expected log-likelihood of image blocks. Advantages of the present invention: Compared with the computational feature matching method and neural network method for detecting watermarks in the prior art, the present invention is concise and efficient, and the watermark detection effect is also very good; compared with the method of filling the missing pixels at the watermark in the prior art, the effect of removing the watermark and restoring the original image is good; compared with the neural network-based method in the prior art, it does not require a large amount of data to train the model, nor does it require training different models for different watermark styles, the solution cost is low, and it also has better flexibility when removing different watermark styles.

Description

Technical Field

[0001] The invention relates to a method for detecting and removing an image watermark, and belongs to the technical field of image processing. Background Art

[0002] In modern society and the Internet environment, watermarks are widely used in multimedia data such as images, videos, texts, and voices. It is an effective technical means to add some additional digital information to these multimedia data in order to protect copyrights and identify authenticity.

[0003] However, in some scenarios, without affecting the copyright of the original data, the presence of watermarks will have some impact on people's use. For example, watermarks in images will affect people's perception and bring certain difficulties to subsequent image processing and recognition. Therefore, it is necessary to detect and remove watermarks in images.

[0004] There are many different watermark detection and removal methods in the prior art for different types of image watermarks.

[0005] As for the situation where watermarks appear at different positions of an image as a style template, this form of watermarks exists in large numbers in reality, and its solution depends on the effective detection of the watermark pattern and the subsequent removal and restoration of the original image. The existing technology generally detects the watermark by calculating feature matching first, and then uses the pixels around the watermark to fill in the missing pixels. The processing results are sometimes blurred or have obvious traces of mosaics.

[0006] With the rise of deep learning technology, existing technologies also choose to use neural networks to detect and remove watermarks. This type of solution often requires massive training data, and different network models need to be trained for different watermark templates, which greatly increases the cost of the solution.

[0007] In order to fill the missing image information caused by the presence of watermarks and obtain a clear de-watermarked image, independent component analysis is considered as a prior model of natural image blocks, and the image block expected log-likelihood maximization algorithm is considered as the framework of image restoration.

[0008] Independent component analysis is a statistical method for learning linearly independent data representations. When applied to image data, it can learn structural information used to represent images from general natural images, which is similar to the sparse coding model. Therefore, the learned structural information can be used as a prior model of the image block to restore the local information in the watermark image that is missing due to the occlusion of the watermark.

[0009] The image block expected log-likelihood maximization algorithm is a model for image restoration based on image block priors. Its basic idea is to use the maximum a posteriori estimation method under a pre-selected prior model, hoping that the image blocks of the restored image will be as consistent with the prior as possible, so as to obtain a restored image that conforms to the general natural image structure as much as possible. When using this model to remove watermarks, the key is to be able to accurately detect the watermark pattern contained in the image, thereby determining the precise location of the missing pixel information in the watermark image, and then use this algorithm to obtain a relatively good de-watermarked image. Summary of the invention

[0010] The present invention proposes a method for detecting and removing image watermarks, which aims to overcome the above-mentioned deficiencies in the prior art, effectively solve the problem of detecting watermark patterns, and achieve good image restoration effects after watermark removal.

[0011] The technical solution of the present invention is a method for detecting and removing image watermarks, which includes using a normalized correlation coefficient to detect the position where the watermark appears in the image, and then restoring the original image for the image pixels missing at the watermark position by utilizing a structural representation learned from general natural images and an algorithm for maximizing the expected log-likelihood of image blocks.

[0012] Preferably, the method specifically comprises the following steps:

[0013] Step S1: Obtain a clean standard watermark template: treat an image watermark with a fixed style and repeated arrangement in a certain pattern as a template, and stamp the watermark on an image with a white or black background, or lay the basic elements of the watermark on the background image in a certain pattern to obtain a clean standard watermark template;

[0014] Step S2: Calculate the normalized correlation coefficient between the standard watermark template and the target image, and infer the most likely position of the standard watermark template in the target image: the normalized correlation coefficient is an indicator for template matching. When the normalized template and the normalized area in the target image are the same, the indicator reaches the maximum value, indicating that the most likely match occurs in the area. The position of the standard watermark template in the target image is inferred by the maximum value of the normalized correlation coefficient.

[0015] Step S3: Calculate the watermark pattern appearing in the target image according to the position of the standard watermark template and the size of the target image: According to the position of the standard watermark template determined in step S24, obtain the exact position of a part of the basic watermark elements contained in the target image, and then calculate the entire watermark pattern contained in the target image according to the tiling rule of the basic watermark elements and the size of the target image;

[0016] Step S4: Erasing the pixels at the watermark pattern position in the target image: Erasing is achieved by setting the pixels at the position to zero, indicating that these pixels are missing information of the image;

[0017] Step S5: Use the image block expected log-likelihood maximization algorithm to restore the original watermark-free image: The image block expected log-likelihood maximization algorithm is used as an image restoration algorithm, combined with the structural representation learned from general natural images, to restore the missing local information in the image and remove the image watermark.

[0018] Preferably, in step S2, the normalized correlation coefficient between the standard watermark template and the target image is calculated, and the most likely position of the standard watermark template in the target image is deduced, which specifically includes:

[0019] Step S21: slide the standard watermark template pixel by pixel on the target image: place the standard watermark template within the range of the target image, and slide it one pixel at a time, from left to right and from top to bottom;

[0020] Step S22: At each sliding position, the normalized correlation coefficient between the standard watermark template and the corresponding area on the target image is calculated: For each sliding, the current pixel position of the standard watermark template on the target image is (x, y), and the calculation formula of the normalized correlation coefficient is:

[0021]

[0022] Among them, w is the template, is the average value of the template, f is the corresponding area on the target image, and f xy is the average value of the area, s and t are the traversal coordinates of the pixels in the template area, and the normalized correlation coefficient calculated at each sliding position is saved;

[0023] Step S23: Select the maximum value of the normalized correlation coefficient: Find the maximum value among all the normalized correlation coefficients γ(x, y) obtained in step S22. If there are multiple maximum values, only find one of them and record the pixel coordinates x and y of the maximum value.

[0024] Step S24: Calculate the coordinate position of the standard watermark template in the target image based on the position where the maximum normalized correlation coefficient appears: Calculate the position of the standard watermark template in the target image based on the pixel coordinates of the maximum normalized correlation coefficient obtained in step S23 and the size of the standard watermark template. The calculation method is: subtract half of the width of the standard watermark template from the coordinate x, and subtract half of the height of the standard watermark template from the coordinate y. If the width or height of the standard watermark template is an odd number, the corresponding amount subtracted is half of the width or height minus 1.

[0025] Preferably, in step S3, the watermark pattern appearing in the target image is calculated according to the position of the standard watermark template and the size of the target image: if the specific parameters for generating the template are known, the watermark pattern of the target image is obtained by program calculation, or by using a method of tiling the standard watermark template, specifically including:

[0026] Step S31: Generate a blank image according to the size of the target image;

[0027] Step S32: Paste the standard watermark template obtained in step S2 at the position where the target image appears, which is called the anchor point, and use the standard watermark template matrix to replace the data at the anchor point of the blank image matrix;

[0028] Step S33: Taking the standard watermark template at the anchor point as a reference, tiling the standard watermark template around until the entire blank image is covered. When tiling the standard watermark template, it is ensured that the watermarks in the overlapping areas of different standard watermark templates are consistent, which is also achieved by placing the standard watermark template matrix at the corresponding position;

[0029] Step S34: Crop the redundant parts of the standard watermark template that exceed the blank image range to obtain the watermark style of the target image: when multiple standard watermark template matrices cover the entire blank image matrix, only take the data within the blank image matrix to obtain the watermark style of the target image.

[0030] Preferably, in step S5, the image block expected log-likelihood maximization algorithm is used to restore the original watermark-free image, which specifically includes: the image block expected log-likelihood maximization algorithm is an algorithm model that uses natural image statistical priors to restore images. The algorithm provided by Yair Weiss is used. When the algorithm is used to restore the watermark-free image, the size of the image block is selected to be 8×8, the watermark pattern of the target image obtained in step S4 is used as the mask template, and the prior model of the natural image block of independent component analysis is used. For the pixel position with watermark, its parameter lambda is set to 0, and the other pixel positions are set to 10. 6 ,The entire algorithm outer loop iterates 5 times, and the parameter beta is set to 10, 40, 160, 1280 and 5120 respectively in each iteration. In each outer loop iteration, the inner loop of the algorithm is set to iterate 20 times, and then the final calculation result is obtained.

[0031] The advantages of the present invention are as follows: 1) By utilizing the fixed and repeated characteristics of the watermark pattern, a standard watermark template is first obtained, and then the position of the watermark in the target image is detected by calculating the normalized correlation coefficient between the watermark template and the target image, and then the entire watermark pattern contained in the target image is obtained. Compared with the calculation feature matching method and the neural network method for detecting watermarks in the prior art, the present invention is concise and efficient, and the watermark detection effect is also very good.

[0032] 2) Compared with the existing method of filling missing pixels at the watermark, this paper adopts the image block prior model learned from natural images and uses the algorithm of maximizing the expected log-likelihood of image blocks to restore the watermark-free image, which utilizes the general structural representation in natural images. Therefore, the effect of removing watermarks and restoring the original image is good.

[0033] 3) Compared with the existing neural network-based method, it does not require massive data to train the model, nor does it require training different models for different watermark styles. Therefore, it has a lower solution cost and better flexibility in removing different watermark styles. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 It is a flow chart of the method for detecting and removing image watermarks of the present invention.

[0035] Figure 2 yes Figure 1 A schematic diagram of an embodiment of the standard watermark template in step S1.

[0036] Figure 3 yes Figure 1 A schematic diagram of an embodiment of the target image in step S2.

[0037] Figure 4 yes Figure 1 Specific flow diagram of step S2 in FIG.

[0038] Figure 5 yes Figure 1 Specific flow chart of step S3 in FIG.

[0039] Figure 6 yes Figure 5 FIG. 1 is a schematic diagram of an embodiment of tiling the standard watermark template in step S33 to obtain the target image watermark pattern. DETAILED DESCRIPTION

[0040] The present invention is further described in detail below with reference to examples and specific implementation modes.

[0041] In view of the problems existing in the above-mentioned prior art, the present invention proposes a method for detecting and removing image watermarks. In view of the fixed characteristics of the watermark template style, the normalized correlation coefficient is used to detect the position where the watermark appears in the image; then, for the missing image pixels at the watermark position, the original image is restored by using the structural representation learned from general natural images and the algorithm of maximizing the expected log-likelihood of the image block. This method effectively solves the problem of detecting the watermark style, and the image restoration effect after the watermark is removed is good.

[0042] like Figure 1 Specifically, the method comprises the following steps:

[0043] Step S1: Get a clean standard watermark template.

[0044] Many image watermarks have a common feature, that is, their style is fixed and repeated in a certain pattern, so this type of watermark can be treated as a template. When this type of watermark is printed on different images, especially on multiple consecutive images, the location of the watermark is generally random. Therefore, it is necessary to first obtain a clean standard watermark template. The method of obtaining it can be to print the watermark on an image with a white or black background, or to obtain it by laying the basic elements of the watermark on the background image in a certain pattern.

[0045] In a specific embodiment, Figure 2 The standard watermark template shown in the figure contains the words "watermark" and is arranged according to a certain pattern. In this template, dotted lines are used to represent the border for display effect.

[0046] Step S2: Calculate the normalized correlation coefficient between the standard watermark template and the target image, and infer the most likely position of the standard watermark template in the target image.

[0047] The normalized correlation coefficient is an indicator for template matching. When the normalized template and the normalized area in the target image are the same, the indicator reaches the maximum value, indicating that the maximum possible match occurs in this area. Therefore, the position of the standard watermark template in the target image can be inferred by the maximum value of the normalized correlation coefficient.

[0048] In a specific embodiment, a watermarked target image such as Figure 3 As shown, by normalizing the maximum value of the correlation coefficient, we can get Figure 2 The most likely location of the standard watermark template in the target image is shown.

[0049] like Figure 4As shown, in step S2, the normalized correlation coefficient between the standard watermark template and the target image is calculated, and the most likely position of the standard watermark template in the target image is deduced, which specifically includes:

[0050] Step S21: slide the standard watermark template pixel by pixel on the target image. Place the standard watermark template within the range of the target image, and slide it one pixel at a time, from left to right and from top to bottom.

[0051] Step S22: At each sliding position, the normalized correlation coefficient between the standard watermark template and the corresponding area on the target image is calculated. For each sliding, the current pixel position of the standard watermark template on the target image is (x, y), and the calculation formula of its normalized correlation coefficient is:

[0052]

[0053] Among them, w is the template, is the average value of the template, f is the corresponding area on the target image, and f xy is the average value of the area, s and t are the traversal coordinates of the pixels in the template area. The normalized correlation coefficient calculated at each sliding position is saved.

[0054] Step S23: Select the maximum value of the normalized correlation coefficient. Find the maximum value among all the normalized correlation coefficients γ(x,y) obtained in the previous step. If there are multiple maximum values, only one of them needs to be found. Record the pixel coordinates x and y of the maximum value.

[0055] Step S24: Calculate the coordinate position of the standard watermark template in the target image according to the position where the maximum normalized correlation coefficient appears. According to the pixel coordinates of the maximum normalized correlation coefficient obtained above, combined with the size of the standard watermark template, calculate the position of the standard watermark template in the target image. The calculation method is to subtract half of the width of the standard watermark template from the coordinate x, and subtract half of the height of the standard watermark template from the coordinate y. If the width or height of the standard watermark template is an odd number, then the subtraction is half of the width or height minus 1.

[0056] Step S3: Calculate the watermark pattern in the target image according to the position of the standard watermark template and the size of the target image. According to the position of the standard watermark template determined above, the exact position of a part of the basic watermark elements contained in the target image can be obtained, and then the entire watermark pattern contained in the target image can be calculated according to the tiling rule of the basic watermark elements and the size of the target image.

[0057] like Figure 5As shown, in step S3, the watermark pattern appearing in the target image is calculated according to the position of the standard watermark template and the size of the target image. Specifically, if the specific parameters of the template generation are known, the watermark pattern of the target image can be obtained by program calculation. You can also use the method of tiling the standard watermark template, such as Figure 5 As shown, the process specifically includes:

[0058] Step S31: Generate a blank image according to the size of the target image.

[0059] Step S32: Paste the standard watermark template obtained in step S2 at the position where the target image appears, which is called the anchor point. The standard watermark template matrix can be used to replace the data at the anchor point of the blank image matrix.

[0060] Step S33: Using the standard watermark template at the anchor point as a reference, tile the standard watermark templates around until the entire blank image is covered. When tiling the standard watermark templates, ensure that the watermarks in the overlapping areas of different standard watermark templates (e.g. Figure 6 The word "watermark" in the figure is consistent, and is still achieved by placing the standard watermark template matrix at the corresponding position. The schematic diagram is shown in Figure 6 shown.

[0061] Step S34: Cut off the redundant parts of the standard watermark template that exceed the blank image range to obtain the watermark pattern of the target image. When multiple standard watermark template matrices cover the entire blank image matrix, only take the data within the blank image matrix range to obtain the watermark pattern of the target image.

[0062] Step S4: Erasing the pixels at the watermark pattern position in the target image. Erasing can be achieved by setting the pixels at the position to zero, indicating that these pixels are missing information of the image.

[0063] Step S5: Use the image block expected log-likelihood maximization algorithm to restore the original watermark-free image. The image block expected log-likelihood maximization algorithm is an image restoration algorithm. Combined with the structural representation learned from general natural images, it can restore the missing local information in the image, thereby achieving the purpose of removing the image watermark.

[0064] In step S5, the image block expected log-likelihood maximization algorithm is used to restore the original watermark-free image. Specifically, the image block expected log-likelihood maximization algorithm is an algorithm model that uses natural image statistical priors to restore images. Here, the algorithm provided by the author of the algorithm, Yair Weiss, is used (https: / / www.cs.huji.ac.il / ~yweiss / ). When using this algorithm to restore the watermark-free image, the image block size is selected as 8×8, and the watermark pattern of the target image obtained in step S4 is used as the mask template. The prior model of the natural image block of independent component analysis is used. For the pixel position with the word "watermark", the parameter lambda is set to 0, and the other pixel positions are set to 10. 6 The entire algorithm's outer loop iterates 5 times, and the parameter beta is set to 10, 40, 160, 1280, and 5120 in each iteration. In each outer loop iteration, the algorithm's inner loop is set to iterate 20 times, and then the final calculation result is obtained.

[0065] In a specific embodiment, Figure 3 The watermarked target image shown in the figure is dewatered, and then the dewatermarked image I is compared with the original clear Lena grayscale image K, and the dewatermarking effect is measured by calculating the PSNR. The calculation formula of PSNR is PSNR = 10log 10 MN / ∑ i，j [I(i, j)-K(i, j)] 2 , where M×N is the size of the image, and it is assumed that the image is stored in floating-point data, that is, the range of image pixel values ​​is [0,1]. The technical solution of the present invention is used to detect and remove image watermarks, and the PSNR of the de-watermarked image is 43.71dB, which shows that a relatively good image de-watermarking effect is achieved.

[0066] All the components mentioned above are prior art, and those skilled in the art can use any model and existing design that can achieve their corresponding functions.

[0067] The above is only a preferred embodiment of the present invention. It should be pointed out that a person skilled in the art can make several modifications and improvements without departing from the inventive concept of the present invention, which all belong to the protection scope of the present invention.

Claims

1. A method for detecting and removing image watermarks, characterized in that The method includes using a normalized correlation coefficient to detect the position where the watermark appears in the image, and then restoring the original image by using the structure representation learned from general natural images and an algorithm that maximizes the expected log-likelihood of image blocks for the missing image pixels at the watermark position; The specific steps include: Step S1: Obtain a clean standard watermark template: treat an image watermark with a fixed style and repeated arrangement in a certain pattern as a template, and stamp the watermark on an image with a white or black background, or lay the basic elements of the watermark on the background image in a certain pattern to obtain a clean standard watermark template; Step S2: Calculate the normalized correlation coefficient between the standard watermark template and the target image, and infer the most likely position of the standard watermark template in the target image: the normalized correlation coefficient is an indicator for template matching. When the normalized template and the normalized area in the target image are the same, the indicator reaches the maximum value, indicating that the most likely match occurs in the area. The position of the standard watermark template in the target image is inferred by the maximum value of the normalized correlation coefficient. In the step S2, the normalized correlation coefficient between the standard watermark template and the target image is calculated, and the most likely position of the standard watermark template in the target image is deduced, which specifically includes: Step S21: slide the standard watermark template pixel by pixel on the target image: place the standard watermark template within the range of the target image, and slide it one pixel at a time, from left to right and from top to bottom; Step S22: At each sliding position, the normalized correlation coefficient between the standard watermark template and the corresponding area on the target image is calculated: For each sliding, the current pixel position of the standard watermark template on the target image is (x, y), and the calculation formula of the normalized correlation coefficient is: Among them, w is the template, is the average value of the template, f is the corresponding area on the target image, is the average value of the area, s and t are the traversal coordinates of the pixels in the template area, and the normalized correlation coefficient calculated at each sliding position is saved; Step S23: Select the maximum value of the normalized correlation coefficient: Find the maximum value among all the normalized correlation coefficients γ(x, y) obtained in step S22. If there are multiple maximum values, only find one of them and record the pixel coordinates x and y of the maximum value. Step S24: Calculate the coordinate position of the standard watermark template in the target image according to the position where the maximum normalized correlation coefficient appears: Calculate the position of the standard watermark template in the target image according to the pixel coordinates of the maximum normalized correlation coefficient obtained in step S23 and the size of the standard watermark template. The calculation method is: subtract half of the width of the standard watermark template from the coordinate x, and subtract half of the height of the standard watermark template from the coordinate y. If the width or height of the standard watermark template is an odd number, the corresponding subtraction is half of the width or height minus 1; Step S3: Calculate the watermark pattern appearing in the target image according to the position of the standard watermark template and the size of the target image: According to the position of the standard watermark template determined in step S24, obtain the exact position of a part of the basic watermark elements contained in the target image, and then calculate the entire watermark pattern contained in the target image according to the tiling rule of the basic watermark elements and the size of the target image; Step S4: Erasing the pixels at the watermark pattern position in the target image: Erasing is achieved by setting the pixels at the position to zero, indicating that these pixels are missing information of the image; Step S5: Use the image block expected log-likelihood maximization algorithm to restore the original watermark-free image: The image block expected log-likelihood maximization algorithm is used as an image restoration algorithm, combined with the structural representation learned from general natural images, to restore the missing local information in the image and remove the image watermark.

2. A method for detecting and removing an image watermark as claimed in claim 1, characterized in that In the step S3, the watermark pattern appearing in the target image is calculated according to the position of the standard watermark template and the size of the target image: if the specific parameters of the template generation are known, the watermark pattern of the target image is obtained by program calculation, or by using the method of tiling the standard watermark template, which specifically includes: Step S31: Generate a blank image according to the size of the target image; Step S32: Paste the standard watermark template obtained in step S2 at the position where the target image appears, which is called the anchor point, and use the standard watermark template matrix to replace the data at the anchor point of the blank image matrix; Step S33: Taking the standard watermark template at the anchor point as a reference, tiling the standard watermark template around until the entire blank image is covered. When tiling the standard watermark template, it is ensured that the watermarks in the overlapping areas of different standard watermark templates are consistent, which is also achieved by placing the standard watermark template matrix at the corresponding position; Step S34: Crop the redundant parts of the standard watermark template that exceed the blank image range to obtain the watermark style of the target image: when multiple standard watermark template matrices cover the entire blank image matrix, only take the data within the blank image matrix to obtain the watermark style of the target image.

3. The method for detecting and removing an image watermark as claimed in claim 1, characterized in that In the step S5, the image block expected log-likelihood maximization algorithm is used to restore the original watermark-free image, which specifically includes: the image block expected log-likelihood maximization algorithm is an algorithm model that uses natural image statistical priors to restore the image. When using the algorithm to restore the watermark-free image, the size of the image block is selected to be 8×8, the watermark pattern of the target image obtained in step S4 is used as the mask template, and the prior model of the natural image block of independent component analysis is used. For the pixel position with watermark, its parameter lambda is set to 0, and the other pixel positions are set to 10 6 ,The entire algorithm outer loop iterates 5 times, and the parameter beta is set to 10, 40, 160, 1280 and 5120 respectively in each iteration. In each outer loop iteration, the inner loop of the algorithm is set to iterate 20 times, and then the final calculation result is obtained.

Images