Watermark extraction method and device, electronic equipment and storage medium

By identifying and correcting the watermark area in the processing target image, and using a pre-trained model to extract dark watermarks, the problem of low watermark extraction efficiency and accuracy in the prior art is solved, and the robustness and accuracy of image watermark extraction are improved. .

CN118967416APending Publication Date: 2024-11-15BEIJING VOLCANO ENGINE TECH CO LTD
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202310492537.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-05-04
Publication Date
2024-11-15

AI Technical Summary

Technical Problem

In the prior art, in the process of image watermark extraction, especially in the block embedding method, there are problems such as large time consumption and low accuracy. Especially after image rotation, compression and other modification operations, the success rate and accuracy of the watermark area are further reduced. Influences robustness.

Method used

By acquiring the target image, identifying the watermark area, determining the initial watermark area, calculating image correction information based on the initial area, performing correction processing, eliminating the impact of modification, improving the determination accuracy of the watermark area, and using the pre-trained dark watermark segmentation model to perform watermarking Area identification and extraction.

Benefits of technology

It improves the determination efficiency and accuracy of the watermark area, enhances the success rate and accuracy of extracting dark watermarks, reduces the need for image chunking traversal, and improves the robustness of watermark extraction.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118967416A_ABST
    Figure CN118967416A_ABST
Patent Text Reader

Abstract

The invention provides a watermark extraction method and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining a target image of a to-be-extracted dark watermark; performing watermark area identification on the target image to obtain a watermark mask image; determining each initial watermark region corresponding to the target image according to the watermark mask image, and determining image correction information based on each initial watermark region; performing correction processing on the target image by using the image correction information to obtain a corrected target image, and determining each target watermark area including a to-be-extracted dark watermark in the target image based on the corrected target image; and extracting a dark watermark in the target image according to each target watermark area.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a watermark extraction method, device, electronic device and storage medium. Background Art

[0002] In order to determine the copyright of an image, various dark watermark embedding algorithms are often used to add dark watermarks to images to identify the copyright of the image. However, due to the limitation of computing hardware and other resources, different watermark embedding methods are used for images with different image resolutions. For example, common methods include directly embedding watermarks in images and dividing images into blocks and embedding watermarks in the image blocks.

[0003] For the block embedding method, when extracting watermarks, it is often necessary to determine the watermark area by traversing the image blocks, and then extract the watermark from the watermark area, which not only takes a lot of time but also has a low accuracy rate. And after performing modification operations such as rotation and compression on the watermark image, the success rate and accuracy of determining the watermark area from the modified watermark image will be further reduced, which greatly affects the robustness of watermark extraction. Summary of the invention

[0004] The embodiments of the present disclosure at least provide a watermark extraction method, device, electronic device and storage medium.

[0005] In a first aspect, an embodiment of the present disclosure provides a watermark extraction method, comprising:

[0006] Obtain a target image to be extracted with a dark watermark;

[0007] Performing watermark region recognition on the target image to obtain a watermark mask image;

[0008] Determining each initial watermark region corresponding to the target image according to the watermark mask image, and determining image correction information based on each initial watermark region;

[0009] Using the image correction information to perform correction processing on the target image to obtain a corrected target image, and based on the corrected target image, determining each target watermark region in the target image including the dark watermark to be extracted;

[0010] According to each of the target watermark regions, a dark watermark in the target image is extracted.

[0011] In a possible implementation manner, determining each initial watermark region corresponding to the target image according to the watermark mask image includes:

[0012] Clustering the pixel points according to the pixel value and coordinate information of each pixel point in the watermark mask image to obtain each point cluster corresponding to the watermark mask image;

[0013] According to the coordinate information of each pixel point in the point cluster, each target point cluster having a circumscribed rectangle is determined, and the image area corresponding to the circumscribed rectangle of the target point cluster is used as the initial watermark area.

[0014] In a possible implementation manner, determining the image correction information based on each of the initial watermark regions includes:

[0015] Based on the regional coordinates of each of the initial watermark regions, respectively determine the regional correction information of each of the initial watermark regions;

[0016] The image correction information is determined according to the number of occurrences of each of the area correction information.

[0017] In a possible implementation, the image correction information includes rotation angle information;

[0018] The method of performing correction processing on the target image by using the image correction information to obtain a corrected target image, and determining, based on the corrected target image, each target watermark region in the target image including a dark watermark to be extracted, comprises:

[0019] According to the rotation angle information, the target image and the watermark mask image are respectively rotated to obtain a rotated target image and a rotated watermark mask image;

[0020] Determining each intermediate watermark region related to the dark watermark to be extracted from the rotated watermark mask image;

[0021] From the rotated target image, an image region matching the region coordinates of the intermediate watermark region is determined as the target watermark region.

[0022] In a possible implementation manner, the step of performing watermark region recognition on the target image to obtain a watermark mask image includes:

[0023] In the case where the image size of the target image is inconsistent with the target size, resizing the target image according to the image size and the target size to obtain an adjusted image having the target size;

[0024] Performing watermark area recognition on the adjusted image to obtain the watermark mask image;

[0025] After determining, based on the corrected target image, each target watermark region including the dark watermark to be extracted in the target image, the method further includes:

[0026] According to the image size and the target size, each of the target watermark areas is restored to a target watermark area matching the image size.

[0027] In a possible implementation manner, the watermark mask image is obtained by identifying the watermark region of the target image using a pre-trained dark watermark segmentation model;

[0028] The dark watermark segmentation model is trained according to the following steps:

[0029] Acquire a training data set; the training data set includes sample images with multiple resolutions and standard mask images corresponding to the sample images, the sample images include original sample images without embedded sample dark watermarks and watermark sample images carrying sample dark watermarks;

[0030] Inputting the sample image in the training data set into the dark watermark segmentation model to be trained to obtain a first prediction mask image corresponding to the sample image;

[0031] According to the first prediction mask image and the standard mask image corresponding to the sample image, a first prediction loss of the dark watermark segmentation model to be trained is determined, and the dark watermark segmentation model to be trained is iteratively trained using the first prediction loss until a training cutoff condition is met, thereby obtaining a trained dark watermark segmentation model.

[0032] In a possible implementation manner, after acquiring the training data set, the method further includes:

[0033] Randomly performing at least one data enhancement processing operation on a sample image in the training data set and a standard mask image corresponding to the sample image to obtain a new sample image and a new standard mask image;

[0034] Supplementing the new sample image and the new standard mask image to the training data set to obtain an updated training data set;

[0035] The step of inputting the sample image in the training data set into the dark watermark segmentation model to be trained to obtain a first predicted mask image corresponding to the sample image includes:

[0036] The sample image in the updated training data set is input into the dark watermark segmentation model to be trained to obtain a first predicted mask image corresponding to the sample image.

[0037] In a possible implementation manner, inputting the sample image in the training data set into the dark watermark segmentation model to be trained includes:

[0038] For any of the original sample images in the training data set, determine a watermark sample image carrying a dark watermark corresponding to the original sample image, and use the watermark sample image and the original sample image as a sample image group;

[0039] The sample images in the sample image groups corresponding to the training data set are sequentially input into the dark watermark segmentation model to be trained.

[0040] In a possible implementation manner, the iterative training of the dark watermark segmentation model to be trained by using the first prediction loss until a training cutoff condition is met to obtain a trained dark watermark segmentation model includes:

[0041] Iteratively training the dark watermark segmentation model to be trained by using the first prediction loss until a training cutoff condition is met, thereby obtaining an intermediate watermark segmentation model;

[0042] Compressing the sample images in the training data set to obtain compressed sample images; wherein the original sample images and the watermarked sample images belonging to the same sample image group correspond to the same compression factor;

[0043] Inputting the compressed sample image into the intermediate watermark segmentation model to obtain a second predicted mask image;

[0044] The second prediction mask image and the standard mask image corresponding to the compressed sample image are used to determine the second prediction loss of the intermediate watermark segmentation model, and the intermediate watermark segmentation model is iteratively trained using the second prediction loss until a training cutoff condition is met to obtain a trained dark watermark segmentation model.

[0045] In a second aspect, the present disclosure also provides a watermark extraction device, including:

[0046] An acquisition module, used for acquiring a target image to be extracted with a dark watermark;

[0047] An identification module is used to identify the watermark area of ​​the target image to obtain a watermark mask image;

[0048] A first determination module, configured to determine each initial watermark region corresponding to the target image according to the watermark mask image, and determine image correction information based on each initial watermark region;

[0049] A second determination module is used to perform correction processing on the target image using the image correction information to obtain a corrected target image, and based on the corrected target image, determine each target watermark area in the target image including the dark watermark to be extracted;

[0050] The extraction module is used to extract the dark watermark in the target image according to each target watermark area.

[0051] In a third aspect, an optional implementation of the present disclosure further provides an electronic device, a processor, and a memory, wherein the memory stores machine-readable instructions executable by the processor, and the processor is used to execute the machine-readable instructions stored in the memory, and when the machine-readable instructions are executed by the processor, the machine-readable instructions perform the steps of the above-mentioned first aspect, or any possible implementation of the first aspect.

[0052] In a fourth aspect, an optional implementation of the present disclosure further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed, the steps of the above-mentioned first aspect, or any possible implementation of the first aspect are executed.

[0053] For a description of the effects of the above-mentioned watermark extraction device, electronic device, and computer-readable storage medium, please refer to the description of the above-mentioned watermark extraction method, which will not be repeated here.

[0054] The watermark extraction method, device, electronic device and storage medium provided by the embodiments of the present disclosure can accurately identify the watermark mask image corresponding to the target image by performing image recognition on the target image. Since the watermark mask image can indicate whether each pixel belongs to the watermark area, the watermark mask image can be used to accurately determine the initial watermark areas corresponding to the target image. Since the target image may have been modified, the image correction information is determined based on each initial watermark area, and then the target image is corrected using the image correction information, which can eliminate the influence of the modification operation on the content of the target image and restore the content of the target image, thereby improving the robustness of the determination of the watermark area. Finally, the watermark area is determined based on the corrected target image, which can improve the success rate and rationality of the determined watermark area and obtain an accurate target watermark area. The watermark area of ​​the image can be determined based on the identified watermark mask image without traversing the target image in blocks, which effectively improves the efficiency and speed of watermark area determination. Finally, using the determined accurate target watermark area for watermark extraction can improve the success rate of extracting dark watermarks and improve the accuracy of the extracted dark watermarks.

[0055] Furthermore, the watermark extraction method, device, electronic device, and storage medium provided by the embodiments of the present disclosure can also use a pre-trained dark watermark segmentation model to identify the watermark area of ​​the target image to obtain a watermark mask image. Since the trained neural network model has a high recognition accuracy, the pre-trained dark watermark segmentation model is used for image recognition to determine the accurate watermark mask image corresponding to the target image. Using an accurate watermark mask image to identify the watermark area can improve the accuracy of the identified target watermark area.

[0056] In order to make the above-mentioned objectives, features and advantages of the present disclosure more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following is a brief introduction to the drawings required for use in the embodiments. The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present disclosure and are used together with the specification to illustrate the technical solutions of the present disclosure. It should be understood that the following drawings only illustrate certain embodiments of the present disclosure and should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can also be obtained based on these drawings without creative work.

[0058] Figure 1 A flow chart of a watermark extraction method provided by an embodiment of the present disclosure is shown;

[0059] Figure 2 A schematic diagram of a process of embedding a dark watermark using a block embedding method provided by an embodiment of the present disclosure is shown;

[0060] Figure 3 A specific implementation flow chart of a watermark extraction method provided by an embodiment of the present disclosure is shown;

[0061] Figure 4 A schematic diagram of a watermark extraction process provided by an embodiment of the present disclosure is shown;

[0062] Figure 5 A flow chart of a model training method provided by an embodiment of the present disclosure is shown;

[0063] Figure 6 A specific implementation flow chart of model training and model reasoning provided by an embodiment of the present disclosure is shown;

[0064] Figure 7 A schematic diagram of a watermark extraction device provided by an embodiment of the present disclosure is shown;

[0065] Figure 8A schematic structural diagram of an electronic device provided by an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0066] In order to make the purpose, technical scheme and advantages of the embodiments of the present disclosure clearer, the technical scheme in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all of the embodiments. The components of the embodiments of the present disclosure generally described and shown here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present disclosure is not intended to limit the scope of the present disclosure claimed for protection, but merely represents the selected embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present disclosure.

[0067] In addition, the terms "first", "second", etc. in the description and claims of the embodiments of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchangeable where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein.

[0068] The "multiple or several" mentioned in this article refers to two or more. "And / or" describes the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the associated objects are in an "or" relationship.

[0069] It has been found through research that images, as a common multimedia data, have a wide range of application scenarios. With the widespread use of images, the copyright identification of images has also become an issue worthy of attention. In order to realize the copyright identification of images, dark watermark embedding methods are often used to embed copyright information in images. However, due to the problem of computing resources, conventional dark watermark embedding methods can often only be applied to small-resolution images, and have poor watermark embedding effects for images with a resolution greater than the preset resolution. In order to improve the watermark embedding effect of large-resolution images, a block-based watermark embedding method is adopted. Specifically, multiple sub-regions can be screened out from the image, each sub-region is used as an image block, and then the watermark is embedded in each image block and the watermarked image is re-merged to obtain a watermarked image. However, although the above-mentioned block embedding method can improve the watermark embedding effect, when extracting the watermark, it not only takes a lot of time to determine the watermark area, but also affects the success rate and accuracy of determining the watermark area from the modified watermark image after performing modification operations such as rotation and compression on the watermark image, thereby reducing the success rate and robustness of watermark extraction.

[0070] Based on the above research, the present disclosure provides a watermark extraction method, device, electronic device and storage medium. By performing image recognition on the target image, the watermark mask image corresponding to the target image can be accurately identified. Since the watermark mask image can indicate whether each pixel belongs to the watermark area, the watermark mask image can be used to accurately determine the initial watermark areas corresponding to the target image. Since the target image may have been modified, the image correction information is determined based on each initial watermark area, and then the target image is corrected using the image correction information, which can eliminate the influence of the modification operation on the content of the target image and restore the content of the target image, thereby improving the robustness of the determination of the watermark area. Finally, the watermark area is determined based on the corrected target image, which can improve the success rate and rationality of the determined watermark area and obtain an accurate target watermark area. The watermark area of ​​the image can be determined based on the identified watermark mask image without traversing the target image in blocks, which effectively improves the efficiency and speed of watermark area determination. Finally, using the determined accurate target watermark area for watermark extraction can improve the success rate of extracting dark watermarks and improve the accuracy of the extracted dark watermarks.

[0071] The defects existing in the above solutions are the results obtained by the inventor after practice and careful research. Therefore, the discovery process of the above problems and the solutions proposed by the present disclosure for the above problems below should be the contributions made by the inventor to the present disclosure during the disclosure process.

[0072] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0073] It is understandable that before using the technical solutions disclosed in the embodiments of the present disclosure, the types, scope of use, usage scenarios, etc. of the personal information involved in the present disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0074] It should be noted that the specific terms mentioned in the embodiments of the present disclosure include:

[0075] OpenCV is a cross-platform computer vision and machine learning software library released under the Apache 2.0 license (open source), which can run on Linux, Windows, Android and Mac OS operating systems. It is lightweight and efficient, consisting of a series of C functions and a small number of C++ classes, and provides interfaces for languages ​​such as Python, Ruby, and MATLAB, and implements many common algorithms in image processing and computer vision.

[0076] FCN: Fully Convolutional Networks, FCN, is a framework for image semantic segmentation;

[0077] SegNet: Segmentation Network, a deep convolutional encoder-decoder architecture for image segmentation.

[0078] UNet: is an encoder-decoder structure for image segmentation;

[0079] U 2 Net: is a UNet composed of UNets for image semantic segmentation.

[0080] To facilitate understanding of the present embodiment, a watermark extraction method disclosed in the embodiment of the present disclosure is first introduced in detail. The execution subject of the watermark extraction method provided in the embodiment of the present disclosure is generally a terminal device or other processing device with certain computing capabilities, wherein the terminal device may be a user equipment (UE), a mobile device, a user terminal, a terminal, a personal digital assistant (PDA), a handheld device, an electronic device, etc.; in some possible implementations, the watermark extraction method may be implemented by a processor calling computer-readable instructions stored in a memory.

[0081] The watermark extraction method provided by the embodiment of the present disclosure is described below by taking an electronic device as an example.

[0082] like Figure 1 As shown, it is a flow chart of a watermark extraction method provided by an embodiment of the present disclosure, which may include the following steps:

[0083] S101: Acquire a target image from which a dark watermark is to be extracted.

[0084] Here, the target image can be an image embedded with any dark watermark. For example, the target image can be an image embedded with any dark watermark using the following Figure 2 The block embedding method embeds the dark watermark in the image. Figure 2 In the example, after obtaining the original image, multiple watermark embedding regions can be screened out from the original image. Then, the image blocks corresponding to each watermark embedding region are cropped from the original image (e.g. Figure 2 In the image block 1~n, n is an integer greater than 1). Watermark information is embedded in each image block to obtain multiple watermark blocks (such as Figure 2 Finally, the watermark block can be used to replace the image block, so as to obtain the target image embedded with the dark watermark.

[0085] Optionally, no matter what image resolution and image size the target image has, as long as the dark watermark in the target image is embedded using the block embedding method, the watermark extraction method provided in the embodiment of the present disclosure can be used to determine the target watermark area therein, and then extract the dark watermark from the target watermark area.

[0086] S102: Perform watermark region recognition on the target image to obtain a watermark mask image.

[0087] Here, the watermark mask image is used to indicate whether each pixel belongs to the pixel corresponding to the watermark area. Specifically, the watermark mask image can be a single-channel grayscale image, and the pixel value of the pixel in the image is one of 0 or 255. When the pixel value of the pixel is 0, it means that the pixel does not belong to the watermark area; when the pixel value of the pixel is 255, it means that the pixel belongs to the watermark area.

[0088] During specific implementation, a pre-trained watermark segmentation model may be used to identify the watermark region of a target image, and obtain a watermark mask image corresponding to the target image.

[0089] S103: Determine each initial watermark region corresponding to the target image according to the watermark mask image, and determine image correction information based on each initial watermark region.

[0090] Here, since the target image may be subjected to modification operations such as rotation, cropping, compression, etc., the original watermark area in the target image may also change position. Therefore, the initial watermark area can be each watermark area to be adjusted identified from the target image, and there may be a certain position change between the initial watermark area and the original watermark area corresponding to the target image. In order to improve the accuracy of the target watermark area finally determined, it is necessary to use the initial watermark area to determine the position transformation information of the image (i.e., the above-mentioned image correction information), and then determine the accurate target watermark area based on this information.

[0091] Image correction information is information used to correct the position of an image. Exemplarily, the image correction information may include scaling information, rotation angle information, etc. The image correction information is used to represent the position change information of the target image compared to the image before the correction.

[0092] It is understandable that, when the target image has not been modified, the initial watermark area and the final target watermark area may be consistent, and the image correction information may be empty.

[0093] In specific implementation, each pixel point belonging to the watermark area can be determined according to the pixel value of each pixel point in the watermark mask image, and each initial watermark area corresponding to the target image can be determined based on the pixel points belonging to the watermark area. Then, after each initial watermark area is obtained, the image correction information can be determined according to the area position and deflection angle of each initial watermark area.

[0094] In one embodiment, the above step of determining the initial watermark area may be implemented according to the following steps:

[0095] S1: Cluster the pixels according to the pixel value and coordinate information of each pixel in the watermark mask image to obtain each point cluster corresponding to the watermark mask image.

[0096] S2: According to the coordinate information of each pixel point in the point cluster, each target point cluster having a bounding rectangle is determined, and the image area corresponding to the bounding rectangle of the target point cluster is used as the initial watermark area.

[0097] Here, the coordinate information is the coordinate position of the pixel in the watermark mask image. A point cluster can include at least one pixel, and there is a certain distance between different point clusters. The bounding rectangle of the target point cluster is the smallest rectangle that can include each pixel in the target point cluster.

[0098] Exemplarily, using a pre-trained watermark segmentation model, the predicted pixel value corresponding to each pixel point belonging to the interval [0, 255] can be output. Then, using a preset screening value, the predicted pixel value of the pixel point whose predicted pixel value is greater than the preset screening value can be modified to 255, and the predicted pixel value of the pixel point less than or equal to the preset screening value can be modified to 0. In this way, a watermark mask image composed of pixel points with pixel values ​​of 0 and 255 can be obtained. Then, OpenCV can be used to cluster each pixel point according to the pixel value and coordinate information of the pixel points in the watermark mask image to obtain each point cluster corresponding to the watermark mask image.

[0099] Furthermore, for any point cluster, OpenCV can be used to determine the corresponding bounding rectangles of each target point cluster based on the coordinate information of each pixel in the point cluster, as well as the center coordinates and width and height information of the bounding rectangles corresponding to each target point cluster. At the same time, the area corresponding to each outer rectangle can be used as the initial watermark area.

[0100] Understandably, if no point clusters are clustered, it can be said that there may be no dark watermark in the target image, that is, there is no watermark area, and the process can be terminated. If it is determined that each point cluster does not have a corresponding outer rectangle, it can also be said that there may be no dark watermark in the target image, that is, there is no watermark area, and the process can be terminated.

[0101] In one embodiment, the step of “determining image correction information based on each initial watermark region” may be implemented according to the following steps:

[0102] Step 1: Based on the regional coordinates of each initial watermark region, determine the regional correction information of each initial watermark region.

[0103] Here, the region coordinates are coordinates used to characterize the position of the initial watermark region in the watermark mask image, and specifically may include the four vertex coordinates and the center coordinates of the initial watermark region. The four vertex coordinates of the initial watermark region may be determined based on the center coordinates and width and height information determined by the region determination method provided by OpenCV. The region correction information is information used to correct the position of the initial watermark region, and may include scaling information, rotation angle information, etc. One initial watermark region corresponds to one region correction information, and different initial watermark regions may correspond to the same region correction information.

[0104] Taking the regional correction information as rotation angle information as an example, for each initial watermark area, the rotation angle can be estimated based on the four vertex coordinates, center point coordinates and width and height information of the initial watermark area, so as to determine the rotation angle information corresponding to the initial watermark area.

[0105] Optionally, when OpenCV is used to determine the bounding rectangles corresponding to each target point cluster, OpenCV can simultaneously extract the rotation angle information corresponding to each bounding rectangle, so that the rotation angle information corresponding to each initial watermark area is obtained.

[0106] Taking the regional correction information as scaling information as an example, the original image corresponding to the target image can be obtained, and the scaling information corresponding to the initial watermark area can be determined based on the position information of the watermark area identified from the original image and the position information of the initial watermark area identified from the target image. It can be understood that taking the regional correction information as translation information as an example, the original image corresponding to the target image can be obtained, and then the scaling information corresponding to the initial watermark area can be determined based on the movement distance of the position of the initial watermark area identified from the target image compared to the position of the watermark area identified from the original image.

[0107] Step 2: Determine the image correction information according to the number of occurrences of the correction information of each region.

[0108] Here, in the case where the initial watermark area includes only one, the regional correction information will also include only one, so the regional correction information of the initial watermark area can be directly used as the image correction information. The number of occurrences of a regional correction information is the number of initial watermark areas whose corresponding regional correction information is the regional correction information.

[0109] In the case where the initial watermark area includes multiple regions, the regional correction information will also include multiple regions. Therefore, the regional correction information with the largest number of appearances can be used as the image correction information according to the number of appearances of each regional correction information. For example, in the case where regional correction information 1 appears once, regional correction information 2 appears five times, and regional correction information 3 appears once, regional correction information 2 can be used as the image correction information.

[0110] In this way, the regional correction information that appears the most times has the highest correlation with the modification operation performed on the target image. Therefore, by selecting the regional correction information that appears the most times as the image correction information, the rationality and accuracy of the determined correction information can be improved, thereby improving the accuracy of the determined target watermark area.

[0111] Optionally, when the initial watermark region includes multiple regions, average correction information may be determined based on the correction information of each region, and then the average correction information may be used as the image correction information. For example, when the regional correction information is rotation angle information, the average rotation angle corresponding to each rotation angle information may be used as the image correction information. For another example, when the regional correction information is translation information, the average translation distance corresponding to each initial watermark region may be used as the image correction information.

[0112] S104: performing correction processing on the target image using the image correction information to obtain a corrected target image, and based on the corrected target image, determining each target watermark region in the target image including the dark watermark to be extracted.

[0113] Here, the target watermark region is the final determined watermark region. The target watermark region has the highest possibility of including the dark watermark to be extracted, and the accuracy of the dark watermark extracted from the target watermark region will also be the highest.

[0114] Taking the image correction information as rotation angle information as an example, the target image can be rotated according to the rotation angle information, thereby achieving geometric correction of the target image and obtaining a corrected target image. Taking the image correction information as translation information as an example, the target image can be translated according to the translation information, thereby achieving geometric correction of the target image and obtaining a corrected target image.

[0115] Furthermore, the pre-trained dark watermark segmentation model can be used to re-identify the corrected target image to obtain a watermark mask image corresponding to the corrected target image, and each target watermark area corresponding to the corrected target image can be determined based on the watermark mask image.

[0116] S105: extracting the dark watermark in the target image according to each target watermark area.

[0117] In specific implementation, for each target watermark region, a watermark extraction method can be used to extract the dark watermark in the target watermark region from the target watermark region. Then, if the dark watermarks extracted from each target watermark region are consistent, any dark watermark can be used as the dark watermark in the target image.

[0118] When there are differences in the dark watermarks extracted from the target watermark regions, the dark watermarks extracted from the target watermark regions can be used as the dark watermarks in the target image; or the dark watermark that appears most often can be used as the dark watermark in the target image.

[0119] In this way, by performing image recognition on the target image, the watermark mask image corresponding to the target image can be accurately identified. Since the watermark mask image can indicate whether each pixel belongs to the watermark area, the watermark mask image can be used to accurately determine each initial watermark area corresponding to the target image. Since the target image may have been modified, the image correction information is determined based on each initial watermark area, and then the target image is corrected using the image correction information, which can eliminate the influence of the modification operation on the content of the target image and restore the content of the target image, thereby improving the robustness of the determination of the watermark area. Finally, the watermark area is determined based on the corrected target image, which can improve the success rate and rationality of the determined watermark area and obtain an accurate target watermark area. The determination of the image watermark area can be achieved based on the identified watermark mask image without traversing the target image in blocks, which effectively improves the efficiency and speed of the watermark area determination. Finally, the accurate target watermark area is used to extract the watermark, which can improve the success rate of extracting the dark watermark and the accuracy of the extracted dark watermark.

[0120] In one embodiment, the image correction information may include rotation angle information. The above S104 may be implemented according to the following steps:

[0121] S104-1: Rotate the target image and the watermark mask image respectively according to the rotation angle information to obtain a rotated target image and a rotated watermark mask image.

[0122] Here, the rotated target image is the corrected target image, and the rotated watermark mask image is the corrected watermark mask image.

[0123] It should be noted that the rotation angle information can be the target angle of the target image relative to the image before modification, or it can be the correction angle obtained by negating the target angle. When the rotation angle information is the target angle, the target image and the watermark mask image can be reversely rotated according to the target angle to obtain the rotated target image and the rotated watermark mask image. When the rotation angle information is the correction angle, the target image and the watermark mask image can be directly rotated according to the correction angle to obtain the rotated target image and the rotated watermark mask image.

[0124] The step of rotating the watermark mask image may specifically include the following two implementations:

[0125] Method 1: According to the rotation angle information, the watermark mask image is rotated as a whole to obtain the rotated watermark mask image.

[0126] Method 2: According to the rotation angle information, each initial watermark area in the watermark mask image is rotated respectively to obtain each rotated initial watermark area. At the same time, the pixel value of each pixel in the watermark mask image can be modified to 0 to obtain a modified watermark mask image. Then, the area corresponding to the rotated initial watermark area in the modified watermark mask image can be determined, and the pixel value of each pixel included in the area is modified to 255, thereby obtaining a rotated watermark mask image.

[0127] In specific implementation, the target image can be rotated according to the rotation angle information to obtain a rotated target image. At the same time, the watermark mask image can be rotated in any of the above two ways to obtain a rotated watermark mask image.

[0128] S104-2: Determine, from the rotated watermark mask image, various intermediate watermark regions related to the dark watermark to be extracted.

[0129] Here, the intermediate watermark regions are the respective watermark regions determined from the rotated watermark mask image.

[0130] In specific implementation, pixel clustering can be performed according to the pixel value and coordinate information of each pixel in the rotated watermark mask image, so as to obtain each rotated point cluster corresponding to the rotated watermark mask image. Then, each target rotated point cluster with a bounding rectangle can be determined according to the coordinate information of each pixel in each rotated point cluster, and the image area corresponding to the bounding rectangle of the target rotated point cluster is used as the intermediate watermark area.

[0131] Regarding the specific implementation steps of determining the intermediate watermark area from the rotated watermark mask image, reference may be made to S1 and S2 above, which will not be repeated here.

[0132] S104-3: From the rotated target image, determine an image region that matches the region coordinates of the middle watermark region as the target watermark region.

[0133] In specific implementation, for each intermediate watermark area, the target watermark area corresponding to the intermediate watermark area can be determined from the target image according to the area coordinates of the intermediate watermark area and the coordinate mapping relationship between the rotated watermark mask image and the rotated target image.

[0134] Thus, by rotating the target image and the watermark mask image respectively, the position correction of the target image and the watermark mask image can be achieved. Then, the watermark area can be re-determined using the position-corrected watermark mask image and the target image, which can improve the rationality and accuracy of the target watermark area determined from the target image.

[0135] Optionally, after obtaining the rotation angle information, the target areas corresponding to the initial watermark areas can be determined in the target image. Then, the target image can be rotated using the rotation angle information to obtain a rotated target image. At the same time, the target area in the rotated target image also performs a corresponding rotation, and the rotated target area can be used as an intermediate watermark area.

[0136] Optionally, in the case where the image correction information is scaling information, the target image can be reversely scaled according to the scaling information to obtain a reversely scaled target image. At the same time, the watermark mask image can be reversely scaled according to the scaling information to obtain a reversely scaled watermark mask image. Then, each intermediate watermark region related to the dark watermark to be extracted can be determined from the reversely scaled watermark mask image. Finally, the image region that matches the region coordinates of the intermediate watermark region can be determined from the reversely scaled target image as the target watermark region.

[0137] In one embodiment, since the watermark mask image can be obtained by segmenting the target image using a pre-trained dark watermark segmentation model, and the pre-trained watermark segmentation model can often pre-specify the image size of the image that can be processed (i.e., the target size), therefore, for the above S102, it can be determined first whether the image size of the target image is consistent with the target size. If so, the target image can be directly input into the watermark segmentation model to obtain the watermark mask image.

[0138] On the contrary, when the image size of the target image is inconsistent with the target size, the target image may be resized according to the image size and the target size to obtain an adjusted image having the target size.

[0139] Here, the target size may be, for example, 512×512, 1024×1024, etc.

[0140] Exemplarily, when the image size of the target image is inconsistent with the target size, the conversion relationship between the image size and the target size can be determined first, and then the target image can be resized according to the conversion relationship to obtain an adjusted image with the target size. After that, the adjusted image can be input into the watermark segmentation model to obtain a watermark mask image. The conversion relationship is used to characterize the correspondence between pixels in images of different sizes.

[0141] For example, in order to ensure that the image is not distorted after resizing, when the image size and the target size are inconsistent, it can be determined whether the length and width of the target image can be proportionally adjusted to obtain an adjusted image with the target size. If it can, the adjustment ratio of adjusting the target image can be determined according to the image size and the target size, and then the target image can be resized according to the adjustment ratio to obtain an undistorted adjusted image.

[0142] If not, the target image can be first pixel-filled to obtain a filled target image, wherein after the filled target image is resized, an adjusted image with a target size can be obtained, and at the same time, it can be ensured that the original content in the target image is not distorted before and after the adjustment.

[0143] For example, when the image size is 1000×1000 and the target size is 512×512, 24 pixels with a pixel value of 0 or 255 can be filled in the width direction of the target image, and 24 pixels with a pixel value of 0 or 255 can be filled in the height direction of the target image. In this way, a 1024×1024 filled target image can be obtained. Then, an adjustment ratio can be determined based on 1024×1024 and 512×512, and the filled target image can be resized according to the adjustment ratio to obtain a 512×512 adjusted image.

[0144] Further, with respect to S104, the image correction information may be used to correct the adjusted image to obtain a corrected adjusted image, and based on the corrected adjusted image, each target watermark region including the dark watermark to be extracted in the adjusted image is determined.

[0145] Similarly, after determining the target watermark regions including the dark watermark to be extracted in the target image based on the corrected target image, if the target image is resized, the target watermark regions determined may be the target watermark regions corresponding to the adjusted image. In order to determine the true watermark region in the target image, the target watermark regions may be restored to target watermark regions that match the image size based on the image size and the target size. Here, the target watermark region that matches the image size is the true watermark region in the target image. Specifically, the adjustment ratio when obtaining the adjusted image from the target image may be determined based on the image size and the target size, and then the target watermark regions may be restored to target watermark regions that match the image size based on the adjustment ratio.

[0146] Exemplarily, after the filled target image is resized according to the adjustment ratio, after the target watermark area is determined based on the corrected target image, the target watermark area can be reversely resized according to the adjustment ratio to obtain a target watermark area that matches the image size.

[0147] like Figure 3 As shown, a specific implementation flow chart of a watermark extraction method provided by an embodiment of the present disclosure may include the following steps:

[0148] Get the target image to be extracted from the dark watermark. If the image size of the target image is inconsistent with the target size, perform image resizing to obtain an adjusted image with the target size. Use the dark watermark segmentation model to segment the adjusted image to obtain a watermark mask image. According to the coordinate information of each pixel in the point cluster, determine each target point cluster with a bounding rectangle, and use the bounding rectangle as the initial watermark area. Figure 3 In FIG. 1 , K initial watermark regions are shown, and K is greater than or equal to 0. When K is greater than 0, the regional correction information corresponding to each initial watermark region can be determined respectively (i.e. Figure 3 Here, if the rotation angle is not an integer, the rotation angle corresponding to each initial watermark area can be adjusted to an integer by rounding. Then, the image correction information (i.e. Figure 3 The final rotation angle information in the image). The adjusted image and the watermark mask image are rotated respectively using the rotation angle information to obtain the rotated adjusted image and the rotated watermark mask image. Then, each bounding rectangle in the rotated watermark mask image can be determined, and the middle watermark area can be determined according to the bounding rectangle. Figure 3 In FIG. 1 , K intermediate watermark regions are shown. Then, for the region coordinates of each intermediate watermark region, the corresponding target watermark region (i.e. Figure 3 Finally, the size restoration operation can be performed on each watermark block to restore each watermark block to a target watermark block that matches the image size (i.e. Figure 3 The target watermark block 1-K in the target image is used as the final target watermark area corresponding to the target image, and the dark watermark can be extracted from it.

[0149] like Figure 4 FIG. 1 is a schematic diagram of a watermark extraction process provided by an embodiment of the present disclosure. Specifically, Figure 4The original image and the watermark image and the first standard watermark mask image corresponding to the original image are shown in FIG. After performing the modification operation on the watermark image, the target image is obtained. At the same time, the second standard watermark mask image corresponding to the target image can also be obtained. Then, the target image can be segmented using the dark watermark segmentation model to obtain a predicted watermark mask image. The image is corrected using the determined image correction information to obtain a corrected watermark mask image. Then, the target watermark regions corresponding to the target image can be determined based on the corrected watermark mask image. Finally, the dark watermark in the target image can be extracted from the target watermark region.

[0150] Based on the above embodiments, it can be known that the watermark mask image can be obtained by using a pre-trained dark watermark segmentation model to identify the watermark area of ​​the target image. Therefore, the embodiment of the present disclosure also provides a model training method for training a dark watermark segmentation model. Figure 5 As shown, a flow chart of a model training method provided by an embodiment of the present disclosure may include the following steps:

[0151] S501: Acquire a training data set; the training data set includes sample images with various resolutions and standard mask images corresponding to the sample images, and the sample images include original sample images without embedded sample dark watermarks and watermarked sample images carrying sample dark watermarks.

[0152] Here, the training data set includes sample images with multiple resolutions and a standard mask image corresponding to each sample image. In order to improve the generalization ability of the trained dark watermark segmentation model and to have reliable segmentation accuracy for images of any resolution, the difference between the number of sample images of various resolutions in the training data set can be less than a preset difference. Exemplarily, the number of sample images of various resolutions in the training data set can be consistent. In this way, it can be ensured that the dark watermark segmentation model can learn the dark watermark features of images of different resolutions during the training process, thereby effectively improving the robustness of the model.

[0153] At the same time, in order to avoid overfitting of the trained model, the sample image may include the original sample image without the sample dark watermark embedded and the watermark sample image carrying the sample dark watermark. For the watermark sample image, it can be an image embedded with a dark watermark using any existing watermark embedding method, wherein the number and position of the watermark regions in each watermark sample image are different, the number of watermark regions in a watermark sample image belongs to the interval [1, 10], there is no regional overlap between the watermark regions in the same watermark sample image, and there is a certain preset distance between each other; the dark watermarks in different watermark sample images can be embedded using different watermark embedding methods. In this way, by randomly selecting the watermark embedding position, the watermark region in the training data set can be flexible and changeable, thereby effectively avoiding the trained dark watermark segmentation model from overfitting to the specified region.

[0154] In the case where the sample image includes an original sample image and a watermark sample image, the standard mask image may include a first standard sample mask image corresponding to the original sample image, and a second standard sample mask image corresponding to the watermark sample image. Among them, an original sample image may correspond to at least one watermark sample image, and the position and watermark style of the watermark area in each watermark sample image corresponding to the same original sample image are different. Since the dark sample dark watermark is not embedded in the original sample image, the first standard sample mask image may be a black image with the same resolution as the original sample image, and the pixel value of each pixel in the image is 0 (i.e., a full black mask image of all 0). The second standard sample mask image may be determined according to the position of the real watermark area in the watermark sample image. That is, the training data set may include the following four parts:

[0155] a. Original sample image

[0156] b. Watermark sample image

[0157] c. The first standard sample mask image (i.e., a completely black mask image of all 0s)

[0158] d. Second standard sample mask image

[0159] In this way, since it is within reasonable expectations that the dark watermark segmentation model segments the original sample image without the dark watermark embedded, it should produce a completely black mask image. Therefore, using the original sample image and the first standard sample mask image for model training can control the accuracy of the trained dark watermark segmentation model and avoid the problem of overfitting.

[0160] S502: Input a sample image in the training data set into the dark watermark segmentation model to be trained to obtain a first predicted mask image corresponding to the sample image.

[0161] It should be noted that the dark watermark segmentation model can use, but is not limited to, FCN[4], SegNet[5], UNet[6], 2 Net[7]. The first predicted mask image is the predicted mask image output by the dark watermark segmentation model to be trained.

[0162] After obtaining the training data set, each sample image can be input into the dark watermark segmentation model to be trained to obtain the first prediction mask image corresponding to each sample image. For example, the original sample image can be input into the dark watermark segmentation model to be trained to obtain the first prediction mask image corresponding to the original sample image; and the watermark sample image can be input into the dark watermark segmentation model to be trained to obtain the first prediction mask image corresponding to the watermark sample image.

[0163] S503: Determine a first prediction loss of the dark watermark segmentation model to be trained according to the first prediction mask image and the standard mask image corresponding to the sample image, and iteratively train the dark watermark segmentation model to be trained using the first prediction loss until a training cutoff condition is met, thereby obtaining a trained dark watermark segmentation model.

[0164] Here, the first prediction loss is used to characterize the deviation between the first prediction mask image output by the dark watermark segmentation model to be trained and the standard mask image corresponding to the sample image. The training cutoff condition may include that the accuracy of the trained dark watermark segmentation model reaches a preset accuracy and / or the number of iterative training rounds reaches a preset number of rounds.

[0165] In a specific implementation, when the first prediction mask image is the first prediction mask image corresponding to the original sample image, the first prediction loss can be determined using the first standard sample mask image and the first prediction mask image. When the first prediction mask image is the first prediction mask image corresponding to the watermark sample image, the first prediction loss can be determined using the second standard sample mask image corresponding to the watermark sample image and the first prediction mask image. The first prediction loss can be calculated based on the first prediction mask images corresponding to a large number of sample images output by the dark watermark segmentation model to be trained, and the standard mask images corresponding to the large number of sample images.

[0166] After calculating the first prediction loss, the dark watermark segmentation model to be trained can be iteratively trained using the first prediction loss until a training cutoff condition is met, thereby obtaining a trained dark watermark segmentation model.

[0167] In one embodiment, in order to further improve the prediction accuracy and robustness of the trained dark watermark segmentation model, a data enhancement operation may be performed after the training data set is obtained. Specifically, at least one data enhancement processing operation may be randomly performed on the sample images in the training data set and the standard mask images corresponding to the sample images to obtain new sample images and new standard mask images.

[0168] Here, the data enhancement operation is an operation that does not change the content quality of the image. In other words, the data enhancement operation is a lossy operation. Specifically, the data enhancement operation may include the following:

[0169] Image cropping operation, image rotation operation, image scaling operation, image contrast, brightness or saturation update operation, image Gaussian blur operation, noise addition operation (such as adding Gaussian noise, salt and pepper noise, etc.).

[0170] In a specific implementation, at least part of the sample images can be selected from the training data set, and at least one data enhancement processing operation can be randomly performed on the sample images to obtain new sample images. At the same time, after performing at least one data enhancement processing operation on a sample image, the same data enhancement processing operation can also be performed on the standard mask image corresponding to the sample image to obtain a new standard mask image corresponding to the new sample image. For example, at least one data enhancement processing operation can be performed on the original sample image and the first standard sample mask image corresponding to the original sample image to obtain a new original sample image and a new first standard sample mask image; and at least one data enhancement processing operation can be performed on the watermark sample image and the second standard sample mask image corresponding to the watermark sample image to obtain a new watermark sample image and a new second standard sample mask image.

[0171] Optionally, if an operation such as adding noise is performed on the original sample image, since this operation will not change the position of the watermark area, it is not necessary to perform the same data enhancement processing operation on the standard mask image corresponding to the original sample image, and the standard mask image corresponding to the original sample image can be directly used as a new standard mask image. Conversely, if an operation such as image rotation or image scaling is performed on the original sample image, since this operation will change the position of the watermark area, the same data enhancement processing operation must be performed on the standard mask image corresponding to the original sample image to obtain a new standard mask image.

[0172] Then, the new sample image and the new standard mask image can be added to the training data set to obtain an updated training data set. Afterwards, for the above S502, the sample image in the updated training data set can be input into the dark watermark segmentation model to be trained to obtain the first predicted mask image corresponding to the sample image.

[0173] For example, the original sample image can be input into the dark watermark segmentation model to be trained to obtain a first prediction mask image corresponding to the original sample image; and the watermark sample image can be input into the dark watermark segmentation model to be trained to obtain a first prediction mask image corresponding to the watermark sample image; the new original sample image can be input into the dark watermark segmentation model to be trained to obtain a first prediction mask image corresponding to the new original sample image; the new watermark sample image can be input into the dark watermark segmentation model to be trained to obtain a first prediction mask image corresponding to the new watermark sample image.

[0174] Furthermore, the first prediction loss can be calculated by using the first prediction mask image corresponding to the original sample image and the first standard sample mask image corresponding to the original sample image, the first prediction mask image corresponding to the watermark sample image and the second standard sample mask image corresponding to the watermark sample image, the first prediction mask image corresponding to the new original sample image and the new first standard sample mask image corresponding to the new original sample image, the first prediction mask image corresponding to the new watermark sample image and the new second standard sample mask image corresponding to the new watermark sample image, and the first prediction loss can be used to iteratively train the dark watermark segmentation model to be trained to obtain a trained dark watermark segmentation model.

[0175] In this way, by performing data enhancement processing operations on sample images, not only the number of sample images can be increased, but also the model can learn richer features when training with the enhanced sample images, thereby improving the recognition accuracy of the trained dark watermark segmentation model for images after data enhancement processing.

[0176] In one embodiment, in order to make the model better learn the contrast difference between the dark watermark area and the watermark-free area, the model learning effect can also be improved by adjusting the order in which the sample images are input into the dark watermark segmentation model to be trained. Specifically, the step of "inputting the sample images into the dark watermark segmentation model to be trained" in S502 above can be implemented according to the following sub-steps:

[0177] Sub-step 1: for any original sample image in the training data set, determine a watermark sample image carrying a dark watermark corresponding to the original sample image, and take the watermark sample image and the original sample image as a sample image group.

[0178] Here, a sample image group includes an original sample image and a watermark sample image corresponding to the original sample image. In the case where an original sample image corresponds to multiple different watermark sample images, the original sample image may belong to multiple sample image groups, but the watermark sample images in each sample image group are different.

[0179] Optionally, when the training data set also includes new original sample images and new watermark sample images, a sample image group may also be an image group including a new original sample image and a new watermark sample image corresponding to the new original sample image.

[0180] Sub-step 2: input each sample image in each sample image group corresponding to the training data set into the dark watermark segmentation model to be trained.

[0181] During the model training process, the input received each time may be a batch of image information, wherein the batch size is the number of sample images input to the dark watermark segmentation model to be trained at one time. For example, a batch of image information may include 4 sample image groups (a total of 8 sample images), 8 sample image groups (a total of 16 sample images), 12 sample image groups (a total of 24 sample images), 16 sample image groups (a total of 32 sample images), etc. The number of sample image groups included in a batch of image information can be set based on experience, and the present disclosure embodiment does not specifically limit it.

[0182] Exemplarily, taking the number of sample image groups corresponding to a batch as 4, and the 4 sample image groups corresponding to a batch as sample image groups 1 to 4, when inputting the sample images into the dark watermark segmentation model to be trained, the two images in the sample image group 1 can be first input into the dark watermark segmentation model to be trained in sequence; then the two images in the sample image group 2 can be input, and then, the two images in the sample image group 3 can be input, and finally the two images in the sample image group 4 can be input, thereby completing the training of a batch. For example, sample image group 1 includes original sample image 1 and watermark sample image 1, sample image group 2 includes original sample image 2 and watermark sample image 2, sample image group 3 includes original sample image 3 and watermark sample image 3, and sample image group 4 includes original sample image 4 and watermark sample image 4. The original sample image 1 and watermark sample image 1 can be first input into the dark watermark segmentation model to be trained in sequence, and then the original sample image 2 and watermark sample image 2 can be input into the dark watermark segmentation model to be trained in sequence, and then the original sample image 3 and watermark sample image 3 can be input into the dark watermark segmentation model to be trained in sequence, and finally the original sample image 4 and watermark sample image 4 can be input into the dark watermark segmentation model to be trained in sequence.

[0183] In this way, the sample image group can be used to adjust the input order of the sample images. By adjusting the input order, it can be ensured that the original sample images in multiple sample image groups and the watermark sample images obtained by embedding the dark watermark in the original sample images can be located in the same batch, thereby improving the speed and effect of the model learning the contrast difference between the dark watermark area and the watermark-free area.

[0184] In one embodiment, in order to further improve the recognition accuracy of the trained dark watermark segmentation model, the above S503 can also be implemented according to the following steps:

[0185] S5031: Iteratively train the dark watermark segmentation model to be trained using the first prediction loss until a training cutoff condition is met to obtain an intermediate watermark segmentation model.

[0186] Here, the intermediate watermark segmentation model is a model trained using the first prediction loss.

[0187] S5032: compressing the sample images in the training data set to obtain compressed sample images; wherein the original sample images and the watermarked sample images belonging to the same sample image group correspond to the same compression factor.

[0188] Here, different compression factors correspond to different compression strengths. The way of compressing the sample images in the training data set may include, for example, Joint Photographic Experts Group (JPEG) compression.

[0189] Exemplarily, random compression processing may be directly performed on sample images in the training data set to obtain compressed sample images, or random compression processing may be performed on sample images in the updated training data set to obtain compressed sample images.

[0190] Among them, since JPEG compression is a lossy compression method, in order to ensure that the model can learn the above-mentioned contrast differences to a higher degree, the same compression factor can be set for the original sample images and watermark sample images belonging to the same sample image group to reduce the influence of other factors except compression and dark watermarks on image differences and improve the model learning effect.

[0191] S5033: Input the compressed sample image into the intermediate watermark segmentation model to obtain a second predicted mask image.

[0192] Here, the second predicted mask image is the predicted mask image output by the intermediate watermark segmentation model.

[0193] In specific implementation, each compressed sample image in each sample image group corresponding to the training data set can be sequentially input into the dark watermark segmentation model to be trained, so as to obtain the second prediction mask image corresponding to each compressed sample image.

[0194] S5034: Determine the second prediction loss of the intermediate watermark segmentation model using the second prediction mask image and the standard mask image corresponding to the compressed sample image, and iteratively train the intermediate watermark segmentation model using the second prediction loss until the training cutoff condition is met, thereby obtaining a trained dark watermark segmentation model.

[0195] Here, the standard mask image corresponding to the compressed sample image is the standard mask image of the sample image before compression corresponding to the compressed sample image. The second prediction loss is used to characterize the deviation between the second prediction mask image output by the intermediate watermark segmentation model and the standard mask image corresponding to the compressed sample image.

[0196] Iterative training of the intermediate watermark segmentation model using the second prediction loss can improve the segmentation ability of the trained dark watermark segmentation model for the compressed image, and then improve the accuracy of the watermark mask image corresponding to the output compressed image. Since JPEG compression is a lossy compression, it will change the dark watermark in the image to a certain extent. Therefore, by first using uncompressed sample images for training to obtain the intermediate watermark segmentation model, and then using the compressed images to train the intermediate watermark segmentation model, it can not only improve the training speed, but also improve the training effect, and improve the robustness and prediction accuracy of the trained dark watermark segmentation model.

[0197] Optionally, JPEG compression can also be used as a data enhancement method to perform random JPEG compression processing on the sample images in the training data set and the standard mask images corresponding to the sample images to obtain new sample images and new standard mask images, and the new sample images and new standard mask images are added to the training data set to obtain an updated training data set. Finally, the updated training data set can be used to train the dark watermark segmentation model to be trained to obtain a trained dark watermark segmentation model.

[0198] like Figure 6 As shown, a specific implementation flow chart of model training and model reasoning provided by an embodiment of the present disclosure may include the following steps:

[0199] Obtain an original image set. The original image set includes a large number of original sample images. Use a dark watermark embedding method to embed a dark watermark into the original sample image to obtain a watermark image set. The watermark sample image set includes a large number of watermark sample images. Combine the original image set, the watermark image set, the standard mask image corresponding to the original sample image, and the standard mask image corresponding to the watermark sample image to obtain a training data set.

[0200] At least one data enhancement operation is randomly performed on the sample images in the training data set and the standard mask images corresponding to the sample images to obtain an updated training data set. The dark watermark segmentation model to be trained is trained using the sample images in the updated training data set to obtain a trained watermark segmentation model.

[0201] Then, the target image to be extracted from the dark watermark can be obtained, and the target image can be adjusted to an adjusted image with a target size. The trained watermark segmentation model is used for inference to obtain a watermark mask image corresponding to the adjusted image. Then, a circumscribed rectangle positioning operation is performed to determine whether there is an initial watermark area, wherein this step can refer to the step of "determining the initial watermark area" in S103 above. If not, the process ends and it is determined that the target image does not include a dark watermark.

[0202] If so, the image correction information is determined based on each initial watermark area. The image correction information is used to correct the adjusted image and the watermark mask image to obtain a corrected adjusted image and a corrected watermark mask image. Then, the intermediate watermark area can be determined based on the corrected watermark mask image, and the target watermark area that matches the area coordinates of the intermediate watermark area can be determined from the corrected adjusted image. After that, each target watermark area can be restored to a target watermark area that matches the image size, thereby obtaining each watermark block corresponding to the target image. Finally, the dark watermark included in the target image can be extracted from the watermark block.

[0203] Those skilled in the art will appreciate that, in the above method of specific implementation, the order in which the steps are written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of the steps should be determined by their functions and possible internal logic.

[0204] Based on the same inventive concept, a watermark extraction device corresponding to the watermark extraction method is also provided in the embodiment of the present disclosure. Since the principle of solving the problem by the device in the embodiment of the present disclosure is similar to the above-mentioned watermark extraction method in the embodiment of the present disclosure, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0205] like Figure 7 FIG. 1 is a schematic diagram of a watermark extraction device provided by an embodiment of the present disclosure, comprising:

[0206] An acquisition module 701 is used to acquire a target image to be extracted with a dark watermark;

[0207] The recognition module 702 is used to perform watermark region recognition on the target image to obtain a watermark mask image;

[0208] A first determination module 703 is used to determine each initial watermark region corresponding to the target image according to the watermark mask image, and determine image correction information based on each initial watermark region;

[0209] The second determination module 704 is used to perform correction processing on the target image using the image correction information to obtain a corrected target image, and determine, based on the corrected target image, each target watermark region in the target image including a dark watermark to be extracted;

[0210] The extraction module 705 is used to extract the dark watermark in the target image according to each of the target watermark regions.

[0211] In a possible implementation manner, the first determining module 703, when determining each initial watermark region corresponding to the target image according to the watermark mask image, is configured to:

[0212] Clustering the pixel points according to the pixel value and coordinate information of each pixel point in the watermark mask image to obtain each point cluster corresponding to the watermark mask image;

[0213] According to the coordinate information of each pixel point in the point cluster, each target point cluster having a circumscribed rectangle is determined, and the image area corresponding to the circumscribed rectangle of the target point cluster is used as the initial watermark area.

[0214] In a possible implementation manner, the first determining module 703, when determining the image correction information based on each of the initial watermark regions, is configured to:

[0215] Based on the regional coordinates of each of the initial watermark regions, respectively determine the regional correction information of each of the initial watermark regions;

[0216] The image correction information is determined according to the number of occurrences of each of the area correction information.

[0217] In a possible implementation, the image correction information includes rotation angle information;

[0218] The second determining module 704, when performing correction processing on the target image by using the image correction information to obtain a corrected target image, and determining each target watermark area including the dark watermark to be extracted in the target image based on the corrected target image, is used to:

[0219] According to the rotation angle information, the target image and the watermark mask image are respectively rotated to obtain a rotated target image and a rotated watermark mask image;

[0220] Determining each intermediate watermark region related to the dark watermark to be extracted from the rotated watermark mask image;

[0221] From the rotated target image, an image region matching the region coordinates of the intermediate watermark region is determined as the target watermark region.

[0222] In a possible implementation manner, the recognition module 702, when performing watermark region recognition on the target image to obtain the watermark mask image, is configured to:

[0223] In the case where the image size of the target image is inconsistent with the target size, resizing the target image according to the image size and the target size to obtain an adjusted image having the target size;

[0224] Performing watermark area recognition on the adjusted image to obtain the watermark mask image;

[0225] The second determination module 704, after determining, based on the corrected target image, each target watermark region including the dark watermark to be extracted in the target image, is further configured to:

[0226] According to the image size and the target size, each of the target watermark areas is restored to a target watermark area matching the image size.

[0227] In a possible implementation manner, the watermark mask image is obtained by identifying the watermark region of the target image using a pre-trained dark watermark segmentation model;

[0228] The device further comprises a training module 706, which is used to train the dark watermark segmentation model according to the following steps:

[0229] Acquire a training data set; the training data set includes sample images with multiple resolutions and standard mask images corresponding to the sample images, the sample images include original sample images without embedded sample dark watermarks and watermark sample images carrying sample dark watermarks;

[0230] Inputting the sample image in the training data set into the dark watermark segmentation model to be trained to obtain a first prediction mask image corresponding to the sample image;

[0231] According to the first prediction mask image and the standard mask image corresponding to the sample image, a first prediction loss of the dark watermark segmentation model to be trained is determined, and the dark watermark segmentation model to be trained is iteratively trained using the first prediction loss until a training cutoff condition is met, thereby obtaining a trained dark watermark segmentation model.

[0232] In a possible implementation manner, the training module 706, after obtaining the training data set, is further configured to:

[0233] Randomly performing at least one data enhancement processing operation on a sample image in the training data set and a standard mask image corresponding to the sample image to obtain a new sample image and a new standard mask image;

[0234] Supplementing the new sample image and the new standard mask image to the training data set to obtain an updated training data set;

[0235] Furthermore, the training module 706, when inputting the sample image in the training data set into the dark watermark segmentation model to be trained to obtain the first predicted mask image corresponding to the sample image, is used to:

[0236] The sample image in the updated training data set is input into the dark watermark segmentation model to be trained to obtain a first predicted mask image corresponding to the sample image.

[0237] In a possible implementation manner, the training module 706, when inputting the sample image in the training data set into the dark watermark segmentation model to be trained, is configured to:

[0238] For any of the original sample images in the training data set, determining a watermark sample image carrying a dark watermark corresponding to the original sample image, and taking the watermark sample image and the original sample image as a sample image group;

[0239] The sample images in the sample image groups corresponding to the training data set are sequentially input into the dark watermark segmentation model to be trained.

[0240] In a possible implementation manner, the training module 706, when iteratively training the dark watermark segmentation model to be trained by using the first prediction loss until a training cutoff condition is met to obtain a trained dark watermark segmentation model, is used to:

[0241] Iteratively training the dark watermark segmentation model to be trained by using the first prediction loss until a training cutoff condition is met, thereby obtaining an intermediate watermark segmentation model;

[0242] Compressing the sample images in the training data set to obtain compressed sample images; wherein the original sample images and the watermarked sample images belonging to the same sample image group correspond to the same compression factor;

[0243] Inputting the compressed sample image into the intermediate watermark segmentation model to obtain a second predicted mask image;

[0244] The second prediction mask image and the standard mask image corresponding to the compressed sample image are used to determine the second prediction loss of the intermediate watermark segmentation model, and the intermediate watermark segmentation model is iteratively trained using the second prediction loss until a training cutoff condition is met to obtain a trained dark watermark segmentation model.

[0245] For descriptions of the processing flow of each module in the device and the interaction flow between each module, reference may be made to the relevant descriptions in the above method embodiment, which will not be described in detail here.

[0246] Based on the same technical concept, the embodiment of the present application also provides an electronic device. Figure 8 FIG. 1 is a schematic diagram of a structure of an electronic device provided in an embodiment of the present application, including:

[0247] Processor 81, memory 82 and bus 83. The memory 82 stores machine-readable instructions executable by the processor 81, and the processor 81 is used to execute the machine-readable instructions stored in the memory 82. When the machine-readable instructions are executed by the processor 81, the processor 81 executes the following steps: S101: acquiring a target image to be extracted from a dark watermark; S102: identifying a watermark region of the target image to obtain a watermark mask image; S103: determining each initial watermark region corresponding to the target image according to the watermark mask image, and determining image correction information based on each initial watermark region; S104: correcting the target image using the image correction information to obtain a corrected target image, and determining each target watermark region in the target image including the dark watermark to be extracted based on the corrected target image; and S105: extracting the dark watermark in the target image according to each target watermark region.

[0248] The above-mentioned memory 82 includes internal memory 821 and external memory 822; the memory 821 here is also called internal memory, which is used to temporarily store the calculation data in the processor 81, as well as the data exchanged with the external memory 822 such as a hard disk. The processor 81 exchanges data with the external memory 822 through the internal memory 821. When the electronic device is running, the processor 81 communicates with the memory 82 through the bus 83, so that the processor 81 executes the execution instructions mentioned in the above-mentioned method embodiment.

[0249] The embodiment of the present disclosure also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the watermark extraction method described in the above method embodiment are executed. The storage medium can be a volatile or non-volatile computer-readable storage medium.

[0250] The computer program product of the watermark extraction method provided in the embodiments of the present disclosure includes a computer-readable storage medium storing program code, and the instructions included in the program code can be used to execute the steps of the watermark extraction method described in the above method embodiments. For details, please refer to the above method embodiments, which will not be repeated here.

[0251] The computer program product may be implemented in hardware, software or a combination thereof. In one optional embodiment, the computer program product is embodied as a computer storage medium, and in another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (SDK) and the like.

[0252] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, the specific working process of the device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here. In the several embodiments provided in the present disclosure, it should be understood that the disclosed device and method can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of the device or unit can be electrical, mechanical or other forms.

[0253] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0254] In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0255] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present disclosure, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling an electronic device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present disclosure. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0256] If the technical solution of this application involves personal information, the product using the technical solution of this application has clearly informed the personal information processing rules and obtained the individual's voluntary consent before processing the personal information. If the technical solution of this application involves sensitive personal information, the product using the technical solution of this application has obtained the individual's separate consent before processing the sensitive personal information, and at the same time meets the "explicit consent" requirement. For example, on personal information collection devices such as cameras, clear and prominent signs are set to inform that the personal information collection scope has been entered and personal information will be collected. If the individual voluntarily enters the collection scope, it is deemed that he or she agrees to the collection of his or her personal information; or on the device that processes personal information, the personal information processing rules are notified by obvious signs / information, and the individual's authorization is obtained through pop-up information or by asking the individual to upload his or her personal information; among them, the personal information processing rules may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the type of personal information processed.

[0257] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present disclosure, which are used to illustrate the technical solutions of the present disclosure, rather than to limit them. The protection scope of the present disclosure is not limited thereto. Although the present disclosure is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed in the present disclosure, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should be included in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be based on the protection scope of the claims.

Claims

1. A watermark extraction method, characterized in that: include: Obtain a target image to be extracted with a dark watermark; Performing watermark region recognition on the target image to obtain a watermark mask image; Determining each initial watermark region corresponding to the target image according to the watermark mask image, and determining image correction information based on each initial watermark region; Using the image correction information to perform correction processing on the target image to obtain a corrected target image, and based on the corrected target image, determining each target watermark region in the target image including the dark watermark to be extracted; According to each of the target watermark regions, a dark watermark in the target image is extracted.

2. The method according to claim 1, characterized in that The step of determining each initial watermark area corresponding to the target image according to the watermark mask image comprises: Clustering the pixel points according to the pixel value and coordinate information of each pixel point in the watermark mask image to obtain each point cluster corresponding to the watermark mask image; According to the coordinate information of each pixel point in the point cluster, each target point cluster having a circumscribed rectangle is determined, and the image area corresponding to the circumscribed rectangle of the target point cluster is used as the initial watermark area.

3. The method according to claim 1, characterized in that: The determining of image correction information based on each of the initial watermark regions includes: Based on the regional coordinates of each of the initial watermark regions, respectively determine the regional correction information of each of the initial watermark regions; The image correction information is determined according to the number of occurrences of each of the area correction information.

4. The method according to claim 1, characterized in that The image correction information includes rotation angle information; The method of performing correction processing on the target image by using the image correction information to obtain a corrected target image, and determining, based on the corrected target image, each target watermark region in the target image including a dark watermark to be extracted, comprises: According to the rotation angle information, the target image and the watermark mask image are respectively rotated to obtain a rotated target image and a rotated watermark mask image; Determining each intermediate watermark region related to the dark watermark to be extracted from the rotated watermark mask image; From the rotated target image, an image region matching the region coordinates of the intermediate watermark region is determined as the target watermark region.

5. The method according to claim 1, characterized in that The step of performing watermark area recognition on the target image to obtain a watermark mask image includes: In the case where the image size of the target image is inconsistent with the target size, resizing the target image according to the image size and the target size to obtain an adjusted image having the target size; Performing watermark area recognition on the adjusted image to obtain the watermark mask image; After determining, based on the corrected target image, each target watermark region including the dark watermark to be extracted in the target image, the method further includes: According to the image size and the target size, each of the target watermark areas is restored to a target watermark area matching the image size.

6. The method according to claim 1, characterized in that The watermark mask image is obtained by identifying the watermark area of ​​the target image using a pre-trained dark watermark segmentation model; The dark watermark segmentation model is trained according to the following steps: Acquire a training data set; the training data set includes sample images with multiple resolutions and standard mask images corresponding to the sample images, the sample images include original sample images without embedded sample dark watermarks and watermark sample images carrying sample dark watermarks; Inputting the sample image in the training data set into the dark watermark segmentation model to be trained to obtain a first prediction mask image corresponding to the sample image; According to the first prediction mask image and the standard mask image corresponding to the sample image, a first prediction loss of the dark watermark segmentation model to be trained is determined, and the dark watermark segmentation model to be trained is iteratively trained using the first prediction loss until a training cutoff condition is met, thereby obtaining a trained dark watermark segmentation model.

7. The method according to claim 6, characterized in that After obtaining the training data set, the method further includes: Randomly performing at least one data enhancement processing operation on a sample image in the training data set and a standard mask image corresponding to the sample image to obtain a new sample image and a new standard mask image; Supplementing the new sample image and the new standard mask image to the training data set to obtain an updated training data set; The step of inputting the sample image in the training data set into the dark watermark segmentation model to be trained to obtain a first predicted mask image corresponding to the sample image includes: The sample image in the updated training data set is input into the dark watermark segmentation model to be trained to obtain a first predicted mask image corresponding to the sample image.

8. The method according to claim 6, characterized in that The step of inputting the sample image in the training data set into the dark watermark segmentation model to be trained comprises: For any of the original sample images in the training data set, determining a watermark sample image carrying a dark watermark corresponding to the original sample image, and taking the watermark sample image and the original sample image as a sample image group; The sample images in the sample image groups corresponding to the training data set are sequentially input into the dark watermark segmentation model to be trained.

9. The method according to claim 6, characterized in that The iterative training of the dark watermark segmentation model to be trained by using the first prediction loss until a training cutoff condition is met to obtain a trained dark watermark segmentation model includes: Iteratively training the dark watermark segmentation model to be trained by using the first prediction loss until a training cutoff condition is met, thereby obtaining an intermediate watermark segmentation model; Compressing the sample images in the training data set to obtain compressed sample images; wherein the original sample images and the watermarked sample images belonging to the same sample image group correspond to the same compression factor; Inputting the compressed sample image into the intermediate watermark segmentation model to obtain a second predicted mask image; The second prediction mask image and the standard mask image corresponding to the compressed sample image are used to determine the second prediction loss of the intermediate watermark segmentation model, and the intermediate watermark segmentation model is iteratively trained using the second prediction loss until a training cutoff condition is met to obtain a trained dark watermark segmentation model.

10. A watermark extraction device, characterized in that: include: An acquisition module, used for acquiring a target image to be extracted with a dark watermark; An identification module is used to identify the watermark area of ​​the target image to obtain a watermark mask image; A first determination module, configured to determine each initial watermark region corresponding to the target image according to the watermark mask image, and determine image correction information based on each initial watermark region; A second determination module is used to perform correction processing on the target image using the image correction information to obtain a corrected target image, and based on the corrected target image, determine each target watermark area in the target image including the dark watermark to be extracted; The extraction module is used to extract the dark watermark in the target image according to each target watermark area.

11. An electronic device, characterized in that: include: A processor and a memory, wherein the memory stores machine-readable instructions executable by the processor, and the processor is used to execute the machine-readable instructions stored in the memory. When the machine-readable instructions are executed by the processor, the processor executes the steps of the watermark extraction method as described in any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program. When the computer program is executed by an electronic device, the electronic device performs the steps of the watermark extraction method according to any one of claims 1 to 9.

Citation Information

Cited By

  • Single-layer engineering data scanning document watermark removing method, system and equipment and storage medium

    CN120219511A