Watermark removal method, apparatus, device, medium, and program product

Abstract

The application provides a watermark removing method, device, equipment, medium and program product, which can be applied to the field of artificial intelligence technology, and specifically applied to the field of image processing technology. The method comprises the following steps: converting a business image to be watermarked into a three-channel image; performing watermark detection on the three-channel image based on a preset watermark color feature condition, and generating a preliminary watermark mask based on a detection result; performing frequency domain transformation on the business image to obtain a frequency domain component distribution feature of the business image, fusing a color feature of the business image and the frequency domain component distribution feature to form a joint feature vector; inputting the joint feature vector and the preliminary watermark mask into a pre-trained gradient boosting tree model to correct the preliminary watermark mask, and obtaining a corrected watermark mask; and performing color filling on the business image based on a watermark region indicated by the corrected watermark mask, and obtaining a target image with removed watermark. In this way, the accuracy of removing the watermark is improved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence, specifically to the field of image processing technology, and more specifically to a watermark removal method, apparatus, device, medium, and program product. Background Technology

[0002] Watermarking technology, as an important means of copyright protection and data security, is widely used in various images for copyright ownership verification and source tracking. However, in the fintech field, banks need to process massive amounts of business images such as paper documents, bank statement scans, and paper contracts. These business images are often covered with watermarks such as bank seals, anti-counterfeiting marks, and background patterns. Watermarks on banking business images can seriously interfere with the recognition and processing of image content, leading to errors in the extraction of critical information such as amounts and account numbers, and severely restricting the further improvement of the automation level of banking business processes. Summary of the Invention

[0003] In view of the above problems, embodiments of this application provide a watermark removal method, apparatus, device, medium, and program product.

[0004] According to a first aspect of this application, a watermark removal method is provided, the method comprising: acquiring a business image to be watermarked and converting the business image into a three-channel image; performing watermark detection on the three-channel image based on preset watermark color feature conditions, and generating a preliminary watermark mask based on the detection result; performing frequency domain transformation on the business image to obtain the frequency domain component distribution features of the business image, and fusing the color features of the business image with the frequency domain component distribution features to form a joint feature vector; inputting the joint feature vector and the preliminary watermark mask into a pre-trained gradient boosting tree model, and using the gradient boosting tree model to correct the preliminary watermark mask to obtain a corrected watermark mask; and performing color filling on the business image based on the watermark area indicated by the corrected watermark mask to obtain a target image after watermark removal.

[0005] According to an embodiment of this application, the step of color-filling the service image based on the watermark area indicated by the modified watermark mask to obtain a target image after watermark removal includes: constructing color-filling constraints for the watermark area, wherein the color-filling constraints include: texture continuity conditions, color harmony conditions, and semantic rationality conditions; determining the fill color of the watermark area based on the color-filling constraints; and color-filling the watermark area using the fill color to obtain the target image after watermark removal.

[0006] According to embodiments of this application, the texture continuity condition includes: minimizing the chi-square distance between the local binary mode histogram of the watermark region after color filling and the local binary mode histogram of the neighboring background region of the watermark region; the color harmony condition includes: the color value of the filling color in a preset color space is within the color distribution range of the neighboring background region, wherein the preset color space includes a luminance channel, a first chrominance channel, and a second chrominance channel, and the color distribution range is determined based on the color mean and color standard deviation of the neighboring background region in the luminance channel, the first chrominance channel, and the second chrominance channel; the semantic rationality condition includes: when the watermark region is located in a smooth region, the gradient magnitude corresponding to the filling color is less than the gradient threshold of the smooth region; when the watermark region is located in a textured region, the directional difference between the gradient direction corresponding to the filling color and the texture direction of the textured region is less than a preset directional difference threshold.

[0007] According to an embodiment of this application, determining the fill color of the watermark region based on the color fill constraint includes: using the fill color function of the watermark region as the function to be solved, using the value of the fill color function at the boundary of the watermark region being equal to the color value of the neighboring background at the corresponding boundary as the boundary condition, and establishing a first Poisson equation according to the color fill constraint; solving the first Poisson equation to obtain the fill color corresponding to each pixel in the watermark region, wherein the first Poisson equation is used to constrain the Laplace result of the fill color function to match the divergence of the guiding gradient field, and the guiding gradient field is determined according to the gradient of the neighboring background of the watermark region.

[0008] According to an embodiment of this application, when a reference image associated with the business image is obtained, the method further includes: determining a reference region in the reference image corresponding to the watermark region; determining a fidelity term based on the preset fidelity coefficient and the color difference between the fill color function and the color function of the reference region; using the fill color function of the watermark region as the function to be solved, using the value of the fill color function at the boundary of the watermark region being equal to the color value of the reference region at the corresponding boundary as a boundary condition, and establishing a second Poisson equation based on the color fill constraint condition and the fidelity term; solving the second Poisson equation to obtain the fill color corresponding to each pixel in the watermark region, wherein the second Poisson equation is used to make the Laplace result of the fill color function equal to the combination result of the divergence of the guiding gradient field and the fidelity term.

[0009] According to an embodiment of this application, before performing watermark detection on the three-channel image based on preset watermark color feature conditions and generating a preliminary watermark mask based on the detection results, the method further includes: extracting watermark classification features of the business image and determining the target watermark type of the business image based on the watermark classification features, wherein the watermark classification features include at least one of the following: color uniformity features, transparency features, and edge gradient histogram features; performing watermark detection on the three-channel image based on preset watermark color feature conditions and generating a preliminary watermark mask based on the detection results includes: determining, from pre-stored watermark color feature conditions, the watermark type corresponding to the target watermark type. The process involves: determining the target watermark color feature condition; performing watermark detection on pixels in the business image based on the target watermark color feature condition, and generating a preliminary watermark mask based on the detection results; inputting the joint feature vector and the preliminary watermark mask into a pre-trained gradient boosting tree model, and using the gradient boosting tree model to correct the preliminary watermark mask to obtain a corrected watermark mask, including: determining the gradient boosting tree model corresponding to the target watermark type from multiple pre-trained gradient boosting tree models; inputting the joint feature vector and the preliminary watermark mask into the gradient boosting tree model corresponding to the target watermark type, and correcting the preliminary watermark mask to obtain a corrected watermark mask.

[0010] According to embodiments of this application, the watermark type includes at least one of the following: solid color text watermark, pattern texture watermark, semi-transparent overlay watermark, and fused watermark; when the target watermark type is the solid color text watermark, the target watermark color feature condition includes: the channel values ​​of the pixel in the red channel, green channel, and blue channel are respectively greater than the corresponding solid color text watermark channel threshold; when the target watermark type is the pattern texture watermark, the target watermark color feature condition includes: the channel values ​​of the pixel in the red channel and green channel are respectively greater than the corresponding pattern texture watermark channel threshold, and the channel values ​​in the blue channel are also greater than the corresponding pattern texture watermark channel threshold. The value is less than the corresponding pattern texture watermark channel threshold; when the target watermark type is the semi-transparent overlay watermark, the target watermark color feature conditions include: the channel values ​​of the pixel in the red channel, green channel and blue channel are respectively greater than the corresponding semi-transparent overlay watermark channel threshold, and the color variance of the local area where the pixel is located is less than the local variance threshold; when the target watermark type is the fused watermark, the target watermark color feature conditions include: the difference between the channel values ​​of the pixel in the red channel, green channel and blue channel and the channel mean of the corresponding channel in the neighboring background area is respectively less than the corresponding pixel difference threshold.

[0011] According to an embodiment of this application, the step of extracting watermark classification features of the business image and determining the target watermark type of the business image based on the watermark classification features includes: performing edge detection on the business image to determine regions in the business image with color gradients greater than a preset gradient threshold as first candidate watermark regions; clustering the colors of each pixel in the first candidate watermark region to determine regions where pixel clusters with fewer than a preset number threshold are located as second candidate watermark regions; extracting watermark classification features from the second candidate watermark regions and determining the target watermark type of the business image based on the watermark classification features.

[0012] According to an embodiment of this application, after color-filling the service image based on the watermark region indicated by the modified watermark mask to obtain a target image after watermark removal, the method further includes: performing wavelet transform on the service image and the target image respectively to obtain a first multi-scale frequency domain feature of the service image and a second multi-scale frequency domain feature of the target image; performing integrity verification on the service image and the target image based on the first multi-scale frequency domain feature and the second multi-scale frequency domain feature to obtain a semantic consistency verification result; and outputting the target image if the semantic consistency verification result indicates that the key business semantics in the target image have not been destroyed.

[0013] According to a second aspect of this application, a watermark removal apparatus is provided, the apparatus comprising: an image acquisition module for acquiring a business image to be watermarked and converting the business image into a three-channel image; a mask generation module for performing watermark detection on the three-channel image based on preset watermark color feature conditions and generating a preliminary watermark mask based on the detection result; an image transformation module for performing frequency domain transformation on the business image to obtain the frequency domain component distribution features of the business image, and fusing the color features of the business image with the frequency domain component distribution features to form a joint feature vector; a mask correction module for inputting the joint feature vector and the preliminary watermark mask into a pre-trained gradient boosting tree model, and using the gradient boosting tree model to correct the preliminary watermark mask to obtain a corrected watermark mask; and an image filling module for color filling the business image based on the watermark area indicated by the corrected watermark mask to obtain a target image after watermark removal.

[0014] According to a third aspect of this application, an electronic device is provided, comprising: one or more processors; and a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the method described above.

[0015] According to a fourth aspect of this application, a computer-readable storage medium is also provided, on which a computer program or instructions are stored, wherein the computer program or instructions, when executed by a processor, implement the steps of the above-described method.

[0016] According to a fifth aspect of this application, a computer program product is also provided, including a computer program or instructions that, when executed by a processor, implement the steps of the above-described method.

[0017] According to an embodiment of this application, watermark detection is first performed on the three-channel image of the business image based on preset watermark color feature conditions to generate a preliminary watermark mask to initially divide the watermark region in the business image; then, joint modeling is performed based on the frequency domain component distribution features and color features of the business image, and the preliminary watermark mask is corrected using a gradient boosting tree model to obtain a more accurate corrected watermark mask; finally, the watermark region in the business image is filled based on the corrected watermark mask to obtain the target image after watermark removal, thereby improving the accuracy of watermark removal. Attached Figure Description

[0018] The above-mentioned contents, other objects, features and advantages of this application will become clearer from the following description of embodiments with reference to the accompanying drawings, in which:

[0019] Figure 1 The illustrations depict application scenarios of watermark removal methods, apparatus, devices, media, and program products according to embodiments of this application.

[0020] Figure 2 A flowchart illustrating a first watermark removal method according to an embodiment of this application is shown schematically.

[0021] Figure 3 A flowchart illustrating a second watermark removal method according to an embodiment of this application is shown schematically.

[0022] Figure 4 A flowchart illustrating a third watermark removal method according to an embodiment of this application is shown schematically.

[0023] Figure 5 This schematic diagram illustrates the structural block diagram of a watermark removal apparatus according to an embodiment of the present application;

[0024] Figure 6 A block diagram schematically illustrates an electronic device suitable for implementing a watermark removal method according to an embodiment of this application. Detailed Implementation

[0025] The embodiments of this application will now be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of this application. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of this application for ease of explanation. However, it will be apparent that one or more embodiments may be implemented without these specific details. Furthermore, descriptions of well-known structures and technologies are omitted in the following description to avoid unnecessarily obscuring the concepts of this application.

[0026] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of this application. The terms “comprising,” “including,” etc., as used herein indicate the presence of the stated features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0027] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0028] When using expressions such as "at least one of A, B and C", they should generally be interpreted in accordance with the meaning that is commonly understood by those skilled in the art (e.g., "a system having at least one of A, B and C" should include, but is not limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B and C, etc.).

[0029] As used in this paper, the term "model" refers to a model that learns the relationship between inputs and outputs from training data, enabling it to generate corresponding outputs for a given input after training. Model generation can be based on machine learning techniques. Deep learning is a machine learning algorithm that processes inputs and provides corresponding outputs using multiple layers of processing units. A neural network model is an example of a deep learning-based model. In this paper, "model" may also be referred to as a "machine learning model," "learning model," "machine learning network," or "learning network," and these terms are used interchangeably.

[0030] It's important to note that the term "neural network" can refer to a machine learning network based on deep learning. A neural network processes input and provides corresponding output, typically consisting of an input layer, an output layer, and one or more hidden layers between them. Neural networks used in deep learning applications often include many hidden layers, increasing the network's depth. The layers of a neural network are connected sequentially, so that the output of the previous layer serves as the input to the next layer. The input layer receives the input to the neural network, while the output layer's output becomes the final output. Each layer of a neural network includes one or more nodes (also called processing nodes or neurons), each processing the input from the layer above.

[0031] It should be understood that machine learning generally includes three phases: training, testing, and application (also known as inference). In the training phase, a given model is trained using a large amount of training data, iteratively updating parameter values ​​until the model can consistently generate inferences that meet the expected goals from the training data. Through training, the model can be considered to have learned the relationship between inputs and outputs (also known as the input-output mapping) from the training data. The parameter values ​​of the trained model are determined. In the testing phase, test inputs are applied to the trained model to test whether it can provide the correct output, thus determining the model's performance. In the application phase, the model can be used to process actual inputs based on the trained parameter values ​​to determine the corresponding output.

[0032] Watermarking technology, as an important means of copyright protection and data security, is widely used in various images for copyright ownership verification and source tracking. However, in the fintech field, banks need to process massive amounts of business images such as paper documents, bank statement scans, and paper contracts. These business images are often covered with watermarks such as bank seals, anti-counterfeiting marks, and background patterns. Watermarks on banking business images can seriously interfere with the recognition and processing of image content, leading to errors in the extraction of critical information such as amounts and account numbers, and severely restricting the further improvement of the automation level of banking business processes.

[0033] In view of the above problems, embodiments of this application provide a watermark removal method, the method comprising: acquiring a business image to be watermarked and converting the business image into a three-channel image; performing watermark detection on the three-channel image based on preset watermark color feature conditions, and generating a preliminary watermark mask based on the detection results; performing frequency domain transformation on the business image to obtain the frequency domain component distribution features of the business image, and fusing the color features of the business image with the frequency domain component distribution features to form a joint feature vector; inputting the joint feature vector and the preliminary watermark mask into a pre-trained gradient boosting tree model, and using the gradient boosting tree model to correct the preliminary watermark mask to obtain a corrected watermark mask; and filling the business image with color based on the watermark area indicated by the corrected watermark mask to obtain a target image after watermark removal. According to an embodiment of this application, watermark detection is first performed on the three-channel image of the business image based on preset watermark color feature conditions to generate a preliminary watermark mask to initially divide the watermark region in the business image; then, joint modeling is performed based on the frequency domain component distribution features and color features of the business image, and the preliminary watermark mask is corrected using a gradient boosting tree model to obtain a more accurate corrected watermark mask; finally, the watermark region in the business image is filled based on the corrected watermark mask to obtain the target image after watermark removal, thereby improving the accuracy of watermark removal.

[0034] The following description, in conjunction with the accompanying drawings and textual description, will elaborate on the above-mentioned operations in order to achieve at least one objective of this application.

[0035] Figure 1 The illustrations depict application scenarios of watermark removal methods, apparatuses, devices, media, and program products according to embodiments of this application. For example... Figure 1 As shown, application scenario 100 according to an embodiment of this application may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 serves as a medium for providing a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired or wireless communication links or fiber optic cables. For example, a user can use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 through the network 104 to receive or send information, etc.

[0036] The first terminal device 101, the second terminal device 102, and the third terminal device 103 can be electronic devices such as smartphones, wearable devices, personal computers, intelligent voice interaction devices, smart home appliances, intelligent vehicles, in-vehicle terminals, aircraft, unmanned vending terminals, and extended reality devices. Extended reality devices can include virtual reality devices, augmented reality devices, and mixed reality devices. A client application for the target application can be installed and run on the terminal devices. This target application can include, but is not limited to, financial transaction applications, payment applications, shopping applications, web browser applications, search applications, instant messaging tools, email clients, and social media platform software (these are just examples). Furthermore, this application embodiment does not limit the form of the target application, and it can include, but is not limited to, applications, mini-programs, etc., installed on the terminal devices, and can also be in the form of web pages.

[0037] Server 105 can be a server providing various services, such as a backend management server supporting websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103 (this is just an example). The backend management server can analyze and process received user requests and other data, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal devices. The server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services such as cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks, and basic cloud computing services such as big data. The server can be the backend server of the aforementioned target application, used to provide backend services to the clients of the target application.

[0038] It should be noted that the watermark removal method provided in this application embodiment can generally be executed by server 105 and / or terminal devices 101-103. Accordingly, the watermark removal device provided in this application embodiment can generally be set in server 105 and / or terminal devices 101-103. For ease of description, server 105 will be used as the method execution subject in the following description.

[0039] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0040] The following is based on Figure 1 The scenario described herein provides a detailed description of the watermark removal method provided in the embodiments of this application.

[0041] Figure 2A flowchart illustrating a first watermark removal method according to an embodiment of this application is shown schematically. Figure 2 As shown, the watermark removal method 200 according to the embodiments of this application may include steps S210 to S250.

[0042] In step S210, the business image to be watermarked is acquired and converted into a three-channel image.

[0043] In embodiments of this application, the business image to be watermarked may include banking business images such as invoices, bills, bank statement scans, and paper contracts. A three-channel image is an image including three color channels: R (red channel), G (green channel), and B (blue channel).

[0044] Before acquiring the aforementioned business images, user consent or authorization can be obtained. For example, before step S210, a request to acquire user information can be sent to the user. If the user consents or authorizes the acquisition of user information, step S210 is executed.

[0045] In the embodiments of this application, server 105 can receive image file data uploaded by users, such as by selecting local files or obtaining them via network links, without limitation. After obtaining the business image to be watermarked, server 105 can first preprocess the business image. Server 105 can first convert the business image into an array format, and then use the decoding function in the image processing library to decode the array format business image into a color image that conforms to the data structure of the image processing library. For example, the business image can be converted from the BGR color space to the RGB color space, and the R, G, and B channels can be separated.

[0046] In step S220, watermark detection is performed on the three-channel image based on preset watermark color feature conditions, and a preliminary watermark mask is generated based on the detection results.

[0047] In embodiments of this application, the watermark color feature conditions can be constructed based on at least one of the following factors: the relationship between the channel value of a pixel in each channel and the corresponding channel threshold, the relationship between the color variance of the local region where the pixel is located and the local variance threshold, and the relationship between the difference between the channel value of a pixel in each channel and the channel mean of the corresponding channel in the neighboring background region and the pixel difference threshold.

[0048] The watermark detection result indicates whether each pixel in the image is within the watermark region. For pixels that meet the watermark color feature conditions, the detection result is that the pixel is within the watermark region; for pixels that do not meet the watermark color feature conditions, the detection result is that the pixel is within the background region. The preliminary watermark mask is a specific division of each pixel in the image based on the watermark detection result, indicating whether each pixel is classified as a watermark region pixel or a background region pixel. The preliminary watermark mask corresponding to each pixel can be marked as 0 or 255. For example, the preliminary watermark mask corresponding to a pixel classified as a watermark region can be marked as 255, and the preliminary watermark mask corresponding to a pixel classified as a background region can be marked as 0.

[0049] In the embodiments of this application, server 105 can calculate the values ​​of corresponding physical quantities for each pixel based on physical quantities such as channel threshold, local variance threshold, and pixel difference threshold set in the watermark color feature conditions, and further determine whether the watermark color feature conditions are met. It then determines one by one whether each pixel is a pixel in the watermark area, thereby achieving preliminary watermark detection. After obtaining the watermark detection results, server 105 can generate a preliminary watermark mask based on the watermark detection results.

[0050] In step S230, the service image is transformed in the frequency domain to obtain the frequency domain component distribution features of the service image, and the color features of the service image are fused with the frequency domain component distribution features to form a joint feature vector.

[0051] In the embodiments of this application, the frequency domain transformation can be a discrete cosine transform, and the frequency domain component distribution characteristics can characterize the aggregation and fluctuation of frequency domain components. For the watermark area, there is usually an abnormal aggregation of high-frequency components or abnormal fluctuation of low-frequency components. The above-mentioned frequency domain transformation can extract the energy anomaly characteristics of the watermark area, that is, it can show the frequency component distribution characteristics of the above-mentioned abnormal state. The color features are the channel values ​​of each pixel in the business image in the above three color channels.

[0052] Server 105 can perform frequency domain transformation on the business image to obtain the frequency domain component distribution features of the business image, and then fuse the color features of the business image with the frequency domain component distribution features to form a joint feature vector, so as to subsequently correct the initial watermark mask based on the joint feature vector.

[0053] In step S240, the joint feature vector and the initial watermark mask are input into the pre-trained gradient boosting tree model, and the initial watermark mask is corrected using the gradient boosting tree model to obtain the corrected watermark mask.

[0054] In the embodiments of this application, the gradient boosting tree model can adjust the weights of the frequency domain component distribution features and color features in the joint feature vector according to the watermark contrast represented by the joint feature vector. For example, when a low-contrast watermark is identified, the weight of the frequency domain component distribution features is increased; when a high-contrast watermark is identified, the weight of the color features is increased.

[0055] Server 105 can input the joint feature vector and the initial watermark mask into the gradient boosting tree model to obtain the adjusted watermark mask judgment threshold output by the gradient boosting tree model. For example, by adjusting the channel threshold, local variance threshold or pixel difference threshold set in the above watermark color feature conditions, the initial watermark mask can be corrected to obtain the corrected watermark mask.

[0056] In step S250, the service image is filled with color based on the watermark area indicated by the modified watermark mask to obtain the target image after watermark removal.

[0057] In the embodiments of this application, server 105 can determine whether each pixel in the service image is a watermark region pixel based on the mask value corresponding to each pixel in the corrected watermark mask, thereby determining the watermark region in the service image. Server 105 can fill the watermark region in the service image with white, or it can use other color filling strategies. For example, you can refer to the description of filling the watermark region based on color filling constraints below, which will not be detailed here.

[0058] In the embodiments of this application, after obtaining the target image with the watermark removed, the server 105 can determine the file format based on the file extension of the original business image, and then use the encoding function in the image processing library to encode the processed image into a byte stream. Finally, the server 105 can encapsulate the encoded byte stream, file format, and filename, and return it to the user for download.

[0059] According to an embodiment of this application, watermark detection is first performed on the three-channel image of the business image based on preset watermark color feature conditions to generate a preliminary watermark mask to initially divide the watermark region in the business image; then, joint modeling is performed based on the frequency domain component distribution features and color features of the business image, and the preliminary watermark mask is corrected using a gradient boosting tree model to obtain a more accurate corrected watermark mask; finally, the watermark region in the business image is filled based on the corrected watermark mask to obtain the target image after watermark removal, thereby improving the accuracy of watermark removal.

[0060] In some embodiments, the server 105 can construct color fill constraints for the watermark area and fill the watermark area based on the color fill constraints. Specifically, the server 105 can first construct color fill constraints for the watermark area, then determine the fill color of the watermark area based on the color fill constraints, and use the fill color to fill the watermark area to obtain the target image after removing the watermark.

[0061] In embodiments of this application, color filling constraints may include texture continuity conditions, color harmony conditions, and semantic rationality conditions. Server 105 can establish a fill color function for the watermark region as a function to be solved, and establish an equation to be solved based on the aforementioned color filling constraints to obtain the fill color of the watermark region. In one example, server 105 can obtain the fill color of the watermark region by solving a Dirichlet boundary value problem. By solving the corresponding equation, server 105 can obtain the fill color corresponding to each pixel in the watermark region, and then use the fill color to fill the watermark region.

[0062] According to the embodiments of this application, color filling of the watermark area based on the above-mentioned multiple color filling constraints can ensure the texture continuity, color harmony and semantic rationality of the watermark area and the neighboring background area after filling, thereby improving the visual quality of the target image obtained after removing the watermark.

[0063] In some embodiments, to ensure texture continuity between the filled watermark region and the neighboring background region, the local binary pattern of the filled watermark region can be made to maintain maximum similarity with the local binary pattern of the neighboring background region. Texture continuity conditions may include minimizing the chi-square distance between the histogram of the local binary pattern of the color-filled watermark region and the histogram of the local binary pattern of the neighboring background region.

[0064] For example, if the watermark region is Ω and its neighboring background region is bg, server 105 can calculate the local binary pattern histogram H of the pixels within Ω. Ω The local binary pattern histogram H of bg bg The texture continuity condition is to minimize H Ω With H bg The chi-square distance between them. The constraint formula for the texture continuity condition is as follows:

[0065]

[0066] Where K is the number of Local Binary Patterns (LBPs). It is a very small positive number.

[0067] To ensure color harmony, the fill color of the watermark area can be constrained to maintain statistical consistency with the color distribution of the neighboring background area in the LAB color space, avoiding abnormal color noise. Color harmony conditions can include: the fill color value in a preset color space must fall within the color distribution range of the neighboring background area. The preset color space includes the L (luminance) channel, A (primary chromaticity) channel, and B (secondary chromaticity) channel. The L channel describes the lightness or darkness of the color, the A channel describes the color's bias between red and green, and the B channel describes the color's bias between yellow and blue. The color distribution range is determined based on the color mean and standard deviation of the neighboring background area in the luminance, primary, and secondary chromaticity channels.

[0068] For example, the fill color values ​​in the L, A, and B channels are L, a, and b, respectively, and the mean color values ​​of the neighboring background region in the L, A, and B channels are μ, respectively. L,bg μ a,bg μ b,bg The color standard deviations of the neighboring background region in the L, A, and B channels are respectively: σ L,bg , σ a,bg , σ b,bg The constraint formula for color harmony conditions is as follows:

[0069]

[0070] in, , , These are the absolute values ​​of the difference between the fill color values ​​in the L, A, and B channels and the mean values ​​of the neighboring background area in the corresponding channels.

[0071] To ensure the semantic validity of the filled watermark region, if the watermark region is located in a smooth region, the gradient magnitude of the fill color is constrained to be as small as possible; if the watermark region is located in a region with regular texture, the gradient direction of the fill color is constrained to be consistent with the texture direction. Semantic validity conditions may include: when the watermark region is located in a smooth region, the gradient magnitude corresponding to the fill color is less than the gradient threshold for smooth regions; when the watermark region is located in a textured region, the direction difference between the gradient direction corresponding to the fill color and the texture direction of the textured region is less than a preset direction difference threshold.

[0072] For example, where I is the fill color of the watermark area, the constraint formula for the semantic rationality condition of the smooth area is as follows:

[0073]

[0074] in, The gradient magnitude corresponding to the fill color. This is the gradient threshold for the smooth region.

[0075] The constraint formula for the semantic rationality condition of texture regions is as follows:

[0076]

[0077] in, The gradient direction corresponding to the fill color. This represents the texture direction of the texture region.

[0078] According to the embodiments of this application, the fill color of the watermark area can be determined more accurately according to the above-mentioned color fill constraints, thereby further improving the visual quality of the target image obtained after removing the watermark.

[0079] In some embodiments, server 105 can establish a first Poisson equation to solve for the fill color of the watermark region. Server 105 can first use the fill color function of the watermark region as the function to be solved, and use the value of the fill color function at the boundary of the watermark region being equal to the color value of the neighboring background at the corresponding boundary as the boundary condition, and establish the first Poisson equation according to the color fill constraint condition; then solve the first Poisson equation to obtain the fill color corresponding to each pixel in the watermark region. The first Poisson equation is used to constrain the Laplace result of the fill color function to match the divergence of the guiding gradient field, which is determined based on the gradient of the neighboring background of the watermark region.

[0080] For example, the watermark area is Ω, and the boundary of the watermark area is... The fill color function for the watermark area is f, and f is in The value at that location is Neighborhood background The color value at that location is Server 105 uses f as the solution function, with As a boundary condition, the first Poisson equation is established based on the color-fill constraint as follows:

[0081]

[0082] in, For the Laplace operator, Here, v is the divergence operator, and v is the guiding gradient field. For the Laplacian result of the fill color function, To guide the divergence of the gradient field, the first Poisson equation is restricted to solving within Ω.

[0083] According to the embodiments of this application, a first Poisson equation is established to solve for the fill color of the watermark area, which can more accurately determine the fill color of the watermark area, thereby further improving the visual quality of the target image obtained after removing the watermark.

[0084] In some embodiments, server 105 can obtain reference images associated with the business image, such as reference images associated with the user's historical business records. In this case, server 105 can use the reference image for multi-image collaborative enhancement to further optimize the filling effect of the watermark area color filling. Server 105 can also perform secondary correction using the following second Poisson equation based on the preliminary filling color of the watermark area obtained by solving the first Poisson equation.

[0085] Server 105 can first determine the reference region in the reference image corresponding to the watermark region; then, it determines the fidelity term based on the preset fidelity coefficient and the color difference between the fill color function and the color function of the reference region; after obtaining the fidelity term, the fill color function of the watermark region is used as the function to be solved, and the value of the fill color function at the boundary of the watermark region is equal to the color value of the reference region at the corresponding boundary as the boundary condition. A second Poisson equation is then established based on the color fill constraint and the fidelity term; solving the second Poisson equation yields the fill color corresponding to each pixel in the watermark region. The second Poisson equation is used to make the Laplacian result of the fill color function equal to the combination of the divergence of the guiding gradient field and the fidelity term. The preset fidelity coefficient can also be called the balance coefficient. In one example, the preset fidelity coefficient can be 0.5.

[0086] For example, the watermark area is Ω, and the boundary of the watermark area is... The fill color function for the watermark area is f, and the color function for the reference area is f. ref f in The value at that location is f ref exist The color value at that location is The preset fidelity coefficient is λ. Server 105 first determines the fidelity coefficient based on λ and f and f ref The color difference between them (ff) ref Determine the fidelity term λ×(ff) ref Then, using f as the solution function, with For boundary conditions, based on the color fill constraint and λ×(ff) ref The second Poisson equation is established as follows:

[0087]

[0088] in, For the Laplace operator, Here, v is the divergence operator, and v is the guiding gradient field. For the Laplacian result of the fill color function, To guide the divergence of the gradient field, the second Poisson equation is solved within Ω.

[0089] In embodiments of this application, a corresponding operation entry can be provided to the user, allowing the user to choose to agree or refuse to obtain reference images associated with historically processed business records. That is, before obtaining the reference images, the user can provide an instruction to agree or refuse to obtain them through the corresponding operation entry. If the user agrees to obtain reference images associated with historically processed business records for watermark removal from the business images to be watermarked, the reference images are obtained for subsequent processing, namely, the steps described above—determining the fidelity term based on the reference region in the reference image, further establishing the second Poisson equation based on the fidelity term, and solving the equation—are executed. If the user refuses to obtain reference images associated with historically processed business records, the steps described above—establishing the first Poisson equation and solving the equation—are executed.

[0090] According to the embodiments of this application, by using a reference image associated with the business image for collaborative filling, the details and textures of the corresponding area in the reference image can be transferred to the watermark area of ​​the business image, thereby obtaining a repair effect that conforms to the business logic and improving the reliability of the target image obtained after removing the watermark.

[0091] In some embodiments, before performing watermark detection on the three-channel image based on preset watermark color feature conditions and generating a preliminary watermark mask based on the detection results, the server 105 can first classify the watermarks, and then perform strategy-based traffic splitting based on the watermark type, processing different types of watermarks separately. According to the watermark classification results, the server 105 can switch to an independent processing pipeline pre-configured for the corresponding watermark type, including watermark color feature conditions specific to that watermark type, mask generation rules, and corresponding gradient boosting tree model weights. The model corresponding to each watermark type is stored and updated independently.

[0092] Based on this, the embodiments of this application also provide a watermark removal method. Figure 3 A flowchart illustrating a second watermark removal method according to an embodiment of this application is shown schematically. Figure 3 As shown, the watermark removal method 300 according to an embodiment of this application may include steps S310 to S380.

[0093] In step S310, the service image to be watermarked is acquired and converted into a three-channel image. This is the same as step S210 and will not be described again here.

[0094] In step S320, watermark classification features of the business image are extracted, and the target watermark type of the business image is determined based on the watermark classification features.

[0095] The watermark classification features include at least one of the following: color uniformity feature, transparency feature, and edge gradient histogram feature. The color uniformity feature can be the standard deviation or variance of the color values ​​in a local area, and the transparency feature can be the channel value of the business image in the alpha channel.

[0096] After determining the target watermark type of the service image in step S320, steps S330 and S360 can be executed respectively to determine the target watermark color feature conditions corresponding to the target watermark type and the gradient boosting tree model corresponding to the target watermark type. In the embodiments of this application, the execution order of steps S330 and S360 is not important; step S330 can be executed first, step S360 can be executed first, or steps S330 and S360 can be executed simultaneously. There is no limitation on this.

[0097] In step S330, the target watermark color feature condition corresponding to the target watermark type is determined from the pre-stored watermark color feature conditions.

[0098] In embodiments of this application, server 105 may pre-store target watermark color feature conditions corresponding to different watermark types. After determining the target watermark type of the service image, server 105 can determine the target watermark color feature conditions corresponding to the target watermark type from the pre-stored watermark color feature conditions. Then, step S340 is executed to perform watermark detection on the pixels in the service image based on the target watermark color feature conditions, and generate a preliminary watermark mask based on the detection results.

[0099] In step S350, the service image is transformed in the frequency domain to obtain the frequency domain component distribution features of the service image. The color features of the service image are then fused with the frequency domain component distribution features to form a joint feature vector. This is the same as step S230 above and will not be described again here.

[0100] In step S360, among the pre-trained gradient boosting tree models, the gradient boosting tree model corresponding to the target watermark type is determined.

[0101] In the embodiments of this application, server 105 can pre-train sample images of different watermark types to obtain gradient boosting tree models corresponding to different watermark types. After determining the target watermark type of the business image, server 105 can select the gradient boosting tree model corresponding to the target watermark type from the pre-trained gradient boosting tree models. Then, after obtaining the joint feature vector in step S350, step S370 is executed to input the joint feature vector and the preliminary watermark mask into the gradient boosting tree model corresponding to the target watermark type, and to correct the preliminary watermark mask to obtain the corrected watermark mask.

[0102] In step S380, based on the watermark area indicated by the modified watermark mask, the service image is filled with color to obtain the target image after watermark removal. This is the same as step S250 above, and will not be described again here.

[0103] According to the embodiments of this application, watermarks are first classified, and then watermark detection is performed on the business image based on the watermark color feature conditions and gradient boosting tree model corresponding to different types, thereby obtaining a preliminary watermark mask and a corrected watermark mask in sequence, which further improves the accuracy of watermark removal.

[0104] In some embodiments, the watermark type may include at least one of the following: solid color text watermark, pattern / texture watermark, semi-transparent overlay watermark, and blended watermark. Blended watermarks may include artistic watermarks that are closely integrated with the subject. For different watermark types, the target watermark color feature conditions can be set as follows.

[0105] For solid color text watermarks, the target watermark color feature conditions may include: the channel values ​​of the pixel in the red, green and blue channels are greater than the corresponding solid color text watermark channel thresholds.

[0106] For example, the target watermark color feature conditions corresponding to a solid color text watermark may include: ((R>220) & (G>180) & (B>80)), where R, G, and B represent the channel values ​​of pixels in the red, green, and blue channels of the business image, respectively, and 220, 180, and 80 are the solid color text watermark channel thresholds corresponding to the red, green, and blue channels, respectively.

[0107] For patterned watermarks, the target watermark color feature conditions may include: the channel value of the pixel in the red channel and the green channel is greater than the corresponding patterned watermark channel threshold, and the channel value in the blue channel is less than the corresponding patterned watermark channel threshold.

[0108] For example, the target watermark color feature conditions corresponding to the pattern texture watermark may include: ((R>200) & (G>150) & (B<100)), where R, G, and B represent the channel values ​​of pixels in the red, green, and blue channels of the business image, respectively, and 200, 150, and 100 are the pattern texture watermark channel thresholds corresponding to the red, green, and blue channels, respectively.

[0109] For semi-transparent overlay watermarks, the target watermark color feature conditions may include: the channel values ​​of the pixel in the red, green and blue channels are greater than the corresponding semi-transparent overlay watermark channel thresholds, and the color variance of the local area where the pixel is located is less than the local variance threshold.

[0110] For example, the target watermark color feature conditions corresponding to the semi-transparent overlay watermark may include: ((R>180) & (G>180) & (B>180) & (local variance<15)), where R, G, and B represent the channel values ​​of pixels in the red, green, and blue channels of the business image, respectively; 180, 180, and 180 are the semi-transparent overlay watermark channel thresholds corresponding to the red, green, and blue channels, respectively; and 15 is the local variance threshold.

[0111] For fused watermarks, the target watermark color feature conditions may include: the difference between the channel value of a pixel in the red, green and blue channels and the channel mean value of the corresponding channel in the neighboring background area is less than the corresponding pixel difference threshold.

[0112] For example, the target watermark color feature conditions corresponding to the fused watermark may include: ((the difference between R and the mean value of the red channel of the neighboring background region < 30) & (the difference between G and the mean value of the green channel of the neighboring background region < 30) & (the difference between B and the mean value of the red channel of the neighboring background region < 30)), where 30, 30, and 30 are the pixel difference thresholds corresponding to the red channel, green channel, and blue channel, respectively.

[0113] According to the embodiments of this application, based on the characteristics of different types of watermarks, specific target watermark color feature conditions are set for each type, and a preliminary watermark mask is obtained based on the specific target watermark color feature conditions, which further improves the accuracy of the preliminary watermark mask generation, and further improves the accuracy of subsequent watermark removal.

[0114] In some embodiments, the server 105 may determine the target watermark type of the business image as follows: perform edge detection on the business image to determine the region in the business image whose color gradient is greater than a preset gradient threshold as the first candidate watermark region; cluster the colors of each pixel in the first candidate watermark region to determine the region where the number of pixels is less than a preset number threshold as the second candidate watermark region; extract watermark classification features from the second candidate watermark region, and determine the target watermark type of the business image based on the watermark classification features.

[0115] In embodiments of this application, server 105 may employ an edge detection algorithm to extract high-gradient regions in the business image as first candidate watermark regions. The aforementioned edge detection algorithm may include at least one of the following: Canny edge detection algorithm, Sobel edge detection algorithm, Prewitt edge detection algorithm, etc., without limitation.

[0116] In embodiments of this application, server 105 may use a color clustering method to determine the region where the number of pixels in each pixel in the first candidate watermark region is less than a preset threshold, i.e., the region with abnormal color distribution, and delineate this region as the second candidate watermark region. The aforementioned color clustering method may include at least one of the following: K-means clustering algorithm, mean-shift clustering algorithm, and nearest neighbor classification algorithm based on LAB color space, without limitation.

[0117] In the embodiments of this application, the server 105 can extract watermark classification features from the second candidate watermark region, and based on the watermark classification features, use a support vector machine (SVM) classifier to determine the target watermark type of the business image.

[0118] According to the embodiments of this application, a second candidate watermark region is determined by edge detection, color clustering, etc., and then the watermark classification features in the second candidate watermark region are extracted to determine the target watermark type of the business image, which improves the accuracy of watermark type determination and thus improves the accuracy of subsequent watermark removal.

[0119] In some embodiments, after the watermark removal process of the business image is completed to obtain the target image, the server 105 can use a pre-trained image inpainting model based on a generative adversarial network to perform secondary optimization on the target image. The generative adversarial network can adopt an edge-connect architecture. The image inpainting model based on the generative adversarial network can repair residual watermark traces, edge smoothness, and texture continuity in the target image.

[0120] In some embodiments, for images involving business scenarios, after obtaining the target image or completing the aforementioned secondary optimization, the server 105 can also perform semantic consistency verification on the business image and the obtained target image. The server 105 can perform wavelet transform on the business image and the target image respectively to obtain the first multi-scale frequency domain features of the business image and the second multi-scale frequency domain features of the target image; then, based on the first multi-scale frequency domain features and the second multi-scale frequency domain features, it performs integrity verification on the business image and the target image to obtain the semantic consistency verification result; if the semantic consistency verification result indicates that the key business semantics in the target image have not been destroyed, the target image is output.

[0121] In the embodiments of this application, server 105 uses wavelet transform to extract multi-scale frequency domain features of the service image and the target image respectively, enabling the parsing of semantic information in the image at multiple scales. Then, based on the first and second multi-scale frequency domain features, integrity verification is performed to detect whether the target image completely contains the key business semantics of the service image, avoiding damage to the key business semantics in the service image during the watermark removal process. If the semantic consistency verification result indicates that the key business semantics in the target image have not been destroyed, server 105 can output the target image. This ensures the reliability of the target image obtained after watermark removal.

[0122] In the embodiments of this application, the wavelet transform used to extract multi-scale frequency domain features of the service image and the target image respectively has a different function than the discrete cosine transform used in step S230. Step S230 uses discrete cosine transform to extract energy anomaly features of the watermark region in the service image, thereby assisting in the generation of the corrected watermark mask. Here, wavelet transform is used to extract semantic information at multiple scales in the target image and the service image, assisting in semantic consistency verification.

[0123] It is worth noting that the functional description here represents only a preferred implementation method and does not mean that wavelet transform or other frequency domain transform methods cannot be used in step S230 to extract frequency domain component distribution features to assist in the generation of the corrected watermark mask, nor does it mean that other transform methods cannot be used to extract semantic features in the business image and target image to assist in the semantic consistency verification process.

[0124] In some embodiments, after the watermark removal process of the business image is completed to obtain the target image, the server 105 can also perform a confidence score on the color filling of the watermarked area in the target image. The confidence score is calculated based on the following three indicators: color consistency of the watermarked area, color continuity of the boundary area, and the magnitude of color distribution change before and after watermark removal.

[0125] The watermark-removed region is the watermarked region in the target image after it has been filled. The color consistency of the watermark-removed region can be characterized by calculating the variance between the color values ​​of each pixel in the watermark-removed region and the color values ​​of each pixel in the original business image's watermarked region. The smaller the variance calculated above, the higher the confidence level of color filling in the watermarked region of the target image.

[0126] The color continuity of the boundary region can be characterized by calculating the mean square error of the gradient magnitude at the boundary of the watermarked region. The smaller the mean square error calculated above, the higher the confidence level of color filling of the watermarked region in the target image.

[0127] The magnitude of color distribution change before and after watermark removal can be characterized by calculating the chi-square distance between the three-channel color histograms of the entire image or the watermark-removed region. The smaller the calculated chi-square distance, the higher the confidence level of color filling in the watermarked region of the target image.

[0128] Server 105 can perform a weighted summation of the values ​​corresponding to the above three indicators to obtain the final comprehensive confidence score, and determine whether the comprehensive confidence score is lower than a preset confidence threshold. In one example, the preset confidence threshold can be 0.75. If the comprehensive confidence score is lower than the preset confidence threshold, server 105 can mark the watermark removal case as pending confirmation and send a request to the client to guide the user to evaluate or correct the difficult sample. The client can be the first terminal device 101, the second terminal device 102, or the third terminal device 103 mentioned above.

[0129] After receiving user feedback data or correction data, server 105 can classify and store it according to watermark type. Server 105 can use the user feedback data to update the gradient boosting tree model according to a preset first model update cycle, and can also incrementally train the generative adversarial network-based image inpainting model according to a preset second model update cycle. For example, the first model update cycle can be one week, and the second model update cycle can be one month. The updated gradient boosting tree model and the generative adversarial network-based image inpainting model replace the original model for subsequent watermark removal processing of business images.

[0130] The watermark removal method of this application will be specifically described below by way of preferred embodiments.

[0131] Figure 4 A flowchart illustrating a third watermark removal method according to an embodiment of this application is shown schematically. Figure 4 As shown, the watermark removal method 400 according to the embodiments of this application may include steps S410 to S460.

[0132] In step S410, image data is received and preprocessed.

[0133] As described in step S210 above, server 105 can receive the business image uploaded by the user, use the image processing library to convert the business image into an array, and decode it into a color image that conforms to the data structure of the image processing library, converting the initial business image into a three-channel image in RGB color space.

[0134] In step S420, watermark type identification and policy routing are performed.

[0135] As described in step S320 and its corresponding implementation, server 105 can perform edge detection on the business image, extract high-gradient regions in the business image as the first candidate watermark region, and then obtain the second candidate watermark region through color clustering. For the second candidate region, server 105 can extract watermark classification features such as color uniformity features, transparency features, and edge gradient histogram features, and use SVM to classify and identify watermark types, such as solid color text watermarks, pattern texture watermarks, semi-transparent overlay watermarks, and fused watermarks.

[0136] Based on the watermark classification results, server 105 can switch to an independent processing pipeline pre-configured for the corresponding watermark type, including watermark color feature conditions, mask generation rules, and corresponding gradient boosting tree model weights specific to that watermark type. The model for each watermark type is stored and updated independently.

[0137] In step S430, the watermark removal algorithm that combines color features and frequency domain features is executed.

[0138] As described in steps S330-S370 above, server 105 generates a preliminary watermark mask and a modified watermark mask sequentially according to the watermark color feature conditions corresponding to the watermark type and the gradient boosting tree model. Then, based on the modified watermark mask, the service image is filled with color to obtain the target image after watermark removal. Specifically, the first Poisson equation, the second Poisson equation, etc., can be used to solve for the filling color.

[0139] In step S440, image restoration and enhancement are performed.

[0140] After obtaining the target image, server 105 can use a pre-trained image inpainting model based on generative adversarial networks to perform secondary optimization on the target image, repairing residual watermark traces, edge smoothness, and texture continuity. In constructing the second Poisson equation, server 105 can also add a fidelity term constraint obtained from the reference image to achieve multi-image collaborative enhancement, transferring details and textures of corresponding regions in the reference image to the watermark region of the business image, obtaining a repair effect that conforms to business logic. Server 105 can also extract semantic features from the business image and the target image separately through wavelet transform, thereby performing semantic consistency verification before and after watermark removal.

[0141] In step S450, the image is encoded and output.

[0142] Server 105 can determine the output file format based on the file extension of the original business image, and then use the encoding function in the image processing library to encode the processed image into a byte stream. Finally, it can encapsulate the encoded byte stream, file format, and file name and return it to the user for download.

[0143] In step S460, confidence scoring, active learning feedback, and model optimization are performed.

[0144] Server 105 can assign a confidence score to the color fill of the watermark region in the target image. If the score is lower than a preset confidence threshold, it actively requests the user to provide feedback or correction. After receiving user feedback, server 105 stores the feedback data according to watermark type to avoid interference between feedback data corresponding to different watermark types. Server 105 uses the feedback data for each watermark type to update the corresponding model, such as updating the gradient boosting tree model according to a preset first model update cycle, and incrementally training the image inpainting model based on generative adversarial networks according to a preset second model update cycle. The updated model replaces the original model, and steps S430 and S440 are executed respectively to remove watermarks from subsequent business images.

[0145] The technical solution provided in this application, through its watermark type awareness and diversion processing strategy, can accurately adapt to the characteristics of different watermarks, avoiding inconsistent processing results caused by a single logic. For artworks where the watermark is closely integrated with the subject, semi-transparent watermarks, and low-contrast watermarks, the solution effectively identifies and accurately removes them by modeling the combined color features and frequency domain component distribution features, greatly improving the accuracy and reliability of watermark removal.

[0146] The technical solution provided in this application is highly automated. By changing the model optimization method that passively relies on user feedback, an active learning design based on confidence scores is introduced. The machine learning model outputs its confidence score for judging the watermark region when processing each business image. The active learning mechanism based on confidence scores efficiently collects high-quality user evaluation data and continuously optimizes the model corresponding to each watermark type. It has significant advantages for batch processing tasks, with stable and consistent processing results, unaffected by user operating experience.

[0147] The technical solution provided in this application is computationally friendly. By analyzing watermarked image data, specific watermark color feature conditions for different watermark types are obtained without the need for massive training data. The image inpainting model based on generative adversarial networks can be deployed on servers equipped with graphics processors, supporting higher-quality watermark removal effects.

[0148] Based on the above watermark removal method, embodiments of this application also provide a watermark removal device. The following will combine... Figure 5 The device is described in detail.

[0149] Figure 5 A schematic block diagram of a watermark removal apparatus according to an embodiment of this application is shown.

[0150] like Figure 5 As shown, the watermark removal device 500 in this embodiment includes an image acquisition module 510, a mask generation module 520, an image transformation module 530, a mask correction module 540, and an image filling module 550.

[0151] The image acquisition module 510 is used to acquire the business image to be watermarked and convert the business image into a three-channel image. In one embodiment, the image acquisition module 510 can be used to perform step S210 described above, which will not be repeated here.

[0152] The mask generation module 520 is used to perform watermark detection on the three-channel image based on preset watermark color feature conditions, and generate a preliminary watermark mask based on the detection results. In one embodiment, the mask generation module 520 can be used to perform step S220 described above, which will not be repeated here.

[0153] The image transformation module 530 is used to perform frequency domain transformation on the service image to obtain the frequency domain component distribution features of the service image, and to fuse the color features of the service image with the frequency domain component distribution features to form a joint feature vector. In one embodiment, the image transformation module 530 can be used to execute step S230 described above, which will not be repeated here.

[0154] The mask correction module 540 is used to input the joint feature vector and the preliminary watermark mask into a pre-trained gradient boosting tree model, and use the gradient boosting tree model to correct the preliminary watermark mask to obtain a corrected watermark mask. In one embodiment, the mask correction module 540 can be used to perform step S240 described above, which will not be repeated here.

[0155] The image filling module 550 is used to fill the service image with color based on the watermark area indicated by the corrected watermark mask, to obtain the target image after watermark removal. In one embodiment, the mask correction module 550 can be used to perform step S250 described above, which will not be repeated here.

[0156] According to an embodiment of this application, the image filling module 550 can be specifically used to construct color filling constraints for the watermark area, wherein the color filling constraints include: texture continuity conditions, color harmony conditions, and semantic rationality conditions; determine the filling color of the watermark area based on the color filling constraints; and fill the watermark area with the filling color to obtain the target image after removing the watermark.

[0157] According to embodiments of this application, the texture continuity condition may include: minimizing the chi-square distance between the local binary mode histogram of the watermark region after color filling and the local binary mode histogram of the neighboring background region of the watermark region; the color harmony condition may include: the color value of the filling color in a preset color space is within the color distribution range of the neighboring background region, wherein the preset color space includes a luminance channel, a first chrominance channel, and a second chrominance channel, and the color distribution range is determined based on the color mean and color standard deviation of the neighboring background region in the luminance channel, the first chrominance channel, and the second chrominance channel; the semantic rationality condition may include: when the watermark region is located in a smooth region, the gradient magnitude corresponding to the filling color is less than the gradient threshold of the smooth region; when the watermark region is located in a textured region, the directional difference between the gradient direction corresponding to the filling color and the texture direction of the textured region is less than a preset directional difference threshold.

[0158] According to an embodiment of this application, the image filling module 550 can be specifically used to use the fill color function of the watermark region as the function to be solved, and the value of the fill color function at the boundary of the watermark region being equal to the color value of the neighboring background at the corresponding boundary as the boundary condition, and to establish a first Poisson equation according to the color filling constraint condition; to solve the first Poisson equation to obtain the fill color corresponding to each pixel in the watermark region, wherein the first Poisson equation is used to constrain the Laplacian result of the fill color function to match the divergence of the guiding gradient field, and the guiding gradient field is determined according to the gradient of the neighboring background of the watermark region.

[0159] According to an embodiment of this application, the image filling module 550 can also be used to determine a reference region in the reference image corresponding to the watermark region when a reference image associated with the business image is obtained; determine a fidelity term based on the preset fidelity coefficient and the color difference between the filling color function and the color function of the reference region; use the filling color function of the watermark region as the function to be solved, and use the value of the filling color function at the boundary of the watermark region being equal to the color value of the reference region at the corresponding boundary as the boundary condition, and establish a second Poisson equation based on the color filling constraint condition and the fidelity term; solve the second Poisson equation to obtain the filling color corresponding to each pixel in the watermark region, wherein the second Poisson equation is used to make the Laplace result of the filling color function equal to the combination result of the divergence of the guiding gradient field and the fidelity term.

[0160] According to an embodiment of this application, the watermark removal device further includes a watermark classification module, which can be used to extract watermark classification features of the business image before performing watermark detection on the three-channel image based on preset watermark color feature conditions and generating a preliminary watermark mask based on the detection results, and to determine the target watermark type of the business image based on the watermark classification features. The watermark classification features include at least one of the following: color uniformity features, transparency features, and edge gradient histogram features. The mask generation module 520 can specifically be used to generate a mask based on pre-stored watermark color features. The conditions include determining the target watermark color feature conditions corresponding to the target watermark type; based on the target watermark color feature conditions, watermark detection is performed on the pixels in the business image, and a preliminary watermark mask is generated based on the detection results; the mask correction module 540 can be specifically used to determine the gradient boosting tree model corresponding to the target watermark type among multiple pre-trained gradient boosting tree models; the joint feature vector and the preliminary watermark mask are input into the gradient boosting tree model corresponding to the target watermark type to correct the preliminary watermark mask and obtain a corrected watermark mask.

[0161] According to embodiments of this application, the watermark type may include at least one of the following: solid color text watermark, pattern texture watermark, semi-transparent overlay watermark, and fused watermark; when the target watermark type is the solid color text watermark, the target watermark color feature condition includes: the channel values ​​of the pixel in the red channel, green channel, and blue channel are respectively greater than the corresponding solid color text watermark channel threshold; when the target watermark type is the pattern texture watermark, the target watermark color feature condition includes: the channel values ​​of the pixel in the red channel and green channel are respectively greater than the corresponding pattern texture watermark channel threshold, and the channel values ​​in the blue channel are also greater than the corresponding pattern texture watermark channel threshold. The channel value is less than the corresponding pattern texture watermark channel threshold; when the target watermark type is the semi-transparent overlay watermark, the target watermark color feature conditions include: the channel values ​​of the pixel in the red channel, green channel, and blue channel are respectively greater than the corresponding semi-transparent overlay watermark channel threshold, and the color variance of the local area where the pixel is located is less than the local variance threshold; when the target watermark type is the fused watermark, the target watermark color feature conditions include: the difference between the channel values ​​of the pixel in the red channel, green channel, and blue channel and the channel mean of the corresponding channel in the neighboring background area is respectively less than the corresponding pixel difference threshold.

[0162] According to an embodiment of this application, the watermark classification module can specifically be used to perform edge detection on the business image, determine the region in the business image whose color gradient is greater than a preset gradient threshold as a first candidate watermark region; cluster the colors of each pixel in the first candidate watermark region, determine the region where the number of pixels is less than a preset number threshold as a second candidate watermark region; extract watermark classification features from the second candidate watermark region, and determine the target watermark type of the business image based on the watermark classification features.

[0163] According to an embodiment of this application, the watermark removal device further includes an integrity verification module, which can be used to perform wavelet transform on the service image and the target image respectively after color filling of the service image based on the watermark area indicated by the modified watermark mask to obtain the target image after watermark removal, to obtain a first multi-scale frequency domain feature of the service image and a second multi-scale frequency domain feature of the target image; perform integrity verification on the service image and the target image based on the first multi-scale frequency domain feature and the second multi-scale frequency domain feature to obtain a semantic consistency verification result; and output the target image if the semantic consistency verification result indicates that the key business semantics in the target image have not been destroyed.

[0164] According to embodiments of this application, any multiple modules among the image acquisition module 510, mask generation module 520, image transformation module 530, mask correction module 540, and image filling module 550 can be combined into one module, or any one of these modules can be split into multiple modules. Alternatively, at least some of the functions of one or more of these modules can be combined with at least some of the functions of other modules and implemented in one module. According to embodiments of this application, at least one of the image acquisition module 510, mask generation module 520, image transformation module 530, mask correction module 540, and image filling module 550 can be at least partially implemented as hardware circuitry, such as field-programmable gate arrays, programmable logic arrays, systems-on-a-chip, systems-on-a-substrate, systems-on-package, application-specific integrated circuits, or any other reasonable means of integrating or packaging circuitry, or implemented in software, hardware, or firmware, or in any suitable combination of any of these three implementation methods. Alternatively, at least one of the image acquisition module 510, mask generation module 520, image transformation module 530, mask correction module 540, and image filling module 550 may be implemented at least partially as a computer program module, which can perform corresponding functions when the computer program module is run.

[0165] Figure 6 A block diagram schematically illustrates an electronic device suitable for implementing a watermark removal method according to an embodiment of this application.

[0166] like Figure 6 As shown, an electronic device 600 according to an embodiment of this application includes a processor 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory 602 or a program loaded from a storage portion 608 into a random access memory 603. The processor 601 may include, for example, a general-purpose microprocessor, an instruction set processor and / or an associated chipset and / or a dedicated microprocessor. The processor 601 may also include onboard memory for caching purposes. The processor 601 may include a single processing unit or multiple processing units for executing different steps of the method flow according to an embodiment of this application.

[0167] Random access memory 603 stores various programs and data required for the operation of electronic device 600. Processor 601, read-only memory 602, and random access memory 603 are interconnected via bus 604. Processor 601 executes various steps of the method flow according to embodiments of this application by executing programs in read-only memory 602 and / or random access memory 603. It should be noted that the programs may also be stored in one or more memories other than read-only memory 602 and random access memory 603. Processor 601 may also execute various steps of the method flow according to embodiments of this application by executing programs stored in said one or more memories.

[0168] According to embodiments of this application, the electronic device 600 may further include an input / output interface 605, which is also connected to a bus 604. The electronic device 600 may also include one or more of the following components connected to the input / output interface 605: an input section 606 including a keyboard, mouse, etc.; an output section 607 including a cathode ray tube, liquid crystal display, etc., and a speaker, etc.; a storage section 608 including a hard disk, etc.; and a communication section 609 including a network interface card, such as a local area network card, modem, etc. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the input / output interface 605 as needed. A removable medium 611, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 610 as needed so that computer programs read from it can be installed into the storage section 608 as needed.

[0169] Embodiments of this application also provide a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs, which, when executed, implement the watermark removal method according to the embodiments of this application.

[0170] According to embodiments of this application, the computer-readable storage medium can be a non-volatile computer-readable storage medium, such as including but not limited to: portable computer disks, hard disks, random access memory, read-only memory, erasable programmable read-only memory, portable compact disk read-only memory, optical storage devices, magnetic storage devices, or any suitable combination thereof. In embodiments of this application, the computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, according to embodiments of this application, the computer-readable storage medium may include the read-only memory 602 described above, and / or random access memory 603, and / or one or more memories other than read-only memory 602 and random access memory 603.

[0171] Embodiments of this application also include a computer program product comprising a computer program containing program code for performing the methods shown in the flowchart. When the computer program product is run on a computer system, the program code is used to enable the computer system to implement the watermark removal method provided in the embodiments of this application.

[0172] In one embodiment, the computer program may rely on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may also be transmitted and distributed in the form of signals over a network medium, and downloaded and installed via the communication section 609, and / or installed from the removable medium 611. The program code contained in the computer program can be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination thereof.

[0173] In embodiments of this application, the computer program can be downloaded and installed from a network via communication section 609, and / or installed from removable medium 611. When the computer program is executed by processor 601, it performs the functions defined in the system of embodiments of this application. According to embodiments of this application, the systems, devices, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0174] According to embodiments of this application, program code for executing the computer programs provided in the embodiments of this application can be written in any combination of one or more programming languages. Specifically, these computational programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. The program code can be executed entirely on the user's computing device, partially on the user's device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0175] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0176] Those skilled in the art will understand that the features described in the various embodiments of this application can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in this application. In particular, the features described in the various embodiments of this application can be combined and / or combined in various ways without departing from the spirit and teachings of this application. All such combinations and / or combinations fall within the scope of this application.

Claims

1. A method for removing watermarks, characterized in that, The method includes: Obtain the business image to be watermarked and convert the business image into a three-channel image; Based on preset watermark color feature conditions, watermark detection is performed on the three-channel image, and a preliminary watermark mask is generated based on the detection results. The service image is subjected to frequency domain transformation to obtain the frequency domain component distribution features of the service image, and the color features of the service image are fused with the frequency domain component distribution features to form a joint feature vector; The joint feature vector and the initial watermark mask are input into a pre-trained gradient boosting tree model, and the initial watermark mask is corrected using the gradient boosting tree model to obtain the corrected watermark mask. Based on the watermark area indicated by the modified watermark mask, the service image is filled with color to obtain the target image after watermark removal.

2. The method according to claim 1, characterized in that, The step of color-filling the service image based on the watermark area indicated by the modified watermark mask to obtain the target image after watermark removal includes: Color filling constraints are constructed for the watermark area, including: texture continuity conditions, color harmony conditions, and semantic rationality conditions. The fill color of the watermark area is determined based on the color fill constraint conditions. The watermark area is filled with the specified fill color to obtain the target image after watermark removal.

3. The method according to claim 2, characterized in that, The texture continuity condition includes minimizing the chi-square distance between the local binary pattern histogram of the watermark region after color filling and the local binary pattern histogram of the neighboring background region of the watermark region; The color harmony condition includes: the color value of the fill color in the preset color space is within the color distribution range of the neighboring background area, wherein the preset color space includes a luminance channel, a first chrominance channel and a second chrominance channel, and the color distribution range is determined based on the color mean and color standard deviation of the neighboring background area in the luminance channel, the first chrominance channel and the second chrominance channel; The semantic rationality conditions include: when the watermark area is located in a smooth area, the gradient magnitude corresponding to the fill color is less than the gradient threshold of the smooth area; when the watermark area is located in a textured area, the directional difference between the gradient direction corresponding to the fill color and the texture direction of the textured area is less than a preset directional difference threshold.

4. The method according to claim 2, characterized in that, Determining the fill color of the watermark region based on the color fill constraint includes: The fill color function of the watermark region is used as the function to be solved. The value of the fill color function at the boundary of the watermark region is equal to the color value of the neighboring background at the corresponding boundary as the boundary condition. The first Poisson equation is established according to the color fill constraint condition. Solving the first Poisson equation yields the fill color corresponding to each pixel in the watermark region. The first Poisson equation is used to constrain the Laplace result of the fill color function to match the divergence of the guiding gradient field, which is determined based on the neighborhood background gradient of the watermark region.

5. The method according to claim 2, characterized in that, If a reference image associated with the service image is obtained, the method further includes: Determine the reference region in the reference image that corresponds to the watermark region; The fidelity term is determined based on the preset fidelity coefficient and the color difference between the fill color function and the color function of the reference area. The fill color function of the watermark region is used as the function to be solved, and the value of the fill color function at the boundary of the watermark region is equal to the color value of the reference region at the corresponding boundary as the boundary condition. A second Poisson equation is established based on the color fill constraint condition and the fidelity term. Solving the second Poisson equation yields the fill color corresponding to each pixel in the watermark region. The second Poisson equation is used to make the Laplace result of the fill color function equal to the combination of the divergence of the guiding gradient field and the fidelity term.

6. The method according to claim 1, characterized in that, Before performing watermark detection on the three-channel image based on preset watermark color feature conditions and generating a preliminary watermark mask based on the detection results, the method further includes: Extract watermark classification features from the business image, and determine the target watermark type of the business image based on the watermark classification features, wherein the watermark classification features include at least one of the following: color uniformity features, transparency features, and edge gradient histogram features; The step of performing watermark detection on the three-channel image based on preset watermark color feature conditions and generating a preliminary watermark mask based on the detection results includes: determining a target watermark color feature condition corresponding to the target watermark type from pre-stored watermark color feature conditions; performing watermark detection on pixels in the business image based on the target watermark color feature conditions and generating a preliminary watermark mask based on the detection results. The step of inputting the joint feature vector and the preliminary watermark mask into a pre-trained gradient boosting tree model, and using the gradient boosting tree model to correct the preliminary watermark mask to obtain a corrected watermark mask, includes: determining, among multiple pre-trained gradient boosting tree models, a gradient boosting tree model corresponding to the target watermark type; inputting the joint feature vector and the preliminary watermark mask into the gradient boosting tree model corresponding to the target watermark type, and correcting the preliminary watermark mask to obtain a corrected watermark mask.

7. The method according to claim 6, characterized in that, The watermark type includes at least one of the following: solid color text watermark, pattern texture watermark, semi-transparent overlay watermark, and fused watermark; When the target watermark type is the solid color text watermark, the target watermark color feature conditions include: the channel values ​​of the pixel in the red channel, green channel and blue channel are respectively greater than the corresponding solid color text watermark channel threshold; When the target watermark type is the pattern texture watermark, the target watermark color feature conditions include: the channel value of the pixel in the red channel and the green channel is greater than the corresponding pattern texture watermark channel threshold, and the channel value in the blue channel is less than the corresponding pattern texture watermark channel threshold. When the target watermark type is the semi-transparent overlay watermark, the target watermark color feature conditions include: the channel values ​​of the pixel in the red channel, green channel and blue channel are greater than the corresponding semi-transparent overlay watermark channel threshold, and the color variance of the local area where the pixel is located is less than the local variance threshold. When the target watermark type is the fused watermark, the target watermark color feature conditions include: the difference between the channel value of the pixel in the red channel, green channel and blue channel and the channel mean value of the corresponding channel in the neighboring background area is less than the corresponding pixel difference threshold.

8. The method according to claim 6, characterized in that, The step of extracting watermark classification features from the business image and determining the target watermark type of the business image based on the watermark classification features includes: Edge detection is performed on the business image to determine the regions in the business image whose color gradient is greater than a preset gradient threshold, which are then used as the first candidate watermark regions. Cluster the colors of each pixel in the first candidate watermark region and determine the region where the number of pixels is less than a preset threshold as the second candidate watermark region. Watermark classification features are extracted from the second candidate watermark region, and the target watermark type of the business image is determined based on the watermark classification features.

9. The method according to claim 1, characterized in that, After color-filling the service image based on the watermark area indicated by the modified watermark mask to obtain the target image after watermark removal, the method further includes: Wavelet transform is performed on the service image and the target image respectively to obtain the first multi-scale frequency domain features of the service image and the second multi-scale frequency domain features of the target image; Based on the first multi-scale frequency domain features and the second multi-scale frequency domain features, integrity verification is performed on the service image and the target image to obtain semantic consistency verification results. If the semantic consistency verification result indicates that the key business semantics in the target image have not been destroyed, the target image is output.

10. A watermark removal device, characterized in that, The device includes: The image acquisition module is used to acquire the business image to be dewatered and convert the business image into a three-channel image; The mask generation module is used to perform watermark detection on the three-channel image based on preset watermark color feature conditions, and generate a preliminary watermark mask based on the detection results. The image transformation module is used to perform frequency domain transformation on the service image to obtain the frequency domain component distribution features of the service image, and to fuse the color features of the service image with the frequency domain component distribution features to form a joint feature vector. A mask correction module is used to input the joint feature vector and the initial watermark mask into a pre-trained gradient boosting tree model, and use the gradient boosting tree model to correct the initial watermark mask to obtain a corrected watermark mask; and The image filling module is used to fill the service image with color based on the watermark area indicated by the modified watermark mask to obtain the target image after watermark removal.

11. An electronic device, comprising: One or more processors; Memory, used to store one or more computer programs. The characteristic feature is that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1 to 9.

12. A computer-readable storage medium having a computer program or instructions stored thereon, characterized in that, When the computer program or instructions are executed by a processor, they implement the steps of the method according to any one of claims 1 to 9.

13. A computer program product, comprising a computer program or instructions, characterized in that, When the computer program or instructions are executed by a processor, they implement the steps of the method according to any one of claims 1 to 9.

Images