A watermark removal method and related apparatus

Abstract

The application discloses a watermark removing method and related device, and relates to the technical field of image processing, and comprises the following steps: acquiring a to-be-repaired image and target prompt information, performing watermark target detection on the to-be-repaired image to obtain a watermark region, performing watermark target segmentation on the watermark region according to the target prompt information to obtain a watermark pixel, obtaining a target mask image of the to-be-repaired image based on the watermark pixel, and performing watermark removing processing on the to-be-repaired image according to the target mask image to obtain a watermark-free image corresponding to the to-be-repaired image. More watermark information can be obtained based on the target prompt information, the accuracy of watermark target segmentation is improved, and the watermark removing effect is improved.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a watermark removal method and related apparatus. Background Technology

[0002] Images or videos sometimes contain watermarks. In some scenarios, it is necessary to remove the watermarks when using these images or videos.

[0003] For example, to obtain a high-quality image generation model, it is necessary to train it using high-quality, watermark-free images to avoid the training data containing watermarked images, which would cause the trained image generation model to tend to generate watermarked images. However, some images in open-source datasets on the internet contain watermarks, which need to be removed before they can be used to train the image generation model. Summary of the Invention

[0004] In view of the above problems, this application provides a watermark removal method and related apparatus to achieve the purpose of accurately removing watermarks from images. The specific solution is as follows:

[0005] The first aspect of this application provides a watermark removal method, including:

[0006] Obtain the image to be repaired and the target prompt information, wherein the target prompt information is information describing the morphological features of the watermark in the image to be repaired;

[0007] Perform watermark target detection on the image to be repaired to obtain the area where the watermark is located;

[0008] Based on the target prompt information, the watermark target area is segmented to obtain watermark pixels, and the target mask image of the image to be repaired is obtained based on the watermark pixels.

[0009] Watermark removal is performed on the image to be repaired and the target mask image to obtain a watermark-free image corresponding to the image to be repaired.

[0010] In one possible implementation, the step of performing watermark target detection on the image to be repaired to obtain the region where the watermark is located includes:

[0011] A target detection model is used to detect the watermark region in the image to be repaired, and the region description information of the watermark region is obtained.

[0012] The region where the watermark is located is obtained based on the region description information.

[0013] In one possible implementation, the step of segmenting the watermark target area based on the target prompt information to obtain watermark pixels includes:

[0014] A first target segmentation model is used to segment the watermark target area according to the target prompt information to obtain the watermark pixels. The first target segmentation model includes a first image encoding module, a first prompt information encoding module, and a first mask decoding module.

[0015] The step of using a first target segmentation model to segment the watermark target area based on the target prompt information to obtain the watermark pixels includes:

[0016] The first image encoding module maps the watermark region to the first target image feature space to obtain the first image embedding feature corresponding to the watermark region.

[0017] The first prompt information encoding module maps the target prompt information to the first target prompt feature space to obtain the first prompt embedding feature corresponding to the target prompt information.

[0018] The watermark pixels are obtained by the first mask decoding module performing watermark pixel segmentation processing based on the first image embedding features and the first prompt embedding features.

[0019] In one possible implementation, the step of segmenting the watermark target area based on the target prompt information to obtain watermark pixels includes:

[0020] A second target segmentation model is used to segment the watermark target area according to the target prompt information to obtain the watermark pixels. The second target segmentation model includes a second image encoding module, a second prompt information encoding module, a second mask decoding module, and a feature decoding module.

[0021] The step of using a second target segmentation model to segment the watermark target area based on the target prompt information to obtain the watermark pixels includes:

[0022] The second image encoding module maps the watermark region to the second target image feature space to obtain the second image embedding feature corresponding to the watermark region, and obtains the output token corresponding to the watermark region based on the second image embedding feature.

[0023] The second prompt information encoding module generates a first fusion embedding feature based on the target prompt information and the output token;

[0024] The second mask decoding module obtains the second fusion embedding feature based on the second image embedding feature and the first fusion embedding feature, and then obtains the mask feature based on the second fusion embedding feature.

[0025] The feature decoding module obtains the watermark pixels based on the second image embedding feature, the mask feature, and the second fusion embedding feature.

[0026] In one possible implementation, obtaining the watermark pixel by the feature decoding module based on the second image embedding feature, the mask feature, and the second fused embedding feature includes:

[0027] The information extraction module in the feature decoding module obtains image edge features and image global features based on the second image embedding features;

[0028] The feature fusion module in the feature decoding module fuses the image edge features, the image global features, and the mask features to obtain the target fused features;

[0029] The target perception features are obtained by the perception module in the feature decoding module based on the second fused embedded features;

[0030] The watermark pixels are obtained by performing a dot product operation on the target fusion features and the target perception features through the dot product operation module in the feature decoding module.

[0031] In one possible implementation, the step of performing watermark removal processing based on the image to be repaired and the target mask image to obtain a watermark-free image corresponding to the image to be repaired includes:

[0032] A first image restoration model is used to perform watermark removal processing on the image to be restored and the target mask image to obtain the watermark-free image. The first image restoration model includes a mask module and a first restoration module.

[0033] The step of using a first image restoration model to perform watermark removal processing on the image to be restored and the target mask image to obtain the watermark-free image includes:

[0034] The masking module performs watermark pixel masking on the image to be repaired based on the target mask image to obtain the masked image corresponding to the image to be repaired.

[0035] The first repair module performs watermark pixel reconstruction processing on the masked image to obtain the watermark-free image.

[0036] In one possible implementation, the step of performing watermark removal processing based on the image to be repaired and the target mask image to obtain a watermark-free image corresponding to the image to be repaired includes:

[0037] A second image restoration model is used to perform watermark removal processing on the image to be restored and the target mask image to obtain the watermark-free image. The second image restoration model includes a noise-adding module, a noise-removing module, and a second restoration module.

[0038] The step of using a second image restoration model to perform watermark removal processing based on the image to be restored and the target mask image to obtain the watermark-free image includes:

[0039] The noise-adding module adds noise to the image to be repaired based on the target mask image to obtain the noise diffusion features corresponding to the image to be repaired.

[0040] The noise diffusion features are denoised by random noise using the denoising module to obtain the target latent representation features corresponding to the image to be repaired.

[0041] The watermark-free image is obtained by decoding the latent representation features of the target through the second repair module.

[0042] A second aspect of this application provides a watermark removal device, comprising:

[0043] The data acquisition module is used to acquire the image to be repaired and the target prompt information, wherein the target prompt information is information describing the morphological features of the watermark in the image to be repaired;

[0044] The watermark target detection module is used to perform watermark target detection on the image to be repaired, and obtain the area where the watermark is located.

[0045] The mask image generation module is used to segment the watermark target area according to the target prompt information to obtain watermark pixels, and to obtain the target mask image of the image to be repaired based on the watermark pixels.

[0046] The watermark removal module is used to perform watermark removal processing on the image to be repaired and the target mask image to obtain a watermark-free image corresponding to the image to be repaired.

[0047] A third aspect of this application provides a computer program product including computer-readable instructions that, when executed on an electronic device, cause the electronic device to implement the watermark removal method of the first aspect or any implementation thereof.

[0048] A fourth aspect of this application provides an electronic device, including at least one processor and a memory connected to the processor, wherein:

[0049] The memory is used to store computer programs;

[0050] The processor is used to execute the computer program so that the electronic device can implement the watermark removal method of the first aspect or any implementation thereof.

[0051] The fifth aspect of this application provides a computer storage medium carrying one or more computer programs, which, when executed by an electronic device, enable the electronic device to implement the watermark removal method of the first aspect or any implementation thereof.

[0052] By utilizing the above technical solution, the watermark removal method provided in this application, considering the diverse forms of watermarks, and in order to provide more watermark details for the subsequent watermark target segmentation process, obtains target prompt information describing the morphological features of the watermark. First, watermark target detection is performed on the image to be repaired containing the watermark to obtain the watermark location area. Then, watermark target segmentation is performed on the watermark location area based on the target prompt information. The segmented watermark pixels better reflect the true form of the watermark. Based on the watermark pixels, a target mask image of the image to be repaired is obtained, making the target mask image more accurate. Then, watermark removal processing is performed based on the image to be repaired and the target mask image to obtain the watermark-free image corresponding to the image to be repaired. This can accurately remove the watermark from the image to be repaired, improving the watermark removal effect. Attached Figure Description

[0053] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale.

[0054] Figure 1 A schematic diagram of a system architecture is provided for this application;

[0055] Figure 2 A schematic diagram of an optional hardware structure for the terminal 100 provided in this application;

[0056] Figure 3 This application provides a schematic diagram of the structure of a server 200;

[0057] Figure 4 A flowchart illustrating a watermark removal method provided in this application;

[0058] Figure 5 This is a schematic diagram of the structure of a first-target segmentation model;

[0059] Figure 6 This is a schematic diagram of the structure of a second target segmentation model;

[0060] Figure 7This is a schematic diagram of the structure of a first image restoration model;

[0061] Figure 8 This is a schematic diagram of the structure of a second image restoration model;

[0062] Figure 9 This is a schematic diagram of the structure of a watermark removal device provided in this application;

[0063] Figure 10 This is a schematic diagram of the structure of an electronic device provided in this application. Detailed Implementation

[0064] The embodiments of this application are described below with reference to the accompanying drawings. The terminology used in the implementation section of this application is for explaining specific embodiments only and is not intended to limit the scope of this application.

[0065] The embodiments of this application will now be described with reference to the accompanying drawings. Those skilled in the art will recognize that, with technological advancements and the emergence of new scenarios, the technical solutions provided in the embodiments of this application are equally applicable to similar technical problems.

[0066] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms are interchangeable where appropriate; this is merely a way of distinguishing objects with the same attributes in the embodiments of this application. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, so that a process, method, system, product, or apparatus that comprises a series of elements is not necessarily limited to those elements, but may include other elements not explicitly listed or inherent to those processes, methods, products, or apparatuses.

[0067] See Figure 1 , Figure 1 A schematic diagram of a system architecture is shown. The system may include a terminal 100 and a server 200. The server 200 may include one or more servers (…). Figure 1 (The example includes a server), and the server 200 can provide the methods provided in the embodiments of this application to one or more terminals.

[0068] The terminal 100 may have an application installed on it. The application and webpage can provide an interface. The terminal 100 can receive relevant parameters input by the user on the interface and send the parameters to the server 200. The server 200 can obtain the processing result based on the received parameters and return the processing result to the terminal 100.

[0069] It should be understood that in some optional implementations, the terminal 100 can also complete the action of obtaining the processing result based on the received parameters on its own, without the need for the server to cooperate. This application embodiment is not limited to this.

[0070] The following description Figure 1 The product form of the mid-terminal 100;

[0071] The terminal 100 in this application embodiment can be a mobile phone, tablet computer, wearable device, vehicle device, augmented reality (AR) / virtual reality (VR) device, laptop computer, ultra-mobile personal computer (UMPC), netbook, personal digital assistant (PDA), etc., and this application embodiment does not impose any restrictions on it.

[0072] Figure 2 A schematic diagram of an optional hardware structure for terminal 100 is shown.

[0073] refer to Figure 2 As shown, the terminal 100 may include a radio frequency unit 110, a memory 120, an input unit 130, a display unit 140, a camera 150 (optional), an audio circuit 160 (optional), a speaker 161 (optional), a microphone 162 (optional), a headphone jack 163 (optional), a processor 170, an external interface 180, a power supply 190, and other components. Those skilled in the art will understand that... Figure 2 These are merely examples of terminals or multi-functional devices and do not constitute a limitation on terminals or multi-functional devices. They may include more or fewer components than shown in the illustration, or combine certain components, or use different components.

[0074] The input unit 130 can be used to receive input numerical or character information, and to generate key signal inputs related to user settings and function control of the portable multi-functional device. Specifically, the input unit 130 may include a touchscreen 131 (optional) and / or other input devices 132. The touchscreen 131 can collect touch operations performed by the user on or near it (such as operations performed by the user using fingers, knuckles, styluses, or any suitable object on or near the touchscreen), and drive the corresponding connection devices according to a pre-set program. The touchscreen can detect the user's touch actions, convert the touch actions into touch signals and send them to the processor 170, and can receive and execute commands sent by the processor 170; the touch signal includes at least touch point coordinate information. The touchscreen 131 can provide an input interface and an output interface between the terminal 100 and the user. In addition, various types of touchscreens, such as resistive, capacitive, infrared, and surface acoustic wave, can be used to implement the touchscreen. Besides the touchscreen 131, the input unit 130 may also include other input devices. Specifically, other input devices 132 may include, but are not limited to, one or more of the following: physical keyboard, function keys (such as volume control buttons, power buttons, etc.), trackball, mouse, joystick, etc.

[0075] Among them, the input device 132 can receive input data, etc.

[0076] The display unit 140 can be used to display information input by the user or information provided to the user, various menus of the terminal 100, interactive interfaces, file display, and / or playback of any multimedia file.

[0077] The memory 120 can be used to store instructions and data. The memory 120 may primarily include an instruction storage area and a data storage area. The data storage area can store various types of data, such as multimedia files and text. The instruction storage area can store software units such as operating systems, applications, and instructions required for at least one function, or subsets or extended sets thereof. It may also include non-volatile random access memory. It provides the processor 170 with hardware, software, and data resources for managing the computing device, supporting control software and applications. It is also used for storing multimedia files, as well as storing running programs and applications.

[0078] The processor 170 is the control center of the terminal 100. It connects various parts of the terminal 100 via various interfaces and lines. By running or executing instructions stored in the memory 120 and calling data stored in the memory 120, it performs various functions and processes data of the terminal 100, thereby controlling the terminal device as a whole. Optionally, the processor 170 may include one or more processing units; preferably, the processor 170 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 170. In some embodiments, the processor and memory can be implemented on a single chip; in some embodiments, they can also be implemented separately on independent chips. The processor 170 can also be used to generate corresponding operation control signals, send them to the corresponding components of the computing processing device, read and process data in the software, especially read and process data and programs in the memory 120, so that the various functional modules therein perform corresponding functions, thereby controlling the corresponding components to act according to the instructions.

[0079] The memory 120 can be used to store software code related to the watermark removal method, and the processor 170 can execute the steps of the watermark removal method. It can also schedule other units (such as the input unit 130 and the display unit 140 mentioned above) to achieve the corresponding functions.

[0080] The radio frequency unit 110 (optional) can be used for receiving and transmitting signals during information transmission or calls. For example, it can receive downlink information from the base station and process it for the processor 170; additionally, it can transmit uplink data to the base station. Typically, the RF circuit includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low-noise amplifier (LNA), a duplexer, etc. Furthermore, the radio frequency unit 110 can also communicate wirelessly with network devices and other devices. This wireless communication can use any communication standard or protocol, including but not limited to Global System for Mobile Communication (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Long Term Evolution (LTE), email, Short Messaging Service (SMS), etc.

[0081] In this embodiment of the application, the radio frequency unit 110 can send data to the server 200 and receive the processing results sent by the server 200.

[0082] It should be understood that the radio frequency unit 110 is optional and can be replaced with other communication interfaces, such as a network port.

[0083] The terminal 100 also includes a power supply 190 (such as a battery) that supplies power to various components. Preferably, the power supply can be logically connected to the processor 170 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system.

[0084] Terminal 100 also includes an external interface 180, which can be a standard Micro USB interface or a multi-pin connector, which can be used to connect terminal 100 to other devices for communication or to connect a charger to charge terminal 100.

[0085] Although not shown, terminal 100 may also include a flash, a Wireless Fidelity (WiFi) module, a Bluetooth module, sensors with various functions, etc., which will not be described in detail here. Some or all of the methods described below can be applied to, for example... Figure 2 In the terminal 100 shown.

[0086] The following description Figure 1 The product form of the mid-range server 200;

[0087] Figure 3 A structural diagram of a server 200 is provided, as follows: Figure 3 As shown, server 200 includes bus 201, processor 202, communication interface 203, and memory 204. Processor 202, memory 204, and communication interface 203 communicate with each other via bus 201.

[0088] Bus 201 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of representation, Figure 3 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0089] The processor 202 can be any one or more of the following processors: a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0090] Memory 204 may include volatile memory, such as random access memory (RAM). Memory 204 may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid state drive (SSD).

[0091] The memory 204 can be used to store software code related to the watermark removal method, and the processor 202 can execute the steps of the chip's watermark removal method, and can also schedule other units to achieve the corresponding functions.

[0092] It should be understood that the above-mentioned terminal 100 and server 200 may be centralized or distributed devices, and the processors (for example, the processor 170 and the processor 202) in the above-mentioned terminal 100 and server 200 may be hardware circuits (such as Application Specific Integrated Circuit (ASIC), Field-Programmable Gate Array (FPGA), general-purpose processor, DSP, microprocessor or microcontroller, etc.), or a combination of these hardware circuits. For example, the processor may be a hardware system with an instruction execution function, such as a CPU, a DSP, etc., or a hardware system without an instruction execution function, such as an ASIC, an FPGA, etc., or a combination of the above-mentioned hardware system without an instruction execution function and the hardware system with an instruction execution function.

[0093] The present application provides a watermark removal method. The watermark removal method according to embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0094] Referring to Figure 4 , Figure 4 it is a schematic flowchart of a watermark removal method provided by an embodiment of the present application, and the method may include:

[0095] Step S401: Acquire an image to be restored and target prompt information.

[0096] Wherein, the target prompt information is information describing morphological features of a watermark in the image to be restored, and the image to be restored is an image containing a watermark.

[0097] Optionally, the morphological feature may be one or more of a size feature, a shape feature, and a texture feature. Of course, the morphological feature may also be other features, which is not specifically limited in the present application.

[0098] Taking the shape feature as an example, the target prompt information may be: the watermark in the image to be restored is a Chinese character; or the target prompt information may be: the watermark in the image to be restored is the Chinese character "watermark", and so on.

[0099] Step S402: Perform watermark target detection on the image to be restored to obtain a region where the watermark is located.

[0100] In a possible implementation, watermark region detection may be performed on the image to be restored to obtain region description information of the region where the watermark is located, and then the region where the watermark is located is obtained based on the region description information.

[0101] Wherein, the region description information is information describing the region where the watermark is located in the image to be restored.

[0102] Optionally, the region description information describes the bounding box of the watermark. In this case, the region where the watermark is located refers to the region within the bounding box.

[0103] Optionally, the bounding box can be a rectangle, other regular borders, or irregular borders.

[0104] Taking a rectangle as an example, optionally, the rectangle can be the smallest bounding rectangle of the watermark.

[0105] Optionally, the region description information can also be the pixel coordinates of each boundary pixel within the region where the watermark is located.

[0106] Of course, the area description information can be other than that, and this application does not specify any particular limitation.

[0107] In one possible implementation, an object detection model can be used to detect the watermark region in the image to be repaired, thereby obtaining the region description information of the area where the watermark is located.

[0108] Optionally, the object detection model can be either the MMOCR (Multi-modal Open-Source OCR Toolkit) model or the CRAFT (Character Region Awareness For Text detection) model. Here, both MMOCR and CRAFT are models used for text detection.

[0109] Considering that the CRAFT model can detect almost all Chinese character watermarks compared to the MMOCR model, and has better detection performance for Chinese characters, this embodiment can optionally use the CRAFT model to detect the watermark in the image to be repaired if the target information indicates that the watermark in the image to be repaired is a Chinese character, and use the MMOCR model to detect the watermark in the image to be repaired if the target information indicates that the watermark in the image to be repaired is not a Chinese character.

[0110] Of course, the above-mentioned target detection model can be other than that, and this application does not impose any specific limitations.

[0111] Step S403: Perform watermark target segmentation on the watermark area according to the target prompt information to obtain watermark pixels, and obtain the target mask image of the image to be repaired based on the watermark pixels.

[0112] Specifically, in this embodiment, the watermark target in the image to be repaired can be segmented at the pixel level according to the target prompt information to obtain the segmented watermark pixels. Then, a target mask image of the image to be repaired is obtained based on the segmented watermark pixels. Here, watermark pixels refer to the pixels occupied by the watermark.

[0113] This application provides target prompting information so that when segmenting the watermark target in the area where the watermark is located, the real shape of the watermark is used as the segmentation target. The graphic shape formed by combining all the watermark pixels obtained by segmentation is closer to the real shape of the watermark, thus improving the accuracy of watermark target segmentation.

[0114] In this application embodiment, different mask values ​​can be set for pixels in the image to be repaired based on the segmented watermark pixels to obtain a target mask image of the image to be repaired. For example, if the mask value of the watermark pixels in the image to be repaired is 1 and the mask value of the non-watermark pixels is 0, a target mask image composed of 0 and 1 can be obtained.

[0115] Step S404: Perform watermark removal processing on the image to be repaired based on the target mask image to obtain a watermark-free image corresponding to the image to be repaired.

[0116] It should be understood that the pixel value of a watermark pixel is composed of the original pixel value superimposed with the pixel value of the watermark graphic. In this embodiment, the watermark removal process refers to removing the pixel value of the watermark graphic superimposed on the original pixel value.

[0117] The watermark removal method provided in this application takes into account the diverse forms of watermarks. To provide more watermark details for subsequent watermark target segmentation, this application obtains target cue information describing the morphological features of the watermark. First, watermark target detection is performed on the image to be repaired containing the watermark to obtain the watermark location area. Then, watermark target segmentation is performed on the watermark location area based on the target cue information. The segmented watermark pixels more accurately reflect the true form of the watermark. Based on the watermark pixels, a target mask image of the image to be repaired is obtained, making the target mask image more accurate. Finally, watermark removal processing is performed on the image to be repaired and the target mask image to obtain a watermark-free image corresponding to the image to be repaired. This method can accurately remove the watermark from the image to be repaired, improving the watermark removal effect.

[0118] In some embodiments of this application, the process of "step S403, segmenting the watermark target area according to the target prompt information to obtain watermark pixels" described above is introduced.

[0119] In this embodiment, two target segmentation models are provided to achieve watermark target segmentation. For ease of distinction, the two target segmentation models provided in this embodiment are referred to as the first target segmentation model and the second target segmentation model, respectively.

[0120] Therefore, in this embodiment, a first target segmentation model can be used to segment the watermark target area according to the target prompt information to obtain watermark pixels, or a second target segmentation model can be used to segment the watermark target area according to the target prompt information to obtain watermark pixels.

[0121] First, we will introduce the process of segmenting the watermark target area based on the target prompt information using the first target segmentation model to obtain the watermark pixels.

[0122] like Figure 5 The figure shown is a schematic diagram of a first target segmentation model.

[0123] Optionally, the first target segmentation model consists of a first image encoding module 11, a first prompt information encoding module 12, and a first mask decoding module 13.

[0124] In this embodiment, the watermark region can be mapped to the first target image feature space by the first image encoding module 11 to obtain the first image embedding feature corresponding to the watermark region. Then, the target prompt information can be mapped to the first target prompt feature space by the first prompt information encoding module 12 to obtain the first prompt embedding feature corresponding to the target prompt information. Finally, the watermark pixel segmentation process is performed by the first mask decoding module 13 based on the first image embedding feature and the first prompt embedding feature to obtain the segmented watermark pixels.

[0125] The aforementioned first target image feature space is a preset high-dimensional feature space. The first image encoding module 11 can determine the watermark location from the image to be repaired based on the region description information, and then map the watermark location to the first target image feature space so as to extract more watermark information from the watermark location. Thus, the first image embedding feature can contain more watermark information, improving the accuracy of subsequent watermark target segmentation. Similarly, the aforementioned first target prompt feature space is also a preset high-dimensional feature space. By mapping the target prompt information to the first target prompt feature space, more semantic information (i.e., watermark detail information) of the watermark can be retained in the first prompt embedding feature, further improving the accuracy of watermark target segmentation.

[0126] Optionally, the process of "segmenting watermark pixels by the first mask decoding module 13 according to the first image embedding feature and the first prompt embedding feature to obtain the segmented watermark pixels" may include: fusing the first image embedding feature and the first prompt embedding feature by the first mask decoding module to obtain the third fused embedding feature, and then decoding the third fused embedding feature to obtain the watermark pixels.

[0127] Here, the first target segmentation model is trained using the first training watermark region description information and the first training prompt information marked with the first training watermark pixels as training data.

[0128] Optionally, the training process of the first target segmentation model may include: mapping the first training watermark region to the feature space of the first target image to obtain the training image embedding features corresponding to the first training watermark region; performing convolution processing on the first training watermark pixels to obtain the mask embedding features corresponding to the training watermark pixels; concatenating the training image embedding features and the mask embedding features to obtain the training concatenated embedding features; mapping the first training prompt information to the feature space of the first target prompt information to obtain the training prompt embedding features corresponding to the first training prompt information; obtaining the predicted watermark pixels output by the pre-built model based on the training prompt embedding features and the training concatenated embedding features; calculating the loss value based on the predicted watermark pixels and the first training watermark pixels; and training the parameters of the pre-built model based on the loss value to obtain the first target segmentation model.

[0129] In this embodiment, the training data of the first target segmentation model contains a large number of watermarks of various forms, which enables the trained first target segmentation model to learn more information about the watermarks. As a result, the first target segmentation model has good performance in various situations such as unknown watermarks, unfamiliar watermarks, and fuzzy watermarks, and realizes the powerful zero-shot generalization ability of the first target segmentation model.

[0130] The following section describes the process of segmenting the watermark target area based on the target prompt information using the second target segmentation model to obtain the watermark pixels.

[0131] like Figure 6 The diagram shown is a structural schematic of a second target segmentation model.

[0132] Optionally, the second target segmentation model consists of a second image encoding module 21, a second prompt information encoding module 22, a second mask decoding module 23, and a feature decoding module 24.

[0133] In this embodiment, the watermark region can be mapped to the second target image feature space by the second image encoding module 21 to obtain the second image embedding feature corresponding to the watermark region. The output token corresponding to the watermark region is obtained based on the second image embedding feature. Then, the first fusion embedding feature is generated by the second prompt information encoding module 22 based on the target prompt information and the output token. Next, the second fusion embedding feature is obtained by the second mask decoding module 23 based on the second image embedding feature and the first fusion embedding feature. The mask feature is obtained based on the second fusion embedding feature. Finally, the segmented watermark pixels are obtained by the feature decoding module 24 based on the second image embedding feature, the mask feature, and the second fusion embedding feature.

[0134] Optionally, the process of "mapping the watermark region to the feature space of the second target image based on the region description information to obtain the second image embedding feature corresponding to the watermark region" may include: dividing the watermark region into multiple sub-images of the same size based on the region description information, mapping the multiple sub-images to the feature space of the second target image respectively to obtain the sub-image embedding features corresponding to the multiple sub-images respectively, and obtaining the second image embedding feature corresponding to the watermark region based on the sub-image embedding features corresponding to the multiple sub-images respectively.

[0135] The second target image feature space described above serves the same purpose as the first target image feature space mentioned earlier: to map the watermark region to a higher-dimensional feature space, thereby extracting more watermark information and improving the accuracy of subsequent watermark target segmentation. Here, the second target image feature space and the first target image feature space can be the same or different.

[0136] Unlike the first target segmentation model, the second target segmentation model obtains the output token corresponding to the watermark region based on the second image embedding features. Optionally, the process of obtaining the output token may include: for each feature in the second image embedding features, calculating the attention weight between the feature and all other features, multiplying the calculated attention weights by the corresponding other features and summing them to obtain the updated feature corresponding to the feature; and obtaining the output token based on the updated features corresponding to all features in the second image embedding features.

[0137] Optionally, the process of "generating a first fused embedding feature based on the target prompt information and the output token" may include: mapping the target prompt information to a second target prompt feature space to obtain a second prompt embedding feature corresponding to the target prompt information, and fusing the output token and the second prompt embedding feature to obtain the first fused embedding feature.

[0138] The second target suggestion feature space described above serves the same purpose as the first target suggestion feature space mentioned earlier: to map the input target suggestion information to a higher-dimensional feature space. This allows for the extraction of more semantic information from the input target suggestion information, thereby improving the accuracy of subsequent watermark target segmentation. Here, the second target suggestion feature space can be the same as or different from the first target suggestion feature space.

[0139] Optionally, the process of “obtaining the second fusion embedding feature based on the second image embedding feature and the first fusion embedding feature” may include: first performing self-attention calculation on the first fusion embedding feature, and then performing attention calculation on the self-attention result of the first fusion embedding feature and the second image embedding feature (this attention calculation process is a token-to-image and image-to-token attention calculation, which can better fuse the first fusion embedding feature and the second image embedding feature) to obtain the second fusion embedding feature.

[0140] In an optional embodiment, the feature decoding module 24 may include an information extraction module 241, a feature fusion module 242, a perception module 243, and a dot product operation module 244. Then, the process of "obtaining watermark pixels by the feature decoding module 24 based on the second image embedding features, mask features, and the second fused embedding features" may include: obtaining image edge features and image global features by the information extraction module 241 based on the second image embedding features; fusing the image edge features, image global features, and mask features by the feature fusion module 242 to obtain target fused features; obtaining target perceived features by the perception module 243 based on the second fused embedding features; and finally, performing a dot product operation on the target fused features and the target perceived features by the dot product operation module to obtain the watermark pixels.

[0141] Optionally, the information prompting module 241 may include an edge extraction network and a global extraction network. The edge extraction network is used to obtain image edge features based on the second image embedding features, and the global extraction network is used to obtain image global features based on the second image embedding features.

[0142] Optionally, the perception module 243 is a three-layer perceptron (MLP) that can generate dynamic convolutional kernels (i.e., target perception features) from the second fused embedding features. The dynamic convolutional kernels are spatially pointwise multiplied with the target fused features, thereby accurately correcting masking errors in the output token and obtaining high-quality watermark pixels.

[0143] Here, the second target segmentation model is trained using the second training watermark region description information and the second training prompt information marked with the second training watermark pixels as training data.

[0144] Optionally, the training data for the second object segmentation model can be the same as or different from the training data for the first object segmentation model.

[0145] It should be noted that, in addition to the first target segmentation model and the second target segmentation model mentioned above, other target segmentation models may also be used in this application, and this application does not impose any limitations.

[0146] In summary, this application uses a target detection model for pixel-level watermark target segmentation, which can accurately identify the pixels occupied by the watermark to be removed in the image to be repaired, improve the accuracy of the target mask image, and thus improve the watermark removal effect.

[0147] In some other embodiments of this application, the process of "step S404, performing watermark removal processing on the image to be repaired based on the target mask image to obtain the watermark-free image corresponding to the image to be repaired" is described.

[0148] To enable watermark removal based on the original image of the image to be repaired, this embodiment employs an image inpainting model capable of directly processing high-resolution images. This embodiment provides a first image inpainting model and a second image inpainting model. The first image inpainting model can be used to remove the watermark from the image to be repaired based on the target mask image, resulting in a watermark-free image. Alternatively, the second image inpainting model can be used to remove the watermark from the image to be repaired based on the target mask image, also resulting in a watermark-free image.

[0149] First, we will introduce the process of using the first image restoration model to remove the watermark from the image to be restored based on the target mask image, so as to obtain a watermark-free image.

[0150] like Figure 7 The image shown is a schematic diagram of a first image restoration model. Figure 7 The first image restoration model includes a mask module 31 and a first restoration module 32.

[0151] In this embodiment, the masking module 31 can perform watermark pixel masking processing on the image to be repaired based on the target mask image to obtain the masked image corresponding to the image to be repaired. Then, the first repair module 32 can perform watermark pixel reconstruction processing on the masked image to obtain a watermark-free image.

[0152] Optionally, watermark pixel masking refers to setting the watermark pixels in the image to be repaired to a preset pixel value, such as 0 pixels.

[0153] As mentioned earlier, "the pixel value of a watermark pixel is composed of the original pixel value superimposed with the pixel value of the watermark graphic." Therefore, the above watermark pixel reconstruction process is the process of reconstructing the original pixel value, for example, reconstructing the original pixel value from a 0 pixel value.

[0154] Optionally, the first repair module 32 may include: a downsampling module 321, a Fourier convolution residual module 322, and an upsampling module 323.

[0155] Therefore, the process of "reconstructing watermark pixels based on the masked image by the first repair module 32 to obtain a watermark-free image" can include: downsampling the masked image by the downsampling module 321 to obtain a downsampled image; performing at least one fast Fourier convolution on the downsampled image by the Fourier convolution residual module 322 to obtain a processed image, generating a residual image between the processed image and the downsampled image; and upsampling the residual image by the upsampling module 323 to obtain a watermark-free image.

[0156] In this embodiment, the first repair module 32 downsamples the masked image, so that the downsampled image can be optimized in a preset feature space, resulting in higher restoration of the watermark-free image and better watermark removal effect.

[0157] The following section describes the process of using the second image inpainting model to remove the watermark from the image to be inpainted based on the target mask image, resulting in a watermark-free image.

[0158] like Figure 8 The image shown is a schematic diagram of a second image restoration model. Figure 8 The second image restoration model includes a noise-adding module 41, a noise-reducing module 42, and a second restoration module 43.

[0159] In this embodiment, the noise-adding module 41 adds noise to the image to be repaired based on the target mask image to obtain the noise diffusion features corresponding to the image to be repaired. The noise-reducing module 42 uses random noise to denoise the noise diffusion features to obtain the target latent representation features corresponding to the image to be repaired. The second repair module 43 decodes the target latent representation features to obtain a watermark-free image.

[0160] Optionally, the process of "adding noise to the image to be repaired based on the target mask image to obtain the noise diffusion features corresponding to the image to be repaired" may include: generating initial latent representation features of the watermark pixels in the latent space of the image to be repaired based on the target mask image; applying preset noise to the initial latent representation features through at least one iteration to obtain the noise diffusion features corresponding to the image to be repaired. Here, the initial latent representation features are the low-dimensional representation of the image to be repaired in the latent space, and the latent space is obtained by compressing the image to be repaired using a preset autoencoder.

[0161] Optionally, the process of “using random noise to denoise the noise diffusion features to obtain the target latent representation features corresponding to the image to be repaired” may include: using random noise and a U-Net network to predict the preset noise added to the noise diffusion features, and denoising the noise diffusion features based on the predicted noise to obtain the target latent representation features corresponding to the image to be repaired.

[0162] This embodiment gradually increases noise in the latent space through noise addition processing, making the initial latent representation features increasingly difficult to identify. Then, through noise reduction processing, the target latent representation features that conform to the probability distribution are obtained. Finally, through the decoding process, the target latent representation features are decoded back to the original image space, so that the watermark in the image to be repaired can be accurately removed in the latent space, thus improving the watermark removal effect.

[0163] In summary, each image restoration model in this application embodiment can directly remove watermarks from the image to be restored without reducing the resolution of the image to be restored, thus improving the watermark removal effect.

[0164] The above describes a watermark removal method provided by the embodiments of this application. The following describes the apparatus for performing the above watermark removal method.

[0165] Please see Figure 9 , Figure 9 This is a schematic diagram of a watermark removal device provided in an embodiment of this application. Figure 9 As shown, the device may include:

[0166] The data acquisition module 501 is used to acquire the image to be repaired and the target prompt information, wherein the target prompt information is information describing the morphological features of the watermark in the image to be repaired;

[0167] The watermark target detection module 502 is used to perform watermark target detection on the image to be repaired, and to obtain the area where the watermark is located.

[0168] The mask image generation module 503 is used to segment the watermark target area according to the target prompt information to obtain watermark pixels, and obtain the target mask image of the image to be repaired based on the watermark pixels.

[0169] The watermark removal module 504 is used to perform watermark removal processing on the image to be repaired based on the target mask image, so as to obtain the watermark-free image corresponding to the image to be repaired.

[0170] In one possible implementation, the watermark target detection module described above can be used to: use a target detection model to detect the watermark target in the image to be repaired, obtain the region description information of the area where the watermark is located, and obtain the area where the watermark is located based on the region description information.

[0171] In one possible implementation, the watermark target detection module described above can be used to: use a first target segmentation model to segment the watermark target in the area where the watermark is located based on the target prompt information, and obtain the watermark pixels.

[0172] The first target segmentation model includes a first image encoding module, a first prompt information encoding module, and a first mask decoding module. Therefore, when the watermark target detection module uses the first target segmentation model to segment the watermark target area based on the target prompt information to obtain the watermark pixels, it can specifically be used for:

[0173] The first image encoding module maps the watermark region to the feature space of the first target image to obtain the first image embedding feature corresponding to the watermark region.

[0174] The target prompt information is mapped to the first target prompt feature space by the first prompt information encoding module to obtain the first prompt embedding feature corresponding to the target prompt information;

[0175] The watermark pixels are obtained by segmenting the watermark pixels using the first mask decoding module based on the first image embedding features and the first prompt embedding features.

[0176] In one possible implementation, the watermark target detection module described above can be used to: use a second target segmentation model to segment the watermark target in the area where the watermark is located based on the target prompt information, and obtain the watermark pixels.

[0177] The second target segmentation model includes a second image encoding module, a second prompt information encoding module, a second mask decoding module, and a feature decoding module.

[0178] Therefore, when the watermark target detection module uses the second target segmentation model to segment the watermark target area based on the target prompt information and obtain the watermark pixels, it can be specifically used for:

[0179] The second image encoding module maps the watermark region to the feature space of the second target image to obtain the second image embedding feature corresponding to the watermark region, and obtains the output token corresponding to the intermediate image based on the second image embedding feature.

[0180] The second prompt information encoding module generates the first fused embedding feature based on the target prompt information and the output token;

[0181] The second mask decoding module obtains the second fusion embedding feature based on the second image embedding feature and the first fusion embedding feature, and then obtains the mask feature based on the second fusion embedding feature.

[0182] The watermark pixels are obtained by the feature decoding module based on the second image embedding features, mask features, and second fused embedding features.

[0183] In one possible implementation, the process by which the watermark target detection module obtains the watermark pixels based on the second image embedding features, mask features, and second fused embedding features through the feature decoding module may include:

[0184] The information extraction module in the feature decoding module obtains the image edge features and the image global features based on the second image embedding features;

[0185] The feature fusion module in the feature decoding module fuses image edge features, image global features, and mask features to obtain target fused features;

[0186] The target perception features are obtained by the perception module in the feature decoding module based on the second fused embedded features;

[0187] The dot product operation module in the feature decoding module performs dot product operation on the target fusion feature and the target perception feature to obtain the watermark pixel.

[0188] In one possible implementation, the above-mentioned watermark removal module can be used to: use a first image restoration model to perform watermark removal processing on the image to be restored based on the target mask image, and obtain a watermark-free image.

[0189] The first image restoration model includes a mask module and a first restoration module.

[0190] Therefore, when the watermark removal module uses the first image restoration model to perform watermark removal processing on the image to be restored based on the target mask image to obtain a watermark-free image, it can be specifically used for:

[0191] The masking module performs watermark pixel masking on the image to be repaired based on the target mask image, and obtains the masked image corresponding to the image to be repaired.

[0192] The first repair module performs watermark pixel reconstruction processing on the masked image to obtain a watermark-free image.

[0193] In one possible implementation, the above-mentioned watermark removal module can be used to: use a second image restoration model to perform watermark removal processing on the image to be restored based on the target mask image, and obtain a watermark-free image.

[0194] The second image restoration model includes a noise-adding module, a noise-reducing module, and a second restoration module.

[0195] Therefore, when the watermark removal module uses the second image restoration model to remove the watermark from the image to be restored based on the target mask image, and obtains a watermark-free image, it can be specifically used for:

[0196] The noise-adding module adds noise to the image to be repaired based on the target mask image, thereby obtaining the noise diffusion features corresponding to the image to be repaired.

[0197] The noise diffusion feature is denoised by random noise using the denoising module, thereby obtaining the target latent representation feature corresponding to the image to be repaired.

[0198] The second repair module decodes the latent representation features of the target to obtain a watermark-free image.

[0199] The watermark removal device provided in this application corresponds to the watermark removal method described above. For details, please refer to the above description, which will not be repeated here.

[0200] This application also provides an electronic device in its embodiments. (See reference...) Figure 10 The diagram illustrates a structural schematic suitable for implementing the electronic device in the embodiments of this application. The electronic device in the embodiments of this application may include, but is not limited to, fixed terminals such as mobile phones, laptops, PDAs (personal digital assistants), PADs (tablet computers), desktop computers, etc. Figure 10 The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0201] like Figure 10 As shown, the electronic device may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage device 608 into a random access memory (RAM) 603. When the electronic device is powered on, the RAM 603 also stores various programs and data required for the operation of the electronic device. The processing unit 601, ROM 602, and RAM 603 are interconnected via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0202] Typically, the following devices can be connected to I / O interface 605: input devices 606 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 607 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 608 including, for example, memory cards, hard drives, etc.; and communication devices 609. Communication device 609 allows electronic devices to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 10 Electronic devices with various devices are shown, but it should be understood that it is not required to implement or have all of the devices shown. More or fewer devices may be implemented or have instead.

[0203] This application also provides a computer program product including computer-readable instructions, which, when executed on an electronic device, cause the electronic device to implement any of the watermark removal methods provided in this application.

[0204] This application also provides a computer-readable storage medium that carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement any of the watermark removal methods provided in this application.

[0205] It should also be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. In addition, in the device embodiment drawings provided in this application, the connection relationship between modules indicates that they have a communication connection, which can be implemented as one or more communication buses or signal lines.

[0206] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware, or it can be implemented by special-purpose hardware including application-specific integrated circuits, special-purpose CPUs, special-purpose memory, special-purpose components, etc. Generally, any function performed by a computer program can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be diverse, such as analog circuits, digital circuits, or special-purpose circuits. However, for this application, software program implementation is more often the preferred implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a readable storage medium, such as a computer floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk, or optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, training equipment, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0207] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product.

[0208] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, training device, or data center to another website, computer, training device, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device or data center that integrates one or more available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state drives (SSDs)).

Claims

1. A method for removing watermarks, characterized in that, include: Obtain the image to be repaired and the target prompt information, wherein the target prompt information is information describing the morphological features of the watermark in the image to be repaired; Perform watermark target detection on the image to be repaired to obtain the area where the watermark is located; Based on the target prompt information, the watermark target area is segmented to obtain watermark pixels, and the target mask image of the image to be repaired is obtained based on the watermark pixels. The watermark removal process is performed on the image to be repaired based on the target mask image to obtain a watermark-free image corresponding to the image to be repaired.

2. The watermark removal method according to claim 1, characterized in that, The step of performing watermark target detection on the image to be repaired to obtain the area where the watermark is located includes: A target detection model is used to detect the watermark region in the image to be repaired, and the region description information of the watermark region is obtained. The region where the watermark is located is obtained based on the region description information.

3. The watermark removal method according to any one of claims 1 to 2, characterized in that, The step of segmenting the watermark target area according to the target prompt information to obtain watermark pixels includes: A first target segmentation model is used to segment the watermark target area according to the target prompt information to obtain the watermark pixels. The first target segmentation model includes a first image encoding module, a first prompt information encoding module, and a first mask decoding module. The step of using a first target segmentation model to segment the watermark target area based on the target prompt information to obtain the watermark pixels includes: The first image encoding module maps the watermark region to the first target image feature space to obtain the first image embedding feature corresponding to the watermark region. The first prompt information encoding module maps the target prompt information to the first target prompt feature space to obtain the first prompt embedding feature corresponding to the target prompt information. The watermark pixels are obtained by the first mask decoding module performing watermark pixel segmentation processing based on the first image embedding features and the first prompt embedding features.

4. The watermark removal method according to any one of claims 1 to 2, characterized in that, The step of segmenting the watermark target area according to the target prompt information to obtain watermark pixels includes: A second target segmentation model is used to segment the watermark target area according to the target prompt information to obtain the watermark pixels. The second target segmentation model includes a second image encoding module, a second prompt information encoding module, a second mask decoding module, and a feature decoding module. The step of using a second target segmentation model to segment the watermark target area based on the target prompt information to obtain the watermark pixels includes: The second image encoding module maps the watermark region to the second target image feature space to obtain the second image embedding feature corresponding to the watermark region, and obtains the output token corresponding to the watermark region based on the second image embedding feature. The second prompt information encoding module generates a first fusion embedding feature based on the target prompt information and the output token; The second mask decoding module obtains the second fusion embedding feature based on the second image embedding feature and the first fusion embedding feature, and then obtains the mask feature based on the second fusion embedding feature. The feature decoding module obtains the watermark pixels based on the second image embedding feature, the mask feature, and the second fusion embedding feature.

5. The watermark removal method according to claim 4, characterized in that, The step of obtaining the watermark pixel by the feature decoding module based on the second image embedding feature, the mask feature, and the second fused embedding feature includes: The information extraction module in the feature decoding module obtains image edge features and image global features based on the second image embedding features; The feature fusion module in the feature decoding module fuses the image edge features, the image global features, and the mask features to obtain the target fused features; The target perception features are obtained by the perception module in the feature decoding module based on the second fused embedded features; The dot product operation module in the feature decoding module performs a dot product operation on the target fusion feature and the target perception feature to obtain the watermark pixel.

6. The watermark removal method according to any one of claims 1 to 5, characterized in that, The step of performing watermark removal processing on the image to be repaired based on the target mask image to obtain a watermark-free image corresponding to the image to be repaired includes: A first image restoration model is used to remove the watermark from the image to be restored based on the target mask image to obtain the watermark-free image. The first image restoration model includes a mask module and a first restoration module. The step of using a first image inpainting model to remove the watermark from the image to be inpainted based on the target mask image to obtain the watermark-free image includes: The masking module performs watermark pixel masking on the image to be repaired based on the target mask image to obtain the masked image corresponding to the image to be repaired. The first repair module performs watermark pixel reconstruction processing on the masked image to obtain the watermark-free image.

7. The watermark removal method according to any one of claims 1 to 5, characterized in that, The step of performing watermark removal processing on the image to be repaired based on the target mask image to obtain a watermark-free image corresponding to the image to be repaired includes: A second image restoration model is used to remove the watermark from the image to be restored based on the target mask image to obtain the watermark-free image. The second image restoration model includes a noise-adding module, a noise-removing module, and a second restoration module. The step of using a second image inpainting model to remove the watermark from the image to be inpainted based on the target mask image to obtain the watermark-free image includes: The noise-adding module adds noise to the image to be repaired based on the target mask image to obtain the noise diffusion features corresponding to the image to be repaired. The noise diffusion features are denoised by random noise using the denoising module to obtain the target latent representation features corresponding to the image to be repaired. The watermark-free image is obtained by decoding the latent representation features of the target through the second repair module.

8. A watermark removal device, characterized in that, include: The data acquisition module is used to acquire the image to be repaired and the target prompt information, wherein the target prompt information is information describing the morphological features of the watermark in the image to be repaired; The watermark target detection module is used to perform watermark target detection on the image to be repaired, and obtain the area where the watermark is located. The mask image generation module is used to segment the watermark target area according to the target prompt information to obtain watermark pixels, and to obtain the target mask image of the image to be repaired based on the watermark pixels. The watermark removal module is used to perform watermark removal processing on the image to be repaired based on the target mask image, so as to obtain a watermark-free image corresponding to the image to be repaired.

9. An electronic device, characterized in that, It includes at least one processor and a memory connected to the processor, wherein: The memory is used to store computer programs; The processor is used to execute the computer program to enable the electronic device to implement the watermark removal method as described in any one of claims 1 to 7.

10. A computer storage medium, characterized in that, The storage medium carries one or more computer programs that, when executed by an electronic device, enable the electronic device to implement the watermark removal method as described in any one of claims 1 to 7.

Images