Watermark processing method and device, electronic equipment, medium and program product

Abstract

The embodiment of the invention discloses a watermark processing method and device, electronic equipment, a medium and a program product. The method comprises the following steps: converting to-be-added watermark information corresponding to a target original picture into a target barcode; fusing the target original picture with the target barcode picture to generate a target picture carrying the to-be-added watermark information; the target barcode picture is composed of a plurality of target barcodes; the size of the target bar code picture is consistent with that of the target original picture; decoding the to-be-detected picture, and extracting a watermark picture in the to-be-detected picture; the watermark adding mode of the original picture corresponding to the to-be-detected picture is consistent with the watermark adding mode of the target original picture; and under the condition that a non-barcode region exists in the watermark picture, determining a tampered region corresponding to the to-be-detected picture based on the non-barcode region, and determining target watermark information corresponding to the to-be-detected picture based on the barcode region in the watermark picture.

Description

Technical Field

[0001] This specification relates to the field of computer technology, and in particular to a watermarking method, apparatus, electronic device, medium, and program product. Background Technology

[0002] With the rapid development of artificial intelligence generative model technology, the efficiency of image production and editing has been greatly improved. While people enjoy the convenience of technology, it also brings new security challenges to copyright tracing and content protection. For example, it is difficult to distinguish between real and fake images, the adverse effects caused by the spread of fake image news, and the misuse and alteration of artworks without authorization. Summary of the Invention

[0003] This specification provides a watermarking method, apparatus, electronic device, medium, and program product. It adds a watermark by fusing a target barcode image composed of multiple target barcodes with the original target image. Simultaneously, it uses decoding to extract the watermark image from the image to be detected, enabling localized image tampering location and information tracing. The above technical solution is as follows: Firstly, embodiments of this specification provide a watermark processing method, including: Convert the watermark information corresponding to the original target image into a target barcode; The original target image and the target barcode image are merged to generate a target image carrying the watermark information to be added; the target barcode image is composed of multiple target barcodes; the target barcode image has the same size as the original target image. The image to be detected is decoded to extract the watermark image from the image to be detected; the watermark addition method of the original image corresponding to the image to be detected is the same as the watermark addition method of the original image of the target. In the case where there are non-barcode areas in the watermarked image, the tampered area corresponding to the image to be detected is determined based on the non-barcode areas, and the target watermark information corresponding to the image to be detected is determined based on the barcode areas in the watermarked image.

[0004] In one possible implementation, after converting the watermark information corresponding to the original target image into a target barcode, and before fusing the original target image and the target barcode image to generate a target image carrying the watermark information, the method further includes: Based on the dimensions of the original target image, the target barcode is copied, and the copied target barcodes are combined to obtain the target barcode image.

[0005] In one possible implementation, the above-mentioned fusion of the original target image and the target barcode image to generate a target image carrying the watermark information to be added includes: The original target image and the target barcode image are input into the image embedding model. Based on the fusion of the original target image and the target barcode image, a target image carrying the watermark information to be added is generated. The image embedding model includes at least one of the following loss functions: YUV mean square error loss function, frequency domain loss function, and image difference loss function.

[0006] In one possible implementation, the weights corresponding to the luminance and chrominance channels in the YUV mean square error loss function are determined based on the human eye's sensitivity to luminance and chrominance; and / or, the weights corresponding to the frequency domain loss function are determined based on the frequency domain residual between the image generated by the image embedding model and the original input image.

[0007] In one possible implementation, the above-mentioned decoding of the image to be detected and extraction of the watermark image from the image to be detected includes: The image to be detected is input into the image decoding model for decoding to extract the watermark image from the image to be detected. The image decoding model is trained based on known noise information and sample noise images of the original barcode image. The sample noise images are obtained by adding noise to the sample generated image based on at least one attack method. The sample generated image is generated by the image embedding model based on the fusion of the original sample image and the original barcode image. The original barcode image carries the watermark information to be added to the original sample image.

[0008] In one possible implementation, determining the target watermark information corresponding to the image to be detected based on the barcode region in the watermark image includes: Extract the barcode information from the barcode area in the above watermark image; The above barcode information is combined to obtain the watermark barcode corresponding to the image to be detected. Based on the watermark barcode corresponding to the image to be detected, obtain the target watermark information corresponding to the image to be detected.

[0009] In one possible implementation, after decoding the image to be detected and extracting the watermark image from the image to be detected, the method further includes: If the watermarked image contains only a barcode area and there is an invalid barcode within that barcode area, the image to be detected is determined to be a cropped image; the invalid barcode is used to represent a barcode that cannot be properly recognized.

[0010] Secondly, embodiments of this specification provide a watermark processing apparatus, the apparatus comprising: The conversion module is used to convert the watermark information corresponding to the original target image into the target barcode; The generation module is used to merge the original target image and the target barcode image to generate a target image carrying the watermark information to be added; the target barcode image is composed of multiple target barcodes; the target barcode image has the same size as the original target image. The extraction module is used to decode the image to be detected and extract the watermark image from the image to be detected; the watermark addition method of the original image corresponding to the image to be detected is the same as the watermark addition method of the target original image. The determination module is used to determine the tampered area corresponding to the image to be detected based on the non-barcode area in the watermarked image, and to determine the target watermark information corresponding to the image to be detected based on the barcode area in the watermarked image.

[0011] Thirdly, embodiments of this specification provide an electronic device, including: a processor and a memory; The processor is connected to the memory. The aforementioned memory is used to store executable program code; The processor reads the executable program code stored in the memory to run a program corresponding to the executable program code, so as to execute the method provided in the first aspect of the embodiments of this specification.

[0012] Fourthly, embodiments of this specification provide a computer storage medium storing a plurality of instructions adapted for loading by a processor and executing the method provided by the first aspect of the embodiments of this specification or any possible implementation thereof.

[0013] Fifthly, embodiments of this specification provide a computer program product containing instructions that, when run on a computer or processor, cause the computer or processor to execute the watermarking method provided by the first aspect of the embodiments of this specification or any possible implementation thereof.

[0014] In this embodiment, on the one hand, by converting the watermark information to be added to the original target image into a target barcode, and fusing the original target image with a target barcode image composed of multiple target barcodes of the same size as the original target image, a target image carrying the watermark information to be added is generated. This utilizes multiple target barcodes as watermark information for full coverage fusion, increasing watermark capacity and avoiding the problem of image information traceability being compromised due to damage to a single target barcode, thus improving the ability of images to trace information through watermarks to a certain extent. On the other hand, by decoding the image to be detected and extracting the watermark image from the image to be detected, the watermark information corresponding to the original image of the image to be detected is added. The watermarking method is consistent with that of the original target image. In cases where non-barcode areas exist in the watermarked image, the tampered area corresponding to the image to be detected is determined based on these non-barcode areas, and the target watermark information corresponding to the image to be detected is determined based on the barcode areas in the watermarked image. This method utilizes the non-barcode areas extracted from the decoded watermarked image to accurately locate the tampered position in the image to be detected, overcoming the limitation of traditional watermarks which can only determine "whether it has been tampered with" but lacks the ability to locate the tampered area. Furthermore, the target watermark information (such as, but not limited to, the copyright owner) of the image to be detected can be traced using the barcode areas extracted from the decoded watermarked image. This simultaneously satisfies two user needs related to watermarking technology: information tracing and tampered area location. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of this specification, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 A schematic diagram of the architecture of a watermarking system provided for an exemplary embodiment of this specification; Figure 2 A flowchart illustrating a watermarking method provided for an exemplary embodiment of this specification; Figure 3 A schematic diagram illustrating the implementation process of a watermarking method provided as an exemplary embodiment of this specification; Figure 4 A flowchart illustrating another watermarking method provided as an exemplary embodiment of this specification; Figure 5 A schematic diagram of a watermarking device provided for an exemplary embodiment of this specification; Figure 6This is a schematic diagram of the structure of an electronic device provided as an exemplary embodiment of this specification. Detailed Implementation

[0017] The technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings.

[0018] The terms "first," "second," "third," etc., used in this specification, claims, and the foregoing drawings are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to such processes, methods, products, or apparatus.

[0019] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.), and signals involved in the embodiments of this specification are all authorized by the user or fully authorized by all parties, and the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. For example, the watermark information to be added and the original target image involved in this specification were obtained with full authorization.

[0020] Digital watermarking, as an effective copyright protection technology, embeds copyright or event information into images, videos, and audio without affecting the quality of the original content, thereby fulfilling the needs of tracing the source, content authentication, and copyright protection.

[0021] In relevant watermarking technologies, watermark information is generally embedded in bit sequences or through image steganography. However, the former has two drawbacks: firstly, the bit sequence lacks location information when merging with the image to be embedded (the original image), leading to extraction failure when a local area is maliciously tampered with, making it impossible to determine the source; secondly, bit values ​​are usually 0 or 1, so when decoding fails, it's difficult to determine whether the failure is due to the absence of a watermark or a failure in watermark bit extraction. The latter requires embedding a fixed image as a copyright marker, while the steganographic image must be predetermined, resulting in a limited number that cannot meet large-scale usage needs. Furthermore, the decoder needs to know the steganographic image in advance, which can easily lead to information leakage.

[0022] Based on this, embodiments of this specification provide a watermarking method. On one hand, by converting the watermark information to be added to the target original image into a target barcode, and fusing the target original image with a target barcode image composed of multiple target barcodes of the same size as the target original image, a target image carrying the watermark information to be added is generated. This utilizes multiple target barcodes as watermark information for full-coverage fusion, increasing watermark capacity and avoiding the problem of image information traceability failure due to damage to a single target barcode, thus improving the ability of images to trace information through watermarks to a certain extent. On the other hand, by decoding the image to be detected, the watermark image in the image to be detected is extracted. The image to be detected corresponds to the original... The watermarking method for the image is the same as that for the original target image. In cases where non-barcode areas exist in the watermarked image, the tampered area corresponding to the image to be detected is determined based on these non-barcode areas, and the target watermark information corresponding to the image to be detected is determined based on the barcode areas in the watermarked image. This approach utilizes the non-barcode areas extracted from the decoded watermarked image to accurately locate the tampered position in the image to be detected, overcoming the limitation of traditional watermarks which can only determine "whether it has been tampered with" but lacks the ability to locate the tampered area. Furthermore, the barcode areas extracted from the decoded watermarked image can be used to trace the target watermark information of the image to be detected (e.g., but not limited to the copyright owner). This simultaneously satisfies two user needs related to watermarking technology: information tracing and tampered area location.

[0023] Please refer to the following. Figure 1 , Figure 1 This is a schematic diagram of the architecture of a watermarking system provided for an exemplary embodiment of this specification. Figure 1 As shown, the watermarking system may include: a terminal 110 and a server 120. Wherein: Terminal 110 may include one or more user terminals. One or more user-version applications may be installed on terminal 110. Terminal 110 can establish a data relationship with a network and, through that network, establish a data connection with server 120. For example, it can send the original target image to be watermarked and the corresponding watermark information; send an image to be detected for tampering location detection and information tracing; receive a target image carrying the aforementioned watermark information; and receive the tampering location detection results (whether it has been tampered with, the tampered area, etc.) and information tracing results (target watermark information) corresponding to the image to be detected. Terminal 110 may be, but is not limited to, a mobile phone, tablet computer, laptop computer, or other device with user-version applications installed.

[0024] Optionally, users can directly upload the original target image to be watermarked and the watermark information to be added to the application on terminal 110. The application can directly convert the watermark information corresponding to the original target image into a target barcode, and merge the original target image with the target barcode image to generate a target image carrying the watermark information. The target barcode image consists of multiple target barcodes, and the size of the target barcode image is the same as that of the original target image. At the same time, users can also directly upload the image to be detected to the application on terminal 110. The application can decode the image to be detected, extract the watermark image from the image to be detected, and, if there is a non-barcode area in the watermark image, determine the tampered area corresponding to the image to be detected based on the non-barcode area, and determine the target watermark information corresponding to the image to be detected based on the barcode area in the watermark image. The watermark addition method for the original image corresponding to the image to be detected is the same as the watermark addition method for the original target image.

[0025] Server 120 can be a server that can provide various watermarking processes. Server 120 can be, but is not limited to, a hardware server, a virtual server, a cloud server, etc.

[0026] Optionally, after receiving the target original image sent by the terminal 110, the server 120 can convert the watermark information to be added to the target original image into a target barcode, and fuse the target original image with the target barcode image to generate a target image carrying the watermark information to be added. The target barcode image is composed of multiple target barcodes, and the target barcode image has the same size as the target original image. After receiving the image to be detected sent by the terminal 110, the server 120 can first decode the image to be detected, extract the watermark image from the image to be detected, and then, if there is a non-barcode area in the watermark image, determine the tampered area corresponding to the image to be detected based on the non-barcode area, and determine the target watermark information corresponding to the image to be detected based on the barcode area in the watermark image. The watermark addition method of the image to be detected corresponding to the original image is the same as the watermark addition method of the target original image.

[0027] It is understood that the above watermarking process is not limited to being executed by the server 120, but can also be executed by the terminal 110. This specification does not specifically limit this embodiment. All the following embodiments are illustrated by the example of the watermarking process being executed by the terminal 110.

[0028] The network can be a medium that provides a communication link between server 120 and terminal 110, or it can be the Internet, which includes network devices and transmission media, and is not limited thereto. The transmission media can be a wired link, such as, but not limited to, coaxial cable, fiber optic cable, and digital subscriber line (DSL), or a wireless link, such as, but not limited to, wireless fidelity (WIFI), Bluetooth, and mobile device networks.

[0029] Understandably, Figure 1 The number of terminals 110 and servers 120 in the watermarking system shown is merely an example. In a specific implementation, the watermarking system can contain any number of terminals 110 and servers. This specification does not specifically limit this. For example, but not limited to, terminal 110 can be a terminal cluster composed of multiple terminals, and server 120 can be a server cluster composed of multiple servers.

[0030] Next, combine Figure 1 This document describes the watermarking method provided in the embodiments of this specification. Please refer to the following for details. Figure 2 This is a flowchart illustrating a watermarking method provided in an exemplary embodiment of this specification. Figure 2 As shown, this watermarking method includes the following steps: S202, convert the watermark information corresponding to the original target image into a target barcode.

[0031] Specifically, the target original image can be, but is not limited to, the original carrier image to which the watermark is to be embedded (e.g., but not limited to, digital images in JPEG or PNG format). The watermark information to be added can be, but is not limited to, additional information to be hidden in the target original image, such as, but not limited to, copyright notices, user identification identifiers, or anti-counterfeiting codes, and can be, but is not limited to, text, numbers, binary sequences, etc. The target barcode can be, but is not limited to, a visual encoding form of the watermark information to be added, with a specific structure, such as, but not limited to, one-dimensional barcodes, two-dimensional barcodes (i.e., QR codes), etc.

[0032] After obtaining the original image to which a watermark needs to be added and the corresponding watermark information, the watermark information can be directly converted into a target barcode (such as, but not limited to, one-dimensional barcodes, QR codes, or custom matrix codes) using an encoding algorithm. The encoding process must ensure information reversibility, meaning the original information can be recovered without loss through the decoding algorithm. For example, when using QR code encoding, the watermark information can be, but is not limited to, converted into a matrix composed of black and white modules, with each module corresponding to a binary bit.

[0033] S204, merge the original target image with the target barcode image to generate a target image carrying the watermark information to be added. The target barcode image consists of multiple target barcodes and has the same size as the original target image.

[0034] Specifically, the aforementioned target barcode image is an image composed of multiple target barcodes stitched together, serving as the watermark carrier. After obtaining the target barcode, additive fusion (e.g., but not limited to alpha mixing) or frequency domain fusion (e.g., but not limited to DCT domain overlay) can be used to embed the target barcode image composed of multiple target barcodes into the original target image. For example, but not limited to, in the RGB channels, the pixel values ​​of the target barcode image are added to the corresponding pixels of the original target image according to weights (e.g., but not limited to 0.1 or 0.3, etc.) to generate a target image containing the watermark information to be added. Thus, by converting the discrete watermark information to be added into a structured image (target barcode image) with the same size as the original target image and fusing it with the original target image, it is ensured that the watermark is fully covered and invisible, while preserving the visual quality of the original target image.

[0035] Optionally, after converting the watermark information corresponding to the original target image into a target barcode in S202 above, and before fusing the original target image and the target barcode image in S204 above to generate a target image carrying the watermark information to be added, the watermark processing method may also include, but is not limited to, copying the target barcode based on the size of the original target image, and combining the copied multiple target barcodes to obtain a target barcode image, ensuring that the target barcode image can cover the entire original target image. During the copying process, the size and number of target barcodes can be adjusted according to the aspect ratio or specific area size of the original target image to match the size of the target barcode image with that of the original target image. Furthermore, when combining multiple target barcodes, different arrangement methods can be used, such as horizontal, vertical, or staggered arrangements, to increase the complexity and concealment of the target barcode image. In the aforementioned original target image, the multiple target barcodes can all be of the same type, or they can include multiple different barcodes. That is, the original target image can be composed of at least one one-dimensional barcode and at least one two-dimensional barcode corresponding to the watermark information to be added. The specific settings can be made according to actual needs, and this embodiment does not limit this. The sizes of the multiple target barcodes in the aforementioned original target image can be the same or different, and this embodiment does not limit this.

[0036] Optionally, such as Figure 3As shown, the process of fusing the original target image and the target barcode image to generate a target image carrying the watermark information to be added, as described in S204 above, may include, but is not limited to: inputting the original target image and the target barcode image into an image embedding model, and generating a target image carrying the watermark information to be added based on the fusion of the original target image and the target barcode image. The image embedding model may include, but is not limited to, at least one of the following loss functions: YUV mean square error loss function, frequency domain loss function, and image difference loss function. Among them, the YUV mean square error loss function is mainly used to measure the difference in brightness (Y) and chromaticity (U, V) between the generated target image and the original target image, ensuring that the overall visual effect of the target image is not seriously affected after embedding the target barcode image; the frequency domain loss function focuses on evaluating the changes of the original target image before and after embedding the target barcode image in the frequency domain space, so as to preserve the texture and detail information of the original target image; the image difference loss function further ensures the concealment of the target barcode image embedding and the integrity of the watermark information by calculating the pixel-level differences between the images before and after embedding the target barcode image. By combining at least one of the above loss functions, the image embedding model can preserve the visual quality of the original target image to the maximum extent while ensuring the readability of the target barcode in the generated target image, thereby achieving effective embedding of watermark information.

[0037] Furthermore, in the aforementioned YUV mean squared error loss function, the weights corresponding to the luminance channel and the chrominance channel are determined based on the human eye's sensitivity to luminance and chrominance; and / or, the weights corresponding to the aforementioned frequency domain loss function are determined based on the frequency domain residual between the image generated by the image embedding model and the input original image.

[0038] Specifically, the human eye is generally more sensitive to brightness than to color. Therefore, in the YUV mean squared error loss function, the weight corresponding to the brightness channel can be set relatively high to ensure that the generated target image has minimal variation in brightness, thus maintaining good visual quality. Meanwhile, the weights corresponding to the frequency domain loss function can be determined based on the degree of difference between the image generated by the image embedding model and the original input image in the frequency domain, to further preserve the texture and detail information of the original image. Through this weight setting, the image embedding model can better balance the concealment of the watermark (target barcode image) and the visual quality of the original target image while embedding watermark information, exhibiting good robustness.

[0039] S206, Decode the image to be detected and extract the watermark image from the image to be detected. The watermark addition method of the original image corresponding to the image to be detected is consistent with the watermark addition method of the target original image.

[0040] Specifically, the images to be detected are those requiring tampering location detection and / or information tracing. This means they may be watermarked images that have been tampered with or contaminated, and their integrity needs to be verified. The watermarked image is a barcode image extracted from the image to be detected, used for subsequent tampering detection and information tracing.

[0041] For example, if additive fusion is used when adding a watermark to the image to be detected, the barcode image (watermark image) in the image to be detected can be separated by inverse operation (e.g., but not limited to subtracting the estimated value of the original image); if frequency domain fusion is used when adding a watermark to the image to be detected, the barcode image (watermark image) in the image to be detected can be separated by inverse transform (e.g., but not limited to IDCT).

[0042] Optionally, such as Figure 3 As shown, the process of S206 above, which decodes the image to be detected and extracts the watermark image from it, may include, but is not limited to, the following: inputting the image to be detected into an image decoding model for decoding, and extracting the watermark image from the image to be detected. The image decoding model is trained based on known noise information and sample noise images of the original barcode image. By decoding the image to be detected using the image decoding model, it can effectively detect whether there is noise interference in the image to be detected, thereby accurately extracting the watermark image from the image to be detected, and precisely distinguishing the barcode region and the noise region in the extracted watermark image. The sample noise image is obtained by adding noise to the sample generated image using at least one attack method; the sample generated image is generated by the image embedding model based on the fusion of the original sample image and the original barcode image; the original barcode image carries the watermark information to be added to the original sample image. The noise information may include, but is not limited to, the added noise location information, noise attack method information, etc.

[0043] The image embedding model described above can be trained using deep learning techniques, such as Convolutional Neural Networks (CNNs) or Generative Adversarial Networks (GANs). This model fuses the original sample image with the original barcode image corresponding to the watermark information to be added, generating a sample image with watermark information. This sample image is then input into a noise layer, where at least one attack method is used, such as but not limited to image enhancement, noise reduction, blurring, compression, cropping, cropping, barcode scaling, and mask region manipulation (e.g., but not limited to original image replacement, SD1.5 inpainting, etc.), to add noise to the sample image, resulting in a noisy sample image. These noisy sample images are used to train an image decoding model to improve its robustness and accuracy in detecting noise interference. The image decoding model described above can be trained based on, but is not limited to, a barcode loss function. The barcode loss function is used to calculate the difference between the watermark image extracted by the image decoding model from the sample noise image and the original barcode image corresponding to the sample noise image. The training objective of the image decoding model can be, but is not limited to, ensuring that the difference between the barcode loss value and the corresponding noise information is within a preset range.

[0044] For example, since the barcode image is composed of 0 (black) and 255 (white), in order to distinguish the tampered watermark-free area from the watermark image, during the image decoding model training process, the grayscale value corresponding to the tampered area (noise area) of the sample noise image in the watermark image can be set to any value different from 0 (black) and 255 (white), such as, but not limited to, 200, 100, 150, 127, etc. To ensure the effectiveness of image tampering localization detection, during the image decoding model training process, the grayscale value of the noise-added area (tampered area) in the barcode image extracted from the sample noise image can be set to 127 (gray). This ensures that the watermark image extracted by the image decoding model can not only achieve tampering localization through the grayscale value (e.g., 127) area, but also achieve information traceability through the barcode area, that is, the barcode area composed of grayscale values ​​0 (black) and 255 (white).

[0045] Furthermore, to improve the accuracy and robustness of watermark image extraction, the image decoding model can be optimized by incorporating machine learning or deep learning algorithms. For example, deep learning models such as convolutional neural networks can be used to model the image decoding process, learning the distribution characteristics of noise in the image and the potential structure of the barcode in the watermark image through a large amount of training data, thereby achieving effective extraction of the watermark image. In addition, advanced technologies such as attention mechanisms can be introduced to further enhance the sensitivity and recognition ability of the image decoding model to watermark images and noise added to the image.

[0046] S208, when there is a non-barcode area in the watermark image, determine the tampered area corresponding to the image to be detected based on the non-barcode area, and determine the target watermark information corresponding to the image to be detected based on the barcode area in the watermark image.

[0047] Specifically, the aforementioned non-barcode areas can be, but are not limited to, regions in the watermark image that do not conform to the barcode structure rules. For example, regions whose grayscale values ​​differ from the grayscale values ​​0 (black) and 255 (white) that make up the barcode may have been tampered with by an attacker. The target watermark information is the original watermark content decoded from the barcode area of ​​the watermark image, used to verify the source or ownership of the image to be detected. If non-barcode areas exist in the watermark image, they are determined to be traces of tampering (i.e., tampered areas). Simultaneously, the corresponding target watermark information can also be obtained by decoding the barcode area in the watermark image. That is, by separating the barcode area and non-barcode area of ​​the watermark image extracted from the image to be detected, the dual functions of accurately locating the image tampering location and tracing the image information's origin can be achieved.

[0048] Optionally, in step S202 above, the process of determining the target watermark information corresponding to the image to be detected based on the barcode region in the watermark image may include, but is not limited to: extracting barcode information from the barcode region in the watermark image; combining the barcode information to obtain the watermark barcode corresponding to the image to be detected; and obtaining the target watermark information corresponding to the image to be detected based on the watermark barcode. This may involve, but is not limited to, first identifying regions in the watermark image that conform to barcode structure rules using image recognition technology, and then extracting the barcode information within those regions. Then, the identified barcode information is spliced ​​or combined according to preset rules or order to form a complete watermark barcode corresponding to the image to be detected. Finally, the watermark barcode can be decoded using a specific decoding algorithm to obtain the original watermark content, i.e., the target watermark information. This target watermark information can be used to verify the source or ownership of the image to be detected, ensuring the authenticity and legality of the image.

[0049] In this embodiment, on the one hand, by converting the watermark information to be added to the original target image into a target barcode, and fusing the original target image with a target barcode image composed of multiple target barcodes of the same size as the original target image, a target image carrying the watermark information to be added is generated. This utilizes multiple target barcodes as watermark information for full coverage fusion, increasing watermark capacity and avoiding the problem of image information traceability being compromised due to damage to a single target barcode, thus improving the ability of images to trace information through watermarks to a certain extent. On the other hand, by decoding the image to be detected and extracting the watermark image from the image to be detected, the watermark information corresponding to the original image of the image to be detected is added. The watermarking method is consistent with that of the original target image. In cases where non-barcode areas exist in the watermarked image, the tampered area corresponding to the image to be detected is determined based on these non-barcode areas, and the target watermark information corresponding to the image to be detected is determined based on the barcode areas in the watermarked image. This method utilizes the non-barcode areas extracted from the decoded watermarked image to accurately locate the tampered position in the image to be detected, overcoming the limitation of traditional watermarks which can only determine "whether it has been tampered with" but lacks the ability to locate the tampered area. Furthermore, the target watermark information (such as, but not limited to, the copyright owner) of the image to be detected can be traced using the barcode areas extracted from the decoded watermarked image. This simultaneously satisfies two user needs related to watermarking technology: information tracing and tampered area location.

[0050] Please refer to the following. Figure 4 This is a flowchart illustrating another watermarking method provided in an exemplary embodiment of this specification. Figure 4 As shown, this watermarking method includes the following steps: S402, convert the watermark information corresponding to the original target image into a target barcode.

[0051] Specifically, S402 is the same as S202 above, and will not be repeated here.

[0052] S404, merge the original target image with the target barcode image to generate a target image carrying the watermark information to be added. The target barcode image consists of multiple target barcodes and has the same size as the original target image.

[0053] Specifically, S404 is the same as S204 mentioned above, and will not be repeated here.

[0054] S406, decode the image to be detected, extract the watermark image from the image to be detected, and the watermark addition method of the original image corresponding to the image to be detected is consistent with the watermark addition method of the target original image.

[0055] Specifically, S406 is the same as S206 mentioned above, and will not be repeated here.

[0056] S408, when there is a non-barcode area in the watermark image, determine the tampered area corresponding to the image to be detected based on the non-barcode area, and determine the target watermark information corresponding to the image to be detected based on the barcode area in the watermark image.

[0057] Specifically, S408 is the same as S208 mentioned above, and will not be repeated here.

[0058] Please continue to refer to the following. Figure 4 ,like Figure 4 As shown, in step S406 above, after decoding the image to be detected and extracting the watermark image from the image to be detected, the watermark processing method may also include, but is not limited to, the following: S410, if the watermarked image contains only a barcode area and there is an invalid barcode within the barcode area, the image to be detected is determined to be a cropped image.

[0059] Specifically, the aforementioned invalid barcode is used to characterize barcodes that cannot be properly recognized, such as incomplete barcodes. If the watermark image contains only a barcode area, and an invalid barcode exists within that area, it can be determined that the image to be detected is a cropped image. Simultaneously, based on the location information of the invalid barcode, the cropped area of ​​the image to be detected relative to the original image can be determined, and this cropped area is considered the tampered area.

[0060] Please refer to the following. Figure 5 , Figure 5 This is a schematic diagram of a watermarking device provided as an exemplary embodiment of this specification. Figure 5 As shown, the watermark processing device 500 includes: The conversion module 510 is used to convert the watermark information to be added to the original target image into a target barcode. The generation module 520 is used to merge the original target image and the target barcode image to generate a target image carrying the watermark information to be added; the target barcode image is composed of multiple target barcodes; the target barcode image has the same size as the original target image. The extraction module 530 is used to decode the image to be detected and extract the watermark image from the image to be detected; the watermark addition method of the original image corresponding to the image to be detected is the same as the watermark addition method of the target original image. The determining module 540 is used to determine the tampered area corresponding to the image to be detected based on the non-barcode area in the watermarked image, and to determine the target watermark information corresponding to the image to be detected based on the barcode area in the watermarked image.

[0061] In one possible implementation, the watermark processing device 500 further includes: The copy and combine module is used to copy the target barcode based on the size of the original target image, and combine the copied target barcodes to obtain a target barcode image.

[0062] In one possible implementation, the above-mentioned generation module 520 is specifically used for: The original target image and the target barcode image are input into the image embedding model. Based on the fusion of the original target image and the target barcode image, a target image carrying the watermark information to be added is generated. The image embedding model includes at least one of the following loss functions: YUV mean square error loss function, frequency domain loss function, and image difference loss function.

[0063] In one possible implementation, the weights corresponding to the luminance and chrominance channels in the YUV mean square error loss function are determined based on the human eye's sensitivity to luminance and chrominance; and / or, the weights corresponding to the frequency domain loss function are determined based on the frequency domain residual between the image generated by the image embedding model and the original input image.

[0064] In one possible implementation, the extraction module 530 is specifically used for: The image to be detected is input into the image decoding model for decoding to extract the watermark image from the image to be detected. The image decoding model is trained based on known noise information and sample noise images of the original barcode image. The sample noise images are obtained by adding noise to the sample generated image based on at least one attack method. The sample generated image is generated by the image embedding model based on the fusion of the original sample image and the original barcode image. The original barcode image carries the watermark information to be added to the original sample image.

[0065] In one possible implementation, the determining module 540 is specifically used for: Extract the barcode information from the barcode area in the watermark image; combine the barcode information to obtain the watermark barcode corresponding to the image to be detected; obtain the target watermark information corresponding to the image to be detected based on the watermark barcode.

[0066] In one possible implementation, the determining module 540 is further configured to: If the watermarked image contains only a barcode area and there is an invalid barcode within that barcode area, the image to be detected is determined to be a cropped image; the invalid barcode is used to represent a barcode that cannot be properly recognized.

[0067] The division of modules in the above-described watermarking device is for illustrative purposes only. In other embodiments, the watermarking device can be divided into different modules as needed to complete all or part of the functions of the watermarking device. The implementation of each module in the watermarking device provided in the embodiments of this specification can be in the form of a computer program. This computer program can run on a terminal or server. The program modules constituted by this computer program can be stored in the memory of the terminal or server. When the computer program is executed by a processor, it implements all or part of the steps of the watermarking method described in the embodiments of this specification.

[0068] Please refer to the following. Figure 6 , Figure 6 This is a schematic diagram of the structure of an electronic device provided as an exemplary embodiment of this specification. For example... Figure 6 As shown, the electronic device 600 may include: at least one processor 610, at least one communication bus 620, a user interface 630, at least one network interface 640, and a memory 650. The communication bus 620 can be used to enable communication between the aforementioned components.

[0069] The user interface 630 may include a display screen and a camera. Optionally, the user interface 630 may also include a standard wired interface and a wireless interface.

[0070] The network interface 640 may optionally include a Bluetooth module, a Near Field Communication (NFC) module, a Wireless Fidelity (Wi-Fi) module, etc.

[0071] The processor 610 may include one or more processing cores. The processor 610 connects to various parts within the electronic device 600 using various interfaces and lines, and performs various functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in the memory 650, and by calling data stored in the memory 650. Optionally, the processor 610 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor 610 may integrate one or a combination of several of the following: a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), and a modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content required for display; and the modem handles wireless communication. It is understood that the modem may also be implemented as a separate chip without being integrated into the processor 610.

[0072] The memory 650 may include random access memory (RAM) or read-only memory (ROM). Optionally, the memory 650 may include a non-transitory computer-readable medium. The memory 650 may be used to store instructions, programs, code, code sets, or instruction sets. The memory 650 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as watermarking, image tampering detection, etc.), instructions for implementing the various method embodiments described above, etc.; the data storage area may store data involved in the various method embodiments described above, etc. Optionally, the memory 650 may also be at least one storage device located remotely from the aforementioned processor 610. Figure 6 As shown, the memory 650, which serves as a computer storage medium, may include an operating system, a network communication module, a user interface module, and application programs.

[0073] Specifically, the processor 610 can be used to call the application stored in the memory 650 and perform the following operations: The process involves converting the watermark information corresponding to the original target image into a target barcode; fusing the original target image with the target barcode image to generate a target image carrying the watermark information; the target barcode image consists of multiple target barcodes; the target barcode image has the same size as the original target image; decoding the image to be detected to extract the watermark image; the watermark addition method of the original image corresponding to the image to be detected is the same as that of the original target image; in the case where there is a non-barcode area in the watermark image, determining the tampered area corresponding to the image to be detected based on the non-barcode area, and determining the target watermark information corresponding to the image to be detected based on the barcode area in the watermark image.

[0074] In some possible embodiments, after the processor 610 performs the above-mentioned conversion of the watermark information to be added corresponding to the original target image into a target barcode, and before performing the above-mentioned fusion of the original target image and the target barcode image to generate a target image carrying the watermark information to be added, it is further configured to perform: Based on the dimensions of the original target image, the target barcode is copied, and the copied target barcodes are combined to obtain the target barcode image.

[0075] In some possible embodiments, when the processor 610 performs the above-mentioned fusion of the original target image and the target barcode image to generate a target image carrying the watermark information to be added, it specifically performs the following: The original target image and the target barcode image are input into the image embedding model. Based on the fusion of the original target image and the target barcode image, a target image carrying the watermark information to be added is generated. The image embedding model includes at least one of the following loss functions: YUV mean square error loss function, frequency domain loss function, and image difference loss function.

[0076] In some possible embodiments, the weights corresponding to the luminance and chrominance channels in the YUV mean square error loss function are determined based on the human eye's sensitivity to luminance and chrominance; and / or, the weights corresponding to the frequency domain loss function are determined based on the frequency domain residual between the image generated by the image embedding model and the input original image.

[0077] In some possible embodiments, when the processor 610 performs the above-described decoding of the image to be detected and extracts the watermark image from the image to be detected, it specifically performs the following: The image to be detected is input into the image decoding model for decoding to extract the watermark image from the image to be detected. The image decoding model is trained based on known noise information and sample noise images of the original barcode image. The sample noise images are obtained by adding noise to the sample generated image based on at least one attack method. The sample generated image is generated by the image embedding model based on the fusion of the original sample image and the original barcode image. The original barcode image carries the watermark information to be added to the original sample image.

[0078] In some possible embodiments, when the processor 610 executes the above-described method of determining the target watermark information corresponding to the image to be detected based on the barcode region in the watermark image, it is specifically used to perform the following: Extract the barcode information from the barcode area in the watermark image; combine the barcode information to obtain the watermark barcode corresponding to the image to be detected; obtain the target watermark information corresponding to the image to be detected based on the watermark barcode.

[0079] In some possible embodiments, after the processor 610 performs the above-described decoding of the image to be detected and extracts the watermark image from the image to be detected, it is further configured to perform: If the watermarked image contains only a barcode area and there is an invalid barcode within that barcode area, the image to be detected is determined to be a cropped image; the invalid barcode is used to represent a barcode that cannot be properly recognized.

[0080] This specification also provides a computer-readable storage medium storing instructions that, when executed on a computer or processor, cause the computer or processor to perform one or more steps in the above embodiments. If the constituent modules of the above watermarking processing apparatus are implemented as software functional units and sold or used as independent products, they can be stored in the above-described computer-readable storage medium.

[0081] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When these computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this specification are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in or transmitted through a computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired means, such as coaxial cable, fiber optic cable, Digital Subscriber Line (DSL), or wireless means (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The aforementioned available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., Digital Versatile Discs, DVDs), or semiconductor media (e.g., Solid State Disks, SSDs).

[0082] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. The aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks. Unless otherwise specified, the technical features of this embodiment and its implementation can be combined arbitrarily.

[0083] The embodiments described above are merely preferred embodiments of this specification and are not intended to limit the scope of this specification. Any modifications and improvements made to the technical solutions of this specification by those skilled in the art without departing from the spirit of this specification should fall within the protection scope defined by the claims.

[0084] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims and specification may be performed in a different order than in the embodiments described in the specification and still achieve the desired results. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

Claims

1. A watermark processing method, the method comprising: Convert the watermark information corresponding to the original target image into a target barcode; The original target image and the target barcode image are merged to generate a target image carrying the watermark information to be added; the target barcode image is composed of multiple target barcodes; the target barcode image has the same size as the original target image; The image to be detected is decoded to extract the watermark image from the image to be detected; the watermark addition method of the original image corresponding to the image to be detected is the same as the watermark addition method of the target original image. In the case where there is a non-barcode area in the watermark image, the tampered area corresponding to the image to be detected is determined based on the non-barcode area, and the target watermark information corresponding to the image to be detected is determined based on the barcode area in the watermark image.

2. The method as described in claim 1, wherein after converting the watermark information to be added corresponding to the original target image into a target barcode, and before fusing the original target image with the target barcode image to generate a target image carrying the watermark information to be added, the method further includes: Based on the dimensions of the original target image, the target barcode is copied, and the copied target barcodes are combined to obtain a target barcode image.

3. The method as described in claim 1, wherein fusing the original target image with the target barcode image to generate a target image carrying the watermark information to be added comprises: The target original image and the target barcode image are input into the image embedding model, and a target image carrying the watermark information to be added is generated based on the fusion of the target original image and the target barcode image; the image embedding model includes at least one of the following loss functions: YUV mean square error loss function, frequency domain loss function, and image difference loss function.

4. The method as described in claim 3, wherein the weights corresponding to the luminance channel and the chrominance channel in the YUV mean square error loss function are determined based on the sensitivity of the human eye to luminance and chrominance; and / or, the weights corresponding to the frequency domain loss function are determined based on the frequency domain residual between the image generated by the image embedding model and the input original image.

5. The method as described in claim 1, wherein decoding the image to be detected and extracting the watermark image from the image to be detected includes: The image to be detected is input into the image decoding model for decoding, and the watermark image is extracted from the image to be detected. The image decoding model is trained based on known noise information and sample noisy images of the original barcode image; The sample noise image is obtained by adding noise to the sample generated image based on at least one attack method; The sample image is generated by fusing the original sample image and the original barcode image using an image embedding model; the original barcode image carries the watermark information that needs to be added to the original sample image.

6. The method as described in claim 1, wherein determining the target watermark information corresponding to the image to be detected based on the barcode region in the watermark image includes: Extract the barcode information from the barcode area in the watermark image; The barcode information is combined to obtain the watermark barcode corresponding to the image to be detected; The target watermark information corresponding to the image to be detected is obtained based on the watermark barcode corresponding to the image to be detected.

7. The method as described in claim 1, wherein after decoding the image to be detected and extracting the watermark image from the image to be detected, the method further includes: If the watermarked image contains only a barcode area and there is an invalid barcode within the barcode area, the image to be detected is determined to be a cropped image. The invalid barcode is used to characterize a barcode that cannot be properly recognized.

8. A watermark processing apparatus, the apparatus comprising: The conversion module is used to convert the watermark information corresponding to the original target image into the target barcode; A generation module is used to fuse the original target image with the target barcode image to generate a target image carrying the watermark information to be added; the target barcode image is composed of multiple target barcodes; the target barcode image has the same size as the original target image; The extraction module is used to decode the image to be detected and extract the watermark image from the image to be detected; the watermark addition method of the original image corresponding to the image to be detected is the same as the watermark addition method of the target original image. The determination module is used to determine the tampered area corresponding to the image to be detected based on the non-barcode area when there is a non-barcode area in the watermark image, and to determine the target watermark information corresponding to the image to be detected based on the barcode area in the watermark image.

9. An electronic device, comprising: Processor and memory; The processor is connected to the memory; The memory is used to store executable program code; The processor runs a program corresponding to the executable program code stored in the memory to perform the method as described in any one of claims 1-7.

10. A computer storage medium storing a plurality of instructions adapted for loading by a processor and executing the steps of the method as claimed in any one of claims 1-7.

11. A computer program product comprising instructions that, when run on a computer or processor, causes the computer or processor to perform the watermarking method as described in any one of claims 1-7.

Images