Methods, apparatus, electronic devices, and media for watermark embedding and extraction

Abstract

A method, apparatus, electronic device, and medium for watermark embedding and extraction belong to the field of computer technology. The watermark embedding method includes: encrypting and encoding information corresponding to an image to be displayed to obtain encrypted character codes, and splitting the character codes into multiple sub-codes (S11); determining the sub-codes corresponding to multiple encoding regions of the image to be displayed according to a preset arrangement rule (S12); determining the change in pixel value of a pixel in the encoding region based on the pixel value of a pixel in any encoding region and the sub-code corresponding to the encoding region (S13); adjusting the pixel value of a pixel in the encoding region based on the change in pixel value of the encoding region to obtain an adjusted encoding region (S14); and obtaining a watermark image of the image to be displayed based on each adjusted encoding region of the image to be displayed (S15).

Description

Technical Field

[0001] This disclosure belongs to the field of computer technology, and specifically relates to a method, apparatus, electronic device, and computer-readable medium for watermark embedding and watermark extraction. Background Technology

[0002] With the continuous development of technology and the widespread use of terminal devices (such as smartphones), people are increasingly inclined to take various photos and videos using these devices to share on different social media platforms. However, some individuals may use photos and videos shared by others to commit infringements, such as misappropriating and disseminating others' works, causing loss and harm to the rights of others. Therefore, how to protect the rights of photographic and video works has become an urgent issue to be addressed. Summary of the Invention

[0003] This disclosure aims to at least solve one of the technical problems existing in the related art, and to provide a watermark embedding method that has less impact on images and higher traceability.

[0004] In a first aspect, embodiments of this disclosure provide a watermark embedding method, the method comprising: encrypting and encoding information to be encoded corresponding to an image to be displayed, obtaining encrypted character encoding, and splitting the character encoding into multiple sub-codes; determining sub-codes corresponding to multiple encoding regions of the image to be displayed according to preset arrangement rules; determining the change in pixel value of a pixel in the encoding region based on the pixel value of a pixel in any encoding region and the sub-code corresponding to the encoding region; adjusting the pixel value of a pixel in the encoding region based on the change in pixel value of the encoding region, obtaining an adjusted encoding region; and obtaining a watermark image of the image to be displayed based on each adjusted encoding region of the image to be displayed.

[0005] According to embodiments of this disclosure, the character encoding obtained through encryption and encoding can be split into multiple sub-codes. By changing the pixel values ​​of the regions, the sub-codes are embedded into the encoding regions of the image to be displayed to obtain a watermark image. This enables the complete recovery of all the information to be encoded in subsequent processing, improving the success rate and accuracy of information extraction and enhancing the traceability of the image.

[0006] Secondly, this disclosure provides a watermark extraction method, which includes: performing information decoding processing on an image to be extracted to obtain an information decoding result of the image to be extracted; verifying and correcting the character encoding in the information decoding result according to a preset verification rule to determine the verification result of the information decoding result; and decrypting the verified third encoding to obtain the watermark information of the image to be extracted when the verification result is successful.

[0007] Thirdly, embodiments of this disclosure provide a watermark embedding device, which includes: an encryption module for encrypting and encoding information to be encoded corresponding to an image to be displayed, obtaining encrypted character encoding, and splitting the character encoding into multiple sub-codes; a sub-code determination module for determining sub-codes corresponding to multiple encoding regions of the image to be displayed according to preset arrangement rules; a change amount determination module for determining the change amount of pixel values ​​of pixels in the encoding region based on the pixel values ​​of pixels in any encoding region and the sub-codes corresponding to the encoding region; a pixel adjustment module for adjusting the pixel values ​​of pixels in the encoding region according to the change amount of pixel values ​​in the encoding region, obtaining adjusted encoding regions; and an image acquisition module for obtaining a watermark image of the image to be displayed based on each adjusted encoding region of the image to be displayed.

[0008] Fourthly, this disclosure provides a watermark extraction device, which includes: a decoding module for performing information decoding processing on an image to be extracted to obtain an information decoding result of the image to be extracted; a verification module for verifying and correcting the character encoding in the information decoding result according to a preset verification rule to determine the verification result of the information decoding result; and a decryption module for decrypting the verified third encoding when the verification result is successful to obtain the watermark information of the image to be extracted.

[0009] Fifthly, embodiments of this disclosure provide an electronic device, including: one or more processors; a memory for storing one or more programs; and when the one or more programs are executed by the one or more processors, causing the one or more processors to implement the watermark embedding method or watermark extraction method described above. In some embodiments, the processor includes a field-programmable gate array (FPGA).

[0010] Sixthly, embodiments of this disclosure provide a computer-readable medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps in the above-described watermark embedding method or watermark extraction method. Attached Figure Description

[0011] Figure 1 This is a flowchart of a watermark embedding method according to an embodiment of the present disclosure.

[0012] Figure 2 This is a schematic diagram of a watermark embedding method according to an embodiment of the present disclosure.

[0013] Figure 3 This is a schematic diagram of the encoding region of the watermark embedding method according to an embodiment of the present disclosure.

[0014] Figure 4This is a schematic diagram of the encoding region of the watermark embedding method according to an embodiment of the present disclosure.

[0015] Figure 5a and Figure 5b This is a schematic diagram of the encoded sub-region of the watermark embedding method according to an embodiment of the present disclosure.

[0016] Figure 6 This is a flowchart of a watermark extraction method according to an embodiment of the present disclosure.

[0017] Figure 7 This is a schematic diagram of a watermark extraction method according to an embodiment of the present disclosure.

[0018] Figure 8 This is a schematic diagram illustrating pixel value adjustment according to an embodiment of the present disclosure.

[0019] Figure 9 This is a block diagram of a watermark embedding device according to an embodiment of the present disclosure.

[0020] Figure 10 This is a block diagram of a watermark extraction apparatus according to an embodiment of the present disclosure.

[0021] Figure 11 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present disclosure. Detailed Implementation

[0022] To enable those skilled in the art to better understand the technical solutions of this disclosure, the disclosure will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0023] Unless otherwise defined, the technical or scientific terms used in this disclosure shall have the ordinary meaning understood by one of ordinary skill in the art to which this disclosure pertains. The terms “first,” “second,” and similar terms used in this disclosure do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Similarly, the terms “an,” “a,” or “the,” and similar terms do not indicate a quantity limitation, but rather indicate the presence of at least one. The terms “including,” “comprising,” or “containing,” and similar terms mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. The terms “connected,” “linked,” or similar terms are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. The terms “upper,” “lower,” “left,” and “right,” etc., are used only to indicate relative positional relationships, and these relative positional relationships may change accordingly when the absolute position of the described objects changes.

[0024] To effectively protect the rights of photographic and video works, researchers have been actively engaged in research and practice. Embedding watermarks into images for provenance tracking is a promising solution. However, these technologies often fail to address the issue of image or video alterations. If the image or video has been modified or tampered with—for example, captured from a screen or cropped—its origin cannot be determined based on the image or video itself. This makes it impossible to identify the original image's creation time and location, hindering provenance tracking and protection, resulting in poor protection effectiveness.

[0025] According to embodiments of this disclosure, a watermark embedding method is provided. This method encrypts and encodes information to obtain character codes. Based on preset arrangement rules, it determines the sub-codes of character codes corresponding to multiple encoded regions of the image to be displayed. Then, by changing the pixel values ​​of the regions, the character codes are embedded into the image to obtain a watermark image, thus completing the watermark embedding process. This allows for image traceability in subsequent processing by extracting the character codes from the image, thereby reducing the impact of the watermark on the image, achieving a seamless watermark, and enabling easy extraction of character codes from the image, improving recognition efficiency and accuracy, and thus enhancing image traceability.

[0026] The watermark embedding method according to embodiments of this disclosure can be applied to a terminal. The terminal can be an in-vehicle device, user equipment (UE), mobile device, user terminal, terminal, cellular phone, cordless phone, personal digital assistant (PDA), handheld device, computing device, in-vehicle device, wearable device, etc., and this disclosure does not limit it.

[0027] Figure 1 This is a flowchart illustrating a watermark embedding method according to an embodiment of this disclosure. Figure 1 As shown, the watermark embedding method according to an embodiment of this disclosure includes:

[0028] In step S11, the information to be encoded corresponding to the image to be displayed is encrypted and encoded to obtain the encrypted character code, and the character code is split into multiple sub-codes;

[0029] In step S12, according to the preset arrangement rules, the sub-codes corresponding to the multiple encoding regions of the image to be displayed are determined respectively;

[0030] In step S13, the change in pixel value of a pixel in the encoding region is determined based on the pixel value of a pixel in any encoding region and the subcode corresponding to the encoding region.

[0031] In step S14, the pixel values ​​of the pixels in the encoding region are adjusted according to the amount of change in the pixel values ​​of the encoding region to obtain the adjusted encoding region.

[0032] In step S15, the watermark image of the image to be displayed is obtained according to each adjusted encoding region of the image to be displayed.

[0033] For example, the watermark embedding method can be implemented by a processing component of a display component (such as a display or display screen) in the terminal, such as a field-programmable gate array (FPGA), an application-specific integrated circuit (ASIC), etc., and this disclosure does not limit it.

[0034] In some possible implementations, the image to be displayed can be any image displayed on the display component (e.g., a monitor) of the terminal. Before output, the image to be displayed is processed in the processor of the display component.

[0035] In some possible implementations, the information to be encoded corresponding to the image to be displayed can be any information to be embedded in the image to be displayed. The information to be encoded may include, for example, at least one of the following: the generation time information of the image to be displayed, the generation location information of the image to be displayed, the display time information of the image to be displayed, the device information of the terminal, and the location information of the terminal.

[0036] In the example, the generation time information and generation location information of the image to be displayed can be the time information when the image to be displayed was taken and the geographical location information when the image to be displayed was taken; the display time information can be the time information when the image to be displayed is displayed on the display component, such as a timestamp; the terminal's device information can be the terminal's device ID, etc.; and the terminal's location information can be the terminal's current geographical location information. It should be understood that those skilled in the art can set the specific content of the information to be encoded according to the actual situation, and this disclosure does not impose any restrictions on this.

[0037] In some possible implementations, each image to be displayed may correspond to one piece of information to be encoded, thereby improving the accuracy of the information to be encoded; alternatively, multiple images to be displayed within a preset display time period may correspond to the same piece of information to be encoded, thereby reducing the computational load required to generate the information to be encoded, as well as the computational load required to encrypt and encode the information to be encoded. The preset display time period may be, for example, 10 seconds, 1 minute, etc., and this disclosure does not impose any limitations on it.

[0038] In some possible implementations, in step S11, the information to be encoded corresponding to the image to be displayed can be encrypted and encoded to obtain encrypted character encoding. Any encryption method of relevant technology can be used to encrypt the information to be encoded to obtain encrypted information; then, the encrypted information is encoded to obtain encrypted character encoding; this character encoding can be binary code or encoding of other number bases (e.g., ternary). This reduces the risk of information leakage. This disclosure does not limit the specific encryption and encoding methods, or the number base of the character encoding.

[0039] In some possible implementations, prior to step S11, the method according to embodiments of this disclosure further includes: dividing the image to be displayed into n×m image regions, each image region serving as an encoding region, where n and m are integers and n≥3, m≥4, wherein the second number of the encoding regions is greater than the first number of the subcodes of the character encoding.

[0040] In other words, the image to be displayed can be divided into multiple encoding regions for subsequent processing, for example, into n×m encoding regions (n≥3, m≥4). In step S11, the character encoding can also be split into multiple sub-codes, which can be embedded into multiple encoding regions of the image to be displayed. The second number of encoding regions of the image to be displayed, n×m, is greater than the first number of sub-codes, k. Thus, even if the generated watermark image is cropped, as long as the cropped image includes k sub-codes of the character encoding, all the information to be encoded can be completely recovered in subsequent processing, thereby improving the success rate and accuracy of information extraction.

[0041] In some possible implementations, in step S12, sub-codes corresponding to multiple encoded regions of the image to be displayed can be determined according to a preset arrangement rule. This arrangement rule could be, for example, that the sub-code corresponding to any encoded region is different from the sub-code corresponding to the encoded regions adjacent to it; that is, adjacent encoded regions correspond to different sub-codes. This allows all sub-codes of the character encoding to be included in fewer encoded regions, thus better resisting image cropping and improving the success rate of information extraction.

[0042] In some possible implementations, each coded region comprises multiple coded sub-regions, each corresponding to a codeword of the sub-code. That is, the coded region can be divided into multiple smaller image blocks (called coded sub-regions), and the number of coded sub-regions can correspond to the number of codewords in the sub-code; for example, the number of coded sub-regions is greater than or equal to the number of codewords in the sub-code. This allows each coded sub-region to correspond to a codeword of the sub-code, enabling separate embedding.

[0043] In some possible implementations, in step S13, the change in pixel value of a pixel in any encoded region is determined based on the pixel value of the pixel in that encoded region and the sub-code corresponding to the encoded region. Based on the sub-code corresponding to the encoded region, the codeword corresponding to each encoded sub-region of that encoded region can be determined separately. For example, when the character encoding is binary, if the codeword corresponding to the encoded sub-region is 0, the change in pixel value of the pixel in the encoded sub-region can be directly determined to be 0; if the codeword corresponding to the encoded sub-region is 1, processing continues to determine the change in pixel value. That is, when the codeword is 0, the pixel value remains unchanged; when the codeword is 1, the pixel value changes. It should be understood that it is also possible to set the pixel value to remain unchanged when the codeword is 1 and to change when the codeword is 0; this disclosure does not limit this.

[0044] In some possible implementations, when the character encoding is binary, if the codeword corresponding to the encoded sub-region is 1, the average pixel value of the pixel in the encoded sub-region across the three color channels R (red), G (green), and B (blue) can be determined; the target color channel is then determined based on the average value. This target color channel can be the color channel with the largest average pixel value, or the color channel with the smallest average pixel value, or one or more of the three RGB color channels; this disclosure does not impose any limitations on this.

[0045] In some possible implementations, the change in pixel value for a target color channel can be determined. This change in pixel value can be a fixed value, such as a positive fixed value X1, indicating that the pixel value of the target color channel is uniformly increased by X1; or a negative fixed value X2, indicating that the pixel value of the target color channel is uniformly increased by X2, thereby reducing the computational load. In the example, the change in pixel value can be calculated based on the average pixel value of the target color channel. For example, a larger average pixel value results in a larger change in pixel value, and a smaller average pixel value results in a smaller change in pixel value, thereby reducing the impact on the image display effect. This disclosure does not limit the specific method for determining the change in pixel value.

[0046] In some possible implementations, if the character encoding is in a different base, corresponding rules can also be set. For example, if the character encoding is in 3-base (codewords are 1, 0, and -1), the pixel value change can be set to 0 when the codeword is 0; the pixel value change can be positive when the codeword is 1, i.e., the pixel value is increased; and the pixel value change can be negative when the codeword is -1, i.e., the pixel value is decreased. This disclosure does not impose any limitations on this.

[0047] In some possible implementations, in step S14, the pixel values ​​of the pixels in the encoding region are adjusted according to the change in pixel values ​​of the encoding region to obtain an adjusted encoding region. For any pixel, if the sum of the pixel value in the target color channel and the change in pixel value is greater than 0 or less than the maximum pixel value (e.g., 255), the sum is used as the adjusted pixel value; if the sum is less than or equal to 0, or greater than or equal to the maximum pixel value, 0 or the maximum pixel value is used as the adjusted pixel value, that is, the adjustment is at most 0 or the maximum pixel value. In this way, after adjustment, an adjusted encoding sub-region can be obtained; by adjusting each encoding sub-region of the encoding region, the adjusted encoding region can be obtained.

[0048] In some possible implementations, in step S15, after adjusting the coded sub-regions of each coded region of the image to be displayed, the adjusted coded regions can be obtained, thereby obtaining the watermark image of the image to be displayed.

[0049] According to embodiments of this disclosure, the character encoding obtained through encryption and encoding can be split into multiple sub-codes. By changing the pixel values ​​of the regions, the sub-codes are embedded into the encoding regions of the image to be displayed to obtain a watermark image. This enables the complete recovery of all the information to be encoded in subsequent processing, improving the success rate and accuracy of information extraction and enhancing the traceability of the image.

[0050] The watermark embedding method according to embodiments of this disclosure will now be described in detail.

[0051] Figure 2 This is a schematic diagram of a watermark embedding method according to an embodiment of this disclosure. Figure 2 As shown, the watermark embedding method according to embodiments of this disclosure can be implemented by a processing unit 21 in the display component of a terminal. The display component also includes a display unit 22 (e.g., a display screen). Before entering the processing unit 21, the analog signal of the image to be displayed can be converted into a digital signal by an analog-to-digital converter (not shown), and then enter the processing unit 21 after preprocessing.

[0052] like Figure 2 As shown, the image to be displayed and the corresponding information to be encoded can be input into the processing unit 21 respectively. In step S11, the information to be encoded is encrypted and encoded to obtain the encrypted character code. According to the image to be displayed and the character code, watermark generation processing is performed in steps S12-S15 to obtain the watermark image. The watermark image is then output to the display unit 22 for display.

[0053] When extracting character codes from an image, the image to be extracted may differ from the watermark image. For example, the image to be extracted might be a screen capture, where noise is introduced during the capture process to convey the information carried by the watermark. The image to be extracted may also have undergone compression and transmission, resulting in information loss. Furthermore, it may have been cropped, flipped, or rotated, further reducing information. Therefore, the extracted codes may be inaccurate. To address this, information verification and error correction techniques can be designed to identify and correct the correctness of the extracted codes, thereby improving the accuracy of information extraction.

[0054] In some possible implementations, a checksum can be added to the encoding during the encryption and encoding stage of step S11. Step S11 may include:

[0055] The information to be encoded is encrypted and encoded to obtain an initial first code; the first code is verified using a first verification method to generate a first verification code and add it to the first code to obtain a second code; the second code is verified using a second verification method to generate a second verification code and add it to the second code to obtain the character code.

[0056] In other words, after encryption and encoding, an initial character encoding of 'a' bits is obtained (called the first encoding). The first encoding can be verified using a first verification method to generate a first checksum of length 'i' bits. This checksum is then appended to the end of the first encoding to obtain a second encoding of length 'n+i' bits. Here, 'a' and 'i' are integers greater than 1. The first verification method can be, for example, a Hamming code verification method; this disclosure does not limit the specific type of the first verification method. For example, if 'a' = 4, the first encoding is 1010, and the generated checksum is 101, then the resulting second encoding (a+i = 7 bits) is 1010101.

[0057] In some possible implementations, the second code can be verified using a second check method to generate a second check code of length j, which is then appended to the end of the second code to obtain a character code of length a+i+j. Here, j is an integer greater than 1. The second check method can be, for example, a CRC (Cyclic Redundancy Check) check. This disclosure does not limit the specific type of the second check method. For example, if a+i = 6 and the second code is 110101, and the CRC check uses the generator polynomial x^3 + x + 1, the generated check code after the CRC check is 001, then the resulting character code (a+i+j = 9 bits) is 110101001.

[0058] In this way, during the subsequent extraction of character encoding from the image, the extracted a+i+j-bit character encoding can be verified in two steps. First, a CRC check is used to verify the a+i+j-bit character encoding, detect and correct errors, and obtain the a+i-bit encoding; then, a Hamming code check is used to verify the a+i-bit encoding; if the verification is correct, it means that the information extraction is correct.

[0059] By using a two-layer verification method, the ability to correctly identify and correct the extracted character encoding can be improved, thereby enhancing the data integrity, accuracy, and reliability of watermark recognition.

[0060] In some possible implementations, the image to be displayed can be divided into n×m encoding regions. In step S11, the character encoding can also be split into multiple sub-codes, which can be embedded into multiple encoding regions of the image to be displayed. The second number of encoding regions of the image to be displayed, n×m, is greater than the first number of sub-codes, k. Thus, even if the generated watermark image is cropped, as long as the cropped image includes k sub-codes of the character encoding, all the information to be encoded can be completely recovered in subsequent processing, thereby improving the success rate and accuracy of information extraction.

[0061] In some possible implementations, a preset layout rule can be determined first, so that in the subsequent step S12, the sub-codes corresponding to the multiple encoded regions of the image to be displayed can be determined according to the layout rule. Prior to step S12, the method further includes:

[0062] The arrangement rule is determined based on a first number of subcodes of the character encoding and a second number of encoding regions, wherein the arrangement rule includes the subcodes corresponding to any encoding region being different from the subcodes corresponding to the encoding regions adjacent to that encoding region.

[0063] For example, when extracting character codes from an image, the image to be extracted is often a cropped version of the watermark image. To recover the complete encoding information even after the image has been cropped, the encoding regions in the image can be designed so that the complete encoding information can be recovered from any few adjacent encoding regions in subsequent processing, thereby improving the success rate and accuracy of information extraction.

[0064] In some possible implementations, the preset arrangement rules may include that the subcode corresponding to any coded region is different from the subcode corresponding to the coded regions adjacent to it. This can reduce the size of the image region required to recover the complete coded information.

[0065] In some possible implementations, the arrangement rule can be determined based on the first number k of subcodes according to the character encoding and the second number n×m of the encoding region. The process of determining the arrangement rule can be described as a combinatorial optimization problem, that is, for an n×m table, each cell in the table can be filled with any value from 1 to k, requiring that no two adjacent k cells have the same value, and n≥3, m≥4, find the minimum n, m, k, and give one solution.

[0066] In the example, search methods can be used to solve this combinatorial optimization problem. In an n×m table, each cell can be filled with any value from 1 to k, resulting in a total of k^(n×m) possible solutions. Since no two adjacent k cells can have the same value, it is necessary to evaluate all possible solutions. This problem can be transformed into a graph coloring problem: treat each cell as a node, and connect adjacent cells with an edge. If node p and node q are adjacent, it means that the corresponding cells are in the same row or column, and the edge weight between them is 1; otherwise, the edge weight is 0. The intuitive solution to the coloring problem is recursive search. Assuming the number of nodes is N, the search time complexity is O(k^N). Of course, pruning and other optimization strategies can be combined during the solution process to effectively optimize the search time complexity. It should be understood that those skilled in the art can set the method for determining the arrangement rules according to the actual situation, and this disclosure does not impose any restrictions on this.

[0067] In the example, let n = 4, m = 5, k = 3, and the subcodes of the character encoding can be represented as 1, 2, and 3 respectively. Then, we can obtain an example of the arrangement rule: 3 1 2 3 1 2 3 1 2 3 1 2 3 1 2 3 1 2 3 1

[0072] Based on the example of this arrangement rule, the sub-codes corresponding to each encoding region can be determined in step S12.

[0073] Figure 3 This is a schematic diagram of the encoding region of the watermark embedding method according to an embodiment of this disclosure. Figure 3 As shown, the image to be displayed is divided into 20 4×5 encoding regions, each corresponding to subcode 1, 2, or 3, such that the subcode corresponding to any encoding region is different from the subcode corresponding to its adjacent encoding region. Thus, when extracting character codes from the image later, if the image to be extracted (e.g., ...) Figure 3 If 31, 32 or 33 in the code includes subcodes 1, 2 and 3, then the complete encoded information can be recovered.

[0074] In some possible implementations, each coding region includes multiple coding sub-regions, each corresponding to a codeword of the subcode. That is, the coding region can be further divided into multiple smaller coding sub-regions to embed the codewords of the subcode separately. The number of coding sub-regions in the coding region can be determined based on the number of codewords in the subcode, and this disclosure does not impose any limitations on this.

[0075] Figure 4 This is a schematic diagram of the encoding region of the watermark embedding method according to an embodiment of this disclosure. Figure 4 As shown, the 48-bit character encoding "1001011…110100110" is split into three 16-bit subcodes "100…1011", "011…0011", and "100…0110", which correspond to... Figure 4 The image to be displayed is divided into 20 4×5 coded regions, each corresponding to a subcode 1, 2 or 3, such that the subcode corresponding to any coded region is different from the subcode corresponding to the adjacent coded regions.

[0076] Each encoded region is further divided into 4×4=16 encoded sub-regions, each corresponding to a codeword of the sub-code. Figure 4 In the image to be displayed, the 16 coded sub-regions in the upper right corner of the coded area correspond to the code words of sub-code 1 "100...1011". Each coded sub-region corresponds to one code word. The dark coded sub-region represents the code word 1, and the white coded sub-region represents the code word 0.

[0077] In some possible implementations, step S13 may include: for any coded sub-region in the coded region, determining the codeword corresponding to the coded sub-region based on the sub-code corresponding to the coded region; and determining the change in pixel value of the pixel in the coded sub-region based on the pixel value of the pixel in the coded sub-region and the codeword corresponding to the coded sub-region.

[0078] In other words, for any coded sub-region in the coded region, the codeword corresponding to the coded sub-region can be determined based on the sub-code corresponding to the coded sub-region; then, based on the codeword corresponding to the coded sub-region, the pixel values ​​of the pixels in the coded sub-region are adjusted in order to embed the watermark.

[0079] In some possible implementations, the step of determining the change in pixel value of a pixel in the encoded sub-region based on the pixel value of the pixel in the encoded sub-region and the codeword corresponding to the encoded sub-region includes:

[0080] Based on the codeword corresponding to the encoded sub-region, determine whether the pixel value of the pixel in the encoded sub-region needs to be adjusted; if the pixel value of the pixel in the encoded sub-region needs to be adjusted, determine the average pixel value of the pixel in the encoded sub-region on each color channel based on the pixel value of the pixel in the encoded sub-region; determine the target color channel and the amount of change in the pixel value of the target color channel based on the average pixel value of the pixel in the encoded sub-region on each color channel.

[0081] For example, based on the codeword corresponding to the encoded sub-region, it can be determined whether the pixel value of a pixel in that encoded sub-region needs to be adjusted. For instance, when the character encoding is binary, it is set that no adjustment is needed when the codeword is 0, and adjustment is needed when the codeword is 1. This disclosure does not limit the specific conditions under which pixel values ​​need to be adjusted.

[0082] In the example, if it is determined that the pixel values ​​of pixels in the encoded sub-region need to be adjusted, the average pixel value of the pixels in the encoded sub-region across the R, G, and B color channels can be determined based on the pixel values ​​of the pixels in the encoded sub-region. Then, based on the average pixel value, the target color channel to be adjusted is determined from the R, G, and B color channels, and the change in pixel value for that target color channel is calculated.

[0083] In the example, the target color channel can include any of the following: the color channel with the largest average pixel value; the color channel with the smallest average pixel value; or one or more of the R, G, and B color channels. That is, one or more channels can be arbitrarily selected from the R, G, and B color channels as the target color channel, or the color channel with the largest average pixel value and / or the color channel with the smallest average pixel value can be selected as the target color channel.

[0084] Choosing the color channel with the largest average pixel value as the target color channel and increasing its pixel value; or choosing the color channel with the smallest average pixel value as the target color channel and decreasing its pixel value, can minimize the impact on the visual effect of the adjusted image, improve the image display quality, and not affect the recognition algorithm's ability to recognize the embedded codewords. This disclosure does not impose any restrictions on the specific method of selecting the target color channel or the specific method of adjusting the pixel value.

[0085] In some possible implementations, the change in pixel value for a target color channel can be determined. This change in pixel value can be a fixed value, such as a positive fixed value X1, which means that the pixel value of the target color channel is uniformly increased by X1; or a negative fixed value X2, which means that the pixel value of the target color channel is uniformly increased by X2, thereby reducing the amount of computation.

[0086] In some possible implementations, the change in pixel value can also be calculated based on the average pixel value of the target color channel. For example, a larger average pixel value results in a larger change in pixel value, and a smaller average pixel value results in a smaller change in pixel value, thereby reducing the impact on image display. This disclosure does not limit the specific method for determining the change in pixel value.

[0087] In some possible implementations, the step of determining the change in pixel value of the target color channel may include: determining the change in pixel value of the target color channel based on the average pixel value of the target color channel and a pixel change coefficient; wherein the pixel change coefficient is a positive or negative value.

[0088] In other words, a pixel change coefficient can be preset, and the pixel value change of the target color channel can be determined based on this pixel change coefficient and the average pixel value of the target color channel. The pixel value change b can be expressed as follows:

[0089] b = round(y*z / 255) (1)

[0090] In formula (1), z represents the average pixel value of the target color channel, y represents the pixel change coefficient, and round() rounds the result to an integer. The pixel change coefficient y can be set to a positive value between 1 and 20, which indicates an increase in pixel value; or a negative value between -1 and -20, which indicates a decrease in pixel value. For example, the pixel change coefficient y can be set to 3, -5, etc.

[0091] The smaller the pixel change coefficient, the weaker the watermark strength, resulting in fewer traces displayed on the screen, less impact on image display, and less perceptibility to the human eye, leading to a better user experience; however, it also places higher demands on subsequent extraction. Conversely, the larger the pixel change coefficient, the greater the impact on image display, the easier it is to extract character codes from the image, and the higher the success rate and accuracy of extraction. Therefore, the display effect can be optimized as much as possible while ensuring recognition accuracy. Those skilled in the art can set the size and sign of the pixel change coefficient according to actual conditions, and this disclosure does not impose any restrictions on this.

[0092] By analyzing each sub-region of the coding region separately, the change in pixel value of each pixel in the entire coding region can be obtained.

[0093] In some possible implementations, in step S14, the pixel values ​​of the pixels in the encoding region can be adjusted according to the change in pixel values ​​of the encoding region to obtain an adjusted encoding region. Specifically, step S14 may include: adjusting the pixel values ​​of the pixels in each encoding sub-region of the encoding region according to the change in pixel values ​​of the pixels in each sub-region of the encoding region to obtain an adjusted encoding region.

[0094] For example, for any pixel in the coded sub-region to be adjusted, if the sum of the pixel value in the target color channel and the change in pixel value is greater than 0 or less than the maximum pixel value (e.g., 255), then the sum is used as the adjusted pixel value; if the sum is less than or equal to 0, or greater than or equal to the maximum pixel value, then 0 or the maximum pixel value is used as the adjusted pixel value, meaning the adjustment will be up to 0 or the maximum pixel value. In this way, the adjusted coded sub-region can be obtained; by adjusting each coded sub-region of this coded region, the adjusted coded region can be obtained.

[0095] In some possible implementations, in step S15, after adjusting the coded sub-regions of each coded region of the image to be displayed, the adjusted coded regions can be obtained, thereby obtaining the watermark image of the image to be displayed.

[0096] Figure 5a and Figure 5b This is a schematic diagram of the encoded sub-region of the watermark embedding method according to an embodiment of this disclosure. Figure 5a As shown, this is an encoded sub-region of the image to be displayed. The average pixel values ​​of the three color channels R, G, and B are calculated as follows: R channel = 180; G channel = 56; B channel = 155. Among them, the average pixel value of the R channel is the largest, and the average pixel value of the G channel is the smallest. The R red channel and G green channel can be selected as the target color channels. Let the codeword corresponding to this encoded sub-region be 1. The pixel change coefficient of the target color channel with the largest average pixel value is 3, and the pixel change coefficient of the target color channel with the smallest average pixel value is -5. Then, according to formula (1), the pixel value change of the R channel is 2, and the pixel value change of the G channel is -1. In this case, the pixel value of the R channel of the pixels in this encoded sub-region is uniformly increased by 2, and the pixel value of the G channel is uniformly decreased by 1. This completes the encoding of the encoded sub-region, and the adjusted encoded sub-region is obtained, as shown below. Figure 5b As shown.

[0097] In this way, by encoding each sub-region of any encoding region separately, the information encryption of that encoding region is completed; by encrypting each encoding region of the image to be displayed separately, the information encryption of the entire image to be displayed is completed, and the watermark image of the image to be displayed is obtained.

[0098] In some possible implementations, the watermark image of the image to be displayed can be cached in the RAM memory of the processing unit 21; the watermark image in the RAM memory is converted into a standard video format by a digital signal conversion chip (not shown), and the complete video stream is aggregated and output to the display unit 22 (e.g., a display screen) for display.

[0099] In the example, the watermark embedding method of this disclosure embodiment is implemented by the processing unit 21 in the display component of the terminal, which can generate watermark images in real time and improve the efficiency of watermark embedding. However, this disclosure does not limit the specific components for implementing the watermark embedding method of this disclosure embodiment.

[0100] According to embodiments of this disclosure, information can be encrypted and encoded to obtain character codes. Subcodes of character codes corresponding to multiple encoded regions of the image to be displayed are determined according to preset arrangement rules. Then, the character codes are embedded into the image to be displayed by changing the pixel values ​​of the regions to obtain a watermark image, thus completing the watermark embedding process. This allows for image traceability by extracting character codes from the image in subsequent processing, thereby reducing the impact of the watermark on the image, improving the efficiency and accuracy of subsequent identification, and enhancing the traceability of the image.

[0101] According to embodiments of this disclosure, a watermark extraction method is also provided, which can be applied to a server (e.g., a cloud server). Figure 6 This is a flowchart illustrating a watermark extraction method according to an embodiment of this disclosure. Figure 6 As shown, the watermark extraction method according to an embodiment of this disclosure includes:

[0102] In step S61, the image to be extracted is subjected to information decoding processing to obtain the information decoding result of the image to be extracted;

[0103] In step S62, the character encoding in the information decoding result is verified and corrected according to the preset verification rules, and the verification result of the information decoding result is determined.

[0104] In step S63, if the verification result is successful, the third code after verification is decrypted to obtain the watermark information of the image to be extracted.

[0105] For example, the image to be extracted can be any image that may contain infringing content, obtained from any channel (public internet channels, channels reported by the infringed party, etc.). Specifically, the image to be extracted can be a screen capture image obtained by photographing a display screen; an image obtained by screenshotting, downloading, recording, printing, etc.; or an image after compression, cropping, flipping, rotating, etc., of the original image. The image to be extracted can be manually selected or automatically selected through certain rules or algorithms. This disclosure does not restrict the acquisition channel or image type of the image to be extracted.

[0106] In some possible implementations, in step S61, the image to be extracted can be decoded using an information decoding network or other possible algorithms to obtain the information decoding result of the image to be extracted. This information decoding result may include whether character encoding exists in the image, and if so, the character encoding identified. The radix of this character encoding is consistent with the radix used in the watermark embedding method, such as binary code, ternary code, etc. In this example, the character encoding may be binary code.

[0107] The information decoding result may also include other content, such as the decoded binary image, the timestamp of the decoding process, etc. This disclosure does not limit the specific content included in the information decoding result.

[0108] In some possible implementations, the information decoding network may include a deep learning network model based on Transformer (a type of neural network architecture), such as a segmentation model or a classification model; it may also include a deep learning network model based on CNN (convolutional neural network) for image segmentation / classification, etc. This disclosure does not limit the specific model type of the information decoding network.

[0109] When the information decoding network includes a segmentation model and the character encoding is binary, the segmentation model outputs a binary image of the segmented image to be extracted, which is then post-processed to obtain the recognized character encoding. When the information decoding network includes a classification model and the character encoding is binary, the classification model can directly output the character encoding of the classified image to be extracted.

[0110] In some possible implementations, in step S62, the character encoding in the information decoding result can be verified and corrected according to preset verification rules to determine the verification result and the verified information decoding result. When the character encoding uses two layers of verification, the outer second verification method, such as CRC verification, can be used to verify and correct the character encoding, resulting in the corrected encoding; then, the inner first verification method, such as Hamming code verification, can be used to verify the corrected encoding, resulting in the verification result and the verified third encoding.

[0111] In some possible implementations, if the verification result is "verification failed," it means that the correct encoded information could not be extracted from the image to be extracted, and the source of the image to be extracted cannot be traced. The output "no valid information was identified" can be displayed. If the verification result is "verification passed," it means that the correct encoded information could be extracted from the image to be extracted, and the decryption process can continue.

[0112] In some possible implementations, if the verification result is successful, in step S63, the verified third code can be decrypted using a decryption method corresponding to the preset encryption method to obtain the watermark information of the image to be extracted, thereby completing the watermark extraction process. Furthermore, image tracing can be performed on the image to be extracted based on the display time information, terminal device information, etc., contained in the watermark information.

[0113] According to embodiments of this disclosure, the image to be extracted can be decoded, the decoded character encoding can be verified and corrected according to verification rules, and the verified encoding can be decrypted if the verification passes, thereby obtaining the watermark information of the image to be extracted, thus realizing the watermark extraction process of the image. According to embodiments of this disclosure, the efficiency and success rate of watermark extraction can be improved, and the reliability and accuracy of the extracted encoding can be improved through verification and error correction.

[0114] The watermark extraction method according to the embodiments of this disclosure will now be described in detail.

[0115] Figure 7 This is a schematic diagram of a watermark extraction method according to an embodiment of this disclosure. Figure 7 As shown, the watermark extraction method according to the embodiments of this disclosure can decode the input image to be extracted; verify and correct the character encoding obtained by decoding; if the verification fails, output "no valid information was identified"; if the verification passes, decrypt the verified encoding and output the decrypted watermark information, thereby realizing the entire process of watermark extraction.

[0116] In some possible implementations, according to the method of this disclosure, the image to be extracted can be decoded in step S61 by an information decoding network, which includes a decoding segmentation network or a decoding classification network. For example, the information decoding network may include a Transformer-based deep learning network model, such as a segmentation model (referred to as a decoding segmentation network) or a classification model (referred to as a decoding classification network). It may also include a CNN (Convolutional Neural Network)-based deep learning network model for image segmentation / classification, etc. This disclosure does not limit the specific model type of the information decoding network.

[0117] In some possible implementations, step S61 may include: if the information decoding network includes a decoding segmentation network, inputting the image to be extracted into the decoding segmentation network for processing to obtain a decoded binary image; performing image post-processing on the binary image to obtain a character encoding corresponding to the binary image, wherein the information decoding result includes the character encoding.

[0118] In other words, if the information decoding network includes a decoding segmentation network, when the character encoding is binary, inputting the image to be extracted into the decoding segmentation network will output a decoded binary image, meaning that the pixel values ​​of the pixels in the image are only 0 and 255 (or other set values). Post-processing the binary image, such as using pixel values ​​0 and 255 as 0 and 1 in the encoding respectively and sorting them, yields the character encoding corresponding to the binary image. This character encoding is then used as the information decoding result.

[0119] In the example, if the information decoding network includes a decoding classification network, and the character encoding is binary, the image to be extracted will be directly output as the character encoding after being processed by the decoding classification network. This character encoding can be used as the information decoding result.

[0120] In this way, the information of the image to be extracted can be decoded, improving the success rate and accuracy of decoding.

[0121] In some possible implementations, the information decoding network can be trained before inference is performed. Specifically, before step S61, the method further includes: training the information decoding network according to a preset sample set to obtain a trained information decoding network, wherein the sample set includes multiple sample pairs, and each sample pair includes a sample encoded image and a label image of the sample encoded image.

[0122] For example, the sample encoded image can be input into the information decoding network for processing to obtain the sample decoding result; based on the sample decoding result and the label image of the sample encoded image, the network loss of the information decoding network can be determined; then, the parameters of the information decoding network can be adjusted according to the network loss; under the condition of meeting the training conditions, the trained information decoding network is obtained. This disclosure does not limit the specific training method.

[0123] However, the number of directly obtainable labeled samples is limited, while training the information decoding network requires a large number of high-quality labeled samples. According to embodiments of this disclosure, sample encoded images and corresponding label images can be generated efficiently.

[0124] In some possible implementations, prior to step S61, the method further includes: adjusting the pixel values ​​of pixels in one or more color channels in the first image to obtain an adjusted second image; cropping the first image and the second image respectively to obtain multiple first cropping regions of the first image and multiple second cropping regions of the second image, wherein the sizes and positions of the first cropping regions and the second cropping regions correspond; stitching together a portion of the first cropping regions of the first image and a portion of the second cropping regions of the second image to obtain a sample encoded image, wherein the size of the sample encoded image is the same as that of the first image; and generating a label image of the sample encoded image, wherein the label image is used to indicate whether the pixels of the sample encoded image correspond to the first image or the second image.

[0125] For example, any watermark-free image (referred to as the first image) displayed on the screen can be captured using a camera (such as a smartphone). This capture process can be performed manually or using an automated script. This disclosure does not limit the method of capturing the first image.

[0126] Since the watermark embedding method described above embeds the watermark by adjusting the pixel values ​​of the target color channel, the same method of selecting and adjusting the target color channel can be used during sample generation to improve the relevance of the samples and thus enhance the training effect of the network.

[0127] In some possible implementations, the target color channel of the first image can be determined based on the selection and adjustment method of the target color channel during watermark embedding. For example, this could be the color channel with the highest average pixel value; the color channel with the lowest average pixel value; or any one or more of the R, G, and B color channels. Then, the pixel values ​​of each pixel in the target color channel are adjusted to obtain the adjusted second image. Specifically, based on the adjustment method during watermark embedding (increasing or decreasing pixel values), the pixel values ​​of the target color channel can be directly adjusted to a preset value, or the pixel value change amount can be determined according to a preset pixel change coefficient, and the pixel values ​​can be adjusted based on the pixel value change amount.

[0128] Figure 8 This is a schematic diagram illustrating pixel value adjustment according to an embodiment of this disclosure. Figure 8 As shown, the first image 81 can be split into channel images 82, 83, and 84 for three color channels: R, G, and B. For the channel image of the target color channel, such as channel image 82 for the B channel, the pixel value of each pixel in channel image 82 can be increased by a preset value to obtain the adjusted channel image 85. Then, the three channel images 85, 83, and 84 can be recombine to obtain the adjusted second image 86. This second image 86 can be regarded as a full-screen image with a watermark added.

[0129] In some possible implementations, the first image and the second image can be cropped separately to obtain multiple first cropped regions of the first image and multiple second cropped regions of the second image, with the sizes and positions of the first and second cropped regions corresponding. In the example, to increase randomness, a random number generator can be used to determine the cropped regions, and the sizes and positions of the cropped regions of the first and second images can be made to correspond so that they can be stitched together to form a correct new image.

[0130] In some possible implementations, a portion of a first cropped region of the first image and a portion of a second cropped region of the second image can be stitched together to obtain a new image, namely the sample-coded image. The positions of the first and second cropped regions involved in the stitching are complementary, so that a complete image can be formed, and the size of the stitched sample-coded image is the same as that of the first image.

[0131] In some possible implementations, a label image for the sample encoded image can be generated based on the first and second cropped regions involved in the stitching. This label image indicates whether a pixel in the sample encoded image corresponds to the first or second image. For example, a pixel corresponding to the first image might have a value of 0 in the label image, and a pixel corresponding to the second image might have a value of 1. This helps the information decoding network accurately identify each region during training.

[0132] In this way, high-quality sample encoded images and corresponding label images can be generated efficiently.

[0133] In some possible implementations, sample-encoded images and corresponding label images can be combined into sample pairs and added to the sample set for network training. Before training, the sample-encoded images can be preprocessed, such as through normalization or data augmentation. Data augmentation can include operations such as random cropping, flipping, rotating, and compression to further increase the number of samples, enriching the training data in the sample set, improving sample diversity, and enhancing the robustness of the trained information decoding network. This disclosure does not limit the specific augmentation methods used.

[0134] In some possible implementations, the sample set can be divided into a training dataset, a validation dataset, and a test dataset. The training dataset is used to train the information decoding network, the validation dataset is used to evaluate the performance of the information decoding network, and the test dataset is used to test the generalization ability of the information decoding network. It is important to ensure that the dataset allocation is appropriate to avoid overfitting and underfitting during network training.

[0135] By using the above processing method, multiple high-quality sample pairs can be generated efficiently, thereby improving the training effect of the information decoding network and enabling the trained information decoding network to obtain better decoding performance and generalization ability.

[0136] After training, the information decoding network can be used to perform information decoding processing on the image to be extracted in step S61 to obtain the information decoding result of the image to be extracted.

[0137] In some possible implementations, in step S62, the character encoding in the information decoding result is verified and corrected according to a preset verification rule. This verification rule can correspond to the verification method added in the watermark embedding method described above. For example, if a two-layer verification method of Hamming code verification + CRC verification is used to add a verification code during the watermark embedding process, then a Hamming code verification + CRC verification method is also used for verification during the watermark extraction process.

[0138] In some possible implementations, step S62 may include: verifying the character encoding according to the second verification method to obtain a first verification result; if the first verification result is a verification failure, correcting the character encoding according to the second verification method to obtain a corrected fourth encoding; verifying the fourth encoding according to the first verification method to obtain a second verification result; if the second verification result is a verification success, determining that the verification result of the information decoding result is a verification success, and determining the verified third encoding.

[0139] For example, when character encoding uses two layers of verification, the outer second verification method, such as CRC check, can be used to verify the character encoding (a+i+j bits) in the decoded information. For instance, if a=4, i=j=3, the character encoding is a 10-bit binary code 1010 101 001. The same generator polynomial (e.g., x^3+x+1) used in the watermark embedding process can be used to perform division on the character encoding to obtain the remainder. If the remainder is 0, it means the character encoding is correct and no error correction is needed; the first verification result is successful, and the checksum can be removed to obtain the a+i-bit encoding (called the fifth encoding), for example, a 7-bit encoding 1010 101. If the remainder is not 0, it means the received data contains an error and error correction is needed; the first verification result is unsuccessful.

[0140] In some possible implementations, if the first verification result is a failure, the character encoding is corrected. The step of correcting the character encoding according to the second verification method to obtain the corrected fourth encoding includes: multiplying each codeword of the character encoding by the generator polynomial corresponding to the second verification method, and adding the multiple multiplication results to obtain a target polynomial, which indicates the error position of the character encoding; and performing an XOR operation between the target polynomial and the character encoding to obtain the corrected fourth encoding.

[0141] For example, each bit of the character encoding can be multiplied bitwise by the generator polynomial of the CRC checksum, and the results can be added together to obtain a target polynomial. This target polynomial indicates the error position of the character encoding. XORing the target polynomial indicating the error position with the character encoding achieves error correction, resulting in the corrected character encoding. The corrected character encoding is then subjected to another CRC check. If the remainder is 0, the error correction is successful; otherwise, it fails, and "no valid information detected" can be directly output. If the error correction is successful, the checksum can be removed, resulting in the a+i-bit encoding, which is the fourth encoding after error correction.

[0142] Furthermore, based on the first verification method of the inner layer, the fourth code (a+i bits, such as the 7-bit code 1010101) or the fifth code (a+i bits, such as the 7-bit code 1010 101) when the first verification result is a pass verification can be verified to obtain the second verification result.

[0143] In some possible implementations, if the second verification result is a failure, it indicates an error in information extraction, and "No valid information detected" can be output; if the second verification result is a success, it indicates correct information extraction, and the verification result of the information decoding result is confirmed as a success. Furthermore, the checksum can be removed from the fourth or fifth encoding (a+i bits, e.g., a 7-bit encoding 1010 101) to obtain the verified third encoding (a bits, e.g., a 4-bit encoding 1010).

[0144] This method improves the ability to identify the correctness of extracted character encodings, corrects errors in data transmission, ensures data reliability, and thus improves the accuracy of watermark recognition.

[0145] If the verification result of the information decoding result is successful, in step S63, the verified third code can be decrypted using a decryption method corresponding to the preset encryption method to obtain the watermark information of the image to be extracted, thereby completing the watermark extraction process. Furthermore, image tracing can be performed on the image to be extracted based on the display time information, terminal device information, etc., contained in the watermark information.

[0146] According to embodiments of this disclosure, the image to be extracted can be decoded, the decoded character encoding can be verified and corrected according to verification rules, and the verified encoding can be decrypted if the verification passes, thereby obtaining the watermark information of the image to be extracted, thus realizing the watermark extraction process of the image. According to embodiments of this disclosure, the efficiency and success rate of watermark extraction can be improved, and the reliability and accuracy of the extracted encoding can be improved through verification and error correction.

[0147] According to embodiments of this disclosure, a watermark embedding device is also provided. Figure 9 This is a block diagram of a watermark embedding device according to an embodiment of this disclosure. Figure 9 As shown, the device includes:

[0148] The encryption module 91 is used to encrypt and encode the information to be encoded corresponding to the image to be displayed, to obtain the encrypted character encoding, and to split the character encoding into multiple sub-codes.

[0149] The subcode determination module 92 is used to determine the subcodes corresponding to multiple encoded regions of the image to be displayed according to a preset arrangement rule.

[0150] The change amount determination module 93 is used to determine the change amount of the pixel value of the pixel in the encoding region based on the pixel value of any pixel in the encoding region and the subcode corresponding to the encoding region.

[0151] The pixel adjustment module 94 is used to adjust the pixel values ​​of the pixels in the encoding region according to the amount of change in the pixel values ​​of the encoding region, so as to obtain the adjusted encoding region.

[0152] The image acquisition module 95 is used to obtain the watermark image of the image to be displayed based on each adjusted coded region of the image to be displayed.

[0153] In some possible implementations, the encryption module 91 is used to: encrypt and encode the information to be encoded to obtain an initial first code; verify the first code using a first verification method, generate a first verification code and add it to the first code to obtain a second code; verify the second code using a second verification method, generate a second verification code and add it to the second code to obtain a character code.

[0154] In some possible implementations, prior to the encryption module 91, the device further includes: a region division module, used to divide the image to be displayed into n×m image regions, each image region being an encoding region, where n and m are integers and n≥3, m≥4; wherein the second number of the encoding regions is greater than the first number of the subcodes of the character encoding.

[0155] In some possible implementations, before the subcode determination module 92, the apparatus further includes: a layout rule determination module, configured to determine the layout rule based on a first number of subcodes of the character encoding and a second number of encoding regions, wherein the layout rule includes subcodes corresponding to any encoding region being different from subcodes corresponding to encoding regions adjacent to that encoding region.

[0156] In some possible implementations, each encoding region includes multiple encoding sub-regions, and each encoding sub-region corresponds to a codeword of the sub-code. The change determination module 93 is configured to: for any encoding sub-region in the encoding region, determine the codeword corresponding to the encoding sub-region based on the sub-code corresponding to the encoding region; and determine the change in pixel value of the pixel in the encoding sub-region based on the pixel value of the pixel in the encoding sub-region and the codeword corresponding to the encoding sub-region.

[0157] The pixel adjustment module 94 is used to adjust the pixel values ​​of the pixels in the encoding region according to the change in pixel values ​​of the pixels in each encoding sub-region of the encoding region, so as to obtain the adjusted encoding region.

[0158] In some possible implementations, the change determination module 93 is used to: determine whether the pixel value of the pixel in the encoded sub-region needs to be adjusted based on the codeword corresponding to the encoded sub-region; if the pixel value of the pixel in the encoded sub-region needs to be adjusted, determine the average pixel value of the pixel in the encoded sub-region on each color channel based on the pixel value of the pixel in the encoded sub-region; and determine the target color channel and the change amount of the pixel value of the target color channel based on the average pixel value of the pixel in the encoded sub-region on each color channel.

[0159] In some possible implementations, the target color channel includes any of the following: the color channel with the largest average pixel value; the color channel with the smallest average pixel value; one or more of the three color channels R, G, and B; wherein, the change amount determination module 93 is used to: determine the change amount of the pixel value of the target color channel based on the average pixel value of the target color channel and the pixel change coefficient; the pixel change coefficient is a positive or negative value.

[0160] In some possible implementations, the device is applied to a terminal, and the information to be encoded includes at least one of the following: the generation time information of the image to be displayed, the generation location information of the image to be displayed, the display time information of the image to be displayed, the device information of the terminal, and the location information of the terminal; wherein, multiple images to be displayed within a preset display time period correspond to the same information to be encoded.

[0161] According to embodiments of this disclosure, a watermark extraction device is also provided. Figure 10 This is a block diagram of a watermark extraction apparatus according to an embodiment of this disclosure. Figure 10 As shown, the device includes:

[0162] Decoding module 71 is used to perform information decoding processing on the image to be extracted, and obtain the information decoding result of the image to be extracted;

[0163] The verification module 72 is used to verify and correct the character encoding in the information decoding result according to the preset verification rules, and determine the verification result of the information decoding result;

[0164] The decryption module 73 is used to decrypt the third code after verification if the verification result is successful, so as to obtain the watermark information of the image to be extracted.

[0165] In some possible implementations, the device performs information decoding processing on the image to be extracted through an information decoding network, which includes a decoding segmentation network or a decoding classification network.

[0166] In some possible implementations, the decoding module 71 is configured to: input the image to be extracted into the decoding segmentation network for processing, when the information decoding network includes a decoding segmentation network, to obtain a decoded binary image; perform image post-processing on the binary image to obtain a character encoding corresponding to the binary image, wherein the information decoding result includes the character encoding.

[0167] In some possible implementations, the device further includes: a training module, used to train the information decoding network according to a preset sample set to obtain a trained information decoding network, wherein the sample set includes multiple sample pairs, and each sample pair includes a sample encoded image and a label image of the sample encoded image;

[0168] In some possible implementations, the apparatus further includes: an adjustment module for adjusting the pixel values ​​of pixels in one or more color channels of a first image to obtain an adjusted second image; a cropping module for cropping the first image and the second image respectively to obtain multiple first cropping regions of the first image and multiple second cropping regions of the second image, wherein the size and position of the first cropping regions and the second cropping regions correspond; a stitching module for stitching together a portion of the first cropping regions of the first image and a portion of the second cropping regions of the second image to obtain a sample encoded image, wherein the size of the sample encoded image is the same as that of the first image; and a label generation module for generating a label image of the sample encoded image, wherein the label image is used to indicate that the pixels of the sample encoded image correspond to the first image or the second image.

[0169] In some possible implementations, the verification module 72 is configured to: verify the character encoding according to a second verification method to obtain a first verification result; if the first verification result is a verification failure, correct the character encoding according to the second verification method to obtain a corrected fourth encoding; verify the fourth encoding according to the first verification method to obtain a second verification result; if the second verification result is a verification success, determine that the verification result of the information decoding result is a verification success, and determine the verified third encoding.

[0170] In some possible implementations, the verification module 72 is used to: multiply each codeword of the character encoding with the generator polynomial corresponding to the second verification method, and add the multiple multiplication results to obtain a target polynomial, which is used to indicate the error position of the character encoding; and perform an XOR operation on the target polynomial and the character encoding to obtain the error-corrected fourth encoding.

[0171] Figure 11This is a schematic diagram of the structure of an electronic device according to an embodiment of this disclosure. Figure 11 As shown, this disclosure provides an electronic device including: one or more processors 101, a memory 102, and one or more I / O interfaces 103. The memory 102 stores one or more programs, which, when executed by the one or more processors, cause the one or more processors to implement any of the watermark embedding or watermark extraction methods described in the above embodiments; the one or more I / O interfaces 103 are connected between the processor and the memory, configured to enable information interaction between the processor and the memory.

[0172] The processor 101 is a device with data processing capabilities, including but not limited to a central processing unit (CPU); the memory 102 is a device with data storage capabilities, including but not limited to random access memory (RAM, more specifically SDRAM, DDR, etc.), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), and flash memory (FLASH); the I / O interface (read / write interface) 103 is connected between the processor 101 and the memory 102, enabling information exchange between the processor 101 and the memory 102, including but not limited to a data bus (Bus).

[0173] In some embodiments, the processor 101, memory 102, and I / O interface 103 are interconnected via bus 104, and thus connected to other components of the computing device.

[0174] In some embodiments, when performing the watermark embedding method, the one or more processors 101 include a field-programmable gate array (FPGA). The one or more processors 101 may also include hardware with high image processing capabilities, such as a graphics processing unit (GPU) or an image signal processor (ISP), as long as they can implement the watermark embedding method according to the embodiments of this disclosure. This disclosure does not limit the specific hardware type of the processor.

[0175] According to embodiments of this disclosure, a computer-readable medium is also provided. This computer-readable medium stores a computer program, wherein, when executed by a processor, the program implements the steps of any of the watermark embedding or watermark extraction methods described in the above embodiments.

[0176] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a machine-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication component, and / or installed from a removable medium. When the computer program is executed by a central processing unit (CPU), it performs the functions defined above in the system of this disclosure.

[0177] It should be noted that the computer-readable medium disclosed herein may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium may be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0178] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0179] The circuits or sub-circuits described in the embodiments of this disclosure can be implemented in software or hardware. The described circuits or sub-circuits can also be housed in a processor; for example, it can be described as: a processor including: a receiving circuit and a processing circuit, the processing module including a writing sub-circuit and a reading sub-circuit. The names of these circuits or sub-circuits do not necessarily constitute a limitation on the circuit or sub-circuit itself; for example, a receiving circuit can also be described as "receiving video signals".

[0180] It is understood that the above embodiments are merely exemplary embodiments used to illustrate the principles of this disclosure, and this disclosure is not limited thereto. For those skilled in the art, various modifications and improvements can be made without departing from the spirit and substance of this disclosure, and these modifications and improvements are also considered to be within the scope of protection of this disclosure.

Claims

1. A watermark embedding method, characterized in that, The method includes: The information to be encoded corresponding to the image to be displayed is encrypted and encoded to obtain the encrypted character code, and the character code is split into multiple subcodes; According to the preset arrangement rules, the sub-codes corresponding to multiple encoding regions of the image to be displayed are determined respectively; wherein, each encoding region includes multiple encoding sub-regions, and each encoding sub-region corresponds to a codeword of the sub-code; Determining the change in pixel value of a pixel in an encoding region based on the pixel value of a pixel in any encoding region and the subcode corresponding to the encoding region includes: for any encoding sub-region in the encoding region, determining the codeword corresponding to the encoding sub-region based on the subcode corresponding to the encoding region, and determining the change in pixel value of a pixel in the encoding sub-region based on the pixel value of a pixel in the encoding sub-region and the codeword corresponding to the encoding sub-region. Adjusting the pixel values ​​of pixels in the encoding region based on the change in pixel values ​​of the encoding region to obtain an adjusted encoding region includes: adjusting the pixel values ​​of pixels in each encoding sub-region of the encoding region based on the change in pixel values ​​of pixels in each encoding sub-region of the encoding region to obtain an adjusted encoding region; The watermark image of the image to be displayed is obtained based on each adjusted encoding region of the image to be displayed.

2. The method according to claim 1, characterized in that, The process of encrypting and encoding the information to be encoded corresponding to the image to be displayed, to obtain the encrypted character encoding, includes: The information to be encoded is encrypted and encoded to obtain the initial first code; The first code is verified using the first verification method, a first verification code is generated and added to the first code, and a second code is obtained. The second encoding is verified using a second verification method, a second verification code is generated and added to the second encoding, and the character encoding is obtained.

3. The method according to claim 1, characterized in that, Before encrypting and encoding the information to be encoded corresponding to the image to be displayed, the method further includes: The image to be displayed is divided into n×m image regions, and each image region is a coding region, where n and m are integers and n≥3, m≥4; Wherein, the second number of the encoded regions is greater than the first number of the subcodes of the character encoding.

4. The method according to claim 1, characterized in that, Before determining the sub-codes corresponding to the multiple encoded regions of the image to be displayed according to a preset arrangement rule, the method further includes: The arrangement rule is determined based on a first number of subcodes of the character encoding and a second number of encoding regions, wherein the arrangement rule includes the subcodes corresponding to any encoding region being different from the subcodes corresponding to the encoding regions adjacent to that encoding region.

5. The method according to claim 1, characterized in that, Based on the pixel values ​​of the pixels in the encoded sub-region and the codeword corresponding to the encoded sub-region, the change in pixel values ​​of the pixels in the encoded sub-region is determined, including: Based on the codeword corresponding to the encoded sub-region, determine whether the pixel value of the pixel in the encoded sub-region needs to be adjusted; When the pixel value of a pixel in the encoded sub-region needs to be adjusted, the average pixel value of the pixel in the encoded sub-region on each color channel is determined based on the pixel value of the pixel in the encoded sub-region. The target color channel and the change in pixel value of the target color channel are determined based on the average pixel value of the pixels in each color channel in the encoded sub-region.

6. The method according to claim 5, characterized in that, The target color channel includes any of the following: the color channel with the largest average pixel value; the color channel with the smallest average pixel value; or one or more of the three color channels: R, G, and B. The step of determining the target color channel and the change in pixel value of the target color channel based on the average pixel value of the pixels in each color channel in the encoded sub-region includes: The pixel value change of the target color channel is determined based on the average pixel value of the target color channel and the pixel change coefficient; the pixel change coefficient can be positive or negative.

7. The method according to claim 1, characterized in that, The method is applied to a terminal, and the information to be encoded includes at least one of the following: the generation time information of the image to be displayed, the generation location information of the image to be displayed, the display time information of the image to be displayed, the device information of the terminal, and the location information of the terminal; Among them, multiple images to be displayed within a preset display time period correspond to the same information to be encoded.

8. A watermark extraction method, characterized in that, include: The image to be extracted is processed by an information decoding network to obtain the information decoding result of the image to be extracted; According to the preset verification rules, the character encoding in the information decoding result is verified and corrected to determine the verification result of the information decoding result; If the verification result is successful, the third code after verification is decrypted to obtain the watermark information of the image to be extracted; The method further includes: The information decoding network is trained according to a preset sample set to obtain the trained information decoding network. The sample set includes multiple sample pairs, and each sample pair includes a sample encoded image and a label image of the sample encoded image. The method further includes: The pixel values ​​of pixels in one or more color channels in the first image are adjusted to obtain the adjusted second image; The first image and the second image are cropped respectively to obtain multiple first cropping regions of the first image and multiple second cropping regions of the second image, wherein the size and position of the first cropping regions and the second cropping regions correspond to each other; A portion of a first cropped area of ​​the first image and a portion of a second cropped area of ​​the second image are stitched together to obtain a sample encoded image, wherein the size of the sample encoded image is the same as that of the first image; Generate a label image for the sample encoded image, the label image being used to indicate that the pixels of the sample encoded image correspond to the first image or the second image.

9. The method according to claim 8, characterized in that, The information decoding network includes a decoding segmentation network or a decoding classification network.

10. The method according to claim 9, characterized in that, in, The information decoding process of the image to be extracted, to obtain the information decoding result of the image to be extracted, includes: When the information decoding network includes a decoding segmentation network, the image to be extracted is input into the decoding segmentation network for processing to obtain a decoded binary image; The binary image is post-processed to obtain a character encoding corresponding to the binary image, and the information decoding result includes the character encoding.

11. The method according to claim 8, characterized in that, The step of verifying and correcting the character encoding in the information decoding result according to preset verification rules, and determining the verification result and the third encoding after verification, includes: The character encoding is verified according to the second verification method to obtain the first verification result; If the first verification result is that the verification fails, the character encoding is corrected according to the second verification method to obtain the corrected fourth encoding. The fourth code is verified according to the first verification method to obtain the second verification result; If the second verification result is successful, the verification result of the information decoding result is determined to be successful, and the third encoding after verification is determined.

12. The method according to claim 11, characterized in that, The second verification method includes a cyclic redundancy check (CRC) method, wherein the step of correcting the character encoding according to the second verification method to obtain the corrected fourth encoding includes: Each codeword of the character encoding is multiplied by the generator polynomial corresponding to the second verification method, and the results of multiple multiplications are added together to obtain the target polynomial, which is used to indicate the error position of the character encoding. The target polynomial and the character encoding are XORed to obtain the fourth encoding after error correction.

13. A watermark embedding device, characterized in that, The device includes: An encryption module is used to encrypt and encode the information to be encoded corresponding to the image to be displayed, to obtain the encrypted character encoding, and to split the character encoding into multiple sub-codes; The sub-code determination module is used to determine the sub-codes corresponding to multiple encoded regions of the image to be displayed according to a preset arrangement rule; wherein each encoded region includes multiple encoded sub-regions, and each encoded sub-region corresponds to a codeword of the sub-code; The change amount determination module is used to determine the change amount of pixel values ​​in the encoding region based on the pixel values ​​of any pixel in the encoding region and the subcode corresponding to the encoding region, including: For any coded sub-region in the coded region, the codeword corresponding to the coded sub-region is determined according to the sub-code corresponding to the coded region; The change in pixel value of the pixel in the encoded sub-region is determined based on the pixel value of the pixel in the encoded sub-region and the codeword corresponding to the encoded sub-region. A pixel adjustment module is used to adjust the pixel values ​​of pixels in the encoding region according to the change in pixel values ​​of the encoding region, so as to obtain an adjusted encoding region. The module includes: adjusting the pixel values ​​of pixels in the encoding region according to the change in pixel values ​​of pixels in each encoding sub-region of the encoding region, so as to obtain an adjusted encoding region. The image acquisition module is used to obtain the watermark image of the image to be displayed based on each adjusted coded region of the image to be displayed.

14. A watermark extraction device, characterized in that, The device includes: The decoding module is used to perform information decoding processing on the image to be extracted through the information decoding network to obtain the information decoding result of the image to be extracted; The verification module is used to verify and correct the character encoding in the information decoding result according to the preset verification rules, and determine the verification result of the information decoding result; The decryption module is used to decrypt the third code after verification if the verification result is successful, so as to obtain the watermark information of the image to be extracted. The device further includes: The training module is used to train the information decoding network according to a preset sample set to obtain the trained information decoding network. The sample set includes multiple sample pairs, and each sample pair includes a sample encoded image and a label image of the sample encoded image. The adjustment module is used to adjust the pixel values ​​of pixels in one or more color channels in the first image to obtain the adjusted second image; The cropping module is used to crop the first image and the second image respectively to obtain multiple first cropping regions of the first image and multiple second cropping regions of the second image, wherein the size and position of the first cropping regions and the second cropping regions correspond to each other. The stitching module is used to stitch together a portion of a first cropped area of ​​the first image and a portion of a second cropped area of ​​the second image to obtain a sample encoded image, wherein the size of the sample encoded image is the same as that of the first image. A label generation module is used to generate a label image for the sample encoded image, wherein the label image is used to indicate that the pixels of the sample encoded image correspond to the first image or the second image.

15. An electronic device, characterized in that, include: One or more processors; Memory, used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the watermark embedding method as described in any one of claims 1 to 7 or the watermark extraction method as described in any one of claims 8 to 12.

16. The electronic device according to claim 15, characterized in that, The processor includes a field-programmable gate array (FPGA).

17. A computer-readable medium having a computer program stored thereon, wherein, When the computer program is executed by a processor, it implements the steps of the watermark embedding method as described in any one of claims 1 to 7 or the watermark extraction method as described in any one of claims 8 to 12.

Images