Watermark embedding method and apparatus, watermark extraction method and apparatus, and electronic device and medium

Abstract

A watermark embedding method and apparatus, a watermark extraction method and apparatus, and an electronic device and a medium, which belong to the technical field of computers. The watermark embedding method comprises: encrypting and encoding information to be encoded that corresponds to an image to be displayed, so as to obtain an encrypted character code, and splitting the character code into a plurality of sub-codes (S11); on the basis of a preset arrangement rule, respectively determining sub-codes corresponding to a plurality of code regions of said image (S12); on the basis of pixel values of pixel points in any code region and sub-codes corresponding to the code region, determining pixel value variations of the pixel points in the code region (S13); on the basis of the pixel value variations of the code region, adjusting the pixel values of the pixel points in the code region, so as to obtain an adjusted code region (S14); and on the basis of the adjusted code regions of said image, obtaining a watermark image of said image (S15).

Description

Watermark embedding and watermark extraction method, device, electronic device, and medium Technical Field

[0001] The present disclosure belongs to the field of computer technology, and particularly relates to a watermark embedding and watermark extraction method, device, electronic device, and computer-readable medium. Background Art

[0002] With the continuous development of technology and the popularization of terminal devices (such as smartphones), people are increasingly inclined to use these devices to capture various photos and videos for sharing on various social platforms. However, some people may use photos and videos shared by others to commit infringements, such as misappropriating other people's works for dissemination, causing losses and damage to the rights and interests of others. Therefore, how to protect the rights and interests of photos and videos has become a pressing issue.

[0003] Summary of the Invention

[0004] The present disclosure aims to solve at least one of the technical problems existing in the related art and provide a watermark embedding method with less impact on images and higher traceability.

[0005] In a first aspect, an embodiment of the present disclosure provides a watermark embedding method, the method comprising: encrypting and encoding information to be encoded corresponding to an image to be displayed to obtain an encrypted character code, and splitting the character code into multiple subcodes; determining, according to a preset arrangement rule, the subcodes corresponding to multiple encoding regions of the image to be displayed; determining, according to the pixel values ​​of the pixels in any encoding region and the subcodes corresponding to the encoding region, the amount of change in the pixel values ​​of the pixels in the encoding region; adjusting the pixel values ​​of the pixels in the encoding region according to the amount of change in the pixel values ​​of the encoding region to obtain an adjusted encoding region; and obtaining a watermark image of the image to be displayed according to each adjusted encoding region of the image to be displayed.

[0006] According to the embodiments of the present disclosure, the character code obtained by encryption and encoding can be split into multiple sub-codes, and the sub-codes can be embedded into the coding area of ​​the image to be displayed by changing the regional pixel values ​​to obtain a watermark image, so that all the information to be encoded can be completely restored in subsequent processing, thereby improving the success rate and accuracy of information extraction and improving the traceability of the image.

[0007] In a second aspect, an embodiment of the present disclosure provides a watermark extraction method, which includes: performing information decoding processing on an image to be extracted to obtain an information decoding result of the image to be extracted; verifying and correcting the character encoding in the information decoding result according to a preset verification rule to determine a verification result of the information decoding result; and when the verification result is a passed verification, decrypting the verified third code to obtain the watermark information of the image to be extracted.

[0008] In a third aspect, an embodiment of the present disclosure provides a watermark embedding device, which includes: an encryption module for encrypting and encoding information to be encoded corresponding to an image to be displayed, obtaining an encrypted character code, and splitting the character code into multiple subcodes; a subcode determination module for determining, according to a preset arrangement rule, the subcodes corresponding to multiple encoding regions of the image to be displayed; a change amount determination module for determining, according to the pixel value of the pixel point in any encoding region and the subcode corresponding to the encoding region, the change amount of the pixel value of the pixel point in the encoding region; a pixel adjustment module for adjusting the pixel value of the pixel point in the encoding region according to the change amount of the pixel value of the encoding region, to obtain an adjusted encoding region; and an image acquisition module for obtaining a watermark image of the image to be displayed according to each adjusted encoding region of the image to be displayed.

[0009] In a fourth aspect, an embodiment of the present disclosure provides a watermark extraction device, which includes: a decoding module, used to perform information decoding processing on an image to be extracted to obtain an information decoding result of the image to be extracted; a verification module, used to verify and correct the character encoding in the information decoding result according to a preset verification rule, and determine the verification result of the information decoding result; a decryption module, used to decrypt the verified third code when the verification result is verification passed, to obtain the watermark information of the image to be extracted.

[0010] In a fifth aspect, embodiments of the present disclosure provide an electronic device comprising: one or more processors; and a memory for storing one or more programs; wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the aforementioned watermark embedding method or watermark extraction method. In some embodiments, the processor comprises a field programmable gate array (FPGA).

[0011] In a sixth aspect, an embodiment of the present disclosure provides a computer-readable medium having a computer program stored thereon, wherein the computer program implements the steps in the above-mentioned watermark embedding method or watermark extraction method when executed by a processor. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] FIG1 is a flow chart of a watermark embedding method according to an embodiment of the present disclosure.

[0013] FIG2 is a schematic diagram of a watermark embedding method according to an embodiment of the present disclosure.

[0014] FIG3 is a schematic diagram of a coding area of ​​a watermark embedding method according to an embodiment of the present disclosure.

[0015] FIG4 is a schematic diagram of a coding area of ​​a watermark embedding method according to an embodiment of the present disclosure.

[0016] FIG5a and FIG5b are schematic diagrams of coding sub-regions of the watermark embedding method according to an embodiment of the present disclosure.

[0017] FIG6 is a flowchart of a watermark extraction method according to an embodiment of the present disclosure.

[0018] FIG7 is a schematic diagram of a watermark extraction method according to an embodiment of the present disclosure.

[0019] FIG8 is a schematic diagram of pixel value adjustment according to an embodiment of the present disclosure.

[0020] FIG9 is a block diagram of a watermark embedding apparatus according to an embodiment of the present disclosure.

[0021] FIG10 is a block diagram of a watermark extraction apparatus according to an embodiment of the present disclosure.

[0022] FIG11 is a schematic structural diagram of an electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0023] In order to enable those skilled in the art to better understand the technical solutions of the present disclosure, the present disclosure is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0024] Unless otherwise defined, the technical or scientific terms used in this disclosure should have the usual meanings understood by people with ordinary skills in the field to which this disclosure belongs. The words "first", "second" and similar words used in this disclosure do not indicate any order, quantity or importance, but are only used to distinguish different components. Similarly, words such as "one", "an" or "the" do not indicate a quantity limitation, but rather indicate the existence of at least one. Words such as "include" or "comprise" mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. Words such as "connect" or "connected" are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the object being described changes, the relative positional relationship may also change accordingly.

[0025] To effectively protect the rights of photographic and video works, researchers have been actively engaged in research and practice. Embedding watermarks in images for traceability is a promising solution. However, if an image or video has been modified or tampered with to some extent, such as by capturing a screen shot or cropping the original image, the source cannot be determined based on the image or video itself. For example, the time and location of the original image cannot be identified, making it impossible to trace its origin and protect it, resulting in poor protection.

[0026] According to an embodiment of the present disclosure, a watermark embedding method is provided, which can encrypt and encode information to obtain character codes, determine the subcodes of the character codes corresponding to multiple coding areas of the image to be displayed according to a preset arrangement rule, and then embed the character codes into the image to be displayed by changing the regional pixel values ​​to obtain a watermark image, thereby completing the watermark embedding process, so that in subsequent processing, the image can be traced by extracting the character codes in the image, thereby reducing the impact of the watermark on the image and realizing a traceless watermark, and subsequently the character codes in the image can be easily extracted, thereby improving the efficiency and accuracy of recognition, thereby improving the traceability of the image.

[0027] The watermark embedding method according to the embodiment of the present disclosure can be applied to a terminal. The terminal can be an in-vehicle device, user equipment (UE), mobile device, user terminal, terminal, cellular phone, cordless phone, personal digital assistant (PDA), handheld device, computing device, in-vehicle device, wearable device, etc., and the present disclosure does not limit this.

[0028] FIG1 is a flow chart of a watermark embedding method according to an embodiment of the present disclosure. As shown in FIG1 , the watermark embedding method according to an embodiment of the present disclosure includes:

[0029] In step S11, the information to be encoded corresponding to the image to be displayed is encrypted and encoded to obtain an encrypted character code, and the character code is split into multiple subcodes;

[0030] In step S12, subcodes corresponding to the plurality of coding regions of the image to be displayed are respectively determined according to a preset arrangement rule;

[0031] In step S13, a pixel value change amount of a pixel point in any coding region is determined according to the pixel value of the pixel point in the coding region and the subcode corresponding to the coding region;

[0032] In step S14, the pixel values ​​of the pixels in the coding area are adjusted according to the pixel value change amount of the coding area to obtain an adjusted coding area;

[0033] In step S15, a watermark image of the image to be displayed is obtained according to each adjusted coding region of the image to be displayed.

[0034] For example, the watermark embedding method can be implemented by a processing component of a display component (such as a display or display screen) in a terminal, such as a field programmable gate array FPGA, an application-specific integrated circuit ASIC, etc., and the present disclosure does not limit this.

[0035] In some possible implementations, the image to be displayed may be any image to be displayed on a display component (eg, a display) of a terminal. Before being output, the image to be displayed may be input to a processor of the display component for processing.

[0036] In some possible implementations, the information to be encoded corresponding to the image to be displayed may be any information to be embedded in the image to be displayed. The information to be encoded may, for example, include at least one of the following: generation time information of the image to be displayed, generation location information of the image to be displayed, display time information of the image to be displayed, device information of the terminal, and location information of the terminal.

[0037] In this example, the generation time information and generation location information of the image to be displayed can be the time information when the image to be displayed is captured and the geographical location information at the time the image to be displayed is captured; the display time information can be the time information when the image to be displayed is displayed on the display component, such as a timestamp; the terminal device information can be information such as the terminal device ID; and the terminal location information can be information about the terminal's current geographical location. It should be understood that those skilled in the art can set the specific content of the information to be encoded based on actual circumstances, and this disclosure does not impose any limitations thereto.

[0038] In some possible implementations, each image to be displayed may correspond to one piece of information to be encoded, thereby improving the accuracy of the information to be encoded. Alternatively, multiple images to be displayed within a preset display time period may correspond to the same piece of information to be encoded, thereby reducing the computational effort required to generate the information to be encoded, as well as the computational effort required to encrypt and encode the information to be encoded. The preset display time period may be, for example, 10 seconds, 1 minute, etc., and this disclosure is not limited thereto.

[0039] In some possible implementations, in step S11, the information to be encoded corresponding to the image to be displayed may be encrypted and encoded to obtain an encrypted character code. The information to be encoded may be encrypted using any encryption method of the relevant technology to obtain encrypted information; the encrypted information may then be encoded to obtain an encrypted character code; the character code may be a binary code or an encoding of other bases (e.g., base 3). This reduces the risk of information leakage. The present disclosure does not limit the specific methods of encryption and encoding, nor the base of the character code.

[0040] In some possible implementations, before step S11, the method according to an embodiment of the present disclosure further includes: dividing the image to be displayed into n×m image areas, each image area being a coding area, n and m are integers and n≥3, m≥4, wherein the second number of the coding areas is greater than the first number of the subcodes of the character encoding.

[0041] That is, the image to be displayed can be divided into multiple coding regions for subsequent processing, for example, into n×m coding regions (n≥3, m≥4). In step S11, the character code can also be split into multiple subcodes so as to be respectively embedded in multiple coding regions of the image to be displayed. The second number n×m of coding regions of the image to be displayed is greater than the first number k of subcodes. In this way, when the generated watermark image is cropped, as long as the cropped image includes k subcodes of the character code, all the information to be encoded can be completely restored in subsequent processing, thereby improving the success rate and accuracy of information extraction.

[0042] In some possible implementations, in step S12, the subcodes corresponding to the multiple coding regions of the image to be displayed may be determined according to a preset arrangement rule. This arrangement rule may, for example, be such that the subcode corresponding to any coding region is different from the subcode corresponding to the coding region adjacent to it, i.e., adjacent coding regions correspond to different subcodes. This allows all subcodes of the character encoding to be included in fewer coding regions, thereby better combating image cropping and improving the success rate of information extraction.

[0043] In some possible implementations, each coding region includes multiple coding sub-regions, each of which corresponds to a codeword of the subcode. That is, the coding region can be divided into multiple smaller image blocks (referred to as coding sub-regions), and the number of coding sub-regions can correspond to the number of codewords in the subcode. For example, the number of coding sub-regions can be greater than or equal to the number of codewords in the subcode. This allows each coding sub-region to correspond to a codeword of the subcode, allowing for separate embedding.

[0044] In some possible implementations, in step S13, the amount of change in the pixel value of the pixel point in any coding area is determined based on the pixel value of the pixel point in the coding area and the subcode corresponding to the coding area. According to the subcode corresponding to the coding area, the codewords corresponding to each coding sub-area of ​​the coding area can be determined respectively. For example, in the case where the character encoding is a binary code, if the codeword corresponding to the coding sub-area is 0, the amount of change in the pixel value of the pixel point in the coding sub-area can be directly determined to be 0; if the codeword corresponding to the coding sub-area is 1, the processing continues to determine the amount of change in the pixel value. That is, when the codeword is 0, the pixel value remains unchanged, and when the codeword is 1, the pixel value changes. It should be understood that it can also be set that the pixel value remains unchanged when the codeword is 1, and the pixel value changes when the codeword is 0, and the present disclosure does not limit this.

[0045] In some possible implementations, when the character is encoded as a binary code, if the codeword corresponding to the encoding sub-region is 1, the average pixel values ​​of the three color channels R (red), G (green), and B (blue) for the pixels in the encoding sub-region can be determined; and the target color channel can be determined based on the average values. The target color channel can be the color channel with the largest pixel average value, the color channel with the smallest pixel average value, or one or more of the three RGB color channels, and this disclosure is not limited to this.

[0046] In some possible implementations, for a target color channel, the amount of change in the pixel value of the target color channel can be determined. The amount of change in pixel value can be a fixed value, such as a positive fixed value X1, indicating that the pixel values ​​of the target color channel are uniformly increased by X1; or a negative fixed value X2, indicating that the pixel values ​​of the target color channel are uniformly increased by X2, thereby reducing the amount of calculation. In the example, the amount of change in pixel value can be calculated based on the average value of the pixels of the target color channel. For example, when the average value of the pixels is larger, the amount of change in pixel value is also larger, and when the average value of the pixels is smaller, the amount of change in pixel value is also smaller, thereby reducing the impact on the image display effect. The present disclosure does not limit the specific method for determining the amount of change in pixel value.

[0047] In some possible implementations, if the character encoding is in another base, corresponding rules can also be set. For example, if the character encoding is a 3-base code (codewords are 1, 0, and -1), the pixel value change can be set to 0 when the codeword is 0; the pixel value change can be set to a positive value when the codeword is 1, i.e., the pixel value is increased; and the pixel value change can be set to a negative value when the codeword is -1, i.e., the pixel value is decreased. This disclosure does not impose any restrictions on this.

[0048] In some possible implementations, in step S14, the pixel values ​​of the pixels in the coding region are adjusted according to the pixel value change of the coding region to obtain an adjusted coding region. For any pixel, if the sum of the pixel value of the pixel in the target color channel and the pixel value change is greater than 0 or less than the maximum pixel value (e.g., 255), the sum is used as the adjusted pixel value; if the sum is less than or equal to 0, or greater than or equal to the maximum pixel value, 0 or the maximum pixel value is used as the adjusted pixel value, that is, the value is adjusted to 0 or the maximum pixel value at most. In this way, after adjustment, an adjusted coding sub-region can be obtained; by adjusting each coding sub-region of the coding region, an adjusted coding region can be obtained.

[0049] In some possible implementations, in step S15, after adjusting the coding sub-regions of each coding region of the image to be displayed, each adjusted coding region can be obtained, thereby obtaining a watermark image of the image to be displayed.

[0050] According to the embodiments of the present disclosure, the character code obtained by encryption and encoding can be split into multiple sub-codes, and the sub-codes can be embedded into the coding area of ​​the image to be displayed by changing the regional pixel values ​​to obtain a watermark image, so that all the information to be encoded can be completely restored in subsequent processing, thereby improving the success rate and accuracy of information extraction and improving the traceability of the image.

[0051] The watermark embedding method according to the embodiment of the present disclosure is described below.

[0052] Figure 2 is a schematic diagram of a watermark embedding method according to an embodiment of the present disclosure. As shown in Figure 2, the watermark embedding method according to an embodiment of the present disclosure can be implemented by a processing component 21 in a terminal's display assembly, which also includes a display component 22 (e.g., a display screen). Before entering processing component 21, the analog signal of the image to be displayed can be converted into a digital signal via an analog-to-digital converter (not shown) and then pre-processed before entering processing component 21.

[0053] As shown in Figure 2, the image to be displayed and the information to be encoded corresponding to the image to be displayed can be input into the processing component 21 respectively, and the information to be encoded is encrypted and encoded in step S11 to obtain the encrypted character code; according to the image to be displayed and the character code, watermark generation processing is performed in steps S12-S15 to obtain a watermark image; and the watermark image is output to the display component 22 for display.

[0054] When extracting character codes from an image, the extracted image may differ from the watermarked image. For example, if the extracted image is a screen capture of a display screen, the watermarked information may have been subject to noise during capture. The extracted image may also have been compressed or transmitted, resulting in information loss. It may also have been cropped, flipped, or rotated, further losing information. Therefore, the extracted code may be inaccurate. To address this, information verification and error correction can be designed to verify and correct the extracted code, thereby improving the accuracy of information extraction.

[0055] In some possible implementations, during the encryption and encoding phase of step S11, a checksum may be added to the encoding. Step S11 may include:

[0056] The information to be encoded is encrypted and encoded to obtain an initial first code; the first code is verified by a first verification method, a first verification code is generated and added to the first code to obtain a second code; the second code is verified by a second verification method, a second verification code is generated and added to the second code to obtain the character code.

[0057] That is to say, after encryption and encoding, an initial a-bit character code (called the first code) can be obtained. For the first code, the first code can be verified by a first verification method to generate a first verification code with a length of i bits, and the first verification code is added to the end of the first code to obtain a second code of n+i bits. Wherein, a and i are integers greater than 1. The first verification method can be, for example, a Hamming code verification method, and the present disclosure does not limit the specific type of the first verification method. For example, a=4, the first code is 1010, and the verification code generated after verification is 101, then the obtained second code (a+i=7 bits) is 1010101.

[0058] In some possible implementations, for the second code, the second code can be verified by a second verification method to generate a second verification code with a length of j bits, and the second verification code is added to the end of the second code to obtain a character code of a+i+j bits. Wherein, j is an integer greater than 1. The second verification method may be, for example, a CRC (Cyclic Redundancy Check) verification method, and the present disclosure does not limit the specific type of the second verification method. For example, a+i=6, the second code is 110101, and when the CRC verification adopts the generating polynomial x^3+x+1, the verification code generated after the CRC verification is 001, and the resulting character code (a+i+j=9 bits) is 110101001.

[0059] In this way, during the subsequent extraction of the character code from the image, the extracted character code of bits a+i+j can be verified in two steps. First, the character code of bits a+i+j is verified using CRC to detect and correct errors, obtaining the code of bits a+i. Then, the code of bits a+i is verified using Hamming code. If the verification is correct, it means that the information has been extracted correctly.

[0060] Through the two-layer verification method, the subsequent correctness recognition and error correction capabilities of the extracted character codes can be improved, and the data integrity, accuracy and reliability of watermark recognition can be improved.

[0061] In some possible implementations, the image to be displayed can be divided into n×m encoding regions. In step S11, the character code can also be split into multiple subcodes so as to be embedded in the multiple encoding regions of the image to be displayed. The second number n×m of encoding regions of the image to be displayed is greater than the first number k of subcodes. Thus, if the generated watermark image is cropped, as long as the cropped image includes the k subcodes of the character code, all the information to be encoded can be fully recovered in subsequent processing, thereby improving the success rate and accuracy of information extraction.

[0062] In some possible implementations, a preset arrangement rule may be determined first, so that in the subsequent step S12, the subcodes corresponding to the multiple coding regions of the image to be displayed are determined according to the arrangement rule. Before step S12, the method further includes:

[0063] The arrangement rule is determined according to the first number of subcodes of the character code and the second number of coding areas, wherein the arrangement rule includes that the subcode corresponding to any coding area is different from the subcode corresponding to the coding area adjacent to the coding area.

[0064] For example, when extracting character codes from an image, the image to be extracted is often a cropped version of the watermarked image. To recover the complete encoded information even after the image has been cropped, the encoded regions in the image can be designed so that the complete encoded information can be recovered from any number of adjacent encoded regions in subsequent processing, thereby improving the success rate and accuracy of information extraction.

[0065] In some possible implementations, the preset arrangement rule may include that the subcode corresponding to any coding region is different from the subcode corresponding to the coding region adjacent to the coding region. In this way, the size of the image region required to restore the complete coding information can be reduced.

[0066] In some possible implementations, the arrangement rule can be determined based on a first number k of subcodes encoded according to the character code and a second number n×m of encoding regions. The process of determining the arrangement rule can be described as a combinatorial optimization problem. Specifically, for an n×m table, where each grid can be filled with any value from 1 to k, and where no values ​​are repeated between any k adjacent grids, and where n ≥ 3 and m ≥ 4, the minimum n, m, and k values ​​are determined, and a set of solutions is provided.

[0067] In the example, a search method or the like can be used to solve the combinatorial optimization problem. Each grid in the n×m table can be filled with any value from 1 to k, so the total number of solutions is k^(n×m). Since any adjacent k grids cannot have repeated values, it is necessary to judge all possible solutions. This problem can be transformed into a coloring problem in graph theory: each grid is regarded as a node, and an edge is connected between adjacent grids. If node p and node q are adjacent, it means that the corresponding grids are in the same row or column, and the edge weight between them is 1, otherwise the edge weight is 0; the intuitive solution to the coloring problem is a recursive search. Let the number of nodes be N, then the time complexity of the search is O(k^N). Of course, when solving the problem, optimization strategies such as pruning can also be combined to effectively optimize the time complexity of the search. It should be understood that those skilled in the art can set the method for determining the arrangement rules according to actual conditions, and the present disclosure does not limit this.

[0068] In this example, let n = 4, m = 5, k = 3, and the subcodes of the character encoding can be represented as 1, 2, and 3 respectively, then an example of the arrangement rule can be obtained:

[0069] 3 1 2 3 1

[0070] 2 3 1 2 3

[0071] 1 2 3 1 2

[0072] 3 1 2 3 1

[0073] According to the example of the arrangement rule, the subcode corresponding to each coding region can be determined respectively in step S12.

[0074] Figure 3 is a schematic diagram of the coding area of ​​the watermark embedding method according to an embodiment of the present disclosure. As shown in Figure 3, the image to be displayed is divided into 20 coding areas of 4×5, each corresponding to subcode 1, 2, or 3, and the subcode corresponding to any coding area is different from the subcode corresponding to the adjacent coding area. In this way, when the character code in the image is subsequently extracted, if the image to be extracted (for example, 31, 32, or 33 in Figure 3) includes subcodes 1, 2, and 3, the complete coding information can be recovered.

[0075] In some possible implementations, each coding region includes multiple coding sub-regions, each of which corresponds to a codeword of the subcode. In other words, the coding region can be further divided into multiple smaller coding sub-regions to embed each codeword of the subcode. The number of coding sub-regions in a coding region can be determined based on the number of codewords in the subcode, and this disclosure does not impose any limitations on this.

[0076] Figure 4 is a schematic diagram of the coding regions of the watermark embedding method according to an embodiment of the present disclosure. As shown in Figure 4, the 48-bit character code "1001011…110100110" is split into three 16-bit subcodes: "100…1011," "011…0011," and "100…0110," corresponding to 1, 2, and 3 in Figure 4, respectively. The image to be displayed is divided into 20 4×5 coding regions, each corresponding to subcode 1, 2, or 3, ensuring that the subcode corresponding to any coding region is different from the subcode corresponding to the adjacent coding region.

[0077] Each coding region is further divided into 4×4=16 coding sub-regions, each corresponding to a codeword in the subcode. In Figure 4, the 16 coding sub-regions in the upper right corner of the coding region to be displayed correspond to the codewords of subcode 1 "100…1011." Each coding sub-region corresponds to a codeword, with darker coding sub-regions representing codewords of 1 and white coding sub-regions representing codewords of 0.

[0078] In some possible implementations, step S13 may include: for any coding sub-region in the coding region, determining a codeword corresponding to the coding sub-region based on the subcode corresponding to the coding region; and determining a change in the pixel value of the pixel point in the coding sub-region based on the pixel value of the pixel point in the coding sub-region and the codeword corresponding to the coding sub-region.

[0079] That is to say, for any coding sub-region in the coding region, the codeword corresponding to the coding sub-region can be determined according to the subcode corresponding to the coding region; then, according to the codeword corresponding to the coding sub-region, the pixel values ​​of the pixels in the coding sub-region are adjusted to embed the watermark.

[0080] In some possible implementations, the step of determining, based on the pixel value of the pixel in the coding sub-region and the codeword corresponding to the coding sub-region, a change in the pixel value of the pixel in the coding sub-region includes:

[0081] Based on the codeword corresponding to the coding sub-region, determine whether the pixel value of the pixel point in the coding sub-region needs to be adjusted; if the pixel value of the pixel point in the coding sub-region needs to be adjusted, determine the pixel average value of the pixel point in the coding sub-region on each color channel based on the pixel value of the pixel point in the coding sub-region; and determine the target color channel and the pixel value change amount of the target color channel based on the pixel average value of the pixel point in the coding sub-region on each color channel.

[0082] For example, based on the codeword corresponding to the coding sub-region, it can be determined whether the pixel value of a pixel point in the coding sub-region is to be adjusted. For example, when the character encoding is binary code, it is set that no adjustment is made when the codeword is 0, and adjustment is made when the codeword is 1. The present disclosure does not limit the specific conditions for the pixel value to be adjusted.

[0083] In this example, if it is determined that the pixel value of a pixel in the coding subregion needs to be adjusted, the average pixel value of the pixel in the coding subregion across the three color channels R, G, and B can be determined based on the pixel value of the pixel in the coding subregion. Furthermore, based on the average pixel value, a target color channel to be adjusted is determined from the three color channels R, G, and B, and the pixel value change of the target color channel is calculated.

[0084] In this example, the target color channel may include any of the following: the color channel with the largest pixel average value; the color channel with the smallest pixel average value; or one or more of the three color channels R, G, and B. That is, one or more channels may be selected from the three color channels R, G, and B as the target color channel, or the color channel with the largest pixel average value and / or the color channel with the smallest pixel average value may be selected as the target color channel.

[0085] Among them, selecting the color channel with the largest pixel average as the target color channel and increasing the pixel value; or selecting the color channel with the smallest pixel average as the target color channel and decreasing the pixel value can reduce the visual impact of the adjusted image, improve the image display quality, and not affect the recognition algorithm's recognition of the embedded codeword. The present disclosure does not limit the specific method of selecting the target color channel or the specific method of adjusting the pixel value.

[0086] In some possible implementations, a pixel value change amount for a target color channel may be determined. The pixel value change amount may be a fixed value, such as a positive fixed value X1, indicating that the pixel values ​​of the target color channel are uniformly increased by X1; or a negative fixed value X2, indicating that the pixel values ​​of the target color channel are uniformly increased by X2, thereby reducing the amount of computation.

[0087] In some possible implementations, the pixel value change can also be calculated based on the average pixel value of the target color channel. For example, a larger average pixel value results in a larger pixel value change, while a smaller average pixel value results in a smaller pixel value change, thereby reducing the impact on the image display effect. This disclosure does not limit the specific method for determining the pixel value change.

[0088] In some possible implementations, the step of determining the amount of change in the pixel value of the target color channel may include: determining the amount of change in the pixel value of the target color channel based on the pixel average value of the target color channel and a pixel change coefficient; the pixel change coefficient is a positive value or a negative value.

[0089] That is, a pixel change coefficient can be pre-set, and the pixel value change amount of the target color channel is determined based on the pixel change coefficient and the pixel average value of the target color channel. The pixel value change amount b can be expressed as follows:

[0090] b=round(y*z / 255) (1)

[0091] In formula (1), z represents the average pixel value of the target color channel, y represents the pixel change coefficient, and round() represents rounding the result to an integer. The pixel change coefficient y can be set to a positive value between 1 and 20, indicating an increase in the pixel value, or a negative value between -1 and -20, indicating a decrease in the pixel value. The pixel change coefficient y can be set to, for example, 3, -5, etc.

[0092] The smaller the value of the pixel change coefficient, that is, the smaller the watermark strength, the fewer traces displayed on the screen, the smaller the impact on the image display effect, the less likely it is to be noticed by the human eye, and the better the user experience; at the same time, the requirements for subsequent extraction are higher. Conversely, the larger the value of the pixel change coefficient, the greater the impact on the image display effect, the less difficult it is to subsequently extract the character encoding in the image, and the success rate and accuracy of the extraction can be improved. Therefore, the display effect can be optimized as much as possible when the watermark strength reaches a certain level (to ensure recognition accuracy). Those skilled in the art can set the size and positive and negative values ​​of the pixel change coefficient according to actual conditions, and this disclosure does not impose any restrictions on this.

[0093] In this way, by analyzing each coding sub-region of the coding region respectively, the pixel value change of the pixel points in the entire coding region can be obtained.

[0094] In some possible implementations, in step S14, the pixel values ​​of the pixels in the coding region may be adjusted according to the pixel value change in the coding region to obtain an adjusted coding region. Step S14 may include adjusting the pixel values ​​of the pixels in the coding region according to the pixel value change in each coding sub-region of the coding region to obtain the adjusted coding region.

[0095] For example, for any pixel in the coding sub-region to be adjusted, if the sum of the pixel value of the pixel in the target color channel and the pixel value change is greater than 0 or less than the pixel maximum value (e.g., 255), the sum is used as the adjusted pixel value; if the sum is less than or equal to 0, or greater than or equal to the pixel maximum value, 0 or the pixel maximum value is used as the adjusted pixel value, that is, the value is adjusted to 0 or the pixel maximum value at most. In this way, after adjustment, an adjusted coding sub-region can be obtained; by adjusting each coding sub-region of the coding region, an adjusted coding region can be obtained.

[0096] In some possible implementations, in step S15, after adjusting the coding sub-regions of each coding region of the image to be displayed, each adjusted coding region can be obtained, thereby obtaining a watermark image of the image to be displayed.

[0097] Figures 5a and 5b are schematic diagrams of the coding sub-region of the watermark embedding method of the embodiment of the present disclosure. As shown in Figure 5a, it is a coding sub-region of the image to be displayed. The pixel average values ​​of the three color channels R, G, and B are calculated as follows: R channel = 180; G channel = 56; B channel = 155. Among them, the pixel average value of the R channel is the largest, and the pixel average value of the G channel is the smallest. The R red channel and the G green channel can be selected as the target color channels. Assuming that the codeword corresponding to the coding sub-region is 1, the pixel change coefficient of the target color channel with the largest pixel average value is 3, and the pixel change coefficient of the target color channel with the smallest pixel average value is -5, then according to formula (1), the pixel value change of the R channel is 2; the pixel value change of the G channel is -1. In this case, the pixel values ​​of the R channel of the pixels in the coding sub-region are uniformly increased by 2, and the pixel values ​​of the G channel are uniformly reduced by 1, that is, the coding of the coding sub-region is completed, and the adjusted coding sub-region is obtained, as shown in Figure 5b.

[0098] In this way, by encoding each coding sub-region of any coding region separately, the information encryption of the coding region is completed; by encrypting each coding region of the image to be displayed separately, the information encryption of the entire image to be displayed is completed, and the watermark image of the image to be displayed is obtained.

[0099] In some possible implementations, the watermark image of the image to be displayed can be cached in the RAM memory of the processing component 21; the watermark image in the RAM memory is converted into a standard video format by a digital signal conversion chip (not shown), and integrated into a complete video stream, and output to the display component 22 (such as a display screen) for display.

[0100] In this example, the watermark embedding method of the embodiment of the present disclosure is implemented by the processing component 21 in the display component of the terminal, which can generate a watermark image in real time and improve the efficiency of watermark embedding. However, the present disclosure does not limit the specific components that implement the watermark embedding method of the embodiment of the present disclosure.

[0101] According to the embodiments of the present disclosure, information can be encrypted and encoded to obtain character codes, and the subcodes of the character codes corresponding to multiple coding areas of the image to be displayed can be determined according to preset arrangement rules. Then, the character codes can be embedded into the image to be displayed by changing the regional pixel values ​​to obtain a watermark image, completing the watermark embedding process. In order to trace the image by extracting the character codes in the image in subsequent processing, the impact of the watermark on the image can be reduced, the efficiency and accuracy of subsequent recognition can be improved, and the traceability of the image can be improved.

[0102] According to an embodiment of the present disclosure, a watermark extraction method is also provided, which can be applied to a server (e.g., a cloud server). FIG6 is a flow chart of the watermark extraction method according to an embodiment of the present disclosure. As shown in FIG6 , the watermark extraction method according to an embodiment of the present disclosure includes:

[0103] In step S61, information decoding processing is performed on the image to be extracted to obtain an information decoding result of the image to be extracted;

[0104] In step S62, the character encoding in the information decoding result is verified and corrected according to a preset verification rule to determine the verification result of the information decoding result;

[0105] In step S63, when the verification result is that the verification is passed, the verified third code is decrypted to obtain the watermark information of the image to be extracted.

[0106] For example, the image to be extracted can be any image that may contain infringing behavior, obtained from any channel (public channels on the Internet, reporting channels for infringed parties, etc.). The image to be extracted can be a screen shot obtained by photographing a display screen; an image obtained by screenshotting, downloading, recording, printing, etc.; an image obtained by compressing, cropping, flipping, rotating, etc. the original image, etc. The image to be extracted can be obtained by manual screening or automatically screened using certain rules or algorithms. This disclosure does not limit the acquisition channel or image type of the image to be extracted.

[0107] In some possible implementations, in step S61, the image to be extracted can be decoded using an information decoding network or other possible algorithms to obtain an information decoding result of the image to be extracted. The information decoding result may include whether a character code exists in the image, and the character code identified if a character code exists. The character code is encoded in the same base as the base used in the watermark embedding method, such as binary code, ternary code, etc. In this example, the character code can be binary code.

[0108] The information decoding result may also include other contents, such as a decoded binary image, a timestamp of the decoding process, etc. The present disclosure does not limit the specific contents included in the information decoding result.

[0109] In some possible implementations, the information decoding network may include a Transformer (a type of neural network architecture)-based deep learning network model, such as a segmentation model or a classification model; or may include a CNN (convolutional neural network)-based deep learning network model for image segmentation / classification. This disclosure does not limit the specific model type of the information decoding network.

[0110] When the information decoding network includes a segmentation model and the character encoding is a binary code, the segmentation model outputs a binary image after segmentation of the image to be extracted, which is then post-processed to obtain the identified character encoding; when the information decoding network includes a classification model and the character encoding is a binary code, the classification model can directly output the character encoding after the image to be extracted is classified.

[0111] In some possible implementations, in step S62, the character code in the information decoding result may be verified and error-corrected according to a preset verification rule to determine a verification result of the information decoding result and a verified information decoding result. In the case where the character code employs a two-layer verification scheme, the character code may first be verified and error-corrected using an outer second verification scheme, such as a CRC check, to obtain an error-corrected code. The error-corrected code may then be verified using an inner first verification scheme, such as a Hamming code check, to obtain a verification result of the information decoding result and a verified third code.

[0112] In some possible implementations, if the verification result is failure, it means that the correct encoded information cannot be extracted from the image to be extracted, the source of the image to be extracted cannot be traced, and no valid information can be identified. If the verification result is success, it means that the correct encoded information is extracted from the image to be extracted, and the decryption process can continue.

[0113] In some possible implementations, if the verification result is passed, the verified third code can be decrypted in step S63 using a decryption method corresponding to the preset encryption method to obtain the watermark information of the image to be extracted, thereby completing the watermark extraction process. Furthermore, the image to be extracted can be traced based on the display time information and terminal device information in the watermark information.

[0114] According to the embodiments of the present disclosure, the image to be extracted can be decoded, the decoded character encoding can be verified and corrected according to verification rules, and if the verification passes, the verified encoding can be decrypted to obtain the watermark information of the image to be extracted, thereby completing the watermark extraction process of the image. According to the embodiments of the present disclosure, the efficiency and success rate of watermark extraction can be improved, and the reliability and accuracy of the extracted encoding can be improved through verification and error correction.

[0115] The watermark extraction method according to the embodiment of the present disclosure is described below.

[0116] Figure 7 is a schematic diagram of a watermark extraction method according to an embodiment of the present disclosure. As shown in Figure 7, according to the watermark extraction method according to an embodiment of the present disclosure, information can be decoded from an input image to be extracted; the decoded character code can be verified and error-corrected; if the verification fails, "No valid information was identified" is output; if the verification passes, the verified code is decrypted and the decrypted watermark information is output, thereby completing the entire watermark extraction process.

[0117] In some possible implementations, according to the method of the embodiment of the present disclosure, the image to be extracted may be subjected to information decoding processing in step S61 through an information decoding network, and the information decoding network includes a decoding segmentation network or a decoding classification network. For example, the information decoding network may include a Transformer-based deep learning network model, such as a segmentation model (referred to as a decoding segmentation network) or a classification model (referred to as a decoding classification network). It may also include a CNN (convolutional neural network)-based deep learning network model for image segmentation / classification, etc. The present disclosure does not limit the specific model type of the information decoding network.

[0118] In some possible implementations, step S61 may include: when the information decoding network includes a decoding segmentation network, inputting the image to be extracted into the decoding segmentation network for processing to obtain a decoded binary image; performing image post-processing on the binary image to obtain a character code corresponding to the binary image, and the information decoding result includes the character code.

[0119] In other words, if the information decoding network includes a decoding and segmentation network, and the character encoding is binary, the image to be extracted is fed into the decoding and segmentation network for processing, resulting in a decoded binary image. This means that the pixel values ​​in the image can only be 0 and 255 (or other set values). By performing image post-processing on the binary image, for example, treating pixel values ​​0 and 255 as 0 and 1 in the encoding, respectively, and sorting them, the character encoding corresponding to the binary image can be obtained, which is then used as the information decoding result.

[0120] In this example, if the information decoding network includes a decoding classification network, and the character encoding is binary, the image to be extracted is input into the decoding classification network for processing, and the character encoding is directly output. This character encoding can be used as the information decoding result.

[0121] In this way, the information decoding process of the image to be extracted can be achieved, thereby improving the success rate and accuracy of decoding.

[0122] In some possible implementations, the information decoding network may be trained before being used for inference. Prior to step S61, the method further includes: training the information decoding network based on a preset sample set to obtain a trained information decoding network, wherein the sample set includes a plurality of sample pairs, each sample pair including a sample encoded image and a label image of the sample encoded image.

[0123] For example, a sample coded image can be input into an information decoding network for processing to obtain a sample decoding result. Based on the sample decoding result and the label image of the sample coded image, the network loss of the information decoding network can be determined. The parameters of the information decoding network can then be adjusted based on the network loss. If the training conditions are met, a trained information decoding network is obtained. This disclosure does not limit the specific training method.

[0124] However, the number of directly available labeled samples is relatively small, while training the information decoding network requires a large number of high-quality labeled samples. According to the embodiments of the present disclosure, sample encoded images and corresponding label images can be efficiently generated.

[0125] In some possible implementations, before step S61, the method further includes: adjusting the pixel values ​​of the pixel points in the first image on one or more color channels to obtain an adjusted second image; cropping the first image and the second image respectively to obtain multiple first cropping areas of the first image and multiple second cropping areas of the second image, the sizes and positions of the first cropping areas and the second cropping areas corresponding to each other; splicing part of the first cropping areas of the first image and part of the second cropping areas of the second image to obtain a sample coded image, the size of the sample coded image being the same as that of the first image; and generating a label image of the sample coded image, the label image being used to indicate that the pixel points of the sample coded image correspond to the first image or the second image.

[0126] For example, a camera tool (such as a smartphone) can be used to capture any watermark-free image displayed on the screen (referred to as the first image). The capture process can be performed manually or using an automated script. The present disclosure does not limit the method of capturing the first image.

[0127] Since the watermark embedding method in the previous article embeds the watermark by adjusting the pixel value of the target color channel, the selection and adjustment method of the target color channel during the watermark embedding period can also be used during the sample generation to improve the pertinence of the sample and thus improve the training effect of the network.

[0128] In some possible implementations, a target color channel for the first image can be determined based on the selection and adjustment method for the target color channel during watermark embedding. For example, the color channel with the largest pixel average value, the color channel with the smallest pixel average value, or any one or more of the three color channels R, G, and B, etc. Subsequently, the pixel values ​​of each pixel in the target color channel are adjusted to obtain an adjusted second image. Based on the adjustment method used during watermark embedding (increasing or decreasing the pixel value), the pixel value of the target color channel can be directly adjusted by a preset value, or the pixel value change amount can be determined based on a preset pixel change coefficient and the pixel value adjusted based on the pixel value change amount.

[0129] FIG8 is a schematic diagram illustrating pixel value adjustment in an embodiment of the present disclosure. As shown in FIG8 , a first image 81 can be split into channel images 82, 83, and 84 representing the three color channels R, G, and B. For the channel image of the target color channel, for example, channel image 82 representing the B channel, the pixel values ​​of each pixel in channel image 82 can be increased by a preset value to obtain an adjusted channel image 85. The three channel images 85, 83, and 84 can then be recombined to obtain an adjusted second image 86. This second image 86 can be considered a full-screen image with a watermark added.

[0130] In some possible implementations, the first image and the second image may be cropped separately to obtain multiple first cropped regions of the first image and multiple second cropped regions of the second image, with the sizes and positions of the first cropped regions and the second cropped regions corresponding to each other. In this example, to increase randomness, a random number generator may be used to determine the cropped regions, and the sizes and positions of the cropped regions of the first image and the second image may be made corresponding to each other so that they can be stitched together to form a correct new image.

[0131] In some possible implementations, a portion of a first cropped region of a first image and a portion of a second cropped region of a second image may be spliced ​​together to obtain a new image, namely, a sample coded image. The positions of the first cropped region and the second cropped region involved in the splicing are complementary so that they can be spliced ​​together to form a complete image, and the size of the spliced ​​sample coded image is the same as that of the first image.

[0132] In some possible implementations, a label image for the sample coded image can be generated based on the first and second cropped regions involved in the stitching. This label image indicates whether pixels in the sample coded image correspond to the first or second image. For example, pixels corresponding to the first image have a value of 0 in the label image, while pixels corresponding to the second image have a value of 1 in the label image. This facilitates accurate identification of each region by the information decoding network during training.

[0133] In this way, high-quality sample encoded images and corresponding label images can be generated efficiently.

[0134] In some possible implementations, sample encoded images and corresponding labeled images can be combined into sample pairs and added to a sample set for network training. Before training, the sample encoded images can also be preprocessed, such as normalization and data augmentation. Data augmentation can include, for example, random cropping, flipping, rotation, and compression to further increase the number of samples, enrich the training data in the sample set, increase sample diversity, and improve the robustness of the trained information decoding network. This disclosure does not limit the specific methods of data augmentation.

[0135] In some possible implementations, the sample set can be divided into a training dataset, a validation dataset, and a test dataset. The training dataset is used to train the information decoding network, the validation dataset is used to evaluate the performance of the information decoding network, and the test dataset is used to test the generalization ability of the information decoding network. It is important to ensure that the dataset is properly distributed to avoid overfitting and underfitting during network training.

[0136] Through the above processing method, multiple high-quality sample pairs can be efficiently generated, thereby improving the training effect of the information decoding network, so that the trained information decoding network can obtain better decoding performance and generalization ability.

[0137] After training, the image to be extracted can be subjected to information decoding processing by the information decoding network in step S61 to obtain an information decoding result of the image to be extracted.

[0138] In some possible implementations, in step S62, the character encoding in the information decoding result is verified and corrected according to a preset verification rule. The verification rule may correspond to the method of adding verification in the watermark embedding method described above. For example, if the watermark embedding process uses a two-layer verification method of Hamming code verification + CRC verification to add a verification code, the watermark extraction process also uses the Hamming code verification + CRC verification method for verification.

[0139] In some possible implementations, step S62 may include: verifying the character encoding according to a second verification method to obtain a first verification result; if the first verification result is a verification failure, correcting the character encoding according to the second verification method to obtain a fourth encoding after error correction; verifying the fourth encoding according to the first verification method to obtain a second verification result; if the second verification result is a verification pass, determining that the verification result of the information decoding result is a verification pass, and determining the third encoding after verification.

[0140] For example, when character encoding uses two layers of verification, the character encoding (bits a+i+j) in the information decoding result can be verified first using the outer layer's second verification method, such as CRC. For example, if a=4, i=j=3, the character encoding is a 10-bit binary code 1010 101 001. The same generating polynomial as used in the watermark embedding process (e.g., x^3+x+1) can be used to perform a division operation on the character encoding to obtain the remainder. If the remainder is 0, it indicates that the character encoding is error-free and no error correction is required. The first verification result is a pass, and the checksum can be removed to obtain the a+i-bit code (called the fifth code), for example, the 7-bit code 1010 101. If the remainder is not 0, it indicates that the received data has errors and error correction is required. The first verification result is a fail.

[0141] In some possible implementations, if the first check result is a check failure, the character code is error-corrected. The step of error-correcting the character code according to the second check method to obtain a corrected fourth code includes: multiplying each codeword of the character code by a generator polynomial corresponding to the second check method, adding multiple multiplication results to obtain a target polynomial, the target polynomial being used to indicate an error location in the character code; and performing an exclusive-OR operation on the target polynomial and the character code to obtain the corrected fourth code.

[0142] For example, each codeword in the character code can be bitwise multiplied by the CRC check generator polynomial and the results added to obtain a target polynomial, which is used to indicate the error position of the character code. The target polynomial indicating the error position is XORed with the character code to achieve error correction and obtain the corrected character code. The corrected character code is then subjected to a CRC check again. If the remainder is 0, the error correction is successful; otherwise, it indicates a failure, and the output is "No valid information recognized." If the error correction is successful, the checksum can be removed to obtain the a+i bit code, which is the fourth code after error correction.

[0143] Furthermore, according to the first verification method of the inner layer, the fourth code (a+i bits, for example, the 7-bit code 1010 101), or the fifth code (a+i bits, for example, the 7-bit code 1010 101) when the first verification result is verification passed, can be verified to obtain a second verification result.

[0144] In some possible implementations, if the second verification result is a failure, it indicates that the information extraction was erroneous, and "no valid information was identified" may be output; if the second verification result is a success, it indicates that the information extraction was correct, and the verification result of the information decoding result is determined to be a success. Furthermore, the checksum may be removed from the fourth or fifth code (a+i bits, e.g., a 7-bit code 1010 101) to obtain a verified third code (a bit, e.g., a 4-bit code 1010).

[0145] In this way, the ability to identify the correctness of the extracted character encoding can be improved, errors in data transmission can be repaired, and the reliability of the data can be guaranteed, thereby improving the accuracy of watermark recognition.

[0146] If the information decoding result is verified to have passed, the verified third code can be decrypted in step S63 using a decryption method corresponding to the preset encryption method to obtain the watermark information of the image to be extracted, thereby completing the watermark extraction process. Furthermore, the image to be extracted can be traced based on the display time information and terminal device information in the watermark information.

[0147] According to the embodiments of the present disclosure, the image to be extracted can be decoded, the decoded character encoding can be verified and corrected according to verification rules, and if the verification passes, the verified encoding can be decrypted to obtain the watermark information of the image to be extracted, thereby completing the watermark extraction process of the image. According to the embodiments of the present disclosure, the efficiency and success rate of watermark extraction can be improved, and the reliability and accuracy of the extracted encoding can be improved through verification and error correction.

[0148] According to an embodiment of the present disclosure, a watermark embedding device is also provided. FIG9 is a block diagram of the watermark embedding device of an embodiment of the present disclosure. As shown in FIG9 , the device includes:

[0149] The encryption module 91 is used to encrypt and encode the information to be encoded corresponding to the image to be displayed, obtain the encrypted character code, and split the character code into multiple sub-codes;

[0150] A subcode determination module 92 is configured to determine subcodes corresponding to the plurality of coding regions of the image to be displayed, respectively, according to a preset arrangement rule;

[0151] A change amount determination module 93 is configured to determine a pixel value change amount of a pixel point in any coding region based on a pixel value of the pixel point in the coding region and a subcode corresponding to the coding region;

[0152] A pixel adjustment module 94 is configured to adjust the pixel values ​​of the pixels in the coding region according to the pixel value change in the coding region to obtain an adjusted coding region;

[0153] The image obtaining module 95 is configured to obtain a watermark image of the image to be displayed according to each adjusted coding region of the image to be displayed.

[0154] In some possible implementations, the encryption module 91 is used to: encrypt and encode the information to be encoded to obtain an initial first code; verify the first code using a first verification method, generate a first verification code and add it to the first code to obtain a second code; verify the second code using a second verification method, generate a second verification code and add it to the second code to obtain a character code.

[0155] In some possible implementations, before the encryption module 91, the device further includes: a region division module, used to divide the image to be displayed into n×m image regions, each image region serving as a coding region, n and m are integers and n≥3, m≥4; wherein the second number of the coding regions is greater than the first number of the subcodes of the character encoding.

[0156] In some possible implementations, before the subcode determination module 92, the device further includes: an arrangement rule determination module, configured to determine the arrangement rule based on a first number of subcodes of the character encoding and a second number of encoding areas, wherein the arrangement rule includes that the subcode corresponding to any encoding area is different from the subcode corresponding to an encoding area adjacent to the encoding area.

[0157] In some possible implementations, each coding region includes multiple coding sub-regions, each coding sub-region corresponding to a codeword of a subcode. The change amount determination module 93 is configured to: determine, for any coding sub-region in the coding region, the codeword corresponding to the coding sub-region based on the subcode corresponding to the coding region; and determine the pixel value change amount of the pixel point in the coding sub-region based on the pixel value of the pixel point in the coding sub-region and the codeword corresponding to the coding sub-region;

[0158] The pixel adjustment module 94 is configured to adjust the pixel values ​​of the pixels in the coding region according to the pixel value changes of the pixels in each coding sub-region of the coding region to obtain an adjusted coding region.

[0159] In some possible implementations, the change amount determination module 93 is used to: determine whether the pixel value of the pixel point in the coding sub-region needs to be adjusted based on the code word corresponding to the coding sub-region; if the pixel value of the pixel point in the coding sub-region needs to be adjusted, determine the pixel average value of the pixel point in the coding sub-region on each color channel based on the pixel value of the pixel point in the coding sub-region; determine the target color channel and the pixel value change amount of the target color channel based on the pixel average value of the pixel point in the coding sub-region on each color channel.

[0160] In some possible implementations, the target color channel includes any one of the following: a color channel with the largest pixel average value; a color channel with the smallest pixel average value; one or more of the three color channels R, G, and B; wherein the change amount determination module 93 is used to: determine the pixel value change amount of the target color channel based on the pixel average value and the pixel change coefficient of the target color channel; the pixel change coefficient is a positive value or a negative value.

[0161] In some possible implementations, the device is applied to a terminal, and the information to be encoded includes at least one of the following: generation time information of the image to be displayed, generation location information of the image to be displayed, display time information of the image to be displayed, device information of the terminal, and location information of the terminal; wherein, multiple images to be displayed within a preset display time period correspond to the same information to be encoded.

[0162] According to an embodiment of the present disclosure, a watermark extraction device is also provided. FIG10 is a block diagram of the watermark extraction device of an embodiment of the present disclosure. As shown in FIG10 , the device includes:

[0163] A decoding module 71 is used to perform information decoding processing on the image to be extracted to obtain an information decoding result of the image to be extracted;

[0164] A verification module 72 is configured to verify and correct the character encoding in the information decoding result according to a preset verification rule, and determine a verification result of the information decoding result;

[0165] The decryption module 73 is configured to decrypt the verified third code to obtain the watermark information of the image to be extracted when the verification result is that the verification is passed.

[0166] In some possible implementations, the device performs information decoding processing on the image to be extracted through an information decoding network, and the information decoding network includes a decoding segmentation network or a decoding classification network.

[0167] In some possible implementations, the decoding module 71 is used to: when the information decoding network includes a decoding segmentation network, input the image to be extracted into the decoding segmentation network for processing to obtain a decoded binary image; perform image post-processing on the binary image to obtain a character code corresponding to the binary image, and the information decoding result includes the character code.

[0168] In some possible implementations, the apparatus further includes: a training module configured to train the information decoding network based on a preset sample set to obtain a trained information decoding network, wherein the sample set includes a plurality of sample pairs, each sample pair including a sample encoded image and a label image of the sample encoded image;

[0169] In some possible implementations, the device further includes: an adjustment module for adjusting the pixel values ​​of pixel points in the first image on one or more color channels to obtain an adjusted second image; a cropping module for cropping the first image and the second image respectively to obtain multiple first cropping areas of the first image and multiple second cropping areas of the second image, the sizes and positions of the first cropping areas and the second cropping areas corresponding to each other; a splicing module for splicing part of the first cropping areas of the first image and part of the second cropping areas of the second image to obtain a sample coded image, the size of the sample coded image being the same as that of the first image; and a label generation module for generating a label image for the sample coded image, the label image being used to indicate that the pixel points of the sample coded image correspond to the first image or the second image.

[0170] In some possible implementations, the verification module 72 is used to: verify the character encoding according to the second verification method to obtain a first verification result; if the first verification result is a verification failure, correct the character encoding according to the second verification method to obtain a fourth encoding after error correction; verify the fourth encoding according to the first verification method to obtain a second verification result; if the second verification result is a verification pass, determine that the verification result of the information decoding result is a verification pass, and determine the third encoding after verification.

[0171] In some possible implementations, the verification module 72 is used to: multiply each codeword of the character code by a generating polynomial corresponding to the second verification method, and add multiple multiplication results to obtain a target polynomial, wherein the target polynomial is used to indicate an error position of the character code; and perform XOR processing on the target polynomial and the character code to obtain a fourth code after error correction.

[0172] Figure 11 is a schematic diagram of the structure of an electronic device according to an embodiment of the present disclosure. As shown in Figure 11, an electronic device according to an embodiment of the present disclosure includes: one or more processors 101, a memory 102, and one or more I / O interfaces 103. The memory 102 stores one or more programs, which, when executed by the one or more processors, enable the one or more processors to implement any of the watermark embedding methods or watermark extraction methods described in the above embodiments. The one or more I / O interfaces 103 are connected between the processors and the memory and are configured to enable information exchange between the processors and the memory.

[0173] Among them, the processor 101 is a device with data processing capabilities, including but not limited to a central processing unit (CPU); the memory 102 is a device with data storage capabilities, including but not limited to random access memory (RAM, more specifically SDRAM, DDR, etc.), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), and flash memory (FLASH); the I / O interface (read-write interface) 103 is connected between the processor 101 and the memory 102, and can realize information interaction between the processor 101 and the memory 102, including but not limited to a data bus (Bus), etc.

[0174] In some embodiments, the processor 101 , the memory 102 , and the I / O interface 103 are connected to each other via a bus 104 , and further connected to other components of the computing device.

[0175] In some embodiments, when executing the watermark embedding method, the one or more processors 101 include a field programmable gate array (FPGA). The one or more processors 101 may also include hardware with high image processing capabilities, such as a graphics processor (GPU) and an image signal processor (ISP). As long as the watermark embedding method according to the embodiments of the present disclosure can be implemented, the present disclosure does not limit the specific hardware type of the processor.

[0176] According to an embodiment of the present disclosure, a computer-readable medium is further provided, wherein the computer-readable medium stores a computer program, wherein when the program is executed by a processor, the program implements the steps of any of the watermark embedding methods or watermark extraction methods in the above embodiments.

[0177] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a machine-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication part, and / or installed from a removable medium. When the computer program is executed by a central processing unit (CPU), the above-mentioned functions defined in the system of the present disclosure are executed.

[0178] It should be noted that the computer-readable medium described in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device, or component. In the present disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. Program code embodied on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wireline, optical fiber cable, RF, or any suitable combination thereof.

[0179] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the aforementioned module, program segment, or a part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0180] The circuits or sub-circuits described in the embodiments of the present disclosure may be implemented in software or hardware. The described circuits or sub-circuits may also be provided in a processor. For example, they may be described as: a processor comprising: a receiving circuit and a processing circuit, wherein the processing module comprises a writing sub-circuit and a reading sub-circuit. The names of these circuits or sub-circuits do not, in certain circumstances, constitute limitations on the circuits or sub-circuits themselves. For example, a receiving circuit may also be described as "receiving a video signal."

[0181] It is understood that the above embodiments are merely exemplary embodiments for illustrating the principles of the present disclosure, and the present disclosure is not limited thereto. Those skilled in the art may make various modifications and improvements without departing from the spirit and substance of the present disclosure, and such modifications and improvements are also considered to be within the scope of protection of the present disclosure.

Claims

1. A watermark embedding method, characterized in that: The method comprises: Encrypting and encoding the information to be encoded corresponding to the image to be displayed to obtain an encrypted character code, and splitting the character code into multiple subcodes; According to a preset arrangement rule, respectively determining subcodes corresponding to a plurality of coding regions of the image to be displayed; Determine a pixel value change amount of a pixel point in any coding area according to a pixel value of a pixel point in the coding area and a subcode corresponding to the coding area; According to the pixel value change amount of the coding area, the pixel value of the pixel point in the coding area is adjusted to obtain an adjusted coding area; A watermark image of the image to be displayed is obtained according to each adjusted coding region of the image to be displayed.

2. The method according to claim 1, characterized in that The step of encrypting and encoding the information to be encoded corresponding to the image to be displayed to obtain the encrypted character code comprises: Encrypting and encoding the information to be encoded to obtain an initial first code; Verify the first code by a first verification method, generate a first verification code and add it to the first code to obtain a second code; The second code is verified by a second verification method, a second verification code is generated and added to the second code to obtain the character code.

3. The method according to claim 1, characterized in that Before encrypting and encoding the information to be encoded corresponding to the image to be displayed, the method further includes: Divide the image to be displayed into n×m image regions, each image region being a coding region, where n and m are integers and n≥3 and m≥4; The second number of the encoding regions is greater than the first number of the subcodes of the character encoding.

4. The method according to claim 1, characterized in that Before determining the subcodes corresponding to the multiple coding areas of the image to be displayed respectively according to the preset arrangement rule, the method further includes: The arrangement rule is determined according to the first number of subcodes of the character code and the second number of coding areas, wherein the arrangement rule includes that the subcode corresponding to any coding area is different from the subcode corresponding to a coding area adjacent to the coding area.

5. The method according to claim 1, characterized in that Each coding region includes a plurality of coding sub-regions, each coding sub-region corresponding to a codeword of the sub-code; The step of determining the pixel value change amount of the pixel point in any coding area according to the pixel value of the pixel point in the coding area and the subcode corresponding to the coding area includes: For any coding sub-region in the coding region, determine a codeword corresponding to the coding sub-region according to a subcode corresponding to the coding region; Determine a pixel value change amount of a pixel point in the coding sub-region according to a pixel value of a pixel point in the coding sub-region and a codeword corresponding to the coding sub-region; The step of adjusting the pixel values ​​of the pixels in the coding area according to the pixel value change amount of the coding area to obtain the adjusted coding area includes: According to the pixel value changes of the pixel points in each coding sub-region of the coding region, the pixel values ​​of the pixel points in the coding region are adjusted respectively to obtain an adjusted coding region.

6. The method according to claim 5, characterized in that Determining a pixel value change amount of a pixel point in the coding sub-region according to a pixel value of a pixel point in the coding sub-region and a codeword corresponding to the coding sub-region includes: Determining whether a pixel value of a pixel point in the coding sub-region needs to be adjusted according to a codeword corresponding to the coding sub-region; When the pixel value of the pixel point in the coding sub-region is to be adjusted, determining the pixel average value of the pixel point in the coding sub-region on each color channel according to the pixel value of the pixel point in the coding sub-region; According to the pixel average values ​​of the pixel points in the coding sub-region on each color channel, a target color channel and a pixel value change amount of the target color channel are determined.

7. The method according to claim 6, characterized in that The target color channel includes any one of the following: a color channel with the largest pixel average value; a color channel with the smallest pixel average value; one or more of the three color channels of R, G, and B; The step of determining the target color channel and the pixel value change amount of the target color channel according to the pixel average values ​​of the pixel points in the coding sub-region on each color channel includes: The pixel value change amount of the target color channel is determined according to the pixel average value and the pixel change coefficient of the target color channel; the pixel change coefficient is a positive value or a negative value.

8. The method according to claim 1, characterized in that The method is applied to a terminal, and the information to be encoded includes at least one of the following: generation time information of the image to be displayed, generation position information of the image to be displayed, display time information of the image to be displayed, device information of the terminal, and position information of the terminal; The multiple images to be displayed within the preset display time period correspond to the same information to be encoded.

9. A watermark extraction method, characterized in that: include: Performing information decoding processing on the image to be extracted to obtain an information decoding result of the image to be extracted; According to a preset verification rule, the character encoding in the information decoding result is verified and corrected to determine the verification result of the information decoding result; If the verification result is that the verification is passed, the verified third code is decrypted, The watermark information of the image to be extracted is obtained.

10. The method according to claim 9, characterized in that The method performs information decoding processing on the image to be extracted through an information decoding network, and the information decoding network includes a decoding segmentation network or a decoding classification network.

11. The method according to claim 10, characterized in that in, The performing information decoding processing on the image to be extracted to obtain the information decoding result of the image to be extracted includes: In the case where the information decoding network includes a decoding segmentation network, the image to be extracted is input into the decoding segmentation network for processing to obtain a decoded binary image; The binary image is subjected to image post-processing to obtain a character code corresponding to the binary image, and the information decoding result includes the character code.

12. The method according to claim 10, characterized in that The method further comprises: The information decoding network is trained according to a preset sample set to obtain a trained information decoding network, wherein the sample set includes a plurality of sample pairs, and each sample pair includes a sample encoded image and a label image of the sample encoded image.

13. The method according to claim 12, characterized in that The method further comprises: Adjusting pixel values ​​of pixel points in the first image on one or more color channels to obtain an adjusted second image; Cropping the first image and the second image respectively to obtain a plurality of first cropping regions of the first image and a plurality of second cropping regions of the second image, wherein the sizes and positions of the first cropping regions and the second cropping regions correspond to each other; The first image is partially cropped and the second image is partially cropped to obtain a sample coded image. The size of the sample coded image is the same as that of the first image. The images are the same; A label image of the sample coded image is generated, where the label image is used to indicate that a pixel point of the sample coded image corresponds to the first image or the second image.

14. The method according to claim 9, characterized in that The step of checking and correcting the character encoding in the information decoding result according to a preset checking rule, and determining the checking result of the information decoding result and the third encoding after checking, comprises: Verify the character code according to the second verification method to obtain a first verification result; When the first verification result is a verification failure, correcting the character code according to the second verification method to obtain a fourth code after error correction; Verify the fourth code according to the first verification method to obtain a second verification result; In the case where the second verification result is a passed verification, the verification result of the information decoding result is determined to be a passed verification, and a third encoding after the verification is determined.

15. The method according to claim 14, characterized in that The second verification method includes a cyclic redundancy check method, wherein the error correction of the character code according to the second verification method to obtain a fourth code after error correction includes: Multiplying each codeword of the character code by a generator polynomial corresponding to the second verification mode, and adding multiple multiplication results to obtain a target polynomial, wherein the target polynomial is used to indicate an error position of the character code; The target polynomial and the character code are subjected to an XOR process to obtain a fourth code after error correction.

16. A watermark embedding device, characterized in that: The device comprises: An encryption module, used for encrypting and encoding the information to be encoded corresponding to the image to be displayed, obtaining an encrypted character code, and splitting the character code into a plurality of subcodes; A subcode determination module, used to determine the subcodes corresponding to the multiple coding areas of the image to be displayed respectively according to a preset arrangement rule; A change amount determination module, used to determine a change amount of a pixel value of a pixel point in any coding area according to a pixel value of a pixel point in the coding area and a subcode corresponding to the coding area; A pixel adjustment module, used to adjust the pixel values ​​of the pixels in the coding area according to the pixel value change amount of the coding area to obtain an adjusted coding area; The image acquisition module is used to obtain the watermark image of the image to be displayed according to each adjusted coding area of ​​the image to be displayed.

17. A watermark extraction device, characterized in that: The device comprises: A decoding module, used for performing information decoding processing on the image to be extracted to obtain the information decoding result of the image to be extracted; A verification module, used to verify and correct the character encoding in the information decoding result according to a preset verification rule, and determine the verification result of the information decoding result; The decryption module is used to decrypt the verified third code to obtain the watermark information of the image to be extracted when the verification result is that the verification is passed.

18. An electronic device, characterized in that: include: one or more processors; A memory for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the watermark embedding method as described in any one of claims 1 to 8 or the watermark extraction method as described in any one of claims 9 to 15.

19. The electronic device according to claim 18, characterized in that: The processor includes a field programmable gate array FPGA.

20. A computer readable medium having a computer program stored thereon, wherein: The computer program implements the steps in the watermark embedding method according to any one of claims 1 to 8 or the watermark extraction method according to any one of claims 9 to 15 when executed by a processor.

Images