Watermark extraction method, model training method, watermark addition method, and electronic apparatus

Abstract

Provided in the present disclosure are a watermark extraction method, a model training method, a watermark addition method, and an electronic apparatus. The watermark extraction method comprises: acquiring an image to be processed, into which a watermark is embedded, wherein the watermark comprises a plurality of positioning blocks and a plurality of information blocks; performing identification on said image by using a first identification model, so as to identify at least some positioning blocks among the plurality of positioning blocks; performing warping processing on said image by using the current positional relationship of the at least some positioning blocks and a reference positional relationship of the at least some positioning blocks, such that the current positional relationship of the plurality of positioning blocks and a reference positional relationship of the plurality of positioning blocks are consistent in the warped image to be processed; according to the reference positional relationship of the plurality of positioning blocks, extracting a plurality of information blocks from the warped image to be processed; and performing identification on the plurality of information blocks by using a second identification model, so as to acquire watermark information which is carried in the plurality of information blocks.

Description

Watermark extraction method, model training method, watermark adding method and electronic device

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This disclosure is based on and claims priority to an application with CN application number 202310721591.0 and filing date June 16, 2023. The disclosure content of the CN application is hereby incorporated into this disclosure as a whole. Technical Field

[0003] The present disclosure relates to the field of information technology, and in particular to a watermark extraction method, a model training method, a watermark adding method and an electronic device. Background Art

[0004] In order to extract the watermark information embedded in the image, after the user uses the image acquisition device to capture the currently presented image, the user uses the trained machine learning model to identify the captured image to extract the watermark information embedded in the image.

[0005] Summary of the Invention

[0006] According to a first aspect of an embodiment of the present disclosure, a watermark information extraction method is provided, comprising: obtaining an image to be processed in which a watermark is embedded, wherein the watermark includes a plurality of positioning blocks and a plurality of information blocks; identifying the image to be processed using a first recognition model to identify at least some of the plurality of positioning blocks; when the number of at least some of the positioning blocks is greater than a preset threshold, deforming the image to be processed using a current position relationship of the at least some of the positioning blocks and a reference position relationship of the at least some of the positioning blocks, so that in the deformed image to be processed, the current position relationship of the plurality of positioning blocks is consistent with the reference position relationship of the plurality of positioning blocks; extracting the plurality of information blocks from the deformed image to be processed based on the reference position relationship of the plurality of positioning blocks; and identifying the plurality of information blocks using a second recognition model to obtain watermark information carried by the plurality of information blocks.

[0007] In some embodiments, using the first recognition model to identify the image to be processed includes: using the encoder in the first recognition model to encode the image to be processed to obtain multiple first encoding feature vectors, wherein among the multiple first encoding feature vectors, the weight of the first encoding feature vector associated with the positioning block is greater than the weight of the first encoding feature vector associated with the information block; using the decoder in the first recognition model to decode the multiple encoding feature vectors to obtain multiple first decoding feature vectors; and using the multiple first decoding feature vectors to identify at least part of the positioning block.

[0008] In some embodiments, among the multiple first decoded feature vectors, the weight of the first decoded feature vector associated with the positioning block is greater than the weight of the first decoded feature vector associated with the information block.

[0009] In some embodiments, identifying the multiple information blocks by using the second identification model includes: encoding each of the multiple information blocks by using an encoder in the second identification model to obtain a second encoded feature vector; performing decoding processing on the second encoded feature vector by using a decoder in the second identification model to obtain a second decoded feature vector; and obtaining the watermark information of each information block by using the second decoded feature vector.

[0010] In some embodiments, the encoder in the second identification model includes multiple sub-encoders connected in sequence; encoding each of the multiple information blocks by using the encoder in the second identification model includes: processing the input feature vector of the i-th sub-encoder by using the i-th sub-encoder, and using the output feature vector of the i-th sub-encoder as the input feature vector of the (i + 1)-th sub-encoder, where the input feature vector of the first sub-encoder is the embedded feature vector of each information block, and the output feature vector of the N-th sub-encoder is the second encoded feature vector, 1 ≤ i < N, and N is the total number of sub-encoders.

[0011] In some embodiments, each of the multiple sub-encoders includes multiple self-attention models; processing the input feature vector of the i-th sub-encoder by using the i-th sub-encoder includes: processing the input feature vector of the i-th sub-encoder by using each self-attention model in the multiple self-attention models of the i-th sub-encoder to obtain sub-feature vectors, so as to obtain multiple sub-feature vectors; and fusing the multiple sub-feature vectors to obtain the output feature vector of the i-th sub-encoder.

[0012] In some embodiments, fusing the multiple sub-feature vectors includes: concatenating the multiple sub-feature vectors.

[0013] In some embodiments, performing deformation processing on the image to be processed includes: determining affine transformation parameters by using the current position relationship of at least part of the positioning blocks and the reference position relationship of at least part of the positioning blocks; and performing an affine transformation on the image to be processed by using the affine transformation parameters to obtain the deformed image to be processed.

[0014] According to a second aspect of an embodiment of the present disclosure, a model training method is provided, comprising: obtaining a sample image embedded with a watermark, wherein the watermark includes a plurality of positioning blocks and a plurality of information blocks; identifying the sample image using a first recognition model to identify at least some of the plurality of positioning blocks; training the first recognition model using the at least some of the positioning blocks and positioning block annotation information; deforming the sample image using the current position relationship of the at least some of the positioning blocks and the reference position relationship of the at least some of the positioning blocks when the number of the at least some of the positioning blocks is greater than a preset threshold, so that in the deformed sample image, the current position relationship of the plurality of positioning blocks is consistent with the reference position relationship of the plurality of positioning blocks; extracting the plurality of information blocks from the deformed sample image based on the reference position relationship of the plurality of positioning blocks; identifying the plurality of information blocks using a second recognition model to obtain watermark information carried by the plurality of information blocks; and training the second recognition model using the watermark information and watermark annotation information.

[0015] In some embodiments, using the first recognition model to identify the sample image includes: preprocessing the sample image to obtain multiple enhanced sample images, wherein the preprocessing includes at least one of color temperature adjustment or saturation adjustment; and using the first recognition model to identify each enhanced sample image in the multiple enhanced sample images.

[0016] In some embodiments, preprocessing the sample image includes: filtering the sample image; and performing the preprocessing on the filtered sample image.

[0017] In some embodiments, using a second recognition model to identify the multiple information blocks includes: sorting the multiple information blocks in multiple different orders to obtain multiple queues; and using the second recognition model to identify the multiple information blocks in each queue in the multiple queues to obtain the watermark information.

[0018] According to a third aspect of an embodiment of the present disclosure, a watermark adding method is provided, comprising: generating a watermark, wherein the watermark includes a plurality of positioning blocks and a plurality of information blocks, the plurality of positioning blocks having a reference position relationship, and the plurality of information blocks carrying watermark information to be added; and embedding the watermark into a specified area of ​​a target image.

[0019] In some embodiments, the plurality of information blocks include a plurality of first information blocks for representing a first character and a plurality of second information blocks for representing a second character.

[0020] In some embodiments, embedding the watermark in a designated area of ​​the target image includes: superimposing the watermark with an image in the designated area of ​​the target image.

[0021] In some embodiments, the colors of the positioning block, the first information block, and the second information block are colors to which human vision is insensitive and are different from each other.

[0022] In some embodiments, among the positioning block, the first information block, and the second information block, the colors of any two blocks are colors to which human vision is insensitive and are different from each other, and among the positioning block, the first information block, and the second information block, one block other than the any two blocks is colorless.

[0023] In some embodiments, the designated area includes a first sub-area corresponding to the multiple positioning blocks, a second sub-area corresponding to the multiple first information blocks, and a third sub-area corresponding to the multiple second information blocks. Embedding the watermark into the designated area of ​​the target image includes: selecting two sub-areas from the first sub-area, the second sub-area, and the third sub-area; determining a first parameter value using the average grayscale value of pixels in one of the two sub-areas; adjusting the grayscale value of each pixel in one of the two sub-areas using the first parameter value; determining a second parameter value using the average grayscale value of pixels in the other of the two sub-areas, wherein the second parameter value is different from the first parameter value; and adjusting the grayscale value of each pixel in the other of the two sub-areas using the second parameter value.

[0024] In some embodiments, embedding the watermark into a designated area of ​​the target image includes: determining a third parameter value using the average grayscale value of pixels in the first sub-area, the second sub-area, and the remaining sub-areas other than the two sub-areas in the third sub-area, wherein the third parameter value is different from the first parameter value and the second parameter value; and adjusting the grayscale value of each pixel in the remaining sub-area using the third parameter value.

[0025] In some embodiments, the designated area includes a plurality of sub-areas, and embedding the watermark in the designated area of ​​the target image includes: embedding the watermark in each of the plurality of sub-areas.

[0026] According to a fourth aspect of an embodiment of the present disclosure, an electronic device is provided, comprising: a memory; and a processor coupled to the memory, the processor being configured to execute a method as described in any of the above embodiments based on instructions stored in the memory.

[0027] According to a fifth aspect of an embodiment of the present disclosure, a non-volatile computer-readable storage medium is provided, wherein the non-volatile computer-readable storage medium stores computer instructions, and when the instructions are executed by a processor, the method described in any of the above embodiments is implemented.

[0028] Other features and advantages of the present disclosure will become apparent from the following detailed description of exemplary embodiments of the present disclosure with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments of the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0030] The present disclosure can be more clearly understood from the following detailed description with reference to the accompanying drawings, in which:

[0031] FIG1 is a schematic flow chart of a watermark information extraction method according to an embodiment of the present disclosure;

[0032] FIG2 is a schematic diagram of a watermark according to an embodiment of the present disclosure;

[0033] FIG3 is a schematic diagram of a template library according to an embodiment of the present disclosure;

[0034] FIG4 is a schematic structural diagram of a first recognition model according to an embodiment of the present disclosure;

[0035] 5A-5C are schematic diagrams of affine transformation according to some embodiments of the present disclosure;

[0036] FIG6 is a schematic structural diagram of a second recognition model according to an embodiment of the present disclosure;

[0037] FIG7 is a schematic structural diagram of an encoder according to an embodiment of the present disclosure;

[0038] FIG8 is a schematic structural diagram of a sub-encoder according to an embodiment of the present disclosure;

[0039] FIG9 is a flow chart of a model training method according to an embodiment of the present disclosure;

[0040] FIG10 is a schematic flow chart of a watermark adding method according to an embodiment of the present disclosure;

[0041] FIG11 is a schematic diagram of an image with a watermark embedded therein according to an embodiment of the present disclosure;

[0042] FIG12 is a schematic diagram of an image with a watermark embedded therein according to another embodiment of the present disclosure;

[0043] FIG13 is a schematic structural diagram of an electronic device according to an embodiment of the present disclosure.

[0044] It should be understood that the size of each part shown in the drawings is not drawn according to the actual proportional relationship.In addition, the same or similar reference numerals represent the same or similar components. DETAILED DESCRIPTION

[0045] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. The description of the exemplary embodiments is merely illustrative and is in no way intended to limit the present disclosure, its application, or use. The present disclosure can be implemented in many different forms and is not limited to the embodiments described herein. These embodiments are provided to make the present disclosure thorough and complete and to fully convey the scope of the present disclosure to those skilled in the art. It should be noted that unless otherwise specifically stated, the relative arrangement of the components and steps, the composition of the materials, and the numerical values ​​set forth in these embodiments should be interpreted as being merely exemplary and not as limiting.

[0046] The terms "first," "second," and similar terms used in this disclosure do not denote any order, quantity, or importance, but are simply used to distinguish one part from another. Terms such as "include" or "comprise" mean that the elements preceding the term include the elements listed after the term, and do not exclude the possibility of also including other elements.

[0047] All terms (including technical or scientific terms) used in this disclosure have the same meaning as those understood by one of ordinary skill in the art to which this disclosure belongs, unless otherwise specifically defined. It should also be understood that terms defined in, for example, general dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant technology, and should not be interpreted in an idealized or highly formal sense, unless explicitly defined herein.

[0048] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered part of the specification.

[0049] The inventors have noticed that when a user uses an image acquisition device to capture the currently presented image, the captured image will usually be deformed due to the acquisition angle, thereby increasing the difficulty of identifying the watermark information and reducing the success rate of watermark information extraction.

[0050] Based on this, the present disclosure provides a watermark information extraction solution that can effectively extract watermark information embedded in a deformed image.

[0051] Figure 1 is a flow chart of a watermark information extraction method according to an embodiment of the present disclosure. In some embodiments, the watermark information extraction method is executed by a watermark information extraction device.

[0052] In step 101, an image to be processed with a watermark embedded therein is obtained, wherein the watermark includes a plurality of positioning blocks and a plurality of information blocks.

[0053] For example, a watermark is embedded in the currently displayed image. A user uses an image acquisition device to capture the currently displayed image to obtain an image to be processed. The watermark embedded in the image to be processed is shown in FIG2 .

[0054] In FIG2 , the watermark includes multiple image blocks, including multiple positioning blocks 21 and multiple information blocks. The multiple information blocks include multiple information blocks 22 corresponding to the character “0” and multiple information blocks 23 corresponding to the character “1”.

[0055] In some embodiments, after acquiring the image to be processed with the embedded watermark, filtering is performed on the image to be processed to remove moiré patterns generated when the image to be processed is acquired by photographing.

[0056] For example, the filtering process includes filtering processes such as median filtering and mean filtering.

[0057] In step 102 , the image to be processed is recognized using a first recognition model to identify at least some of the plurality of positioning blocks.

[0058] In some embodiments, an encoder in a first recognition model is used to encode an image to be processed to obtain a plurality of first encoded feature vectors, wherein, among the plurality of first encoded feature vectors, the weight of the first encoded feature vector associated with the positioning block is greater than the weight of the first encoded feature vector associated with the information block. Next, a decoder in the first recognition model is used to decode the plurality of encoded feature vectors to obtain a plurality of first decoded feature vectors, and then the plurality of first decoded feature vectors are used to identify at least a portion of the positioning block.

[0059] It should be noted that, due to the deformation of the image to be processed, the positioning blocks in the watermark embedded in the image to be processed will also be deformed. For example, a positioning block originally shaped as a square may become a rectangle, diamond, trapezoid, or other quadrilateral shape after deformation, as shown in Figure 3.

[0060] Therefore, to improve the success rate of identifying the positioning block, a pre-set template library is provided with a variety of quadrilateral templates, as shown in Figure 3. During the encoder's encoding of the image to be processed, the weight of the encoded feature vector that matches the template feature vector in the template library is increased, thereby improving the success rate of identifying the positioning block.

[0061] In some embodiments, among the plurality of first decoded feature vectors, a weight of a first decoded feature vector associated with a positioning block is greater than a weight of a first decoded feature vector associated with an information block.

[0062] It should be noted that, when the decoder decodes multiple encoded feature vectors, the weight of the first decoded feature vector that matches the template feature vector in the template library is further increased to further improve the recognition success rate of the positioning block.

[0063] FIG4 is a schematic diagram of a first recognition model according to an embodiment of the present disclosure. As shown in FIG4 , the first recognition model includes an encoder 41 and a decoder 42. The encoder 41 is provided with three sequentially connected convolutional layers 411-413, and the decoder 42 is provided with three sequentially connected convolutional layers 421-423. In the encoder 41, weighted processing is performed on the output of a specified convolutional layer. For example, for the encoded feature vector output by the convolutional layer 411, the weight of the encoded feature vector that matches the template feature vector in the template library 43 is increased. In the decoder 42, weighted processing is performed on the output of a specified convolutional layer. For example, for the decoded feature vector output by the convolutional layer 422, the weight of the decoded feature vector that matches the template feature vector in the template library 43 is increased.

[0064] It should be noted that, for the sake of simplicity, only three convolutional layers are shown in the decoder 41 and the decoder 42. Those skilled in the art may adjust the number of convolutional layers in the decoder 41 and the decoder 42 as needed.

[0065] In step 103, when the number of at least some of the positioning blocks is greater than a preset threshold, the image to be processed is deformed using the current position relationship of at least some of the positioning blocks and the reference position relationship of at least some of the positioning blocks, so that in the deformed image to be processed, the current position relationship of the multiple positioning blocks and the reference position relationship of the multiple positioning blocks are consistent.

[0066] In some embodiments, the current position relationship of at least part of the positioning blocks and the reference position relationship of at least part of the positioning blocks are used to determine affine transformation parameters, and the image to be processed is affine transformed using the affine transformation parameters to obtain a deformed image to be processed.

[0067] For example, as shown in Figure 5A, a watermark 51 embedded in an image has five positioning blocks 52. A user takes a photo of the image displayed on the screen, resulting in a processed image 53, as shown in Figure 5B. As can be seen in Figure 5B, watermark 51 has changed from a rectangle to a diamond shape, meaning it has been deformed. If four of the five positioning blocks in watermark 51 are identified, affine transformation parameters are determined based on the positional relationships between these four blocks and between these four blocks and the four vertices of the watermark. These affine transformation parameters are then used to perform an affine transformation on processed image 53, resulting in a deformed processed image 54, as shown in Figure 5C.

[0068] It can be seen that after the affine transformation, the positional relationship of the five positioning blocks 52 in Figure 5C is consistent with the reference positional relationship of the five positioning blocks 52 in Figure 5A, thereby effectively overcoming the deformation problem of the watermark 51 and facilitating the effective extraction of watermark information.

[0069] It should be noted that if the number of at least some of the positioning blocks is greater than a preset threshold, correct affine transformation parameters can be obtained based solely on the at least some of the positioning blocks, and in this case, deformation processing can be performed on the image to be processed. Conversely, if the number of identified positioning blocks is small, correct affine transformation parameters cannot be obtained, and in this case, correct deformation processing cannot be performed on the image to be processed.

[0070] In step 104, a plurality of information blocks are extracted from the deformed image to be processed according to the reference position relationship of the plurality of positioning blocks.

[0071] It should be noted that the reference position relationship of the multiple positioning blocks in the watermark is predetermined. Therefore, based on the reference position relationship of the multiple positioning blocks, multiple information blocks can be extracted from the deformed image to be processed.

[0072] For example, according to the reference position relationship of the plurality of positioning blocks 21 in FIG. 2 , a plurality of information blocks 22 corresponding to the character “0” and a plurality of information blocks 23 corresponding to the character “1” can be extracted from FIG. 2 .

[0073] In step 105, the second recognition model is used to recognize the multiple information blocks to obtain watermark information carried by the multiple information blocks.

[0074] FIG6 is a schematic diagram of a second recognition model according to an embodiment of the present disclosure. As shown in FIG6 , the second recognition model includes an encoder 61 and a decoder 62 .

[0075] First, the encoder 61 in the second recognition model is used to encode each of the multiple information blocks to obtain a second encoded feature vector. Next, the decoder 62 in the second recognition model is used to decode the second encoded feature vector to obtain a second decoded feature vector, and then the watermark information of each information block is obtained using the second decoded feature vector.

[0076] Figure 7 is a schematic diagram of an encoder in a second recognition model according to an embodiment of the present disclosure. As shown in Figure 7, the encoder 61 includes multiple sub-encoders 611, 612, ..., 61N connected in sequence. By providing multiple sub-encoders, the success rate of watermark information recognition can be further improved.

[0077] As shown in FIG7 , the input feature vector of subcoder 611 is processed by subcoder 611, and the output feature vector of subcoder 611 is used as the input feature vector of subcoder 612. The input feature vector of subcoder 612 is processed by subcoder 612, and the output feature vector of subcoder 612 is used as the input feature vector of the next subcoder, and so on. The input feature vector of subcoder 611 is the embedded feature vector of each information block, and the output feature vector of subcoder 61N is the second encoded feature vector.

[0078] In some embodiments, multiple self-attention models are set in each sub-encoder, where different self-attention models are used to extract feature vectors of different dimensions, thereby further improving the recognition success rate of watermark information.

[0079] Figure 8 is a schematic diagram of the structure of a sub-encoder according to one embodiment of the present disclosure. As shown in Figure 8, sub-encoder 80 includes multiple self-attention models 811, 812, ..., 81M, and a fusion model 82. Each of the multiple self-attention models 811, 812, ..., 81M processes the input feature vector of sub-encoder 80 to obtain a sub-feature vector, thereby obtaining multiple sub-feature vectors. The multiple sub-feature vectors are then fused using fusion model 82 to obtain the output feature vector of sub-encoder 80.

[0080] In some embodiments, the fusion model 82 concatenates multiple sub-feature vectors to obtain the output feature vector of the sub-encoder 80.

[0081] In some embodiments, after obtaining the watermark information carried by the multiple information blocks, the watermark information is further decrypted using a designated key to obtain plaintext information.

[0082] For example, the obtained watermark information is "0101110101010001010101", and the device number and time information are obtained by decrypting the watermark information using a specified key.

[0083] The watermark information extraction method provided in the above embodiment of the present disclosure deforms the image to be processed by utilizing the current position relationship of multiple positioning blocks in the watermark, so that in the deformed image to be processed, the current position relationship of the multiple positioning blocks is consistent with the reference position relationship of the multiple positioning blocks. On this basis, multiple information blocks are extracted from the image to be processed for identification, thereby effectively extracting the watermark information embedded in the deformed image.

[0084] Figure 9 is a flow chart of a model training method according to an embodiment of the present disclosure. In some embodiments, the following model training method is performed by a model training device.

[0085] In step 901, a sample image embedded with a watermark is obtained, wherein the watermark includes a plurality of positioning blocks and a plurality of information blocks.

[0086] In step 902, a first recognition model is used to recognize the sample image to identify at least part of the plurality of positioning blocks.

[0087] In some embodiments, the first recognition model is as shown in any embodiment of FIG. 4 .

[0088] In some embodiments, the sample image is preprocessed to obtain a plurality of enhanced sample images, wherein the preprocessing includes at least one of color temperature adjustment or saturation adjustment. Next, each of the plurality of enhanced sample images is recognized using a first recognition model.

[0089] It should be noted here that by preprocessing the sample images, the number of sample images used for training can be effectively increased, and at the same time it helps to increase the generalization ability of the first recognition model.

[0090] In some embodiments, when adjusting the color temperature of a sample image, the image is first white-balanced using a grayscale world algorithm to obtain the color temperature of the sample image. The color temperature of the sample image is then adjusted to obtain a different sample image.

[0091] For example, using the grayscale world algorithm to white balance an image, the color temperature of the sample image is obtained to be 6500 K. Next, the color temperature of the sample image is adjusted to 2000 K, 4000 K, 8000 K, and 10000 K, respectively, to obtain five sample images with different color temperatures.

[0092] In some embodiments, the process of adjusting the saturation of a sample image is as follows:

[0093] 1. Set the adjustment parameter percent. The value range of the adjustment parameter percent is [-100, 100]. After normalization, the value range of the adjustment parameter percent is [-1, 1].

[0094] 2. For each pixel of the sample image, select the maximum grayscale value Max and the minimum grayscale value Min in the R (Red) channel, G (Green) channel, and B (Blue) channel, and then obtain the parameters delta and value, as shown in formulas (1) and (2). delta=(Max-Min) / 255 (1) value=(Max+Min) / 255 (2)

[0095] 3. If delta = 0, that is, the maximum grayscale value Max and the minimum grayscale value Min of a pixel are the same, indicating that the pixel is a gray point. In this case, no subsequent processing is performed on the pixel. Otherwise, the subsequent processing is continued for the pixel.

[0096] 4. Calculate the brightness value L of each pixel in the HSL (Hue, Saturation, Lightness) color space, as shown in formula (3). L = (Max-Min) / (2×255) (3)

[0097] 5. Calculate the saturation S of each pixel in the HSL color space, as shown in formula (4).

[0098] 6. If the adjustment parameter percent is greater than or equal to 0, the color saturation needs to be increased. In this case, the parameter alpha is as shown in formula (5).

[0099] Therefore, the grayscale value R of the R channel, the grayscale value G of the G channel, and the grayscale value B of the B channel of each pixel after adjustment are as shown in formulas (6), (7), and (8): R = R + (RL × 255) × alpha (6) G = G + (GL × 255) × alpha (7) B = B + (BL × 255) × alpha (8)

[0100] 7. If the adjustment parameter percent is less than 0, the color saturation needs to be reduced, that is, alpha = percent. At this time, the grayscale value R of the R channel, the grayscale value G of the G channel, and the grayscale value B of the B channel of each pixel after adjustment are as shown in formulas (9), (10), and (11): R = L × 255 + (RL × 255) × (1 + alpha) (9) G = L × 255 + (GL × 255) × (1 + alpha) (10) B = L × 255 + (BL × 255) × (1 + alpha) (11)

[0101] For example, by setting the adjustment parameter percent to 0.2, 0.4, 0.6, and 0.8, and adjusting the saturation of the sample image through the above process, four sample images with different saturations can be obtained.

[0102] It should be noted that the above saturation adjustment process is only an example, and other methods can also be used to adjust the saturation.

[0103] In some embodiments, the sample image is first filtered, and then the filtered sample image is preprocessed.

[0104] In some embodiments, after acquiring the sample image embedded with the watermark, filtering is performed on the sample image to remove moiré patterns generated when the sample image is acquired by photographing.

[0105] For example, the filtering process includes filtering processes such as median filtering and mean filtering.

[0106] In step 903, a first recognition model is trained using at least part of the positioning blocks and the positioning block annotation information.

[0107] For example, a loss function is determined using the positioning block recognition result and the positioning block annotation information in the sample image, and the first recognition model is trained using the loss function.

[0108] In step 904, when the number of at least some of the positioning blocks is greater than a preset threshold, the sample image is deformed using the current position relationship of at least some of the positioning blocks and the reference position relationship of at least some of the positioning blocks, so that in the deformed sample image, the current position relationship of multiple positioning blocks and the reference position relationship of multiple positioning blocks are consistent.

[0109] In some embodiments, affine transformation parameters are determined using the current position relationship of at least part of the positioning blocks and the reference position relationship of at least part of the positioning blocks, and the sample image is affine transformed using the affine transformation parameters to obtain a deformed sample image.

[0110] In step 905, a plurality of information blocks are extracted from the deformed sample image according to the reference position relationship of the plurality of positioning blocks.

[0111] In step 906, the second recognition model is used to recognize the multiple information blocks to obtain watermark information carried by the multiple information blocks.

[0112] For example, according to the reference position relationship of the plurality of positioning blocks 21 in FIG. 2 , a plurality of information blocks 22 corresponding to the character “0” and a plurality of information blocks 23 corresponding to the character “1” can be extracted from FIG. 2 .

[0113] In some embodiments, the second recognition model is as shown in any one of the embodiments in FIG. 6 to FIG. 8 .

[0114] In some embodiments, the plurality of information blocks are sorted in a plurality of different orders to obtain a plurality of queues. Next, a second recognition model is used to recognize the plurality of information blocks in each of the plurality of queues to obtain watermark information.

[0115] It's important to note that each information block in the watermark corresponds to a character, either "0" or "1." Different orders of information blocks fed into the second recognition model produce different outputs. By arranging the watermark's information blocks in different orders, we can generate more information block queues, increasing the amount of training data and boosting the generalization capabilities of the second recognition model.

[0116] In step 907, the second recognition model is trained using the watermark information and the watermark annotation information.

[0117] For example, the loss function is determined using the recognized watermark information and the watermark annotation information, and the second recognition model is trained using the loss function.

[0118] Figure 10 is a flow chart of a watermark adding method according to an embodiment of the present disclosure. In some embodiments, the following watermark adding method is executed by a watermark adding device.

[0119] In step 1001, a watermark is generated, wherein the watermark includes a plurality of positioning blocks and a plurality of information blocks, the plurality of positioning blocks have a reference position relationship, and the plurality of information blocks carry watermark information to be added.

[0120] In some embodiments, the plurality of information blocks include a plurality of first information blocks for representing a first character and a plurality of second information blocks for representing a second character. For example, the first character is "0" and the second character is "1".

[0121] For example, the plain text information is encrypted using a key to obtain the watermark information to be added. Next, each character in the watermark information is converted according to a preset encoding method to obtain multiple information blocks.

[0122] As shown in Figure 2, the watermark includes multiple image blocks. The multiple image blocks include multiple positioning blocks 21 and multiple information blocks. Any two of the multiple positioning blocks and the multiple information blocks do not overlap. The multiple information blocks include multiple information blocks 22 corresponding to the character "0" and multiple information blocks 23 corresponding to the character "1."

[0123] In step 1002, a watermark is embedded in a designated area of ​​a target image.

[0124] In some embodiments, the watermark is superimposed with the image in a designated area of ​​the target image so as to add the watermark to the target image.

[0125] For example, the colors of the positioning block, the first information block, and the second information block are different from each other and are colors to which human vision is insensitive, so that the watermark embedded in the target image cannot be directly observed by human vision.

[0126] For another example, the colors of any two of the positioning block, the first information block, and the second information block are different and are colors to which human vision is insensitive, and one of the other two blocks is colorless. This makes it impossible for humans to directly observe the watermark embedded in the target image, while still facilitating recognition of the watermark information.

[0127] For example, the positioning block is cyan with a grayscale value of (0, 10, 10). The first information block is light blue with a grayscale value of (0, 0, 10). The second information block is colorless.

[0128] In some embodiments, the watermark is added to the target image by adjusting the grayscale values ​​of pixels in a designated area of ​​the target image.

[0129] For example, a designated area of ​​a target image includes a first sub-area corresponding to a plurality of positioning blocks, a second sub-area corresponding to a plurality of first information blocks, and a third sub-area corresponding to a plurality of second information blocks. First, two sub-areas are selected from the first sub-area, the second sub-area, and the third sub-area. Next, a first parameter value is determined using the average grayscale value of pixels in one of the two sub-areas, and the grayscale value of each pixel in the one of the two sub-areas is adjusted using the first parameter value. Next, a second parameter value is determined using the average grayscale value of pixels in the other of the two sub-areas, where the second parameter value is different from the first parameter value, and the grayscale value of each pixel in the other of the two sub-areas is adjusted using the second parameter value.

[0130] For example, in the first sub-region, the second sub-region, and the third sub-region, if the average grayscale value of the pixels in the first sub-region is m1, the first parameter value δ1 is as shown in formula (12). δ1=m1×a / 255 (12)

[0131] Where a is the adjustment value, for example, a = 2. The original grayscale value of each pixel in the first sub-area is x1, and the grayscale value after adjustment is x1', as shown in formula (13). x1' = x1 + δ1 (13)

[0132] Next, if the average grayscale value of the pixels in the second sub-region is m2, the second parameter value δ2 is as shown in formula (14). δ2=m2×a / 255 (14)

[0133] Where a is the adjustment value, for example, a = 2. The original grayscale value of each pixel in the second sub-area is x2, and the grayscale value after adjustment is x2', as shown in formula (15). x2' = x2-δ2 (15)

[0134] It should be noted that, in this embodiment, the grayscale value of the pixels in the third sub-region is not adjusted.

[0135] That is, it is possible to add a watermark to a target image without superimposing the watermark on the image in a designated area of ​​the target image.

[0136] In some embodiments, after adjusting the grayscale values ​​of pixels in two selected sub-regions from among the first, second, and third sub-regions, a third parameter value is determined using the average grayscale value of pixels in the remaining sub-regions other than the first, second, and third sub-regions, wherein the third parameter value is different from the first and second parameter values, and the grayscale values ​​of each pixel in the remaining sub-regions are adjusted using the third parameter value. In other words, the grayscale values ​​of pixels in the first, second, and third sub-regions are all adjusted.

[0137] For example, based on adjusting the pixel grayscale values ​​of the first and second sub-regions, the pixel grayscale value of the third sub-region is adjusted. If the average grayscale value of the pixels in the third sub-region is m3, then the second parameter value δ3 is as shown in formula (16). δ3=m3×a / 255 (16)

[0138] Where a is the adjustment value, for example, a = 2. The original grayscale value of each pixel in the third sub-region is x3, and the grayscale value after adjustment is x3', as shown in formula (17). x3' = x3 + 2 × δ3 (17)

[0139] By adjusting the pixel grayscale value of each sub-region, the watermark can be added to the target image and the recognition of the watermark information can be more facilitated.

[0140] In some embodiments, a watermark is added to a designated area of ​​the target image.

[0141] Figure 11 is a schematic diagram of an image embedded with a watermark according to an embodiment of the present disclosure. As shown in Figure 11 , an image 111 includes a designated area, in which a watermark 112 is embedded.

[0142] In some embodiments, the designated area of ​​the target image includes a plurality of sub-areas, and the watermark is embedded in each of the plurality of sub-areas.

[0143] Figure 12 is a schematic diagram of an image embedded with a watermark according to another embodiment of the present disclosure. Figure 12 differs from Figure 11 in that, in the embodiment shown in Figure 12 , the designated area of ​​image 121 includes four sub-areas, and a watermark is embedded in each of the four sub-areas, so that image 121 includes watermarks 122, 123, 124, and 125.

[0144] It should be noted that, since watermark 122, watermark 123, watermark 124 and watermark 125 are exactly the same, the watermark information can be obtained by simply identifying one of watermark 122, watermark 123, watermark 124 and watermark 125.

[0145] For example, if image 122 is incomplete, the watermark information carried by watermark 122 cannot be successfully obtained, but the watermark information can be obtained from one of watermarks 123, 124, and 125. Therefore, the redundancy setting of watermarks improves the success rate of watermark information acquisition.

[0146] FIG13 is a schematic diagram of the structure of an electronic device according to an embodiment of the present disclosure. As shown in FIG13 , the electronic device includes a memory 131 and a processor 132 .

[0147] The memory 131 is used to store instructions. The processor 132 is coupled to the memory 131. The processor 132 is configured to execute the method involved in any of the embodiments shown in FIG1 , FIG9 or FIG10 based on the instructions stored in the memory.

[0148] For example, when the processor 132 executes the method according to any embodiment of Figure 1 based on the instructions stored in the memory, the electronic device functions as a watermark information extraction device. When the processor 132 executes the method according to any embodiment of Figure 9 based on the instructions stored in the memory, the electronic device functions as a model training device. When the processor 132 executes the method according to any embodiment of Figure 10 based on the instructions stored in the memory, the electronic device functions as a watermark adding device.

[0149] As shown in Figure 13, the electronic device further includes a communication interface 133 for exchanging information with other devices. The electronic device also includes a bus 134 through which the processor 132, the communication interface 133, and the memory 131 communicate with each other.

[0150] Memory 131 may include high-speed RAM (Random Access Memory) or NVM (Non-Volatile Memory). For example, it may include at least one disk storage device. Memory 131 may also be a storage array. Memory 131 may also be divided into blocks, and the blocks may be combined into virtual volumes according to certain rules.

[0151] In addition, the processor 132 may be a central processing unit, or may be an ASIC (Application Specific Integrated Circuit), or may be configured to implement one or more integrated circuits of the embodiments of the present disclosure.

[0152] In some embodiments, the transparent display panel control device may be provided in a SoC (System on Chip) board, an FPGA (Field-Programmable Gate Array) board, or a GPU (Graphics Processing Unit).

[0153] The present disclosure also provides a non-transitory computer-readable storage medium storing computer instructions, which, when executed by a processor, implement the method described in any one of the embodiments shown in FIG. 1 , FIG. 9 , or FIG. 10 .

[0154] Thus far, the embodiments of the present disclosure have been described in detail. To avoid obscuring the concept of the present disclosure, some details known in the art have not been described. Based on the above description, those skilled in the art can fully understand how to implement the technical solutions disclosed herein.

[0155] Although some specific embodiments of the present disclosure have been described in detail through examples, those skilled in the art will understand that the above examples are for illustration only and are not intended to limit the scope of the present disclosure. Those skilled in the art will understand that the above embodiments may be modified or some technical features may be replaced with equivalents without departing from the scope and spirit of the present disclosure. The scope of the present disclosure is defined by the appended claims.

Claims

1. A watermark information extraction method, comprising: Obtain the image to be processed with an embedded watermark, where the watermark includes a plurality of positioning blocks and a plurality of information blocks; Use a first recognition model to recognize the image to be processed to recognize at least some of the plurality of positioning blocks; When the number of at least some of the positioning blocks is greater than a preset threshold, use the current positional relationship of the at least some of the positioning blocks and the reference positional relationship of the at least some of the positioning blocks to perform a deformation process on the image to be processed, so that in the deformed image to be processed, the current positional relationship of the plurality of positioning blocks and the reference positional relationship of the plurality of positioning blocks are consistent; Extract the plurality of information blocks from the deformed image to be processed according to the reference positional relationship of the plurality of positioning blocks; Use a second recognition model to recognize the plurality of information blocks to obtain the watermark information carried by the plurality of information blocks.

2. The method according to claim 1, wherein: Using the first recognition model to recognize the image to be processed includes: Use the encoder in the first recognition model to perform encoding processing on the image to be processed to obtain a plurality of first encoded feature vectors. Among the plurality of first encoded feature vectors, the weight of the first encoded feature vector associated with the positioning block is greater than the weight of the first encoded feature vector associated with the information block; Use the decoder in the first recognition model to perform decoding processing on the plurality of encoded feature vectors to obtain a plurality of first decoded feature vectors; Use the plurality of first decoded feature vectors to recognize the at least some of the positioning blocks.

3. The method according to claim 2, wherein: Among the plurality of first decoded feature vectors, the weight of the first decoded feature vector associated with the positioning block is greater than the weight of the first decoded feature vector associated with the information block.

4. The method according to claim 1, wherein: Using the second recognition model to recognize the plurality of information blocks includes: Use the encoder in the second recognition model to encode each of the plurality of information blocks to obtain a second encoded feature vector; Use the decoder in the second recognition model to perform decoding processing on the second encoded feature vector to obtain a second decoded feature vector; Use the second decoded feature vector to obtain the watermark information of each information block.

5. The method according to claim 4, wherein: The encoder in the second recognition model includes a plurality of sub-encoders connected in sequence; Using the encoder in the second recognition model to encode each of the plurality of information blocks includes: Use the i-th sub-encoder to process the input feature vector of the i-th sub-encoder, and use the output feature vector of the i-th sub-encoder as the input feature vector of the (i + 1)-th sub-encoder, where the input feature vector of the first sub-encoder is the embedded feature vector of each information block, and the output feature vector of the N-th sub-encoder is the second encoded feature vector, 1 ≤ i < N, and N is the total number of sub-encoders.

6. The method according to claim 5, wherein: Each of the plurality of sub-encoders includes a plurality of self-attention models; Using the i-th sub-encoder to process the input feature vector of the i-th sub-encoder includes: Using each of the multiple self-attention models in the i-th sub-encoder, processing the input feature vector of the i-th sub-encoder to obtain a sub-feature vector, thereby obtaining multiple sub-feature vectors; The multiple sub-feature vectors are fused to obtain an output feature vector of the i-th sub-encoder.

7. The method according to claim 6, wherein: The fusion of the multiple sub-feature vectors includes: The multiple sub-feature vectors are concatenated.

8. The method according to claim 1, wherein: Performing deformation processing on the image to be processed includes: Determining affine transformation parameters using the current position relationship of at least part of the positioning blocks and the reference position relationship of at least part of the positioning blocks; The image to be processed is affine transformed using the affine transformation parameters to obtain the deformed image to be processed.

9. A model training method, comprising: Acquire a sample image embedded with a watermark, wherein the watermark includes a plurality of positioning blocks and a plurality of information blocks; Using a first recognition model to recognize the sample image, so as to recognize at least part of the plurality of positioning blocks; Using at least part of the positioning blocks and the positioning block annotation information, training the first recognition model; When the number of at least some of the positioning blocks is greater than a preset threshold, the sample image is deformed by using the current position relationship of at least some of the positioning blocks and the reference position relationship of at least some of the positioning blocks, so that in the deformed sample image, the current position relationship of the plurality of positioning blocks is consistent with the reference position relationship of the plurality of positioning blocks; extracting the plurality of information blocks from the deformed sample image according to the reference position relationship of the plurality of positioning blocks; Using a second recognition model to recognize the multiple information blocks to obtain watermark information carried by the multiple information blocks; The second recognition model is trained using the watermark information and the watermark annotation information.

10. The method according to claim 9, wherein: Using the first recognition model to recognize the sample image includes: Preprocessing the sample images to obtain a plurality of enhanced sample images, wherein the preprocessing includes at least one of color temperature adjustment or saturation adjustment; Each enhanced sample image in the plurality of enhanced sample images is recognized using the first recognition model.

11. The method according to claim 10, wherein: Preprocessing the sample image includes: filtering the sample image; The preprocessing is performed on the filtered sample image.

12. The method according to claim 9, wherein: Using the second recognition model to recognize the multiple information blocks includes: sorting the plurality of information blocks according to a plurality of different orders to obtain a plurality of queues; The second recognition model is used to recognize multiple information blocks in each of the multiple queues to obtain the watermark information.

13. A watermark adding method, comprising: generating a watermark, wherein the watermark comprises a plurality of positioning blocks and the plurality of information blocks, the plurality of positioning blocks have a reference position relationship, and the plurality of information blocks carry watermark information to be added; The watermark is embedded in a designated area of ​​a target image.

14. The method according to claim 13, wherein: The plurality of information blocks include a plurality of first information blocks for representing first characters and a plurality of second information blocks for representing second characters.

15. The method according to claim 14, wherein: Embedding the watermark into a designated area of ​​a target image comprises: The watermark is superimposed on the image in the designated area of ​​the target image.

16. The method according to claim 15, wherein: The colors of the positioning block, the first information block, and the second information block are colors to which human vision is insensitive and are different from each other.

17. The method according to claim 15, wherein: Among the positioning block, the first information block and the second information block, colors of any two blocks are colors to which human vision is insensitive and are different from each other, and one block among the positioning block, the first information block and the second information block except the any two blocks is colorless.

18. The method according to claim 14, wherein: The designated area includes a first sub-area corresponding to the plurality of positioning blocks, a second sub-area corresponding to the plurality of first information blocks, and a third sub-area corresponding to the plurality of second information blocks. Embedding the watermark into a designated area of ​​a target image comprises: selecting two sub-areas from among the first sub-area, the second sub-area and the third sub-area; Determine a first parameter value using an average grayscale value of pixels in one of the two sub-regions; Using the first parameter value to adjust the grayscale value of each pixel in one of the two sub-areas; Determine a second parameter value by using an average grayscale value of pixels in another sub-region of the two sub-regions, wherein the second parameter value is different from the first parameter value; The grayscale value of each pixel in another sub-region of the two sub-regions is adjusted using the second parameter value.

19. The method according to claim 18, wherein: Embedding the watermark into a designated area of ​​a target image comprises: Determine a third parameter value by using the average grayscale value of pixels in the first sub-region, the second sub-region and the remaining sub-regions of the third sub-region except the two sub-regions, wherein the third parameter value is different from the first parameter value and the second parameter value; The grayscale value of each pixel in the remaining sub-area is adjusted using the third parameter value.

20. The method according to claim 13, wherein: The designated area includes a plurality of sub-areas. Embedding the watermark into a designated area of ​​a target image comprises: The watermark is embedded in each of the plurality of sub-regions.

21. An electronic device comprising: a memory configured to store instructions; A processor is coupled to the memory, and the processor is configured to execute the method according to any one of claims 1 to 20 based on instructions stored in the memory.

22. A computer-readable storage medium, wherein: The computer-readable storage medium stores computer instructions, and when the instructions are executed by a processor, the method according to any one of claims 1 to 20 is implemented.

Images