Watermark extraction method, model training method, watermark adding method and electronic device

Abstract

The invention provides a watermark extraction method, a model training method, a watermark adding method and an electronic device. The watermark extraction method comprises the following steps: acquiring a to-be-processed image embedded with a watermark, wherein the watermark comprises a plurality of positioning blocks and a plurality of information blocks; identifying the to-be-processed image by using a first identification model to identify at least part of the positioning blocks in the plurality of positioning blocks; performing deformation processing on the to-be-processed image by using the current position relationship of at least part of the positioning blocks and the reference position relationship of at least part of the positioning blocks, so that the current position relationship of the plurality of positioning blocks is consistent with the reference position relationship of the plurality of positioning blocks in the deformed to-be-processed image; extracting a plurality of information blocks from the deformed to-be-processed image according to the reference position relationship of the plurality of positioning blocks; and identifying the plurality of information blocks by using a second identification model to obtain watermark information carried by the plurality of information blocks.

Description

Technical Field

[0001] The present disclosure relates to the field of information technology, and in particular to a watermark extraction method, a model training method, a watermark adding method and an electronic device. Background Art

[0002] In order to extract the watermark information embedded in the image, after the user uses the image acquisition device to capture the currently presented image, the user uses the trained machine learning model to identify the captured image to extract the watermark information embedded in the image. Summary of the invention

[0003] According to a first aspect of an embodiment of the present disclosure, a method for extracting watermark information is provided, comprising: obtaining an image to be processed in which a watermark is embedded, wherein the watermark comprises a plurality of positioning blocks and a plurality of information blocks; identifying the image to be processed using a first recognition model to identify at least some of the plurality of positioning blocks; in a case where the number of at least some of the positioning blocks is greater than a preset threshold, deforming the image to be processed using a current position relationship of at least some of the positioning blocks and a reference position relationship of at least some of the positioning blocks, so that in the deformed image to be processed, the current position relationship of the plurality of positioning blocks is consistent with the reference position relationship of the plurality of positioning blocks; extracting the plurality of information blocks from the deformed image to be processed according to the reference position relationship of the plurality of positioning blocks; and identifying the plurality of information blocks using a second recognition model to obtain the watermark information carried by the plurality of information blocks.

[0004] In some embodiments, using the first recognition model to recognize the image to be processed includes: using the encoder in the first recognition model to encode the image to be processed to obtain multiple first encoded feature vectors, wherein the weight of the first encoded feature vector associated with the positioning block is greater than the weight of the first encoded feature vector associated with the information block; using the decoder in the first recognition model to decode the multiple encoded feature vectors to obtain multiple first decoded feature vectors; and using the multiple first decoded feature vectors to identify at least part of the positioning block.

[0005] In some embodiments, among the plurality of first decoded feature vectors, a weight of a first decoded feature vector associated with the positioning block is greater than a weight of a first decoded feature vector associated with the information block.

[0006] In some embodiments, identifying the plurality of information blocks by using the second identification model includes: encoding each of the plurality of information blocks by using an encoder in the second identification model to obtain a second encoded feature vector; performing decoding processing on the second encoded feature vector by using a decoder in the second identification model to obtain a second decoded feature vector; and obtaining watermark information of each of the information blocks by using the second decoded feature vector.

[0007] In some embodiments, the encoder in the second identification model includes a plurality of sub-encoders connected in sequence; encoding each of the plurality of information blocks by using the encoder in the second identification model includes: processing an input feature vector of the i-th sub-encoder by using the i-th sub-encoder, and using an output feature vector of the i-th sub-encoder as an input feature vector of the (i + 1)-th sub-encoder, where the input feature vector of the first sub-encoder is an embedded feature vector of each of the information blocks, and the output feature vector of the N-th sub-encoder is the second encoded feature vector, 1 ≤ i < N, and N is the total number of sub-encoders.

[0008] In some embodiments, each of the plurality of sub-encoders includes a plurality of self-attention models; processing the input feature vector of the i-th sub-encoder by using the i-th sub-encoder includes: processing the input feature vector of the i-th sub-encoder by using each of the plurality of self-attention models in the i-th sub-encoder to obtain sub-feature vectors, so as to obtain a plurality of sub-feature vectors; and fusing the plurality of sub-feature vectors to obtain an output feature vector of the i-th sub-encoder.

[0009] In some embodiments, fusing the plurality of sub-feature vectors includes: concatenating the plurality of sub-feature vectors.

[0010] In some embodiments, performing a deformation process on the image to be processed includes: determining affine transformation parameters by using a current position relationship of at least some of the positioning blocks and a reference position relationship of at least some of the positioning blocks; and performing an affine transformation on the image to be processed by using the affine transformation parameters to obtain the deformed image to be processed.

[0011] According to a second aspect of an embodiment of the present disclosure, a model training method is provided, comprising: obtaining a sample image embedded with a watermark, wherein the watermark includes a plurality of positioning blocks and a plurality of information blocks; identifying the sample image using a first recognition model to identify at least some of the plurality of positioning blocks; training the first recognition model using the at least some of the positioning blocks and positioning block annotation information; deforming the sample image using the current position relationship of the at least some of the positioning blocks and the reference position relationship of the at least some of the positioning blocks when the number of the at least some of the positioning blocks is greater than a preset threshold, so that in the deformed sample image, the current position relationship of the plurality of positioning blocks is consistent with the reference position relationship of the plurality of positioning blocks; extracting the plurality of information blocks from the deformed sample image according to the reference position relationship of the plurality of positioning blocks; identifying the plurality of information blocks using a second recognition model to obtain the watermark information carried by the plurality of information blocks; and training the second recognition model using the watermark information and the watermark annotation information.

[0012] In some embodiments, using the first recognition model to recognize the sample image includes: preprocessing the sample image to obtain multiple enhanced sample images, wherein the preprocessing includes at least one of color temperature adjustment or saturation adjustment; and using the first recognition model to recognize each enhanced sample image in the multiple enhanced sample images.

[0013] In some embodiments, preprocessing the sample image includes: filtering the sample image; and performing the preprocessing on the filtered sample image.

[0014] In some embodiments, using the second recognition model to identify the multiple information blocks includes: sorting the multiple information blocks in multiple different orders to obtain multiple queues; using the second recognition model to identify the multiple information blocks in each queue of the multiple queues to obtain the watermark information.

[0015] According to a third aspect of an embodiment of the present disclosure, a watermark adding method is provided, comprising: generating a watermark, wherein the watermark comprises a plurality of positioning blocks and a plurality of information blocks, the plurality of positioning blocks have a reference position relationship, and the plurality of information blocks carry watermark information to be added; and embedding the watermark into a specified area of ​​a target image.

[0016] In some embodiments, the plurality of information blocks include a plurality of first information blocks for representing first characters and a plurality of second information blocks for representing second characters.

[0017] In some embodiments, embedding the watermark in a designated area of ​​the target image includes: superimposing the watermark with an image in the designated area of ​​the target image.

[0018] In some embodiments, the colors of the positioning block, the first information block, and the second information block are colors to which human vision is insensitive and are different from each other.

[0019] In some embodiments, among the positioning block, the first information block and the second information block, the colors of any two blocks are colors to which human vision is insensitive and are different from each other, and among the positioning block, the first information block and the second information block, one block other than the any two blocks is colorless.

[0020] In some embodiments, the designated area includes a first sub-area corresponding to the multiple positioning blocks, a second sub-area corresponding to the multiple first information blocks, and a third sub-area corresponding to the multiple second information blocks, and embedding the watermark into the designated area of ​​the target image includes: selecting two sub-areas from the first sub-area, the second sub-area, and the third sub-area; determining a first parameter value using the average grayscale value of pixels in one of the two sub-areas; adjusting the grayscale value of each pixel in one of the two sub-areas using the first parameter value; determining a second parameter value using the average grayscale value of pixels in the other of the two sub-areas, wherein the second parameter value is different from the first parameter value; and adjusting the grayscale value of each pixel in the other of the two sub-areas using the second parameter value.

[0021] In some embodiments, embedding the watermark into a designated area of ​​the target image includes: determining a third parameter value using the average grayscale value of pixels in the first sub-area, the second sub-area, and the third sub-area except the two sub-areas, wherein the third parameter value is different from the first parameter value and the second parameter value; and adjusting the grayscale value of each pixel in the remaining sub-area using the third parameter value.

[0022] In some embodiments, the designated area includes a plurality of sub-areas, and embedding the watermark in the designated area of ​​the target image includes: embedding the watermark in each of the plurality of sub-areas.

[0023] According to a fourth aspect of an embodiment of the present disclosure, there is provided an electronic device, comprising: a memory; and a processor, coupled to the memory, the processor being configured to execute a method as described in any of the above embodiments based on instructions stored in the memory.

[0024] According to a fifth aspect of an embodiment of the present disclosure, a non-volatile computer-readable storage medium is provided, wherein the non-volatile computer-readable storage medium stores computer instructions, and when the instructions are executed by a processor, the method described in any of the above embodiments is implemented.

[0025] Other features and advantages of the present disclosure will become apparent from the following detailed description of exemplary embodiments of the present disclosure with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The accompanying drawings, which constitute a part of the specification, illustrate embodiments of the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0027] The present disclosure may be more clearly understood from the following detailed description with reference to the accompanying drawings, in which:

[0028] Figure 1 A schematic diagram of a process of extracting watermark information according to an embodiment of the present disclosure;

[0029] Figure 2 A schematic diagram of a watermark according to an embodiment of the present disclosure;

[0030] Figure 3 A schematic diagram of a template library according to an embodiment of the present disclosure;

[0031] Figure 4 This is a schematic diagram of the structure of a first recognition model according to an embodiment of the present disclosure;

[0032] Figure 5A-Figure 5C A schematic diagram of affine transformation of some embodiments of the present disclosure;

[0033] Figure 6 A schematic diagram of the structure of a second recognition model according to an embodiment of the present disclosure;

[0034] Figure 7 A schematic diagram of the structure of an encoder according to an embodiment of the present disclosure;

[0035] Figure 8 A schematic diagram of the structure of a sub-encoder according to an embodiment of the present disclosure;

[0036] Fig. 9 A flowchart of a model training method according to an embodiment of the present disclosure;

[0037] Fig.10 A schematic diagram of a watermark adding method according to an embodiment of the present disclosure;

[0038] Fig.11 A schematic diagram of an image with a watermark embedded therein according to an embodiment of the present disclosure;

[0039] Fig.12A schematic diagram of an image with a watermark embedded therein according to another embodiment of the present disclosure;

[0040] Fig.13 The figure is a schematic diagram of the structure of an electronic device according to an embodiment of the present disclosure.

[0041] It should be understood that the size of each part shown in the accompanying drawings is not drawn according to the actual proportional relationship. In addition, the same or similar reference numerals represent the same or similar components. DETAILED DESCRIPTION

[0042] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. The description of the exemplary embodiments is merely illustrative and is in no way intended to limit the present disclosure and its application or use. The present disclosure can be implemented in many different forms and is not limited to the embodiments described herein. These embodiments are provided to make the present disclosure thorough and complete and to fully express the scope of the present disclosure to those skilled in the art. It should be noted that unless otherwise specifically stated, the relative arrangement of the components and steps, the composition of the materials, and the numerical values ​​set forth in these embodiments should be interpreted as being merely exemplary and not as limiting.

[0043] The words "first", "second" and similar words used in this disclosure do not indicate any order, quantity or importance, but are only used to distinguish different parts. The words "include" or "comprise" and similar words mean that the elements before the word include the elements listed after the word, and do not exclude the possibility of including other elements.

[0044] All terms (including technical terms or scientific terms) used in the present disclosure have the same meanings as those understood by ordinary technicians in the field to which the present disclosure belongs, unless otherwise specifically defined. It should also be understood that terms defined in general dictionaries, such as general dictionaries, should be interpreted as having meanings consistent with their meanings in the context of the relevant technology, and should not be interpreted in an idealized or extremely formal sense, unless explicitly defined herein.

[0045] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered as part of the specification.

[0046] The inventors have noticed that when a user uses an image acquisition device to capture the currently presented image, the captured image will usually be deformed due to the acquisition angle, thereby increasing the difficulty of identifying the watermark information and reducing the success rate of watermark information extraction.

[0047] Based on this, the present disclosure provides a watermark information extraction solution, which can effectively extract the watermark information embedded in the deformed image.

[0048] Figure 1 The figure is a flow chart of a watermark information extraction method according to an embodiment of the present disclosure. In some embodiments, the watermark information extraction method is executed by a watermark information extraction device.

[0049] In step 101, an image to be processed with a watermark embedded therein is obtained, wherein the watermark includes a plurality of positioning blocks and a plurality of information blocks.

[0050] For example, a watermark is embedded in the currently presented image. The user uses an image acquisition device to acquire the currently presented image to obtain the image to be processed. The watermark embedded in the image to be processed is as follows: Figure 2 shown.

[0051] exist Figure 2 In the embodiment, the watermark includes a plurality of image blocks. The plurality of image blocks include a plurality of positioning blocks 21 and a plurality of information blocks. The plurality of information blocks include a plurality of information blocks 22 corresponding to the character "0" and a plurality of information blocks 23 corresponding to the character "1".

[0052] In some embodiments, after acquiring the image to be processed with the embedded watermark, the image to be processed is subjected to filtering processing so as to remove the moiré pattern generated when the image to be processed is acquired by photographing.

[0053] For example, the filtering process includes filtering processes such as median filtering and mean filtering.

[0054] In step 102, the image to be processed is recognized using a first recognition model to recognize at least part of the plurality of positioning blocks.

[0055] In some embodiments, the image to be processed is encoded using an encoder in the first recognition model to obtain a plurality of first encoded feature vectors, wherein among the plurality of first encoded feature vectors, the weight of the first encoded feature vector associated with the positioning block is greater than the weight of the first encoded feature vector associated with the information block. Next, the plurality of encoded feature vectors are decoded using a decoder in the first recognition model to obtain a plurality of first decoded feature vectors, and then at least part of the positioning block is identified using the plurality of first decoded feature vectors.

[0056] It should be noted that, due to the deformation of the image to be processed, the positioning block in the watermark embedded in the image to be processed will also be deformed. For example, the original shape of a positioning block is a square, and after deformation, the positioning block may become a rectangle, diamond, trapezoid or other quadrilaterals, such as Figure 3 shown.

[0057] Therefore, in order to improve the recognition success rate of the positioning block, the following settings are set in the pre-set template library: Figure 3The quadrilateral templates of various shapes are shown. In the process of encoding the image to be processed by the encoder, the weight of the encoding feature vector that matches the template feature vector in the template library is increased, thereby improving the recognition success rate of the positioning block.

[0058] In some embodiments, among the plurality of first decoded feature vectors, a weight of a first decoded feature vector associated with a positioning block is greater than a weight of a first decoded feature vector associated with an information block.

[0059] It should be noted here that, in the process of decoding the multiple encoded feature vectors by the decoder, the weight of the first decoded feature vector matching the template feature vector in the template library is further increased to further improve the recognition success rate of the positioning block.

[0060] Figure 4 FIG. 1 is a schematic diagram of a first recognition model according to an embodiment of the present disclosure. Figure 4 As shown, the first recognition model includes an encoder 41 and a decoder 42. The encoder 41 is provided with three sequentially connected convolutional layers 411-413, and the decoder 42 is provided with three sequentially connected convolutional layers 421-423. In the encoder 41, weighted processing is performed on the output of the specified convolutional layer. For example, for the encoded feature vector output by the convolutional layer 411, the weight of the encoded feature vector that matches the template feature vector in the template library 43 is increased. In the decoder 42, weighted processing is performed on the output of the specified convolutional layer. For example, for the decoded feature vector output by the convolutional layer 422, the weight of the decoded feature vector that matches the template feature vector in the template library 43 is increased.

[0061] It should be noted that, for the sake of simplicity, only three convolutional layers are shown in the decoder 41 and the decoder 42. Those skilled in the art may adjust the number of convolutional layers in the decoder 41 and the decoder 42 as needed.

[0062] In step 103, when the number of at least some of the positioning blocks is greater than a preset threshold, the image to be processed is deformed using the current position relationship of at least some of the positioning blocks and the reference position relationship of at least some of the positioning blocks, so that in the deformed image to be processed, the current position relationship of multiple positioning blocks and the reference position relationship of multiple positioning blocks are consistent.

[0063] In some embodiments, affine transformation parameters are determined using the current position relationship of at least part of the positioning blocks and the reference position relationship of at least part of the positioning blocks, and the image to be processed is affine transformed using the affine transformation parameters to obtain a deformed image to be processed.

[0064] For example, Figure 5AAs shown, the watermark 51 embedded in the image is provided with five positioning blocks 52. The user takes a picture of the image displayed on the screen, and the image to be processed 53 is obtained as shown in FIG. Figure 5B As shown. Figure 5B It can be seen that the watermark 51 changes from a rectangle to a diamond, that is, the watermark 51 is deformed. If four of the five positioning blocks in the watermark 51 are identified, then the affine transformation parameters are determined according to the positional relationship between the four positioning blocks and the positional relationship between the four positioning blocks and the four vertices of the watermark, and then the affine transformation parameters are used to perform affine transformation on the image to be processed 53 to obtain the deformed image to be processed 54, as shown in FIG. Figure 5C shown.

[0065] It can be seen that after affine transformation, Figure 5C The positional relationship of the five positioning blocks 52 in Figure 5A The reference position relationship of the five positioning blocks 52 is consistent, thereby effectively overcoming the deformation problem of the watermark 51 and facilitating the effective extraction of watermark information.

[0066] It should be noted that, when the number of at least some of the positioning blocks is greater than a preset threshold, the correct affine transformation parameters can be obtained based on only the at least some of the positioning blocks, and in this case, the image to be processed can be deformed. On the contrary, if the number of the identified positioning blocks is small, the correct affine transformation parameters cannot be obtained, and in this case, the image to be processed cannot be deformed correctly.

[0067] In step 104, a plurality of information blocks are extracted from the deformed image to be processed according to the reference position relationship of the plurality of positioning blocks.

[0068] It should be noted that the reference position relationship of the multiple positioning blocks in the watermark is predetermined, so according to the reference position relationship of the multiple positioning blocks, multiple information blocks can be extracted from the deformed image to be processed.

[0069] For example, according to Figure 2 The reference position relationship of the plurality of positioning blocks 21 can be obtained from Figure 2 A plurality of information blocks 22 corresponding to the character "0" and a plurality of information blocks 23 corresponding to the character "1" are extracted.

[0070] In step 105, the second recognition model is used to recognize the multiple information blocks to obtain watermark information carried by the multiple information blocks.

[0071] Figure 6 FIG. 1 is a schematic diagram of a second recognition model according to an embodiment of the present disclosure. Figure 6 As shown, the second recognition model includes an encoder 61 and a decoder 62 .

[0072] First, the encoder 61 in the second recognition model is used to encode each of the multiple information blocks to obtain a second encoding feature vector. Next, the decoder 62 in the second recognition model is used to decode the second encoding feature vector to obtain a second decoding feature vector, and then the watermark information of each information block is obtained using the second decoding feature vector.

[0073] Figure 7 FIG. 1 is a schematic diagram of an encoder in a second recognition model according to an embodiment of the present disclosure. Figure 7 As shown, the encoder 61 includes a plurality of sub-encoders 611, 612, ..., 61N connected in sequence. By setting a plurality of sub-encoders, the recognition success rate of the watermark information can be further improved.

[0074] like Figure 7 As shown, the input feature vector of sub-encoder 611 is processed by sub-encoder 611, and the output feature vector of sub-encoder 611 is used as the input feature vector of sub-encoder 612. The input feature vector of sub-encoder 612 is processed by sub-encoder 612, and the output feature vector of sub-encoder 612 is used as the input feature vector of the next sub-encoder, and so on. The input feature vector of sub-encoder 611 is the embedded feature vector of each information block, and the output feature vector of sub-encoder 61N is the second encoded feature vector.

[0075] In some embodiments, multiple self-attention models are set in each sub-encoder, where different self-attention models are used to extract feature vectors of different dimensions, thereby further improving the recognition success rate of watermark information.

[0076] Figure 8 FIG. 1 is a schematic diagram of the structure of a sub-encoder according to an embodiment of the present disclosure. Figure 8 As shown, the sub-encoder 80 includes multiple self-attention models 811, 812, ..., 81M, and a fusion model 82. Each of the multiple self-attention models 811, 812, ..., 81M processes the input feature vector of the sub-encoder 80 to obtain a sub-feature vector, thereby obtaining multiple sub-feature vectors. Next, the multiple sub-feature vectors are fused using the fusion model 82 to obtain the output feature vector of the sub-encoder 80.

[0077] In some embodiments, the fusion model 82 concatenates multiple sub-feature vectors to obtain an output feature vector of the sub-encoder 80 .

[0078] In some embodiments, after the watermark information carried by the multiple information blocks is obtained, the watermark information is further decrypted using a designated key to obtain plaintext information.

[0079] For example, the obtained watermark information is "0101110101010001010101", and the device number and time information are obtained by decrypting the watermark information using a specified key.

[0080] In the watermark information extraction method provided in the above embodiment of the present disclosure, the current position relationship of multiple positioning blocks in the watermark is used to deform the image to be processed, so that in the deformed image to be processed, the current position relationship of the multiple positioning blocks is consistent with the reference position relationship of the multiple positioning blocks. On this basis, multiple information blocks are extracted from the image to be processed for identification, thereby effectively extracting the watermark information embedded in the deformed image.

[0081] Fig. 9 The following is a flow chart of a model training method according to an embodiment of the present disclosure. In some embodiments, the following model training method is performed by a model training device.

[0082] In step 901, a sample image embedded with a watermark is obtained, wherein the watermark includes a plurality of positioning blocks and a plurality of information blocks.

[0083] In step 902, the sample image is recognized using a first recognition model to recognize at least part of the plurality of positioning blocks.

[0084] In some embodiments, the first recognition model is as follows: Figure 4 As shown in any one of the embodiments.

[0085] In some embodiments, the sample images are preprocessed to obtain a plurality of enhanced sample images, wherein the preprocessing includes at least one of color temperature adjustment or saturation adjustment. Next, each of the plurality of enhanced sample images is recognized using a first recognition model.

[0086] It should be noted here that by preprocessing the sample images, the number of sample images used for training can be effectively increased, and at the same time it helps to increase the generalization ability of the first recognition model.

[0087] In some embodiments, in the process of adjusting the color temperature of the sample image, the gray world algorithm is first used to perform white balance processing on the image to obtain the color temperature of the sample image. Next, the color temperature of the sample image is adjusted to obtain a different sample image.

[0088] For example, the gray world algorithm is used to perform white balance processing on the image, and the color temperature of the sample image is obtained to be 6500 K. Next, the color temperature of the sample image is adjusted to 2000 K, 4000 K, 8000 K, and 10000 K, thereby obtaining 5 sample images with different color temperatures.

[0089] In some embodiments, the process of adjusting the saturation of a sample image is as follows:

[0090] 1. Set the adjustment parameter percent. The value range of the adjustment parameter percent is [-100, 100]. After normalization, the value range of the adjustment parameter percent is [-1, 1].

[0091] 2. For each pixel of the sample image, select the maximum grayscale value Max and the minimum grayscale value Min in the R (Red) channel, G (Green) channel, and B (Blue) channel, and then obtain the parameters delta and value, as shown in formulas (1) and (2).

[0092]

[0093] 3. If delta = 0, that is, the maximum grayscale value Max and the minimum grayscale value Min of a pixel are the same, indicating that the pixel is a gray point. In this case, the pixel is not processed subsequently. Otherwise, the pixel is processed subsequently.

[0094] 4. Calculate the brightness value L of each pixel in the HSL (Hue, Saturation, Lightness) color space, as shown in formula (3).

[0095]

[0096] 5. Calculate the saturation S of each pixel in the HSL color space, as shown in formula (4).

[0097]

[0098] 6. If the adjustment parameter percent is greater than or equal to 0, the color saturation needs to be increased. In this case, the parameter alpha is as shown in formula (5).

[0099]

[0100] Therefore, the grayscale value R of the R channel, the grayscale value G of the G channel, and the grayscale value B of the B channel of each pixel are adjusted as shown in formula (6), formula (7), and formula (8):

[0101] R=R+(RL×255)×alpha (6)

[0102] G=G+(GL×255)×alpha (7)

[0103] B=B+(BL×255)×alpha (8)

[0104] 7. If the adjustment parameter percent is less than 0, the color saturation needs to be reduced, that is, alpha = percent. At this time, the grayscale value R of the R channel, the grayscale value G of the G channel, and the grayscale value G of the B channel of each pixel are 灰 After adjustment, the value B is as shown in formula (9), formula (10) and formula (11):

[0105] R=L×255+(RL×255)×(1+alpha) (9)

[0106] G=L×255+(GL×255)×(1+alpha) (10)

[0107] B=L×255+(BL×255)×(1+alpha) (11)

[0108] For example, by setting the adjustment parameter percent to 0.2, 0.4, 0.6, and 0.8 respectively, and adjusting the saturation of the sample image through the above process, four sample images with different saturations can be obtained.

[0109] It should be noted that the above saturation adjustment process is only an example, and other methods can also be used to adjust the saturation.

[0110] In some embodiments, the sample image is first filtered and then the filtered sample image is preprocessed.

[0111] In some embodiments, after acquiring the sample image embedded with the watermark, filtering is performed on the sample image to remove moiré patterns generated when the sample image is acquired by photographing.

[0112] For example, the filtering process includes filtering processes such as median filtering and mean filtering.

[0113] In step 903, a first recognition model is trained using at least part of the positioning blocks and the positioning block annotation information.

[0114] For example, a loss function is determined using the positioning block recognition result and the positioning block annotation information in the sample image, and the first recognition model is trained using the loss function.

[0115] In step 904, when the number of at least some of the positioning blocks is greater than a preset threshold, the sample image is deformed using the current position relationship of at least some of the positioning blocks and the reference position relationship of at least some of the positioning blocks, so that in the deformed sample image, the current position relationship of multiple positioning blocks and the reference position relationship of multiple positioning blocks are consistent.

[0116] In some embodiments, affine transformation parameters are determined using the current position relationship of at least part of the positioning blocks and the reference position relationship of at least part of the positioning blocks, and the sample image is affine transformed using the affine transformation parameters to obtain a deformed sample image.

[0117] In step 905, a plurality of information blocks are extracted from the deformed sample image according to the reference position relationship of the plurality of positioning blocks.

[0118] In step 906, the second recognition model is used to recognize the multiple information blocks to obtain watermark information carried by the multiple information blocks.

[0119] For example, according to Figure 2 The reference position relationship of the plurality of positioning blocks 21 can be obtained from Figure 2 A plurality of information blocks 22 corresponding to the character "0" and a plurality of information blocks 23 corresponding to the character "1" are extracted.

[0120] In some embodiments, the second recognition model is Figure 6-Figure 8 As shown in any one of the embodiments.

[0121] In some embodiments, the plurality of information blocks are sorted in a plurality of different orders to obtain a plurality of queues. Next, a second recognition model is used to recognize a plurality of information blocks in each of the plurality of queues to obtain watermark information.

[0122] It should be noted that each information block in the watermark corresponds to a character, either "0" or "1". Different orders of information blocks input into the second recognition model will result in different outputs of the second recognition model. By arranging multiple information blocks in the watermark in different orders, more information block queues can be obtained, which not only increases the training data, but also helps to increase the generalization ability of the second recognition model.

[0123] In step 907, the second recognition model is trained using the watermark information and the watermark annotation information.

[0124] For example, a loss function is determined using the recognized watermark information and the watermark annotation information, and the loss function is used to train the second recognition model.

[0125] Fig.10 The following is a flow chart of a watermark adding method according to an embodiment of the present disclosure. In some embodiments, the following watermark adding method is executed by a watermark adding device.

[0126] In step 1001, a watermark is generated, wherein the watermark includes a plurality of positioning blocks and a plurality of information blocks, the plurality of positioning blocks have a reference position relationship, and the plurality of information blocks carry watermark information to be added.

[0127] In some embodiments, the plurality of information blocks include a plurality of first information blocks for representing a first character and a plurality of second information blocks for representing a second character. For example, the first character is "0" and the second character is "1".

[0128] For example, the plain text information is encrypted using a key to obtain the watermark information to be added. Next, each character in the watermark information is converted according to a preset encoding method to obtain multiple information blocks.

[0129] like Figure 2 As shown, the watermark includes multiple image blocks. The multiple image blocks include multiple positioning blocks 21 and multiple information blocks. Any two blocks in the multiple positioning blocks and the multiple information blocks do not overlap. The multiple information blocks include multiple information blocks 22 corresponding to the character "0" and multiple information blocks 23 corresponding to the character "1".

[0130] In step 1002, a watermark is embedded in a designated area of ​​a target image.

[0131] In some embodiments, the watermark is superimposed with the image in a designated area of ​​the target image so as to add the watermark to the target image.

[0132] For example, the colors of the positioning block, the first information block, and the second information block are colors that human vision is not sensitive to and are different from each other. Therefore, the watermark embedded in the target image cannot be directly observed by human vision.

[0133] For another example, among the positioning block, the first information block, and the second information block, the colors of any two blocks are colors that human vision is not sensitive to and are different from each other, and among the positioning block, the first information block, and the second information block, one block other than the above two blocks is colorless. Thus, while the watermark embedded in the target image cannot be directly observed by human vision, it can also facilitate the recognition of the watermark information.

[0134] For example, the positioning block is cyan, and the grayscale value is (0, 10, 10). The first information block is light blue, and the grayscale value is (0, 0, 10). The second information block is colorless.

[0135] In some embodiments, the watermark is added to the target image by adjusting the grayscale value of pixels in a designated area of ​​the target image.

[0136] For example, the designated area of ​​the target image includes a first sub-area corresponding to a plurality of positioning blocks, a second sub-area corresponding to a plurality of first information blocks, and a third sub-area corresponding to a plurality of second information blocks. First, two sub-areas are selected from the first sub-area, the second sub-area, and the third sub-area. Next, a first parameter value is determined using the average grayscale value of pixels in one of the two sub-areas, and the grayscale value of each pixel in one of the two sub-areas is adjusted using the first parameter value. Next, a second parameter value is determined using the average grayscale value of pixels in the other of the two sub-areas, wherein the second parameter value is different from the first parameter value, and the grayscale value of each pixel in the other of the two sub-areas is adjusted using the second parameter value.

[0137] For example, in the first sub-region, the second sub-region and the third sub-region, if the average grayscale value of the pixels in the first sub-region is m1, the first parameter value δ1 is as shown in formula (12).

[0138] δ1=m1×a / 255 (12)

[0139] Wherein, a is an adjustment value, for example, a = 2. The original grayscale value of each pixel in the first sub-region is x1, and the grayscale value after adjustment is x1', as shown in formula (13).

[0140] x1′=x1+δ1 (13)

[0141] Next, if the average grayscale value of the pixels in the second sub-region is m2, the second parameter value δ2 is as shown in formula (14).

[0142] δ2=m2×a / 255 (14)

[0143] Wherein, a is an adjustment value, for example, a = 2. The original grayscale value of each pixel in the second sub-region is x2, and the adjusted grayscale value is x2', as shown in formula (15).

[0144] x2′=x2-δ2 (15)

[0145] It should be noted that, in this embodiment, the grayscale value of the pixels in the third sub-region is not adjusted.

[0146] That is, it is possible to add a watermark to a target image without superimposing the watermark with the image in a designated area of ​​the target image.

[0147] In some embodiments, on the basis of adjusting the pixel grayscale values ​​of two sub-regions selected from the first sub-region, the second sub-region, and the third sub-region, the average grayscale value of the pixels in the remaining sub-regions other than the two sub-regions in the first sub-region, the second sub-region, and the third sub-region is further used to determine the third parameter value, wherein the third parameter value is different from the first parameter value and the second parameter value, and the grayscale value of each pixel in the remaining sub-region is adjusted using the third parameter value. In other words, the pixel grayscale values ​​of the first sub-region, the second sub-region, and the third sub-region are all adjusted.

[0148] For example, the pixel grayscale value of the third sub-region is adjusted based on the adjustment of the pixel grayscale value of the first sub-region and the second sub-region. If the average pixel grayscale value of the third sub-region is m3, the second parameter value δ3 is as shown in formula (16).

[0149] δ3=m3×a / 255 (16)

[0150] Wherein, a is the adjustment value, for example, a = 2. The original grayscale value of each pixel in the third sub-region is x3, and the adjusted grayscale value is x3', as shown in formula (17).

[0151] x3′=x3+2×δ3 (17)

[0152] By adjusting the pixel grayscale value of each sub-region, it is possible to add the watermark to the target image and facilitate the recognition of the watermark information.

[0153] In some embodiments, a watermark is added to a designated area of ​​the target image.

[0154] Fig.11 FIG. 1 is a schematic diagram of an image with a watermark embedded therein according to an embodiment of the present disclosure. Fig.11 As shown, the image 111 includes a designated area, in which a watermark 112 is embedded.

[0155] In some embodiments, the designated area of ​​the target image includes a plurality of sub-areas, and a watermark is embedded in each of the plurality of sub-areas.

[0156] Fig.12 A schematic diagram of an image embedded with a watermark according to another embodiment of the present disclosure. Fig.12 and Fig.11 The difference is that in Fig.12 In the illustrated embodiment, the designated area of ​​the image 121 includes four sub-areas, and a watermark is embedded in each of the four sub-areas, so that the image 121 includes watermarks 122 , 123 , 124 , and 125 .

[0157] It should be noted that, since watermark 122, watermark 123, watermark 124 and watermark 125 are exactly the same, the watermark information can be obtained as long as one of watermark 122, watermark 123, watermark 124 and watermark 125 is identified.

[0158] For example, when image 122 is incomplete, the watermark information carried by watermark 122 cannot be successfully obtained, but the watermark information can be obtained from one of watermarks 123, 124 and 125. Therefore, the success rate of obtaining watermark information is improved by setting the watermark redundancy.

[0159] Fig.13 FIG. 1 is a schematic diagram of the structure of an electronic device according to an embodiment of the present disclosure. Fig.13 As shown, the electronic device includes a memory 131 and a processor 132 .

[0160] The memory 131 is used to store instructions. The processor 132 is coupled to the memory 131. The processor 132 is configured to execute the instructions stored in the memory to implement the following. Figure 1 , Fig. 9 or Fig.10 The method of any one of the embodiments.

[0161] For example, the processor 132 executes the instructions stored in the memory to implement Figure 1 In the case of the method involved in any of the embodiments, the electronic device acts as a watermark information extraction device. The processor 132 executes the instructions stored in the memory to implement the following Fig. 9 In the case of the method involved in any embodiment of the present invention, the electronic device serves as a model training device. The processor 132 executes the instructions stored in the memory to implement the following Fig.10 In any of the method embodiments described herein, the electronic device serves as a watermark adding device.

[0162] like Fig.13 As shown, the electronic device further includes a communication interface 133 for information exchange with other devices. At the same time, the electronic device further includes a bus 134, through which the processor 132, the communication interface 133, and the memory 131 communicate with each other.

[0163] The memory 131 may include high-speed RAM (Random Access Memory) and may also include NVM (Non-Volatile Memory). For example, at least one disk storage. The memory 131 may also be a storage array. The memory 131 may also be divided into blocks, and the blocks may be combined into virtual volumes according to certain rules.

[0164] In addition, the processor 132 may be a central processing unit, or may be an ASIC (Application Specific Integrated Circuit), or may be configured to implement one or more integrated circuits of the embodiments of the present disclosure.

[0165] In some embodiments, the transparent display panel control device may be disposed in a SoC (System on Chip) board, a FPGA (Field-Programmable Gate Array) board, or a GPU (Graphics Processing Unit).

[0166] The present disclosure also provides a non-transitory computer-readable storage medium. The computer-readable storage medium stores computer instructions, which are executed by a processor to implement the following Figure 1 , Fig. 9 or Fig.10 The method of any one of the embodiments.

[0167] So far, the embodiments of the present disclosure have been described in detail. In order to avoid obscuring the concept of the present disclosure, some details known in the art are not described. Based on the above description, those skilled in the art can fully understand how to implement the technical solutions disclosed herein.

[0168] Although some specific embodiments of the present disclosure have been described in detail by way of examples, it should be understood by those skilled in the art that the above examples are for illustration only and are not intended to limit the scope of the present disclosure. It should be understood by those skilled in the art that the above embodiments may be modified or some technical features may be replaced by equivalents without departing from the scope and spirit of the present disclosure. The scope of the present disclosure is defined by the appended claims.

Claims

1. A method for extracting watermark information, comprising: Obtaining a to-be-processed image with an embedded watermark, wherein the watermark includes a plurality of positioning blocks and a plurality of information blocks; Using a first recognition model to recognize the to-be-processed image to recognize at least some of the plurality of positioning blocks; When the number of at least some of the positioning blocks is greater than a preset threshold, using the current position relationship of the at least some of the positioning blocks and the reference position relationship of the at least some of the positioning blocks to perform a deformation process on the to-be-processed image, so that in the deformed to-be-processed image, the current position relationship of the plurality of positioning blocks and the reference position relationship of the plurality of positioning blocks are consistent; Extracting the plurality of information blocks from the deformed to-be-processed image according to the reference position relationship of the plurality of positioning blocks; Using a second recognition model to recognize the plurality of information blocks to obtain the watermark information carried by the plurality of information blocks.

2. The method according to claim 1, wherein: Using the first recognition model to recognize the to-be-processed image includes: Using an encoder in the first recognition model to perform encoding processing on the to-be-processed image to obtain a plurality of first encoded feature vectors, wherein among the plurality of first encoded feature vectors, the weight of the first encoded feature vector associated with the positioning block is greater than the weight of the first encoded feature vector associated with the information block; Using a decoder in the first recognition model to perform decoding processing on the plurality of encoded feature vectors to obtain a plurality of first decoded feature vectors; Recognizing the at least some of the positioning blocks using the plurality of first decoded feature vectors.

3. The method according to claim 2, wherein, Among the plurality of first decoded feature vectors, the weight of the first decoded feature vector associated with the positioning block is greater than the weight of the first decoded feature vector associated with the information block.

4. The method according to claim 1, wherein: Using the second recognition model to recognize the plurality of information blocks includes: Using an encoder in the second recognition model to encode each of the plurality of information blocks to obtain a second encoded feature vector; Using a decoder in the second recognition model to perform decoding processing on the second encoded feature vector to obtain a second decoded feature vector; Obtaining the watermark information of each information block using the second decoded feature vector.

5. The method according to claim 4, wherein, The encoder in the second recognition model includes a plurality of sub-encoders connected in sequence; Using the encoder in the second recognition model to encode each of the plurality of information blocks includes: Using the i-th sub-encoder to process the input feature vector of the i-th sub-encoder, and using the output feature vector of the i-th sub-encoder as the input feature vector of the (i + 1)-th sub-encoder, wherein the input feature vector of the first sub-encoder is the embedded feature vector of each information block, and the output feature vector of the N-th sub-encoder is the second encoded feature vector, 1 ≤ i < N, and N is the total number of sub-encoders.

6. The method according to claim 5, wherein, Each of the plurality of sub-encoders includes a plurality of self-attention models; Processing the input feature vector of the i-th sub-encoder by using the i-th sub-encoder includes: Using each of the multiple self-attention models in the i-th sub-encoder, processing the input feature vector of the i-th sub-encoder to obtain a sub-feature vector, thereby obtaining multiple sub-feature vectors; The multiple sub-feature vectors are fused to obtain an output feature vector of the i-th sub-encoder.

7. The method according to claim 6, wherein: The fusion of the multiple sub-feature vectors includes: The multiple sub-feature vectors are concatenated.

8. The method according to claim 1, wherein: Performing deformation processing on the image to be processed includes: Determining affine transformation parameters using the current position relationship of at least part of the positioning blocks and the reference position relationship of at least part of the positioning blocks; The image to be processed is affine transformed using the affine transformation parameters to obtain the deformed image to be processed.

9. A model training method, comprising: Acquire a sample image embedded with a watermark, wherein the watermark includes a plurality of positioning blocks and a plurality of information blocks; Using a first recognition model to recognize the sample image, so as to recognize at least part of the plurality of positioning blocks; Using at least part of the positioning blocks and the positioning block annotation information, training the first recognition model; When the number of at least some of the positioning blocks is greater than a preset threshold, the sample image is deformed by using the current position relationship of at least some of the positioning blocks and the reference position relationship of at least some of the positioning blocks, so that in the deformed sample image, the current position relationship of the plurality of positioning blocks is consistent with the reference position relationship of the plurality of positioning blocks; extracting the plurality of information blocks from the deformed sample image according to the reference position relationship of the plurality of positioning blocks; Using a second recognition model to recognize the multiple information blocks to obtain watermark information carried by the multiple information blocks; The second recognition model is trained using the watermark information and the watermark annotation information.

10. The method according to claim 9, wherein: Using the first recognition model to recognize the sample image includes: Preprocessing the sample images to obtain a plurality of enhanced sample images, wherein the preprocessing includes at least one of color temperature adjustment or saturation adjustment; Each enhanced sample image in the plurality of enhanced sample images is recognized using the first recognition model.

11. The method according to claim 10, wherein: Preprocessing the sample image includes: filtering the sample image; The preprocessing is performed on the filtered sample image.

12. The method according to claim 9, wherein: Using the second recognition model to recognize the multiple information blocks includes: sorting the plurality of information blocks according to a plurality of different orders to obtain a plurality of queues; The second recognition model is used to recognize multiple information blocks in each of the multiple queues to obtain the watermark information.

13. A watermark adding method, comprising: generating a watermark, wherein the watermark comprises a plurality of positioning blocks and the plurality of information blocks, the plurality of positioning blocks have a reference position relationship, and the plurality of information blocks carry watermark information to be added; The watermark is embedded in a designated area of ​​a target image.

14. The method according to claim 13, wherein: The plurality of information blocks include a plurality of first information blocks for representing first characters and a plurality of second information blocks for representing second characters.

15. The method according to claim 14, wherein: Embedding the watermark into a designated area of ​​a target image comprises: The watermark is superimposed on the image in the designated area of ​​the target image.

16. The method according to claim 15, wherein: The colors of the positioning block, the first information block, and the second information block are colors to which human vision is insensitive and are different from each other.

17. The method according to claim 15, wherein: Among the positioning block, the first information block and the second information block, colors of any two blocks are colors to which human vision is insensitive and are different from each other, and one block among the positioning block, the first information block and the second information block except the any two blocks is colorless.

18. The method according to claim 14, wherein: The designated area includes a first sub-area corresponding to the plurality of positioning blocks, a second sub-area corresponding to the plurality of first information blocks, and a third sub-area corresponding to the plurality of second information blocks. Embedding the watermark into a designated area of ​​a target image comprises: selecting two sub-areas from among the first sub-area, the second sub-area and the third sub-area; Determine a first parameter value using an average grayscale value of pixels in one of the two sub-regions; Using the first parameter value to adjust the grayscale value of each pixel in one of the two sub-areas; Determine a second parameter value by using an average grayscale value of pixels in another sub-region of the two sub-regions, wherein the second parameter value is different from the first parameter value; The grayscale value of each pixel in another sub-region of the two sub-regions is adjusted using the second parameter value.

19. The method according to claim 18, wherein: Embedding the watermark into a designated area of ​​a target image comprises: Determine a third parameter value by using the average grayscale value of pixels in the first sub-region, the second sub-region and the remaining sub-regions of the third sub-region except the two sub-regions, wherein the third parameter value is different from the first parameter value and the second parameter value; The grayscale value of each pixel in the remaining sub-area is adjusted using the third parameter value.

20. The method according to claim 13, wherein: The designated area includes a plurality of sub-areas. Embedding the watermark into a designated area of ​​a target image comprises: The watermark is embedded in each of the plurality of sub-regions.

21. An electronic device comprising: a memory configured to store instructions; A processor is coupled to the memory, and the processor is configured to execute the method according to any one of claims 1 to 20 based on instructions stored in the memory.

22. A computer-readable storage medium, wherein: The computer-readable storage medium stores computer instructions, and when the instructions are executed by a processor, the method according to any one of claims 1 to 20 is implemented.

Images