Watermark embedding and extracting method and device, equipment and storage medium
By identifying image key points, determining image blocks and replacing watermark information, the problem of insufficient robustness of watermark embedding and extraction in the prior art under image change is solved, and an efficient and widely applicable watermark solution is realized, and the protection of digital carriers is enhanced. ability.
Patent Information
- Application Number
- CN202310584038.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-22
- Publication Date
- 2024-11-22
AI Technical Summary
The prior art is difficult to realize reliable and robust watermark embedding and extraction of digital carriers without affecting the visual effect of the image, especially in image changes (such as compression, noise addition, rotation and other operations), and Traditional deep learning-based solutions cannot take into account both visual effects and robustness.
By identifying key points in the image, determining the image block to be processed, and generating the target image block based on the watermark information, replacing the original image block to embed the watermark information, and using the key points to locate the image block to extract the watermark information, the image block is used as Units are watermarked embed and extracted, suitable for images of any resolution.
It improves the robustness and scope of application of watermark embedding, realizes efficient watermark embedding and extraction without affecting the image visual effect, and enhances the protection ability of image data.
Smart Images

Figure CN119006254A_ABST
Abstract
Description
Technical Field
[0001] Example embodiments of the present disclosure generally relate to the field of image processing, and more particularly to methods, devices, apparatuses, and computer-readable storage media for watermark embedding and watermark extraction. Background Art
[0002] With the popularization and rapid development of information technology, the speed of digital carriers (e.g., images, videos, etc.) is getting faster and faster and the scope of dissemination is getting wider and wider. In order to better protect digital carriers, digital watermarking technologies such as dark watermarks or hidden watermarks have emerged. People can achieve the purpose of leakage tracing and copyright protection by embedding watermark information into digital carriers without affecting the use value of the original carrier. In view of this, it is expected to achieve a reliable and robust watermarking scheme. Summary of the invention
[0003] In a first aspect of the present disclosure, a watermark embedding method is provided. The method comprises: determining at least one image block to be processed in the image to be processed based on a set of key points identified in the image to be processed; generating at least one target image block corresponding to the at least one image block to be processed based on watermark information and the at least one image block to be processed, each target image block being embedded with at least a portion of the watermark information; and generating a target image for the image to be processed by replacing at least one image block to be processed in the image to be processed with the at least one target image block.
[0004] In a second aspect of the present disclosure, a watermark extraction method is provided. The method comprises: determining at least one target image block in the image to be extracted based on a set of key points identified in the image to be extracted; and determining watermark information embedded in the image to be extracted based on the at least one target image block.
[0005] In a third aspect of the present disclosure, a device for watermark embedding is provided. The device comprises: an image block determination module, configured to determine at least one image block to be processed in the image to be processed based on a set of key points identified in the image to be processed; an image block generation module, configured to generate at least one target image block corresponding to at least one image block to be processed based on watermark information and at least one image block to be processed, each target image block being embedded with at least a portion of the watermark information; and an image block replacement module, configured to generate a target image for the image to be processed by replacing at least one image block to be processed in the image to be processed with at least one target image block.
[0006] In a fourth aspect of the present disclosure, a device for watermark extraction is provided. The device comprises: a target image block determination module, configured to determine at least one target image block in the image to be extracted based on a set of key points identified in the image to be extracted; and a watermark information extraction module, configured to determine the watermark information embedded in the image to be extracted based on the at least one target image block.
[0007] In a fifth aspect of the present disclosure, an electronic device is provided. The device includes at least one processing unit; and at least one memory, the at least one memory is coupled to the at least one processing unit and stores instructions for execution by the at least one processing unit. When the instructions are executed by the at least one processing unit, the device executes the method of the first aspect or the second aspect.
[0008] In a sixth aspect of the present disclosure, a computer-readable storage medium is provided, wherein a computer program is stored on the computer-readable storage medium, and the computer program can be executed by a processor to implement the method of the first aspect or the second aspect.
[0009] It should be understood that the contents described in this content section are not intended to limit the key features or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS
[0010] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. In the accompanying drawings, the same or similar reference numerals represent the same or similar elements, wherein:
[0011] Figure 1A A schematic diagram showing an example environment for watermark embedding in which some embodiments of the present disclosure can be implemented;
[0012] Figure 1B A schematic diagram showing an example environment for watermark extraction in which some embodiments of the present disclosure can be implemented;
[0013] Figure 2 A flowchart showing a process of watermark embedding according to some embodiments of the present disclosure is shown;
[0014] Figure 3 A schematic block diagram of a watermark embedding model according to some embodiments of the present disclosure is shown;
[0015] Figure 4 A flowchart showing a process of watermark extraction according to some embodiments of the present disclosure is shown;
[0016] Figure 5A schematic diagram showing an architecture of model training according to some embodiments of the present disclosure;
[0017] Figure 6 A schematic diagram showing an example process of watermark embedding according to some embodiments of the present disclosure;
[0018] Figure 7 A schematic diagram showing an example process of watermark extraction according to some embodiments of the present disclosure;
[0019] Figure 8 A schematic structural block diagram of an apparatus for watermark embedding according to some embodiments of the present disclosure is shown;
[0020] Fig. 9 A schematic structural block diagram of an apparatus for watermark extraction according to some embodiments of the present disclosure is shown; and
[0021] Fig.10 A block diagram of an electronic device is shown in which one or more embodiments of the present disclosure may be implemented. DETAILED DESCRIPTION
[0022] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.
[0023] In the description of the embodiments of the present disclosure, the term "including" and similar terms should be understood as open inclusion, that is, "including but not limited to". The term "based on" should be understood as "based at least in part on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The term "some embodiments" should be understood as "at least some embodiments". The following may also include other explicit and implicit definitions. As used herein, the term "model" can represent the association relationship between various data. For example, the above-mentioned association relationship can be obtained based on a variety of technical solutions currently known and / or to be developed in the future.
[0024] It is understandable that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and relevant provisions.
[0025] It is understandable that before using the technical solutions disclosed in the embodiments of the present disclosure, the types, scope of use, usage scenarios, etc. of the personal information involved in the present disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.
[0026] For example, in response to receiving an active request from a user, a prompt message is sent to the user to clearly prompt the user that the operation requested to be performed will require obtaining and using the user's personal information. Thus, the user can autonomously choose whether to provide personal information to software or hardware such as an electronic device, application, server, or storage medium that performs the operation of the technical solution of the present disclosure according to the prompt message.
[0027] As an optional but non-limiting embodiment, in response to receiving an active request from the user, the prompt information is sent to the user in a manner such as a pop-up window, in which the prompt information can be presented in text form. In addition, the pop-up window can also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.
[0028] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation to the embodiments of the present disclosure. Other methods that meet relevant laws and regulations may also be applied to the embodiments of the present disclosure.
[0029] As used herein, the term "model" can learn the association between the corresponding input and output from the training data, so that after the training is completed, the corresponding output can be generated for a given input. The generation of the model can be based on machine learning technology. Deep learning is a machine learning algorithm that processes inputs and provides corresponding outputs by using multi-layer processing units. A neural network model is an example of a model based on deep learning. In this article, "model" may also be referred to as "machine learning model", "learning model", "machine learning network" or "learning network", and these terms are used interchangeably in this article.
[0030] A "neural network" is a machine learning network based on deep learning. A neural network is capable of processing inputs and providing corresponding outputs, and typically includes an input layer and an output layer and one or more hidden layers between the input layer and the output layer. Neural networks used in deep learning applications typically include many hidden layers, thereby increasing the depth of the network. The layers of a neural network are connected in sequence so that the output of the previous layer is provided as input to the next layer, where the input layer receives the input of the neural network and the output of the output layer serves as the final output of the neural network. Each layer of a neural network includes one or more nodes (also called processing nodes or neurons), each of which processes input from the previous layer.
[0031] Generally, machine learning can be roughly divided into three stages, namely the training stage, the testing stage, and the application stage (also called the inference stage). In the training stage, a given model can be trained using a large amount of training data, and the parameter values are continuously updated iteratively until the model can obtain consistent inferences that meet the expected goals from the training data. Through training, the model can be considered to be able to learn the association from input to output (also called the mapping of input to output) from the training data. The parameter values of the trained model are determined. In the testing stage, the test input is applied to the trained model to test whether the model can provide the correct output, thereby determining the performance of the model. In the application stage, the model can be used to process the actual input based on the parameter values obtained from the training to determine the corresponding output.
[0032] Figure 1A 1 is a schematic diagram of an example environment 100A for watermark embedding in which some embodiments of the present disclosure can be implemented. Figure 1A As shown, example environment 100A may include electronic device 110 .
[0033] The image to be processed 102 is an image to be embedded with a watermark, which can be an image of any format, any size, any color (such as a color image, a black and white image, a grayscale image), etc. The image to be processed 102 can be an image obtained by the electronic device 110 from other devices, an image stored locally, or an image captured by the electronic device 110 through its own image acquisition component (such as a camera).
[0034] The watermark information 104 may also be generally referred to as a digital watermark. The watermark information 104 may be information in any form, such as a bit string. The electronic device 110 may generate a target image 112 for the image to be processed 102 based on the image to be processed 102 and the watermark information 104. That is, the electronic device 110 may embed the watermark information 104 into the image to be processed 102 to generate the target image 112. In particular, in the target image 112, the watermark information may be invisible, that is, imperceptible to the human eye.
[0035] In environment 100A, electronic device 110 may be any type of device with computing capabilities, including terminal devices or server devices. The terminal device may be any type of mobile terminal, fixed terminal or portable terminal, including mobile phones, desktop computers, laptop computers, notebook computers, netbook computers, tablet computers, media computers, multimedia tablets, personal communication system (PCS) devices, personal navigation devices, personal digital assistants (PDAs), audio / video players, digital cameras / camcorders, positioning devices, television receivers, radio broadcast receivers, e-book devices, gaming devices or any combination of the foregoing, including accessories and peripherals of these devices or any combination thereof.
[0036] The server device can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content distribution networks, and big data and artificial intelligence platforms. Server devices can include computing systems / servers such as mainframes, edge computing nodes, electronic devices in cloud environments, and so on.
[0037] Figure 1B 1 is a schematic diagram of an example environment 100B for watermark extraction in which some embodiments of the present disclosure can be implemented. Figure 1B As shown, example environment 100B may include an electronic device 120 .
[0038] The image to be extracted 106 is an image that may be embedded with a watermark. The image to be extracted 106 may be, for example, any image of any format, any size, any color, etc. The image to be extracted 106 may be an image obtained by the electronic device 120 from other devices through communication or an image stored locally by the electronic device 120. The electronic device 120 may extract watermark information 122 from the image to be extracted 106. The extracted watermark information 122 may be further verified to determine whether the extracted watermark is a correct watermark.
[0039] Similar to the electronic device 110 in the environment 100A, the electronic device 120 in the environment 100B may also be any type of device with computing capabilities, including a terminal device or a server device. In some embodiments, the electronic device 110 and the electronic device 120 may be the same device. For example, the owner of the image to be processed 102 may use the same device to perform watermark embedding on the image to be processed 102 and to perform watermark extraction on the acquired image to be extracted 106. In some embodiments, the electronic device 110 and the electronic device 120 may be different devices. For example, the owner of the image may use the electronic device 110 to embed a watermark in the image, and the user of the image may use the electronic device 120 to extract the watermark information in the image to verify the authenticity of the image.
[0040] It should be understood that Figure 1A and Figure 1B The components and arrangements in the illustrated environments 100A and 100B are merely examples. A computing system suitable for implementing the exemplary embodiments described in the present disclosure may include one or more different components, other components, and / or different arrangements.
[0041] As mentioned above, watermark information that is imperceptible to the human eye can be added to the image, also known as dark watermark. In this way, the intended purpose can be achieved without affecting the normal use of the image, such as leakage tracing and copyright protection. Traditional solutions for dark watermark technology mainly include solutions based on image processing and solutions based on deep learning. Solutions based on image processing often need to be manually designed for different feature information of the image. Therefore, this solution is complex to operate and cannot take into account the robustness of the image under various changes. The deep learning-based solution embeds and extracts watermarks by training an encoder-decoder model, and performs data enhancement by introducing changes in the image during the training process to improve the robustness of the embedded watermark. However, traditional deep learning-based solutions can only embed watermarks in images that are consistent with the model input size, and cannot take into account both visual effects and robustness.
[0042] According to an exemplary embodiment of the present disclosure, an improved scheme for watermark embedding and watermark extraction is provided. According to the scheme, on the watermark embedding side, the image block to be processed can be determined based on the key points identified from the image to be processed. The watermark information can be embedded in the image block to be processed to generate a target image block. Furthermore, the target image block can be used to replace the image block to be processed in the image to be processed to obtain a target image embedded with the watermark information. On the watermark extraction side, the target image block can be determined based on the key points identified from the image to be processed, and the watermark information can be extracted from the target image block.
[0043] In an embodiment of the present disclosure, key points are used to determine image blocks for embedding watermarks. Since key points do not change significantly when an image is changed (e.g., compressed, noisy, rotated, etc.), the robustness of watermark embedding can be improved. Accordingly, key points are used to determine image blocks for watermark extraction during watermark extraction. This can reliably locate image areas that may contain watermarks. In addition, in an embodiment of the present disclosure, watermark embedding and extraction are performed in units of image blocks rather than in units of the entire image. In this way, images of arbitrary resolution can be processed, thereby achieving a watermark embedding and extraction scheme with wide applicability.
[0044] Some example embodiments of the present disclosure will be described below with continued reference to the accompanying drawings.
[0045] Figure 2 1 shows a flow chart of a process 200 of watermark embedding according to some embodiments of the present disclosure. The process 200 may be implemented at the electronic device 110. For ease of discussion, reference will be made to Figure 1A The process 200 is described with reference to the environment 100A.
[0046] In box 210, the electronic device 110 determines at least one image block to be processed in the image to be processed 102 based on a set of key points identified in the image to be processed 102. In the embodiment of the present disclosure, the key point may also be referred to as a feature point, which refers to one or more pixels with visually characteristic properties. In some embodiments, the key point may be a corner point, which represents one or more pixels that are particularly prominent in a certain attribute. For example, the key point may be an extreme point of a certain attribute. The image block to be processed may have a predetermined size, such as 128*128.
[0047] In some embodiments, these key points may be identified by the electronic device 110. The electronic device 110 may determine the key points of the image 102 to be processed using a key point detection algorithm. Any suitable key point detection algorithm may be used, such as but not limited to the feature from accelerated segmented experiments (FAST) algorithm, the scale-invariant feature transform (SIFT) algorithm, the directional FAST and the rotated binary robust basic features (ORB) algorithm. In some embodiments, the electronic device 110 may determine the key points of the image 102 to be processed using an ORB key point detection algorithm. The ORB key point detection algorithm has good robustness. For example, even if the image to be processed undergoes a variety of image processing operations such as compression, noise addition, cropping, translation, and rotation, the key points determined using the ORB key point detection algorithm will not change significantly. Therefore, in this embodiment, using the ORB algorithm helps to further improve the robustness of watermark embedding.
[0048] In some embodiments, the electronic device 110 may also use a trained key point detection model to determine key points in the image to be processed 102. The specific type of model structure may be selected according to actual application needs. In some embodiments, the key points in the image to be processed 102 may be identified by other devices, and the electronic device 110 may obtain information about the identified key points from the device.
[0049] The image blocks to be processed may be determined based on the key points in various suitable ways. In some embodiments, the electronic device 110 may directly determine at least one image block to be processed in the image to be processed 102 based on the corresponding positions of the group of key points in the image to be processed 102. Exemplarily, the electronic device 110 may determine a group of image blocks in the image to be processed 102 centered on each key point as a group of image blocks to be processed.
[0050] In some embodiments, in order to determine the image block to be processed, the eigenvalue of the key point can be taken into consideration. The identified key points can be screened using the eigenvalues, and then the image block to be processed is determined based on the screened key points. The electronic device 110 can also determine the eigenvalue of each key point while determining a set of key points. The eigenvalue can represent the robustness of the corresponding key point, that is, the probability that the point is detected as a key point when the image changes. Exemplarily, the electronic device 110 can use the ORB key point detection algorithm to determine a set of key points in the image 102 to be processed and the corresponding eigenvalue of each key point.
[0051] The electronic device 110 can determine a group of candidate image blocks in the image 102 to be processed based on the corresponding feature values of the identified group of key points and the corresponding positions in the image 102 to be processed. For example, the group of key points can be sorted based on the corresponding feature values of the group of key points. In order to ensure the robustness of the key points obtained by screening, the electronic device 110 can sort the group of key points in descending order based on the level of the corresponding feature values of the group of key points. That is, the corresponding feature values of the key points with a higher ranking are higher, and the corresponding feature values of the key points with a lower ranking are lower. For each key point, the electronic device 110 can determine an image block of a predetermined size (for example, 128*128) including the key point in the image 102 to be processed. For example, the electronic device 110 can determine a group of image blocks in the image to be processed that are centered on each key point in the group of key points. The electronic device 110 can then screen the group of image blocks starting from the image blocks with higher feature values of the corresponding key points according to the sorting results of the key points. Accordingly, the image blocks corresponding to the key points with higher feature values can be determined as candidate image blocks.
[0052] The candidate image blocks determined in the above manner may include more than one key point, and different candidate image blocks may overlap. In some embodiments, when screening based on the feature values of the key points, the overlap between the image blocks can be considered to obtain a set of candidate image blocks in which different candidate image blocks do not overlap with each other. Exemplarily, the electronic device 110 can determine whether the image blocks overlap by using an intersection-over-union (IOU) algorithm.
[0053] Furthermore, the electronic device 110 can determine at least one image block to be processed from the group of candidate image blocks. In some embodiments, these candidate image blocks can be directly determined as image blocks to be processed. In some embodiments, the image blocks to be processed can be determined from the group of candidate image blocks based on the sorting results of the eigenvalues of the corresponding key points. For example, a predetermined number of candidate image blocks can be selected as image blocks to be processed. It should be noted that in order to ensure the visual effect of the target image obtained by the final watermark embedding, the number of image blocks to be processed should not be too large. Exemplarily, the electronic device 110 can determine the top 10 candidate image blocks in the group of candidate image blocks corresponding to the eigenvalues of the key points as image blocks to be processed.
[0054] In order to enhance the visual effect, in some embodiments, the electronic device 110 may also determine the texture complexity of each candidate image block in the set of candidate image blocks. The higher the texture complexity, the more difficult it is for the human eye to perceive changes in the corresponding image block. For example, the electronic device 110 may determine the texture complexity of each candidate image block by any suitable texture calculation algorithm. The texture calculation algorithm may include, for example, a local binary pattern (LBP) algorithm, a just noticeable difference (JND) algorithm, and the like. Exemplarily, the electronic device 110 may determine the texture complexity of each candidate image block by a JND algorithm. Alternatively or additionally, in some embodiments, the electronic device 110 may also determine the texture complexity of each candidate image block using a trained texture calculation model. The specific type of model structure may be selected according to actual application needs.
[0055] Then, the electronic device 110 may select at least one image block to be processed from the group of candidate image blocks based on the corresponding texture complexity of the candidate image blocks. In some embodiments, the electronic device 110 may sort the group of candidate image blocks in descending order based on the corresponding texture complexity, and then select a predetermined number of candidate image blocks from the group of candidate image blocks based on the sorting result as the image blocks to be processed. For example, the electronic device 110 may select 10 candidate image blocks with the top corresponding texture complexity from the group of candidate image blocks, and determine the 10 candidate image blocks as the image blocks to be processed. In some embodiments, the electronic device 110 may compare the corresponding texture complexity of the group of candidate image blocks with a threshold value, and select the image block to be processed from the group of candidate image blocks based on the comparison result. The threshold value may be pre-input to the electronic device 110 by the user, or may be determined by the electronic device 110 itself. The electronic device 110 may determine the candidate image blocks with a texture complexity higher than the threshold value as the image blocks to be processed.
[0056] The higher the texture complexity, the more difficult it is for the human eye to detect changes in the image. In view of this, selecting image blocks with high texture complexity to embed watermarks is conducive to obtaining embedded watermarks that are difficult for the human eye to detect. This can advantageously improve the visual effect of the obtained target image.
[0057] Alternatively or additionally, in some embodiments, in addition to texture complexity, the image block to be processed can be selected from the candidate image blocks based on other attributes of the image block. Exemplarily, the color complexity of each candidate image block can be determined, for example, the more colors, the higher the complexity can be and / or the more chaotic the color distribution, the higher the complexity can be. Then, a predetermined number of candidate image blocks with high color complexity can be selected as the image blocks to be processed. Similar to texture complexity, the higher the color complexity, the more difficult it is for the human eye to perceive changes in the image. In this way, the visual effect of the obtained target image can be advantageously improved.
[0058] It should be understood that the specific number and size of the image blocks mentioned in the above description are only exemplary and are not intended to limit the scope of the present disclosure. Other suitable numbers and sizes of the image blocks are also possible.
[0059] Continue with process 200. In box 220, the electronic device 110 generates at least one target image block corresponding to the at least one image block to be processed based on the watermark information 104 and the at least one image block to be processed. Each target image block is embedded with at least a portion of the watermark information 104. For example, each target image block can be embedded with the complete watermark information 104. For another example, the watermark information 104 can be divided into different parts, and each target image block can be embedded with one or more of these parts. The embodiments of the present disclosure are not limited in this respect.
[0060] The electronic device 110 may use any suitable watermark embedding technology to generate a corresponding target image block based on the watermark information and the image block to be processed. The watermark embedding technology may include, but is not limited to, superposition technology, least significant bit technology (LSB), quantization index modulation technology (QIM), wavelet transform technology, etc.
[0061] In some embodiments, the electronic device 110 may generate a target image block using one or more trained machine learning models. The machine learning model may include, for example, a convolutional neural network (CNN), a feedforward neural network (FNN), a recurrent neural network (RNN), etc. The trained model may be deployed in the electronic device 110, or in other devices accessible to the electronic device 110.
[0062] In some embodiments, in order to further ensure the full integration of the watermark information and the image block to improve the robustness of the watermark embedding, the electronic device 110 can implement watermark embedding by multi-level splicing of the watermark information and the image block. For example, the watermark information 104 (for example, represented by a binary string) can be vectorized or tokenized to obtain the characteristics of the watermark information 104, which can be regarded as a vectorized representation of the watermark information 104, and can also be called a watermark feature. In some embodiments, the binary watermark information can be added with error correction bits using the Bose-Ray-Chadhoury-Houchingam (BCH) code to obtain a watermark binary string. By way of example and without any limitation, a 64-bit watermark binary string can be obtained, which can include 24-bit error correction bits. The characteristics of the watermark binary string with error correction bits added can then be determined.
[0063] Accordingly, for each image block to be processed, feature extraction can be performed on the image block to be processed to obtain the features of the image block to be processed. Any suitable feature extraction algorithm is possible. Further, the electronic device 110 can generate fusion features of the watermark information and the image block to be processed based on the features of the watermark information 104 and the features of the image block to be processed. It can be understood that before the features of the watermark information 104 and the features of the image block to be processed are fused, the electronic device 110 can first adjust the watermark features and the features of the image block to be processed to have the same dimension.
[0064] The electronic device 110 can then generate a target image block based on the fused features, the watermark features, and the image block to be processed. In some embodiments, the electronic device 110 can obtain the target image block by fusing the fused features, the watermark features, and the features of the image block to be processed multiple times. In some embodiments, the electronic device 110 can use a trained watermark embedding model to embed the watermark information into the image block. Figure 3 Such an example embodiment is described.
[0065] Figure 3 FIG. 3 is a schematic block diagram of a watermark embedding model 300 according to some embodiments of the present disclosure. Figure 3 As shown, the watermark embedding model 300 includes a linear layer 310, a feature extraction layer 320-1 and a feature extraction layer 320-2 (also collectively referred to as a feature extraction layer 320), and feature fusion layers 330-1, 330-2, 330-3, 330-4, and 330-5 (also collectively referred to as a feature fusion layer 330). In some embodiments, the feature extraction layer 320 and the feature fusion layer 330 may include at least one or more of a multi-layer channel attention (SeNet) layer, a convolution layer (Conv), an upsampling (Upsampling) layer, a downsampling (Downsampling) layer, a normalization (BN) layer, and an activation (Relu) layer. In general, the watermark embedding model 300 may include a part for extracting features of watermark information (including a linear layer 310 and a feature extraction layer 320-1 in this example), a part for extracting features of image blocks (including a feature extraction layer 320-2 in this example), and a part for multi-level splicing of watermark information and image blocks (including a feature fusion layer 330-1, a feature fusion layer 330-2, and a feature fusion layer 330-3 in this example).
[0066] like Figure 3 The watermark information 301 shown can be regarded as an example of the watermark information 104, and the image block to be processed 303 can be regarded as an example of the image block to be processed determined in block 201. The watermark information 301 and the image block to be processed 303 are input into the watermark embedding model 300. The watermark embedding model 300 obtains the watermark feature 302 of the watermark information 301 through the linear layer 310 and the feature extraction layer 320-1. Exemplarily, the linear layer 310 can be a simple fully connected neural network. The watermark embedding model 300 obtains the feature 304 of the image block to be processed through the feature extraction layer 320-2. The watermark feature 302 and the feature 304 of the image block to be processed can be provided to the feature fusion layer 330-1 together to obtain the fused feature.
[0067] In order to enhance the representation capability of the watermark information 301 in the final target image block 305, in some embodiments, the watermark embedding model 300 may generate an intermediate feature enhanced with the watermark information 301 based on the fusion feature and the watermark feature 302. Figure 3 As shown, the fusion feature output by the feature fusion layer 330-1 is provided to the feature fusion layer 330-2 together with the watermark feature 302 to obtain the first intermediate feature enhanced by the watermark information 301. The first intermediate feature output by the feature fusion layer 330-2 is provided to the feature fusion layer 330-3 together with the watermark feature 302 to obtain the second intermediate feature enhanced by the watermark information 301.
[0068] Then, the intermediate features enhanced with the watermark information can be combined with the image block to be processed 303 to generate the target image block 305. In some embodiments, in order to ensure that the similarity between the target image block 305 and the image block to be processed 303 is high, the watermark embedding model 300 can also generate the target image block 305 based on the intermediate features and the image block to be processed 303, and the target image block 305 generated in this way is enhanced by the image block to be processed 303. Figure 3 As shown, the second intermediate feature output by the feature fusion layer 330-3 is provided to the feature fusion layer 330-4 together with the image block to be processed 303 to obtain the first target feature. The first target feature output by the feature fusion layer 330-4 is provided to the feature fusion layer 330-5 together with the image block to be processed 303 to obtain the target image block 305.
[0069] In this way, the watermark information and the image block to be processed can be fully integrated, which helps to improve the robustness of watermark embedding. Figure 3 The types, numbers and connection relationships of the layers of the watermark embedding model shown are only exemplary and are not intended to limit the scope of the present disclosure. Any suitable structure can be used to implement the watermark embedding model.
[0070] The above describes an example of first using watermark information to enhance the fusion feature and then using the image block to be processed to enhance it. In other embodiments, other methods can also be used to generate the target image block based on the fusion feature, the feature of the watermark information and the feature of the image block to be processed. For example, the fusion feature, the feature of the watermark information and the feature of the image block to be processed can be spliced together, and then the spliced features are used to generate the target image block.
[0071] Return to reference Figure 2 For each of the at least one to-be-processed image blocks determined in block 210 , the electronic device 110 may generate a corresponding target image block through the watermark embedding model 300 , thereby obtaining at least one target image block.
[0072] In block 230, the electronic device 110 generates a target image 112 for the image 102 to be processed by replacing at least one image block to be processed with at least one target image block in the image 102 to be processed. For each image block to be processed, the electronic device 110 may replace the image block to be processed with the generated corresponding target image block, thereby obtaining the target image 112. Since the target image block is embedded with watermark information, the target image 112 containing the target image block after replacement is an image embedded with watermark information.
[0073] Some example embodiments of watermark embedding of the present disclosure are described above. In general, key points in an image can be used to determine image blocks for watermark embedding. In this way, a watermark embedding with high robustness and wide applicability can be achieved. In addition, compared with embedding the watermark in the entire image, this solution embeds the watermark in a partial area that is imperceptible to the human eye, which has obvious improvements in visual effects and image data volume growth. The following will continue to describe some example embodiments of watermark extraction of the present disclosure in conjunction with the accompanying drawings.
[0074] Figure 4 4 is a flowchart of a process 400 for watermark extraction according to some embodiments of the present disclosure. The process 400 may be implemented at the electronic device 120. For ease of discussion, reference will be made to Figure 1B The process 200 is described with reference to the environment 100B.
[0075] In block 410, the electronic device 120 determines at least one target image block in the image to be extracted 106 based on a set of key points identified in the image to be extracted 106. Similar to the identification of key points in the image to be processed 102, the electronic device 120 can use any suitable key point detection algorithm or a trained key point detection model to determine a set of key points in the image to be extracted 106. The identification of key points in the image to be extracted 106 will not be described in detail herein.
[0076] The target image block can be determined based on the key points in various suitable ways. In some embodiments, the electronic device 120 can directly determine at least one target image block in the image to be extracted 106 based on the corresponding positions of the group of key points in the image to be extracted 106. Exemplarily, the electronic device 120 can determine a group of image blocks centered on each key point in the image to be extracted 106 as a group of target image blocks.
[0077] In some embodiments, in order to determine the target image block, the eigenvalue of the key point can be taken into consideration. The identified key points can be screened using the eigenvalues, and then the target image block is determined based on the screened key points. The electronic device 120 can also determine the eigenvalue of each key point while determining a set of key points. The eigenvalue can represent the robustness of the corresponding key point, that is, the probability that the point is detected as a key point when the image changes. Exemplarily, the electronic device 120 can use the ORB key point detection algorithm to determine a set of key points in the image 106 to be extracted and the corresponding eigenvalue of each key point.
[0078] The electronic device 120 may determine a group of candidate image blocks in the image 106 to be extracted based on the corresponding feature values of the identified group of key points and the corresponding positions in the image 106 to be extracted. For example, the group of key points may be sorted based on the corresponding feature values of the group of key points. In order to ensure the robustness of the key points screened, the electronic device 120 may sort the group of key points in descending order based on the corresponding feature values of the group of key points. That is, the corresponding feature values of the key points at the front of the sorting are higher, and the corresponding feature values of the key points at the back of the sorting are lower. For each key point, the electronic device 120 may determine an image block of a first size (for example, 64*64) including the key point in the image 106 to be extracted. For example, the electronic device 120 may determine a group of image blocks in the image 106 to be extracted that are centered on each key point in the group of key points. The electronic device 120 may then screen the group of image blocks starting from the image blocks with higher feature values of the corresponding key points according to the sorting results of the key points. Accordingly, the image blocks corresponding to the key points with higher feature values may be determined as candidate image blocks.
[0079] The candidate image blocks determined in the above manner may include more than one key point, and different candidate image blocks may overlap. In some embodiments, when screening based on the feature values of the key points, the overlap between the image blocks may be considered to obtain a set of candidate image blocks in which different candidate image blocks do not overlap with each other. Exemplarily, the electronic device 110 may determine whether the image blocks overlap by an IOU algorithm.
[0080] Furthermore, the electronic device 120 can determine at least one target image block from the group of candidate image blocks. In some embodiments, these candidate image blocks can be directly determined as image blocks to be processed. In some embodiments, the target image block can be determined from the group of candidate image blocks based on the sorting results of the feature values of the corresponding key points. For example, a predetermined number of candidate image blocks can be selected as target image blocks. It should be noted that the number of target image blocks selected in the watermark extraction process is larger than the number of image blocks to be processed selected in the watermark embedding process. In this way, the probability of hitting the key points used for embedding can be increased. Exemplarily, the electronic device 120 can determine the 50 candidate image blocks whose feature values of the corresponding key points in a group of candidate image blocks are ranked in the top 50 as target image blocks.
[0081] In some embodiments, the size of the target image block may be larger than the size of the candidate image block. If the candidate image block has a first size and the target image block has a second size, the second size is larger than the first size. In this embodiment, for each candidate image block, the electronic device 120 may determine the image block to be extracted 106 as the target image block, based on the position of the candidate image block in the image to be extracted 106, including the key point in the candidate image block and having the second size. For example, an image block centered on the key point and having the second size may be determined as the target image block. Exemplarily, if the first size of the candidate image block is 64*64, the second size of the target image block may be 128*128. In this example, an image area centered on each key point and having a size of 128*128 may be determined as the target image block. In this embodiment, the candidate image block and the corresponding target image block may have the same center, for example, the center may be the corresponding key point. In addition, although different candidate image blocks do not overlap with each other, different target image blocks may overlap.
[0082] Continue with process 400. In block 420, the electronic device 120 determines the watermark information 122 embedded in the image to be extracted 106 based on at least one target image block. That is, the watermark information can be extracted from these target image blocks respectively. For example, if there are multiple target image blocks, multiple binary watermark strings can be obtained. Then, an error correction algorithm (e.g., BCH error correction algorithm) can be used for verification, and the watermark string that succeeds in verification is the correct watermark string.
[0083] In block 420, the electronic device 120 may use any suitable watermark extraction technology to extract watermark information from the target image block. The watermark extraction technology may include, but is not limited to, the least significant bit technology (LSB), discrete cosine transform (DCT), etc.
[0084] In some embodiments, the electronic device 120 may utilize one or more trained machine learning models to extract watermark information 122. Neural networks may include, for example, convolutional neural networks (CNNs), feedforward neural networks (FNNs), recurrent neural networks (RNNs), etc. These trained models may be deployed in the electronic device 120, or in other devices accessible to the electronic device 120. In particular, in some embodiments, the machine learning model for watermark extraction may be trained together with the machine learning model for watermark embedding.
[0085] Some exemplary embodiments of watermark extraction of the present disclosure are described above. In general, key points in an image can be used to determine image blocks for watermark extraction. In this way, watermark extraction can be achieved conveniently and quickly while ensuring accuracy of watermark extraction.
[0086] As mentioned above, in some embodiments, the electronic device 110 may use a trained watermark embedding model to embed a watermark into a to-be-processed image block, and the electronic device 120 may use a trained watermark extraction model to extract a watermark from a target image block. In some embodiments, the watermark embedding model and the watermark extraction model may be trained together.
[0087] Figure 5 FIG. 5 is a schematic diagram of an architecture 500 for model training according to some embodiments of the present disclosure. Figure 5 As shown, the architecture 500 includes a watermark embedding model 510 (eg, the watermark embedding model 300 ), a transformation module 520 , and a watermark extraction model 530 .
[0088] The reference watermark information 502 and the first image 504 are input to a watermark embedding model 510. The watermark embedding model 510 embeds the reference watermark information 502 into the first image 504 to obtain a second image 514.
[0089] The transformation module 520 can perform multiple transformation operations on the second image 514 to generate a third image 524. That is, the second image 514 is changed into a third image 524 using the transformation module 520. The multiple transformation operations performed can be any combination of two or more of the candidate transformation operations. Examples of candidate transformation operations may include, but are not limited to, compression, noise addition (e.g., Gaussian noise addition), cropping, color conversion, rotation, scaling, etc. In a forward propagation, a random combination of multiple transformation operations can be used. In this way, the model can be made to have the ability to resist image changes, which effectively improves the robustness of the model.
[0090] In some embodiments, the degree of change of each of the multiple transformation operations on the second image 514 may remain unchanged. In other embodiments, the degree of change of at least one of the multiple transformation operations on the second image 514 may increase with the increase of the training rounds. For example, the degree of change may be in a linear growth pattern. The degree of change of the second image 514 by the transformation operation is associated with the strength coefficient of the transformation operation. The increase in the degree of change with the increase of the training rounds means that the strength of the transformation operation can grow linearly with the training rounds. For example, if the transformation operation is cropping, the proportion of cropping can increase with the increase of the training rounds. For another example, if the transformation operation is noise addition, the amount or intensity of the added noise can increase with the increase of the training rounds. As an example, one or more transformation operations can be selected from the multiple transformation operations in each round of training to increase their degree of change. As another example, the degree of change can be increased for each transformation operation in each round of training.
[0091] Therefore, in the early stage of model training, weaker transformation operations are used for training. As the number of training rounds increases, the strength of the transformation operations is continuously increased, which can make the model more easily converged and improve the training effect of the model. This can be regarded as a progressive data enhancement strategy, which can effectively improve the robustness of the model to different degrees of image changes.
[0092] Continuing with architecture 500, the third image 524 is provided to a watermark extraction model 530. The watermark extraction model 530 may extract watermark information 532 from the third image 524.
[0093] In some embodiments, the watermark embedding model 510 and the watermark extraction model 530 can be trained based on an objective function related to the difference between the watermark information 532 and the reference watermark information 502. The objective function is to ensure that the extracted watermark information is as consistent as possible with the real watermark information. The objective function here can be a loss function. The loss function used for the difference in watermark information can include, for example, a mean square error (MSE) loss function, but other loss functions are also possible, such as an L1 loss function and an L2 loss function. Figure 5 In the example of , the extraction loss 508 can be determined based on the difference between the reference watermark information 502 and the extracted watermark information 532. The watermark embedding model 510 and the watermark extraction model 530 can be trained by back propagation of the extraction loss 508.
[0094] In addition to the extraction loss 508, other objective functions or loss functions may be used for training the watermark embedding model 510 and / or the watermark extraction model 530. Figure 5 As shown, an embedding loss 541 may be determined based on the difference between the first image 504 and the second image 514. The embedding loss 541 is to ensure that the image embedded with the watermark is as similar as possible to the original image. Exemplarily, the embedding loss 541 may use an MSE loss function. In some embodiments, a multi-scale structural similarity (MS-SSIM) loss function may be additionally used to further enhance the visual effect of the watermarked image.
[0095] At least one of a mean square error (MSE) loss function, a structural similarity (SSIM) loss function, a binary cross entropy (BCE) loss function, etc. Specifically, the watermark embedding model 510 can be trained based on an embedding loss 541 between the first image 504 and the second image 514, and the watermark extraction model 530 can be trained based on an extraction loss 508 between the reference watermark information 502 and the watermark information 532. The training objective of the watermark embedding model 510 at least includes making the embedding loss 541 less than a first threshold, and the training objective of the watermark extraction model 530 at least includes making the extraction loss 508 less than a second threshold.
[0096] In some embodiments, the watermark embedding model 510 can be trained in an adversarial manner. In such an embodiment, the architecture 500 can also include a discriminator 540. Training the discriminator 540 can include two goals. One goal is to make the discriminator 540 judge the first image 504 (i.e., the original image) as true and the second image 514 (i.e., the image embedded with the watermark) as false. This goal is represented by the adversarial loss 542. Another goal is to make the discriminator 540 judge the second image 514 as true. The other goal is represented by the embedding loss 543. Exemplarily, for the discriminator 540, a binary cross entropy (BCE) loss function can be used.
[0097] For model training, any suitable training set and validation set can be used. The images in the training set and validation set are scaled to a predetermined image block size (e.g., 128*128), and the watermark information also has the same length (e.g., 64 bits). Then reasonable weights can be assigned to the various loss functions mentioned above. Any suitable optimizer (e.g., Adam optimizer) and learning rate decay strategy (e.g., exponential decay strategy) can be used.
[0098] References Figures 2 to 5 Some example embodiments of watermark embedding and watermark extraction are described separately. Figure 6 and Figure 7 Exemplary processes of watermark embedding and watermark extraction are described separately.
[0099] Figure 6 FIG. 6 is a schematic diagram of a watermark embedding process 600 according to some embodiments of the present disclosure. The process 600 may be implemented at the electronic device 110. For ease of discussion, reference will be made to FIG. Figure 1A Process 600 is described with reference to environment 100A.
[0100] In block 610, the electronic device 110 obtains the image to be processed 102 and the watermark information 104. The image to be processed 102 may be an image of any resolution.
[0101] In block 620, the electronic device 110 determines key points in the image to be processed 102, and sorts the key points in descending order based on the corresponding feature values of the key points. For example, an ORB key point algorithm may be used to identify key points and determine the feature value of each key point. Then, the key points may be sorted from high to low according to the feature value.
[0102] In block 630, the electronic device 110 screens key points corresponding to high eigenvalues to ensure that candidate image blocks of size a*a centered on each key point do not overlap with each other. a*a may be, for example, 128*128. An IOU algorithm may be used to determine whether image blocks overlap.
[0103] In block 640, the electronic device 110 determines the corresponding texture complexity of each candidate image block of size a*a, and sorts them in descending order based on the complexity, and selects N candidate image blocks with the highest complexity as the image blocks to be processed. N may be, for example, 10. Exemplarily, the JND algorithm may be used to calculate the texture complexity.
[0104] In block 650, the electronic device 110 inputs the image block to be processed into the trained watermark embedding model to obtain a target image block embedded with watermark information, also referred to as a watermark image block. For example, the BCH algorithm is used to add error correction bits to binary watermark information, and a 64-bit watermark binary string is obtained, which includes 24-bit error correction bits. For the selected image block, it is first preprocessed and then input into the trained watermark embedding model together with the watermark information.
[0105] In block 660, the electronic device 110 replaces the corresponding image block to be processed with the generated target image block to generate a watermark image, that is, the original image block is replaced with the watermark image block to obtain the watermark image.
[0106] Figure 7 Schematic diagram of an example process 700 of watermark extraction according to some embodiments of the present disclosure is shown. Process 700 can be implemented at electronic device 120. For ease of discussion, reference will be made to Figure 1B Process 700 is described with reference to environment 100B.
[0107] In block 710, the electronic device 120 obtains the image to be extracted 106. The image to be extracted 106 is an image that may contain a watermark.
[0108] In block 720 , the electronic device 120 determines key points in the image 106 to be extracted, and sorts the key points in descending order based on the corresponding feature values of the key points.
[0109] In block 730, the electronic device 120 screens key points corresponding to high eigenvalues to ensure that candidate image blocks of size b*b centered on each key point do not overlap with each other. b*b may be, for example, 64*64. An IOU algorithm may be used to determine whether image blocks overlap.
[0110] In block 740, the electronic device 120 determines the first M key points of the corresponding candidate image blocks that do not overlap with each other, and determines target image blocks of size c*c centered on the M key points, where M is, for example, 50, and c*c is, for example, 128*128.
[0111] In block 750, the electronic device 120 inputs the target image block into the trained watermark extraction model to obtain watermark information. For example, for the selected image block, it can be pre-processed and then input into the trained watermark extraction model to obtain a probability vector with the same length as the watermark. By setting a suitable threshold, multiple binary watermark strings can be obtained. Then, the BCH error correction algorithm can be used for verification, and the watermark string that succeeds in verification is the correct watermark string.
[0112] Figure 8 A schematic structural block diagram of an apparatus 800 for watermark embedding according to some embodiments of the present disclosure is shown. The apparatus 800 may be implemented as or included in the electronic device 110. Each module / component in the apparatus 800 may be implemented by hardware, software, firmware or any combination thereof.
[0113] As shown in the figure, the device 800 includes an image block determination module 810, which is configured to determine at least one image block to be processed in the image to be processed based on a set of key points identified in the image to be processed. The device 800 also includes an image block generation module 820, which is configured to generate at least one target image block corresponding to the at least one image block to be processed based on the watermark information and the at least one image block to be processed, each target image block is embedded with at least a part of the watermark information. The device 800 also includes an image block replacement module 830, which is configured to generate a target image for the image to be processed by replacing at least one image block to be processed in the image to be processed with at least one target image block.
[0114] In some embodiments, the image block determination module 810 includes: a candidate image block determination module, configured to determine a group of candidate image blocks in the image to be processed based on corresponding feature values of a group of key points and corresponding positions in the image to be processed, each candidate image block includes at least one key point in the group of key points; and a to-be-processed image block determination module, configured to determine at least one to-be-processed image block from the group of candidate image blocks.
[0115] In some embodiments, different candidate image blocks in a set of candidate image blocks do not overlap with each other.
[0116] In some embodiments, the module for determining the image block to be processed includes: a texture complexity determination module configured to determine the corresponding texture complexity of a group of candidate image blocks; and an image block selection module configured to select at least one image block to be processed from the group of candidate image blocks based on the corresponding texture complexity.
[0117] In some embodiments, the image block generation module 820 is further configured to: for at least one image block to be processed, obtain a target image block corresponding to the image block to be processed in the following manner: based on the features of the watermark information and the features of the image block to be processed, generate a fusion feature of the watermark information and the image block to be processed; and based on the fusion feature, the features of the watermark information and the image block to be processed, generate a target image block.
[0118] In some embodiments, the image block generation module 820 includes: an intermediate feature generation module, configured to generate an intermediate feature enhanced with watermark information based on the fused feature and the feature of the watermark information; and a target image block generation module, configured to generate a target image block based on the intermediate feature and the image block to be processed.
[0119] In some embodiments, a target image block corresponding to an image block to be processed is obtained using a watermark embedding model, and the watermark embedding model is trained as follows: using the watermark embedding model, reference watermark information is embedded into a first image to obtain a second image; a third image is generated by performing multiple transformation operations on the second image; the difference between the watermark information extracted from the third image and the reference watermark information is determined; and the watermark embedding model is trained based on an objective function related to the difference.
[0120] In some embodiments, the degree to which at least one of the plurality of transformation operations changes the second image increases as the number of training rounds increases.
[0121] Fig. 9 A schematic structural block diagram of an apparatus 900 for watermark extraction according to some embodiments of the present disclosure is shown. The apparatus 900 may be implemented as or included in the electronic device 120. Each module / component in the apparatus 900 may be implemented by hardware, software, firmware or any combination thereof.
[0122] As shown in the figure, the device 900 includes a target image block determination module 910, which is configured to determine at least one target image block in the image to be extracted based on a set of key points identified in the image to be extracted. The device 900 also includes a watermark information extraction module 920, which is configured to determine the watermark information embedded in the image to be extracted based on the at least one target image block.
[0123] In some embodiments, the target image block determination module 910 includes: a candidate image block determination module, configured to determine a group of candidate image blocks in the image to be extracted based on corresponding feature values of a group of key points and corresponding positions in the image to be extracted, each candidate image block includes at least one key point in the group of key points; and an image block selection module, configured to determine at least one target image block based on the group of candidate image blocks.
[0124] In some embodiments, different candidate image blocks in a set of candidate image blocks do not overlap with each other.
[0125] In some embodiments, a group of candidate image blocks respectively have a first size, and the image block selection module is further configured to: for a candidate image block in the group of candidate image blocks, based on the position of the key points included in the candidate image block in the image to be extracted, determine the image block including the key points and having a second size as one of the at least one target image blocks, and the second size is larger than the first size.
[0126] In some embodiments, watermark information is determined using a watermark extraction model, and the watermark extraction model is trained as follows: a second image is obtained by embedding reference watermark information into a first image; a third image is generated by performing multiple transformation operations on the second image; a difference between the watermark information extracted from the third image using the watermark extraction model and the reference watermark information is determined; and the watermark extraction model is trained based on an objective function related to the difference.
[0127] In some embodiments, the degree to which at least one of the plurality of transformation operations changes the second image increases as the number of training rounds increases.
[0128] Fig.10 1 shows a block diagram of an electronic device 1000 in which one or more embodiments of the present disclosure may be implemented. It should be understood that Fig.10 The electronic device 1000 shown is merely exemplary and should not constitute any limitation on the functionality and scope of the embodiments described herein. Fig.10 The electronic device 1000 shown may be used to implement the electronic device 110 and / or the electronic device 120 of FIG. 1 .
[0129] like Fig.10 As shown, the electronic device 1000 is in the form of a general electronic device. The components of the electronic device 1000 may include, but are not limited to, one or more processors or processing units 1010, a memory 1020, a storage device 1030, one or more communication units 1040, one or more input devices 1050, and one or more output devices 1060. The processing unit 1010 may be an actual or virtual processor and is capable of performing various processes according to a program stored in the memory 1020. In a multi-processor system, multiple processing units execute computer executable instructions in parallel to improve the parallel processing capability of the electronic device 1000.
[0130] The electronic device 1000 typically includes a plurality of computer storage media. Such media may be any accessible media that is accessible to the electronic device 1000, including but not limited to volatile and non-volatile media, removable and non-removable media. The memory 1020 may be a volatile memory (e.g., a register, a cache, a random access memory (RAM)), a non-volatile memory (e.g., a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof. The storage device 1030 may be a removable or non-removable medium, and may include a machine-readable medium, such as a flash drive, a disk, or any other medium, which may be capable of being used to store information and / or data (e.g., training data for training) and may be accessed within the electronic device 1000.
[0131] The electronic device 1000 may further include additional removable / non-removable, volatile / non-volatile storage media. Fig.10 As shown in , a disk drive for reading or writing from a removable, non-volatile disk (e.g., a "floppy disk") and an optical drive for reading or writing from a removable, non-volatile optical disk may be provided. In these cases, each drive may be connected to the bus (not shown) by one or more data media interfaces. Memory 1020 may include a computer program product 1025 having one or more program modules that are configured to perform various methods or actions of various embodiments of the present disclosure.
[0132] The communication unit 1040 implements communication with other electronic devices through a communication medium. Additionally, the functions of the components of the electronic device 1000 can be implemented in a single computing cluster or multiple computing machines that can communicate through a communication connection. Therefore, the electronic device 1000 can operate in a networked environment using a logical connection with one or more other servers, a network personal computer (PC), or another network node.
[0133] The input device 1050 may be one or more input devices, such as a mouse, a keyboard, a tracking ball, etc. The output device 1060 may be one or more output devices, such as a display, a speaker, a printer, etc. The electronic device 1000 may also communicate with one or more external devices (not shown) through the communication unit 1040 as needed, such as a storage device, a display device, etc., communicate with one or more devices that allow a user to interact with the electronic device 1000, or communicate with any device that allows the electronic device 1000 to communicate with one or more other electronic devices (e.g., a network card, a modem, etc.). Such communication may be performed via an input / output (I / O) interface (not shown).
[0134] According to an exemplary implementation of the present disclosure, a computer-readable storage medium is provided, on which computer-executable instructions are stored, wherein the computer-executable instructions are executed by a processor to implement the method described above. According to an exemplary implementation of the present disclosure, a computer program product is also provided, which is tangibly stored on a non-transitory computer-readable medium and includes computer-executable instructions, and the computer-executable instructions are executed by a processor to implement the method described above.
[0135] Various aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices, equipment, and computer program products implemented according to the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.
[0136] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processing unit of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.
[0137] Computer-readable program instructions can be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operational steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, so that the instructions executed on the computer, other programmable data processing apparatus, or other device implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.
[0138] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple implementations of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of a module, program segment or instruction includes one or more executable instructions for realizing the logical function of the specification. In some implementations as replacements, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be realized by a special hardware-based system that performs the function or action of the specification, or can be realized by a combination of special hardware and computer instructions.
[0139] The above descriptions of various implementations of the present disclosure are exemplary, non-exhaustive, and not limited to the disclosed implementations. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described implementations. The selection of terms used herein is intended to best explain the principles of the implementations, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the various implementations disclosed herein.
Claims
1. A watermark embedding method, include: Determining at least one image block to be processed in the image to be processed based on a group of key points identified in the image to be processed; Based on the watermark information and the at least one image block to be processed, generating at least one target image block corresponding to the at least one image block to be processed, each target image block is embedded with at least a part of the watermark information; as well as A target image for the image to be processed is generated by replacing the at least one image block to be processed with the at least one target image block in the image to be processed.
2. The method according to claim 1, wherein determining the at least one image block to be processed include: Based on corresponding feature values of the set of key points and corresponding positions in the image to be processed, determining a set of candidate image blocks in the image to be processed, each candidate image block including at least one key point in the set of key points; as well as The at least one image block to be processed is determined from the group of candidate image blocks. The method according to claim 2 , wherein different candidate image blocks in the set of candidate image blocks do not overlap with each other.
4. The method according to claim 2, wherein determining the at least one image block to be processed include: determining corresponding texture complexities of the set of candidate image blocks; as well as Based on the corresponding texture complexity, the at least one image block to be processed is selected from the group of candidate image blocks.
5. The method according to claim 1, wherein generating the at least one target image block include: For an image block to be processed in the at least one image block to be processed, a target image block corresponding to the image block to be processed is obtained in the following manner: Based on the features of the watermark information and the features of the image block to be processed, generating fusion features of the watermark information and the image block to be processed; and The target image block is generated based on the fusion feature, the feature of the watermark information and the image block to be processed.
6. The method according to claim 5, wherein the target image block is generated include: Based on the fused features and the features of the watermark information, generating an intermediate feature enhanced by the watermark information; as well as The target image block is generated based on the intermediate features and the image block to be processed.
7. The method according to claim 1 or 5, wherein the target image block corresponding to the image block to be processed is obtained by using a watermark embedding model, and the watermark embedding model is trained as follows: Using the watermark embedding model, embedding reference watermark information into the first image to obtain a second image; generating a third image by performing a plurality of transformation operations on the second image; determining a difference between the watermark information extracted from the third image and the reference watermark information; as well as The watermark embedding model is trained based on an objective function related to the difference. 8 . The method according to claim 7 , wherein the degree of change of the second image by at least one transformation operation among the plurality of transformation operations increases with an increase in training rounds.
9. A watermark extraction method, include: Determining at least one target image block in the image to be extracted based on a set of key points identified in the image to be extracted; as well as Based on the at least one target image block respectively, watermark information embedded in the image to be extracted is determined.
10. The method according to claim 9, wherein determining the at least one target image block include: Based on corresponding feature values of the set of key points and corresponding positions in the image to be extracted, determining a set of candidate image blocks in the image to be extracted, each candidate image block including at least one key point in the set of key points; as well as The at least one target image block is determined based on the set of candidate image blocks.
11. The method according to claim 9, wherein the watermark information is determined using a watermark extraction model, and the watermark extraction model is trained as follows: Obtaining a second image by embedding reference watermark information into the first image; generating a third image by performing a plurality of transformation operations on the second image; determining a difference between the watermark information extracted from the third image using the watermark extraction model and the reference watermark information; as well as The watermark extraction model is trained based on an objective function related to the difference.
12. A watermark embedding device, include: An image block determination module is configured to determine at least one image block to be processed in the image to be processed based on a group of key points identified in the image to be processed; an image block generation module, configured to generate at least one target image block corresponding to the at least one image block to be processed based on the watermark information and the at least one image block to be processed, each target image block being embedded with at least a part of the watermark information; as well as The image block replacement module is configured to generate a target image for the image to be processed by replacing the at least one image block to be processed in the image to be processed with the at least one target image block.
13. A watermark extraction device, include: A target image block determination module is configured to determine at least one target image block in the image to be extracted based on a group of key points identified in the image to be extracted; as well as The watermark information extraction module is configured to determine the watermark information embedded in the image to be extracted based on the at least one target image block.
14. An electronic device, include: at least one processing unit; as well as At least one memory, the at least one memory being coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, the instructions, when executed by the at least one processing unit, causing the electronic device to perform a method according to any one of claims 1 to 8 or 9 to 14.
15. A computer-readable storage medium having a computer program stored thereon, wherein the computer program can be executed by a processor to implement the method according to any one of claims 1 to 8 or 9 to 11.
Citation Information
Cited By
Method, apparatus, device and storage medium for watermark embedding and extraction
EP4468692A1
Method, apparatus, device and storage medium for watermark embedding and extraction
EP4468692B1
