Image watermark processing method and device, storage medium and electronic equipment

Abstract

The embodiment of the invention provides an image watermark processing method and device, a storage medium and electronic equipment, and the method comprises the steps: obtaining random watermark information and a to-be-processed image indicated by a watermark processing instruction, inputting the to-be-processed image and the random watermark information into a watermark embedding model, the watermark embedding model comprises a group of first convolution layers and a group of parameter adjustment module layers, and each parameter adjustment module layer is connected with one first convolution layer in the group of first convolution layers; each parameter adjustment module layer is used for generating a modulation parameter of the first convolutional layer connected with each parameter adjustment module layer according to the random watermark information, and the modulation parameter of each first convolutional layer in the group of first convolutional layers is used for modulating a feature map output by each first convolutional layer. Through the watermarking method and device, the problem that watermarking processing cannot be directly carried out on images of any size in the prior art is solved, and the practicability and universality of the watermarking technology are improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of digital watermark technology, and in particular, to an image watermark processing method, device, storage medium and electronic device. Background Art

[0002] Digital watermarking is a method of embedding invisible or semi-visible marks into digital media files for copyright protection, data integrity verification or information tracking. Watermark information can be a copyright logo, serial number or any other form of digital data, and is not easily detected under normal viewing or hearing. This technology is widely used in the fields of images, audio and video.

[0003] In the related technologies, watermark processing methods mainly rely on signal processing technologies, such as wavelet transform, discrete cosine transform (DCT) and Fourier transform, etc. Specifically, the image to be processed is preprocessed to a specific size, and then the preprocessed image to be processed is subjected to signal processing technology to add a watermark, thereby embedding the watermark information into the frequency domain or transform domain of the image to minimize the impact on the visual quality of the image.

[0004] However, the watermark processing method in the above-mentioned related art is only applicable to images of a specific size, and has limited processing capabilities for images of any size, which is a significant limitation in practical applications. Therefore, there is a technical problem in the related art that it is impossible to directly perform watermark processing on images of any size. Summary of the invention

[0005] The embodiments of the present application provide an image watermark processing method, device, storage medium and electronic device to at least solve the technical problem in the related art that it is impossible to directly perform watermark processing on images of any size.

[0006] According to one aspect of an embodiment of the present application, a method for processing an image watermark is provided, comprising:

[0007] In response to the acquired watermark processing instruction, acquiring random watermark information and an image to be processed indicated by the watermark processing instruction, wherein the image to be processed is an image to be watermarked;

[0008] The image to be processed and the random watermark information are input into a watermark embedding model to obtain an embedded watermark image output by the watermark embedding model, wherein the watermark embedding model is used to embed the input watermark information into the input image, and the watermark embedding model includes a group of first convolutional layers and a group of parameter adjustment module layers, each parameter adjustment module layer in the group of parameter adjustment module layers is connected to a first convolutional layer in the group of first convolutional layers; each parameter adjustment module layer is used to generate modulation parameters of the first convolutional layer connected to each parameter adjustment module layer according to the random watermark information, and the modulation parameters of each first convolutional layer in the group of first convolutional layers are used to modulate the feature map output by each first convolutional layer.

[0009] According to another aspect of the embodiment of the present application, there is also provided an image watermark processing device, including:

[0010] an acquisition unit, configured to acquire random watermark information and an image to be processed indicated by the watermark processing instruction in response to the acquired watermark processing instruction, wherein the image to be processed is an image to be watermarked;

[0011] An output unit is used to input the image to be processed and the random watermark information into a watermark embedding model to obtain an embedded watermark image output by the watermark embedding model, wherein the watermark embedding model is used to embed the input watermark information into the input image, and the watermark embedding model includes a group of first convolutional layers and a group of parameter adjustment module layers, each parameter adjustment module layer in the group of parameter adjustment module layers is connected to a first convolutional layer in the group of first convolutional layers; each parameter adjustment module layer is used to generate modulation parameters of the first convolutional layer connected to each parameter adjustment module layer according to the random watermark information, and the modulation parameters of each first convolutional layer in the group of first convolutional layers are used to modulate the feature map output by each first convolutional layer.

[0012] According to another aspect of the embodiments of the present application, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above method embodiments when run.

[0013] According to another aspect of the embodiments of the present application, a computer program product or a computer program is provided, the computer program product or the computer program includes computer instructions, the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the steps in any of the above method embodiments.

[0014] According to another aspect of the embodiments of the present application, there is further provided an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the steps in any one of the above method embodiments through the computer program.

[0015] Through the present application, the image to be processed and the random watermark information are input into the watermark embedding model to obtain the embedded watermark image output by the watermark embedding model, wherein the watermark embedding model is used to embed the input watermark information into the input image, each parameter adjustment module layer in the watermark embedding model is used to generate the modulation parameters of the first convolution layer connected to each parameter adjustment module layer according to the random watermark information, and the modulation parameters of each first convolution layer in a group of first convolution layers are used to modulate the feature map output by each first convolution layer, so as to modulate the output features of the convolution layer through the parameter adjustment module layer in the neural network, and the generation of the modulation parameters is independent of the image size, so that the watermark information can be embedded into an image of any size, solving the problem that the watermark processing of an image of any size cannot be directly performed in the related art, thereby improving the practicality and versatility of the watermark technology. In addition, the output features of the first convolution layer are modulated using the parameter adjustment module layer, so that the watermark information is integrated with the deep features of the image to be processed, which not only maintains the visual quality of the image, but also increases the capacity of the watermark information to a certain extent, thereby improving the information carrying capacity. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 is a schematic diagram of an application scenario of an image watermark processing method according to an embodiment of the present application;

[0017] Figure 2 is a flowchart of an optional image watermark processing method according to an embodiment of the present application;

[0018] Figure 3 is a schematic diagram of an optional watermark embedding model according to an embodiment of the present application;

[0019] Figure 4 is a schematic diagram of an optional parameter adjustment module layer according to an embodiment of the present application;

[0020] Figure 5 is a schematic diagram of an optional watermark extraction model according to an embodiment of the present application;

[0021] Figure 6 is a schematic diagram of an optional target model according to an embodiment of the present application;

[0022] Figure 7 is a structural block diagram of an optional image watermark processing device according to an embodiment of the present application;

[0023] Figure 8It is a block diagram of a computer system structure of an optional electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0024] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0025] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0026] According to one aspect of an embodiment of the present application, a method for processing an image watermark is provided. Optionally, in this embodiment, the above-mentioned method for processing an image watermark can be applied to, but is not limited to, Figure 1 The hardware environment shown includes a terminal device 102 and a server 104. The server 104 can be connected to the terminal device 102 via a network, and can be used to provide services (e.g., application services, etc.) for the terminal device 102 or a client installed on the terminal device 102. A database can be set on the server 104 or independently of the server 104 to provide data storage services for the server 104.

[0027] The above network may include but is not limited to at least one of the following: wired network, wireless network. The above wired network may include but is not limited to at least one of the following: wide area network, metropolitan area network, local area network, and the above wireless network may include but is not limited to at least one of the following: WIFI (Wireless Fidelity), Bluetooth. The terminal device 102 may be but is not limited to a PC (Personal Computer), a mobile phone, a tablet computer, etc. The server 104 may be but is not limited to a cloud server, a server cluster or other server types.

[0028] The image watermark processing method of the embodiment of the present application can be executed by the server 104, or by the terminal device 102, or by both the server 104 and the terminal device 102. The image watermark processing method of the embodiment of the present application can be executed by the terminal device 102 or by a client installed thereon.

[0029] Taking the image watermark processing method in this embodiment executed by the terminal device 102 as an example, Figure 2 is a flow chart of an optional image watermark processing method according to an embodiment of the present application, such as Figure 2 As shown, the process of the method may include the following steps:

[0030] Step S202, in response to the acquired watermark processing instruction, acquiring random watermark information and the image to be processed indicated by the watermark processing instruction, wherein the image to be processed is the image to be watermarked;

[0031] Step S204, input the image to be processed and the random watermark information into the watermark embedding model to obtain an embedded watermark image output by the watermark embedding model, wherein the watermark embedding model is used to embed the input watermark information into the input image, and the watermark embedding model includes a group of first convolutional layers and a group of parameter adjustment module layers, each parameter adjustment module layer in a group of parameter adjustment module layers is connected to a first convolutional layer in a group of first convolutional layers; each parameter adjustment module layer is used to generate modulation parameters of the first convolutional layer connected to each parameter adjustment module layer according to the random watermark information, and the modulation parameters of each first convolutional layer in a group of first convolutional layers are used to modulate the feature map output by each first convolutional layer.

[0032] The image watermark processing method in this embodiment can be applied to a wide range of fields such as digital copyright protection, image authentication and multimedia security. In particular, in scenarios where a large number of image files are shared on the Internet and mobile devices, it can effectively add invisible watermarks to images for subsequent copyright tracking or data verification.

[0033] In the related art, the watermark processing method mainly relies on signal processing technology, such as wavelet transform, discrete cosine transform (DCT) and Fourier transform. Specifically, after the image to be processed is preprocessed to a specific size, the preprocessed image to be processed is subjected to signal processing technology to add a watermark, thereby embedding the watermark information into the frequency domain or transform domain of the image to minimize the impact on the visual quality of the image. However, the related art is only applicable to images of a specific size, and has limited processing capabilities for images of any size, resulting in the problem that the related art cannot directly perform watermark processing on images of any size.

[0034] In order to at least partially solve the above technical problems, in this embodiment, the image to be processed and the random watermark information are input into the watermark embedding model to obtain the embedded watermark image output by the watermark embedding model, wherein the watermark embedding model is used to embed the input watermark information into the input image, each parameter adjustment module layer in the watermark embedding model is used to generate the modulation parameters of the first convolution layer connected to each parameter adjustment module layer according to the random watermark information, and the modulation parameters of each first convolution layer in a group of first convolution layers are used to modulate the feature map output by each first convolution layer, so as to modulate the output features of the convolution layer through the parameter adjustment module layer in the neural network, so that the watermark information can be embedded into an image of any size, which solves the problem that the watermark processing of an image of any size cannot be directly performed in the related art, thereby improving the practicality and versatility of the watermark technology. In addition, the output features of the first convolution layer are modulated by the parameter adjustment module layer, so that the watermark information is integrated with the deep features of the image to be processed, which not only maintains the visual quality of the image, but also increases the capacity of the watermark information to a certain extent, thereby improving the information carrying capacity.

[0035] It should be noted that the watermark processing instruction may refer to the command signal used to start the watermark embedding process. It may originate from the interaction of the user interface or be automatically triggered by the program, and may contain the basic information required to perform the watermark task, such as the path of the image to be processed, the mode of watermark embedding or extraction, etc. Random watermark information may refer to a random binary sequence generated at the beginning of the watermark embedding process, which is used to identify the copyright, authentication or other information of the image. Randomness helps to increase the complexity of the watermark and increase the difficulty of cracking.

[0036] Optionally, the random watermark information generation process can be: using a pseudo-random number generator (PRNG) or a true random number generator (TRNG) to generate a random binary sequence, wherein the PRNG generates a seemingly random sequence based on a seed or initial value, while the TRNG generates true random numbers based on a physical process (such as thermal noise). The seed can be part of the key or a key generated through a secure protocol. During the watermark embedding and extraction process, the same seed needs to be used to ensure the correct generation and extraction of the watermark information. The generated random binary sequence needs to be encoded according to the specific application to obtain the random watermark information. The encoding process may involve data compression, encryption or error correction coding to improve the efficiency, security and robustness of the watermark.

[0037] When the terminal device obtains the watermark processing instruction, it can trigger the random generation of a random watermark information, and obtain the image to be processed according to the instruction of the watermark processing instruction. The image to be processed refers to the original image to be watermarked, which can be a digital image of any form or size, such as JPEG, PNG, BMP and other formats.

[0038] The watermark embedding model can be a deep learning model for embedding watermark information into an image. The watermark embedding model is a pre-trained model that learns how to hide the watermark information in the deep features of the image while maintaining the visual quality of the image during training. The watermark embedding model can include a set of first convolutional layers and a set of parameter adjustment module layers. Among them, the set of first convolutional layers can be used to extract features from the input image. Each convolutional layer in the set of convolutional layers contains multiple convolution kernels for detecting specific features in the image, such as edges, textures, etc.

[0039] The parameter adjustment module layer is a neural network layer for generating modulation parameters, which can dynamically adjust the behavior of the convolution layer according to the random watermark information. The parameter adjustment module layer can be set with a FiLM (Feature-wise Linear Modulation) mechanism. Through the FiLM mechanism, the modulation parameters generated by the parameter adjustment module layer can linearly modulate the feature map output by the first convolution layer, change the weight of the feature, and thus achieve watermark embedding. The modulation parameters allow the watermark information to affect the feature map in a way that is independent of the image size, thereby achieving high-capacity watermark embedding.

[0040] An embedded watermark image refers to an image that has been processed by a watermark embedding model and has watermark information embedded in it. Generally, the image is visually almost the same as the original image, but carries the watermark information. Generally, the watermark information is invisible on the embedded watermark image.

[0041] Through the embodiment provided by the present application, the image to be processed and the random watermark information are input into the watermark embedding model to obtain the embedded watermark image output by the watermark embedding model, wherein the watermark embedding model is used to embed the input watermark information into the input image, each parameter adjustment module layer in the watermark embedding model is used to generate the modulation parameters of the first convolution layer connected to each parameter adjustment module layer according to the random watermark information, and the modulation parameters of each first convolution layer in a group of first convolution layers are used to modulate the feature map output by each first convolution layer, so as to modulate the output features of the convolution layer through the parameter adjustment module layer in the neural network, and the generation of the modulation parameters is independent of the image size, so that the watermark information can be embedded into an image of any size, solving the problem that the related art cannot directly perform watermark processing on an image of any size, thereby improving the practicality and versatility of the watermark technology. In addition, the output features of the first convolution layer are modulated using the parameter adjustment module layer, so that the watermark information is integrated with the deep features of the image to be processed, which not only maintains the visual quality of the image, but also increases the capacity of the watermark information to a certain extent, thereby improving the information carrying capacity.

[0042] In an exemplary embodiment, each first convolutional layer in a group of first convolutional layers is connected in sequence; step S204 includes: inputting the image to be processed into the watermark embedding model to perform the following processing operations in the watermark embedding model: inputting the random watermark information into each parameter adjustment module layer respectively to obtain the modulation parameters of the first convolutional layer connected to each parameter adjustment module layer; using each first convolutional layer as the current first convolutional layer in sequence to perform the following watermark embedding operations, wherein the input image of the current first convolutional layer is the current input image: performing feature processing operations on the current input image through the current first convolutional layer to obtain the feature map output by the current first convolutional layer; using the modulation parameters of the current first convolutional layer to modulate the feature map output by the current first convolutional layer to obtain the feature map output by the modulated current first convolutional layer; wherein, in a group of first convolutional layers, the input image of the first first convolutional layer is the image to be processed, the input images of the other first convolutional layers except the first first convolutional layer are the feature maps output by the previous convolutional layer of the other first convolutional layers after modulation, and the embedded watermark image is the feature map output by the last first convolutional layer in the group of first convolutional layers after modulation.

[0043] It should be noted that the watermark embedding model may include a group of first convolutional layers, and each first convolutional layer in the group of first convolutional layers is connected in sequence, that is, it is serially connected. In the watermark embedding model, a group of first convolutional layers can be multiple convolutional neural network layers, and multiple convolutional neural network layers are connected in series in sequence, which can be used to extract image features. Each first convolutional layer may include multiple convolutional kernels, and different features in the image, such as edges, textures, etc., can be detected by multiple convolutional kernels.

[0044] The sequential connection of each first convolutional layer means that the output of the current first convolutional layer serves as the input of the next first convolutional layer. This structure allows the watermark embedding model to gradually extract more abstract and complex features from the original input.

[0045] Each first convolutional layer is connected to a parameter adjustment module layer. The parameter adjustment module layer may be provided with a FiLM mechanism. The parameter adjustment module layer may be used to generate modulation parameters of the first convolutional layer connected thereto. The modulation parameters may be generated by the parameter adjustment module layer according to random watermark information and used to adjust the feature map output by the convolutional layer. Through these parameters, the watermark information may be embedded into each channel of the feature map.

[0046] The watermark embedding operation refers to the process of embedding binary watermark information into the image feature map.

[0047] Exemplarily, in a group of first convolutional layers of the watermark embedding model, according to the connection order of each first convolutional layer in the group of first convolutional layers, each first convolutional layer is sequentially used as the current first convolutional layer to perform the watermark embedding operation, and the input image of the current first convolutional layer is used as the current input image. Specifically, the step of the current first convolutional layer performing the watermark embedding operation includes: performing a feature processing operation on the current input image through the current first convolutional layer to obtain a feature map output by the current first convolutional layer, and modulating the feature map output by the current first convolutional layer using the modulation parameters of the current first convolutional layer to obtain a modulated feature map output by the current first convolutional layer. Among them, when the current first convolutional layer is the first convolutional layer in a group of first convolutional layers, its corresponding input image is the image to be processed; when the current first convolutional layer is not the first convolutional layer in a group of first convolutional layers, that is, other first convolutional layers except the first first convolutional layer, its corresponding input image is the feature map of the previous convolutional layer output of the modulated other first convolutional layers; when the current first convolutional layer is the last first convolutional layer in a group of first convolutional layers, its corresponding output image is the embedded watermark image.

[0048] Through this embodiment, the watermark information is embedded into the deep features of the image through the serial use of a group of first convolutional layers. This method can embed higher capacity watermark information while keeping the visual quality of the image unchanged. In addition, the modulation parameters are generated for the first convolutional layer connected to it through the parameter adjustment module layer, so as to use the modulation parameters to influence the feature extraction process of each first convolutional layer, so that the embedding of the watermark is based on the joint influence of the image content and the watermark information, thereby improving the concealment and robustness of the watermark.

[0049] In an exemplary embodiment, a group of first convolutional layers includes a group of feature extraction convolutional layers and a group of transposed convolutional layers, and the last feature extraction convolutional layer in the group of feature extraction convolutional layers is connected to the first transposed convolutional layer in the group of transposed convolutional layers. The feature processing operation is performed on the current input image through the current first convolutional layer to obtain a feature map output by the current first convolutional layer, including:

[0050] When the current first convolution layer is a feature extraction convolution layer, a feature extraction operation is performed on the current input image through the current first convolution layer to obtain a feature extraction map output by the current first convolution layer; when the current first convolution layer is a transposed convolution layer, a feature reconstruction operation is performed on the current input image through the current first convolution layer to obtain a feature reconstruction map output by the current first convolution layer.

[0051] It should be noted that a set of first convolutional layers may include a set of feature extraction convolutional layers and a set of transposed convolutional layers. A set of feature extraction convolutional layers may be a set of convolutional layers used in the watermark embedding model to extract features from the input image. Generally, these layers have a small receptive field and a large step size, and are used to identify local features in the image, such as edges and textures, and gradually build more complex image representations. A set of transposed convolutional layers may be convolutional layers used in the watermark embedding model for spatial size recovery of feature maps and generation of feature maps. Transposed convolutional layers are often used to convert low-resolution or highly compressed feature maps into feature maps of high resolution or original image size. The feature extraction map is the output of the feature extraction convolutional layer after processing the input image. It contains a series of abstract image features that are further processed and utilized in the next or subsequent layers to identify high-level features of the image or perform watermark embedding. The feature reconstruction map is the output after the transposed convolutional layer is processed, and its purpose is to restore the low-dimensional or low-resolution feature map to a high-dimensional or high-resolution state. The feature reconstruction map helps to integrate the watermark information into the detail level of the image while maintaining the visual quality of the image.

[0052] Optionally, the watermark embedding model may further include a set of normalization layers and a set of activation functions. Among them, a normalization layer may be set in each first convolution layer in a set of first convolution layers. Except for the last feature extraction convolution layer in a set of feature extraction convolution layers and the last transposed convolution layer in a set of transposed convolution layers, the output end of each first convolution layer may be connected to an activation function, the last feature extraction convolution layer in a set of feature extraction convolution layers is connected to the first transposed convolution layer in a set of transposed convolution layers, and the last transposed convolution layer in a set of transposed convolution layers may directly output the watermark embedded image.

[0053] By setting a normalization layer in each of the first convolutional layers in a group of first convolutional layers, the watermark embedding model can be made more stable during the training process, avoiding the problem of gradient vanishing or gradient exploding, and the normalization layer can reduce the dependence of the output distribution of each layer, which helps to converge faster and improve training efficiency. Using activation functions (such as ReLU, tanh or sigmoid, etc.) to introduce nonlinear relationships enables the watermark embedding model, a neural network, to learn and represent complex function mappings, activate important features in the network, and suppress irrelevant or redundant features, which helps the watermark embedding model focus on key image information, thereby improving the embedding effect and robustness of the watermark. The last convolutional layer in a group of feature extraction convolutional layers is directly connected to the first convolutional layer of a group of transposed convolutional layers, avoiding the computational overhead and information loss introduced by additional normalization or activation layers, and ensuring direct and efficient transfer from high-level features to watermark embedding.

[0054] In summary, the addition of normalization layers and activation functions as well as the optimization of specific layer structures can significantly improve the training efficiency, robustness and generalization ability of the watermark embedding model.

[0055] Specifically, Figure 3 As shown, Figure 3 : is a schematic diagram of an optional watermark embedding model of an embodiment of the present application. A set of first convolutional layers consists of 4 feature extraction convolutional layers and 4 transposed convolutional layers, wherein the last feature extraction convolutional layer in a set of feature extraction convolutional layers is connected to the input of the first transposed convolutional layer in a set of transposed convolutional layers. The watermark embedding process is as follows:

[0056] Random watermark information is input to each parameter adjustment module layer. The input image (the original image to be processed) first passes through the feature extraction convolution layer (Conv+BN) to extract the low-level features of the image and obtain a feature extraction map. The feature extraction map is further abstracted through subsequent feature extraction convolution layers to obtain a higher-level feature representation, that is, a deeper feature extraction map. Then, in the final stage of feature extraction, the obtained feature extraction map is sent to the starting layer of a set of transposed convolution layers, that is, the first transposed convolution layer (Deconv+BN), to start the feature reconstruction process. The transposed convolution layer gradually restores the received high-level feature information to a feature reconstruction map that is closer to the original image size. After multiple feature space expansions and watermark embedding, the output image (embedded watermark image) is finally obtained, that is, an image of the original image size, in which the watermark information has been embedded in an invisible form. In the process of feature extraction and feature reconstruction, the watermark information is embedded into the feature map through modulation parameters.

[0057] Through this embodiment, by combining the feature extraction convolution layer and the transposed convolution layer, images of any size can be effectively processed. The feature extraction convolution layer allows the watermark information to be embedded in the high-level features of the image, rather than being limited to the surface pixel values, which increases the robustness and concealment of the watermark, enabling it to resist common image processing operations without being destroyed. In addition, the use of the transposed convolution layer in the feature reconstruction process can better restore the detail level of the image, ensuring that the image after embedding the watermark maintains good visual quality, and the watermark information can be effectively embedded and extracted.

[0058] In an exemplary embodiment, each parameter adjustment module layer includes a scaling parameter module and an offset parameter module, the scaling parameter module in each parameter adjustment module layer includes a group of first data sensing units, and the offset parameter module in each parameter adjustment module layer includes a group of second data sensing units;

[0059] The random watermark information is respectively input into each parameter adjustment module layer to obtain the modulation parameters of the first convolution layer of each parameter adjustment module layer, including: in each parameter adjustment module layer, the random watermark information is mapped by each first data perception unit in the scaling parameter module of each parameter adjustment module layer to obtain the scaling parameter vector of the first convolution layer connected to each parameter adjustment module layer, and the random watermark information is mapped by each second data perception unit in the offset parameter module of each parameter adjustment module layer to obtain the offset parameter vector of the first convolution layer connected to each parameter adjustment module layer.

[0060] It should be noted that, in the process of watermark embedding, the parameter adjustment module layer can be used to dynamically adjust the parameters of the first convolutional layer connected to it to achieve the embedding of watermark information. The parameter adjustment module layer modulates the output of the first convolutional layer connected to it by generating a scaling parameter vector and an offset parameter vector.

[0061] Each parameter adjustment module layer may include a scaling parameter module and an offset parameter module. The scaling parameter module is part of the parameter adjustment module layer, which is responsible for generating scaling parameters for adjusting the output of the convolution layer. The offset parameter module is used to generate an offset parameter vector for offset adjustment of the feature map output by the convolution layer. The offset parameter, together with the scaling parameter, acts on the feature map through linear modulation to change its value to embed watermark information.

[0062] The scaling parameter module in each parameter adjustment module layer may include a group of first data sensing units, and the offset parameter module in each parameter adjustment module layer may include a group of second data sensing units. Among them, the first data sensing unit and the second data sensing unit can be understood as neurons or nodes in a neural network, which correspond to different computing units and can sense and process different characteristics of the input data. In this embodiment, the first data sensing unit and the second data sensing unit correspond to the computing units in the scaling parameter module and the offset parameter module, respectively, and are used to map the random watermark information into scaling parameter vectors and offset parameter vectors. Both the first data sensing unit and the second data sensing unit can use a multi-layer perceptron (Multi-Layer Perceptron, abbreviated as MLP), which refers to a neural network structure used to generate scaling parameters and offset parameters from watermark information. Specifically, as Figure 4 As shown, Figure 4It is a schematic diagram of an optional parameter adjustment module layer of an embodiment of the present application. Among them, the parameter adjustment module layer includes a scaling parameter module (i.e., module A) and an offset parameter module (i.e., module B). Both the scaling parameter module and the offset parameter module may include a fully connected layer, a normalization layer, and an activation function. The MLP in the scaling parameter module and the offset parameter module works as follows: random watermark information (i.e., binary watermark information) is input, and first linearly transformed through a series of fully connected layers, and then a normalization layer is used after each fully connected layer to keep the statistical characteristics of the data stable, and then a nonlinear transformation is introduced through an activation function. After these processes, the final output is a scaling parameter vector (i.e., A1, ..., A c ) and the offset parameter vector (i.e., B1, ..., B c ). These two vectors will dynamically adjust the output feature map of the convolutional layer to embed the watermark information.

[0063] Through the multi-layer perceptron, the watermark information can be effectively converted into a set of parameters that can modulate the feature map of the convolutional layer to achieve the embedding of the watermark information. This conversion method not only increases the concealment and robustness of the watermark embedding, but also ensures that the generation of scaling and offset parameters is directly related to the watermark information, improving the flexibility and performance of the model.

[0064] Through this embodiment, the parameter adjustment module layer dynamically generates a scaling parameter vector and an offset parameter vector based on the random watermark information, ensuring that the watermark information can be effectively embedded into the deep features of the image without causing obvious visual changes, thereby improving the concealment and robustness of the watermark.

[0065] In an exemplary embodiment, the feature map output by each first convolution layer includes a feature map of each channel in a set of channels, and the scaling parameter vector of the first convolution layer connected to each parameter adjustment module layer includes scaling parameters equal to the number of channels of the feature map output by the first convolution layer connected to each parameter adjustment module layer; the offset parameter vector of the first convolution layer connected to each parameter adjustment module layer includes offset parameters equal to the number of channels of the feature map output by the first convolution layer connected to each parameter adjustment module layer;

[0066] Using the modulation parameters of the current first convolutional layer to modulate the feature map output by the current first convolutional layer to obtain the modulated feature map output by the current first convolutional layer, including: according to the scaling parameter vector of the current first convolutional layer and the offset parameter vector of the current first convolutional layer, modulating the feature map of each channel output by the current first convolutional layer to obtain the modulated feature map of each channel output by the current first convolutional layer, wherein the modulated feature map output by the current first convolutional layer includes the modulated feature map of each channel output by the current first convolutional layer.

[0067] It should be noted that the first convolutional layer refers to the convolutional layer used for feature extraction in the watermark embedding model, or the convolutional layer used for feature reconstruction in the watermark embedding model. After the convolutional layer performs a convolution operation on the input image, it outputs a series of two-dimensional matrices that match the size of the input image (or are reduced in dimension). These matrices are called feature maps. Each feature map corresponds to a channel, representing a specific feature or information in the image. The scaling parameter vector is a set of values, the same as the number of channels of the feature map output by the first convolutional layer. These parameters are used to adjust the value of each channel in the feature map, and increase or decrease the intensity or range of the feature map through scaling operations, thereby achieving specific modulation of the features. The offset parameter vector also contains the same number of values ​​as the number of channels, which is used for offset adjustment of the feature map. By adding or subtracting a specific value, the value of each channel in the feature map is changed to achieve further modulation of the feature map. For example, a specific first convolutional layer in the watermark embedding model outputs a feature map containing 64 channels. The scaling parameter module and the offset parameter module will generate a scaling parameter vector and an offset parameter vector containing 64 elements respectively. These parameter vectors will be dynamically generated according to the random watermark information to ensure the flexibility and robustness of the watermark embedding process.

[0068] Optionally, the feature map of each channel of the modulated current first convolutional layer output can be determined by the following formula (1). Specifically,

[0069] F c =A c F c +B c ; (1)

[0070] Among them, F′ c is the new value of the c-th channel of F, that is, the adjusted value of the c-th channel output of the current first convolutional layer, F c is the value of the cth channel of the original output of the first convolutional layer, A c represents the cth scalar element of the scaling parameter vector; B c Represents the c-th scalar element of the offset parameter vector.

[0071] Through this embodiment, by modulating the feature map output by the first convolutional layer channel by channel (i.e., multiplying the feature map of each channel by the corresponding scaling parameter and adding the offset parameter), the depth characteristics of the image can be changed, and these changes are difficult for the human eye to detect, thereby achieving invisible embedding of the watermark. The generation of scaling parameter vectors and offset parameter vectors substantially expands the information capacity of watermark embedding, because the feature map of each channel can carry part of the watermark information, which significantly increases the overall amount of embedded watermark information. In addition, since the generation of scaling and offset parameter vectors is independent of the image size (only related to the number of channels), the technical solution can process images of any size, thereby improving the scope of application of the watermark embedding model.

[0072] In an exemplary embodiment, the watermark embedding model is connected to a watermark extraction model, the watermark extraction model is used to extract the watermark in the input image, and the watermark extraction model includes a set of second convolutional layers, adaptive pooling layers and fully connected layers;

[0073] After the image to be processed and the random watermark information are input into the watermark embedding model to obtain the embedded watermark image output by the watermark embedding model, the above method also includes: inputting the embedded watermark image into the watermark extraction model so that the watermark extraction model performs the following watermark extraction operations: extracting features of the embedded watermark image through a set of second convolutional layers to obtain image watermark features; performing adaptive conversion processing on the image watermark features through an adaptive pooling layer to obtain a watermark feature vector; extracting information from the watermark feature vector through a fully connected layer to obtain predicted watermark information.

[0074] It should be noted that the watermark embedding model can be directly or indirectly connected to the watermark extraction model, and the watermark embedding model can be used to insert specific watermark information into the image. The watermark extraction model is a set of algorithm models based on neural networks, which can be used to extract watermarks from input images.

[0075] In the watermark extraction model, the second convolutional layer functions similarly to the first convolutional layer in the watermark embedding model, but it is used to extract features from the watermarked image that are associated with the watermark information. Through multi-level convolution operations, the model can capture local and global features in the image, thereby separating the watermark information from the image background. The adaptive pooling layer is a special pooling technique that can adaptively adjust the size of the feature map to a preset size, regardless of the original size of the input feature map. In the watermark extraction process, the adaptive pooling layer is used to compress the extracted image watermark features into a fixed-length feature vector for subsequent fully connected layer processing. The fully connected layer in the watermark extraction model is used to convert the watermark feature vector into predicted watermark information. The fully connected layer performs a linear transformation through the weight matrix and the bias vector, and finally outputs the prediction result through a nonlinear activation function (such as ReLU or Sigmoid). The design of the fully connected layer enables the model to recover the original watermark data from the feature vector.

[0076] Specifically, Figure 5 FIG. 1 is a schematic diagram of an optional watermark extraction model of an embodiment of the present application. Figure 5 As shown, the watermark extraction model may include a set of second convolutional layers, normalization layers, activation functions, adaptive pooling layers, and fully connected layers.

[0077] The input image (i.e., the watermarked image) is first extracted through a set of second convolutional layers to obtain a feature map, which is then normalized and activated to optimize the features and introduce nonlinearity. The adaptive pooling layer converts the input feature map into a fixed-size watermark feature vector for subsequent processing. Finally, the fully connected layer decodes the feature vector into the predicted watermark information, i.e., the original watermark bitstream.

[0078] Through this embodiment, the design of the adaptive pooling layer ensures that the watermark extraction model can process images of any size, so that the extraction of watermarks is no longer limited to the original size of the image, thereby improving the versatility and flexibility of the model.

[0079] In an exemplary embodiment, Figure 6 : is a schematic diagram of an optional target model of an embodiment of the present application. Wherein, the watermark embedding model and the watermark extraction model both belong to the target model, and the target model also includes a differentiable noise module, wherein the watermark embedding model and the watermark extraction model are connected via the differentiable noise module; the above method also includes:

[0080] In the process of using the training image to train the target model, the watermark training image is interfered with by noise through the noise module to obtain the watermark training image after noise interference, wherein the watermark training image is the image obtained after the training image is input into the watermark embedding model for watermark embedding, and the watermark training image after noise interference is input into the watermark extraction model.

[0081] It should be noted that the target model is a general term for the watermark embedding model and the watermark extraction model, and the target model also includes a differentiable noise module. The target model is used to constitute the complete process of watermark embedding and extraction. It can simulate various image interferences that may be encountered in practice through the noise module during the training process to improve the robustness and extraction accuracy of the model. Among them, the differentiable noise module is used to simulate various types of interference that images may suffer in actual environments, such as JPEG compression, noise addition, contrast changes, etc. Differentiable noise modules can provide gradient information during the back propagation process, thereby helping the watermark embedding model and the watermark extraction model to optimize their parameters to better deal with these interferences.

[0082] The watermark training image is an image obtained by embedding randomly generated watermark information into the training image by the watermark embedding model during the training process. The watermark training image can contain embedded watermark information, which is used to train the watermark extraction model to recognize and extract watermarks. After the watermark training image is interfered by the noise module, the watermark training image with noise interference is obtained. This process simulates a variety of interference situations that the watermark image may encounter in actual applications, so that the watermark extraction model can be trained to accurately restore the watermark information under various interferences.

[0083] Optionally, during the training process of the target model, a loss function can be used to determine the convergence of the target model. Specifically, during the training process of the target model, the predicted training watermark information output by the target model is obtained; the cross entropy loss of the predicted training watermark information and the original training watermark information corresponding to the predicted training watermark information, as well as the distortion between the training image and the watermark training image are determined; based on the cross entropy loss and the distortion, the loss function value of the target model is determined, wherein the loss function value is used to indicate that the model parameters of the target model are updated. Specifically, the formula of the loss function can be shown as formula (2):

[0084]

[0085] Among them, L is the loss function value, β is the constant coefficient, Represents the predicted training watermark information The cross entropy loss between the original training watermark information y, Represents the training image x (i.e. input image) and the watermark training image (ie, the output image) is the distortion function between them.

[0086] Optionally, the distortion function may be selected from mean squared error (MSE), LPIPS perceptual similarity (Learned Perceptual Image Patch Similarity), Structure Similarity Index Measure (SSIM), and the like.

[0087] Through this embodiment, by introducing a differentiable noise module during the training process to simulate the interference in the real environment, the target model can learn to extract watermark information under various noise conditions, significantly enhancing the robustness of watermark extraction. In addition, the differentiable characteristics of the noise module allow back propagation through the noise layer, helping the model to adjust parameters to minimize the watermark extraction error, thereby ensuring that the watermark information can be restored with high accuracy even in the disturbed image.

[0088] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0089] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM (Read-Only Memory) / RAM (Random Access Memory), a disk, or an optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0090] According to another aspect of the embodiments of the present application, an image watermark processing device is also provided, which can be used to implement the image watermark processing method provided in the above embodiments, and will not be repeated hereafter. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0091] Figure 7 is a structural block diagram of an optional image watermark processing device according to an embodiment of the present application, such as Figure 7 As shown in , the image watermark processing device includes:

[0092] The acquisition unit 702 is used to acquire random watermark information and the image to be processed indicated by the watermark processing instruction in response to the acquired watermark processing instruction, wherein the image to be processed is the image to be watermarked;

[0093] The output unit 704 is used to input the image to be processed and the random watermark information into the watermark embedding model to obtain the embedded watermark image output by the watermark embedding model, wherein the watermark embedding model is used to embed the input watermark information into the input image, and the watermark embedding model includes a group of first convolutional layers and a group of parameter adjustment module layers, each parameter adjustment module layer in the group of parameter adjustment module layers is connected to a first convolutional layer in the group of first convolutional layers; each parameter adjustment module layer is used to generate modulation parameters of the first convolutional layer connected to each parameter adjustment module layer according to the random watermark information, and the modulation parameters of each first convolutional layer in the group of first convolutional layers are used to modulate the feature map output by each first convolutional layer.

[0094] It should be noted that the acquisition unit 702 in this embodiment can be used to execute the above step S202, and the output unit 704 in this embodiment can be used to execute the above step S204.

[0095] Through the embodiment provided by the present application, the image to be processed and the random watermark information are input into the watermark embedding model to obtain the embedded watermark image output by the watermark embedding model, wherein the watermark embedding model is used to embed the input watermark information into the input image, each parameter adjustment module layer in the watermark embedding model is used to generate the modulation parameters of the first convolution layer connected to each parameter adjustment module layer according to the random watermark information, and the modulation parameters of each first convolution layer in a group of first convolution layers are used to modulate the feature map output by each first convolution layer, so as to modulate the output features of the convolution layer through the parameter adjustment module layer in the neural network, and the generation of the modulation parameters is independent of the image size, so that the watermark information can be embedded into an image of any size, solving the problem that the related art cannot directly perform watermark processing on an image of any size, thereby improving the practicality and versatility of the watermark technology. In addition, the output features of the first convolution layer are modulated using the parameter adjustment module layer, so that the watermark information is integrated with the deep features of the image to be processed, which not only maintains the visual quality of the image, but also increases the capacity of the watermark information to a certain extent, thereby improving the information carrying capacity.

[0096] In an exemplary embodiment, each first convolutional layer in a group of first convolutional layers is connected in sequence; the output unit 704 is further used to: input the image to be processed into the watermark embedding model to perform the following processing operations in the watermark embedding model: input the random watermark information into each parameter adjustment module layer respectively to obtain the modulation parameters of the first convolutional layer connected to each parameter adjustment module layer; use each first convolutional layer as the current first convolutional layer in sequence to perform the following watermark embedding operations, wherein the input image of the current first convolutional layer is the current input image: the current input image is processed by the current first convolutional layer Perform feature processing operations to obtain a feature map output by the current first convolutional layer; use the modulation parameters of the current first convolutional layer to modulate the feature map output by the current first convolutional layer to obtain the modulated feature map output by the current first convolutional layer; wherein, in a group of first convolutional layers, the input image of the first first convolutional layer is the image to be processed, the input images of the other first convolutional layers except the first first convolutional layer are the feature maps output by the previous convolutional layer of the other first convolutional layers after modulation, and the embedded watermark image is the feature map output by the last first convolutional layer in the group of first convolutional layers after modulation.

[0097] In an exemplary embodiment, the set of first convolutional layers includes a set of feature extraction convolutional layers and a set of transposed convolutional layers, and the last feature extraction convolutional layer in the set of feature extraction convolutional layers is connected to the first transposed convolutional layer in the set of transposed convolutional layers;

[0098] The output unit 704 is also used for: when the current first convolutional layer is a feature extraction convolutional layer, performing a feature extraction operation on the current input image through the current first convolutional layer to obtain a feature extraction map output by the current first convolutional layer; when the current first convolutional layer is a transposed convolutional layer, performing a feature reconstruction operation on the current input image through the current first convolutional layer to obtain a feature reconstruction map output by the current first convolutional layer.

[0099] In an exemplary embodiment, each parameter adjustment module layer includes a scaling parameter module and an offset parameter module, the scaling parameter module in each parameter adjustment module layer includes a group of first data sensing units, and the offset parameter module in each parameter adjustment module layer includes a group of second data sensing units;

[0100] The output unit 704 is also used to: in each parameter adjustment module layer, map the random watermark information through each first data perception unit in the scaling parameter module of each parameter adjustment module layer to obtain the scaling parameter vector of the first convolution layer connected to each parameter adjustment module layer, and map the random watermark information through each second data perception unit in the offset parameter module of each parameter adjustment module layer to obtain the offset parameter vector of the first convolution layer connected to each parameter adjustment module layer.

[0101] In an exemplary embodiment, the feature map output by each first convolution layer includes a feature map of each channel in a set of channels, and the scaling parameter vector of the first convolution layer connected to each parameter adjustment module layer contains scaling parameters equal to the number of channels of the feature map output by the first convolution layer connected to each parameter adjustment module layer; the offset parameter vector of the first convolution layer connected to each parameter adjustment module layer includes offset parameters equal to the number of channels of the feature map output by the first convolution layer connected to each parameter adjustment module layer; the output unit 704 is further used to: modulate the feature map of each channel output by the current first convolution layer according to the scaling parameter vector of the current first convolution layer and the offset parameter vector of the current first convolution layer, to obtain a modulated feature map of each channel output by the current first convolution layer, wherein the modulated feature map output by the current first convolution layer includes the modulated feature map of each channel output by the current first convolution layer.

[0102] In an exemplary embodiment, the watermark embedding model is connected to a watermark extraction model, the watermark extraction model is used to extract the watermark in the input image, and the watermark extraction model includes a set of second convolutional layers, adaptive pooling layers and fully connected layers; the image watermark processing device also includes:

[0103] The extraction unit is used to input the image to be processed and the random watermark information into the watermark embedding model to obtain the embedded watermark image output by the watermark embedding model, and then input the embedded watermark image into the watermark extraction model, so that the watermark extraction model performs the following watermark extraction operations: extracting features of the embedded watermark image through a set of second convolutional layers to obtain image watermark features; performing adaptive conversion processing on the image watermark features through an adaptive pooling layer to obtain a watermark feature vector; extracting information from the watermark feature vector through a fully connected layer to obtain predicted watermark information.

[0104] In an exemplary embodiment, the watermark embedding model and the watermark extraction model both belong to the target model, and the target model further includes a differentiable noise module, wherein the watermark embedding model and the watermark extraction model are connected via the differentiable noise module; the image watermark processing device further includes:

[0105] The interference unit is used to perform noise interference on the watermark training image through the noise module during the process of using the training image to train the target model, so as to obtain the watermark training image after noise interference, wherein the watermark training image is the image obtained after the training image is input into the watermark embedding model for watermark embedding, and the watermark training image after noise interference is input into the watermark extraction model.

[0106] It should be noted that the above modules can be implemented by software or hardware. For the latter, it can be implemented in the following ways, but not limited to: the above modules are all located in the same processor; or the above modules are located in different processors in any combination.

[0107] According to another aspect of the embodiments of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium includes a stored program, wherein the steps of any of the above method embodiments are executed when the program is run.

[0108] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, a ROM, a RAM, a mobile hard disk, a magnetic disk, or an optical disk.

[0109] According to another aspect of the embodiments of the present application, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor is configured to execute the steps in any of the above method embodiments through the computer program. In an exemplary embodiment, the electronic device may further comprise a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0110] For specific examples in this embodiment, reference may be made to the examples described in the above embodiments and exemplary implementation modes, and this embodiment will not be described in detail herein.

[0111] According to another aspect of the embodiment of the present application, a computer program product is also provided, and the computer program product includes a computer program / instruction, and the computer program / instruction contains a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 809, and / or installed from the removable medium 811. When the computer program is executed by the central processing unit 801, the various functions provided by the embodiment of the present application are executed. The above-mentioned serial numbers of the embodiments of the present application are only for description and do not represent the advantages and disadvantages of the embodiments.

[0112] Figure 8 The computer system structure block diagram of the electronic device used to implement the embodiment of the present application is schematically shown. Figure 8 As shown, the computer system 800 includes a CPU (Central Processing Unit) 801, which can perform various appropriate actions and processes according to the program stored in the ROM 802 or the program loaded from the storage part 808 to the RAM 803. In the random access memory 803, various programs and data required for system operation are also stored. The central processing unit 801, the read-only memory 802 and the random access memory 803 are connected to each other through a bus 804. An I / O (Input / Output) interface 805 is also connected to the bus 804.

[0113] The following components are connected to the I / O interface 805: an input section 806 including a keyboard, a mouse, etc.; an output section 807 including a CRT (Cathode Ray Tube), an LCD (Liquid Crystal Display), etc., and a speaker, etc.; a storage section 808 including a hard disk, etc.; and a communication section 809 including a network interface card such as a LAN card, a modem, etc. The communication section 809 performs communication processing via a network such as the Internet. A drive 810 is also connected to the input / output interface 805 as needed. A removable medium 811, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 810 as needed so that a computer program read therefrom is installed into the storage section 808 as needed.

[0114] In particular, according to an embodiment of the present application, the process described in each method flow chart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer readable medium, and the computer program contains a program code for executing the method shown in the flow chart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 809, and / or installed from the removable medium 811. When the computer program is executed by the central processor 801, various functions defined in the system of the present application are executed.

[0115] It should be noted that Figure 8 The computer system 800 of the electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0116] Obviously, those skilled in the art should understand that the above modules or steps of the present application can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order from that herein, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present application is not limited to any specific combination of hardware and software.

[0117] The above are only preferred embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the principles of the present application shall be included in the protection scope of the present application.

Claims

1. An image watermark processing method, characterized in that: include: In response to the acquired watermark processing instruction, acquiring random watermark information and an image to be processed indicated by the watermark processing instruction, wherein the image to be processed is an image to be watermarked; The image to be processed and the random watermark information are input into a watermark embedding model to obtain an embedded watermark image output by the watermark embedding model, wherein the watermark embedding model is used to embed the input watermark information into the input image, and the watermark embedding model includes a group of first convolutional layers and a group of parameter adjustment module layers, each parameter adjustment module layer in the group of parameter adjustment module layers is connected to a first convolutional layer in the group of first convolutional layers; each parameter adjustment module layer is used to generate modulation parameters of the first convolutional layer connected to each parameter adjustment module layer according to the random watermark information, and the modulation parameters of each first convolutional layer in the group of first convolutional layers are used to modulate the feature map output by each first convolutional layer.

2. The method according to claim 1, characterized in that Each first convolutional layer in the group of first convolutional layers is connected in sequence; The step of inputting the image to be processed and the random watermark information into a watermark embedding model to obtain an embedded watermark image output by the watermark embedding model comprises: The image to be processed is input into the watermark embedding model to perform the following processing operations in the watermark embedding model: Inputting the random watermark information into each parameter adjustment module layer respectively to obtain the modulation parameters of the first convolution layer connected to each parameter adjustment module layer; Each of the first convolutional layers is sequentially used as the current first convolutional layer to perform the following watermark embedding operation, wherein the input image of the current first convolutional layer is the current input image: Performing a feature processing operation on the current input image through the current first convolutional layer to obtain a feature map output by the current first convolutional layer; Modulating the feature map output by the current first convolutional layer using the modulation parameters of the current first convolutional layer to obtain a modulated feature map output by the current first convolutional layer; Among them, in the group of first convolutional layers, the input image of the first first convolutional layer is the image to be processed, the input images of other first convolutional layers except the first first convolutional layer are the feature maps output by the previous convolutional layer of the other first convolutional layers after modulation, and the embedded watermark image is the feature map output by the last first convolutional layer in the group of first convolutional layers after modulation.

3. The method according to claim 2, characterized in that The set of first convolutional layers includes a set of feature extraction convolutional layers and a set of transposed convolutional layers, and the last feature extraction convolutional layer in the set of feature extraction convolutional layers is connected to the first transposed convolutional layer in the set of transposed convolutional layers; The step of performing a feature processing operation on the current input image through the current first convolutional layer to obtain a feature map output by the current first convolutional layer includes: In a case where the current first convolutional layer is a feature extraction convolutional layer, performing a feature extraction operation on the current input image through the current first convolutional layer to obtain a feature extraction graph output by the current first convolutional layer; In the case where the current first convolutional layer is a transposed convolutional layer, a feature reconstruction operation is performed on the current input image through the current first convolutional layer to obtain a feature reconstruction image output by the current first convolutional layer.

4. The method according to claim 2, characterized in that: Each parameter adjustment module layer includes a scaling parameter module and an offset parameter module, the scaling parameter module in each parameter adjustment module layer includes a group of first data sensing units, and the offset parameter module in each parameter adjustment module layer includes a group of second data sensing units; The step of inputting the random watermark information to each parameter adjustment module layer to obtain a modulation parameter of the first convolution layer of each parameter adjustment module layer comprises: In each of the parameter adjustment module layers, the random watermark information is mapped by each first data perception unit in the scaling parameter module of each of the parameter adjustment module layers to obtain a scaling parameter vector of the first convolutional layer connected to each of the parameter adjustment module layers, and the random watermark information is mapped by each second data perception unit in the offset parameter module of each of the parameter adjustment module layers to obtain an offset parameter vector of the first convolutional layer connected to each of the parameter adjustment module layers.

5. The method according to claim 4, characterized in that The feature map output by each first convolution layer includes a feature map of each channel in a set of channels, and the scaling parameter vector of the first convolution layer connected to each parameter adjustment module layer includes scaling parameters that are the same as the number of channels of the feature map output by the first convolution layer connected to each parameter adjustment module layer; the offset parameter vector of the first convolution layer connected to each parameter adjustment module layer includes offset parameters that are the same as the number of channels of the feature map output by the first convolution layer connected to each parameter adjustment module layer; The step of modulating the feature map output by the current first convolutional layer using the modulation parameter of the current first convolutional layer to obtain the modulated feature map output by the current first convolutional layer includes: According to the scaling parameter vector of the current first convolutional layer and the offset parameter vector of the current first convolutional layer, the feature map of each channel output by the current first convolutional layer is modulated to obtain the modulated feature map of each channel output by the current first convolutional layer, wherein the modulated feature map output by the current first convolutional layer includes the modulated feature map of each channel output by the current first convolutional layer.

6. The method according to claim 1, characterized in that The watermark embedding model is connected to a watermark extraction model, and the watermark extraction model is used to extract the watermark in the input image, and the watermark extraction model includes a set of second convolutional layers, adaptive pooling layers and fully connected layers; After inputting the image to be processed and the random watermark information into the watermark embedding model to obtain the watermarked image output by the watermark embedding model, the method further includes: The watermark-embedded image is input into the watermark extraction model so that the watermark extraction model performs the following watermark extraction operations: Extracting features of the watermarked image through the set of second convolutional layers to obtain image watermark features; Performing adaptive conversion processing on the image watermark feature through the adaptive pooling layer to obtain a watermark feature vector; The watermark feature vector is extracted through the fully connected layer to obtain predicted watermark information.

7. The method according to claim 6, characterized in that The watermark embedding model and the watermark extraction model both belong to a target model, and the target model further includes a differentiable noise module, wherein the watermark embedding model and the watermark extraction model are connected via the differentiable noise module; The method further comprises: In the process of using the training image to train the target model, the watermark training image is interfered with by noise through the noise module to obtain the watermark training image after noise interference, wherein the watermark training image is an image obtained after the training image is input into the watermark embedding model for watermark embedding, and the watermark training image after noise interference is input into the watermark extraction model.

8. An image watermark processing device, characterized in that: include: an acquisition unit, configured to acquire random watermark information and an image to be processed indicated by the watermark processing instruction in response to the acquired watermark processing instruction, wherein the image to be processed is an image to be watermarked; An output unit is used to input the image to be processed and the random watermark information into a watermark embedding model to obtain an embedded watermark image output by the watermark embedding model, wherein the watermark embedding model is used to embed the input watermark information into the input image, and the watermark embedding model includes a group of first convolutional layers and a group of parameter adjustment module layers, each parameter adjustment module layer in the group of parameter adjustment module layers is connected to a first convolutional layer in the group of first convolutional layers; each parameter adjustment module layer is used to generate modulation parameters of the first convolutional layer connected to each parameter adjustment module layer according to the random watermark information, and the modulation parameters of each first convolutional layer in the group of first convolutional layers are used to modulate the feature map output by each first convolutional layer.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program implements the steps of any one of the methods of claims 1 to 7 when executed by a processor.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

Images