Model training method and device, equipment, storage medium and program product

Abstract

The invention discloses a model training method and device, equipment, a storage medium and a program product, relates to the technical field of computers, and discloses a model training method which comprises the following steps: adding noise in a watermark image to generate a noise image; a decoder model is trained based on the noise image, the decoder model comprises a spatial transformation network and a convolutional network, the spatial transformation network is used for performing spatial transformation on the noise image, and the convolutional network is used for extracting watermark features from the noise image after spatial transformation; and taking the trained decoder model as a watermark extraction model, wherein the watermark extraction model is used for extracting watermark information of the image. According to the method, after the watermark image is subjected to a certain degree of noise interference, the initially embedded watermark information can still be obtained, and the accuracy of watermark information extraction is improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a model training method, apparatus, device, storage medium and program product. Background Art

[0002] Digital watermarking is to embed the watermark information and the original data into the content carrier after a series of encryption and calculation processing to achieve deep binding. Digital watermarking technology does not interfere with the daily use of digital content by users, and can quickly locate the source when digital content is leaked. It can meet the needs of enterprises and become an important means to protect the security of digital content.

[0003] In actual image usage scenarios, images may be affected by perspective deformation, motion blur, color processing, etc., resulting in the inability to accurately extract digital watermarks from the image and the inability to quickly locate the source of the digital content. Summary of the invention

[0004] The main purpose of this application is to provide a model training method, device, equipment, storage medium and program product, aiming to solve the technical problem that digital watermarks cannot be accurately extracted from images.

[0005] To achieve the above objectives, the present application proposes a model training method, which includes:

[0006] Adding noise to the watermark image to generate a noise image;

[0007] Training a decoder model based on the noise image, the decoder model comprising a spatial transformation network and a convolutional network, the spatial transformation network being used to perform spatial transformation on the noise image, and the convolutional network being used to extract watermark features from the noise image after spatial transformation;

[0008] The trained decoder model is used as a watermark extraction model, and the watermark extraction model is used to extract watermark information of an image.

[0009] In one embodiment, the step of training a decoder model based on the noisy image comprises:

[0010] Obtain a noise image and the corresponding initial watermark information as a training set;

[0011] Training a decoder model based on the training set, and obtaining watermark information of the noise image output by the decoder model;

[0012] Determine a loss value between the output watermark information of the noise image and the initial watermark information;

[0013] The model parameters of the decoder model are updated according to the loss value until the loss value is less than a preset threshold, thereby obtaining a trained decoder model.

[0014] In one embodiment, before the step of adding noise to the watermark image to generate a noise image, the method further includes:

[0015] Inputting the initial image and the watermark information to be added into a watermark embedding model, wherein the watermark embedding model is used to generate a residual image according to the initial image and the watermark information to be added, and generate the watermark image according to the residual image and the initial image;

[0016] The watermark image output by the watermark embedding model is obtained.

[0017] In one embodiment, before the step of inputting the initial image and the watermark information to be added into the watermark embedding model, the method further includes:

[0018] Obtaining an initial training image and watermark training information to be added as a training image set;

[0019] Training a preset encoder model based on the training image set, and obtaining a watermark training image output by the encoder model;

[0020] Determine a scoring matrix of the initial training image, and determine a loss value according to the scoring matrix and a residual image of the watermark training image, wherein the scoring matrix is ​​used to determine a pixel value of the watermark information in the watermark training image;

[0021] The parameters of the encoder model are updated according to the loss value until the loss value is less than a preset threshold, thereby obtaining a trained watermark embedding model.

[0022] In one embodiment, the step of adding noise to the watermark image to generate a noise image includes:

[0023] When the noise is compressed noise, the watermark image is input into a preset set compression model to obtain a noise image output by the set compression model;

[0024] The set compression model is obtained by training based on a data set of watermark training images and compressed training images, and the set compression model includes an encoder and a decoder.

[0025] In one embodiment, the step of adding noise to the watermark image to generate a noise image includes:

[0026] When the noise is moiré noise, the moiré noise is added to the watermark image to obtain the noise image.

[0027] In addition, to achieve the above objectives, the present application also proposes a model training device, which includes:

[0028] Add a module for adding noise to the watermark image to generate a noise image;

[0029] A training module, used for training a decoder model based on the noise image, wherein the decoder model comprises a spatial transformation network and a convolutional network, wherein the spatial transformation network is used for performing spatial transformation on the noise image, and the convolutional network is used for extracting watermark features from the noise image after spatial transformation;

[0030] The generation module is used to use the trained decoder model as a watermark extraction model, and the watermark extraction model is used to extract watermark information of the image.

[0031] In addition, to achieve the above-mentioned purpose, the present application also proposes a model training device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, and the computer program is configured to implement the steps of the model training method described above.

[0032] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by the processor, the steps of the model training method described above are implemented.

[0033] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, it implements the steps of the model training method described above.

[0034] One or more technical solutions proposed in this application have at least the following technical effects:

[0035] Add noise to the watermark image to generate a noise image; train the decoder model based on the noise image; use the trained decoder model as the watermark extraction model, and the watermark extraction model is used to extract the watermark information of the image. The decoder model is trained with the noise image to obtain the watermark extraction model. After the watermark image is interfered with by a certain degree of noise, the originally embedded watermark message can still be obtained, which improves the accuracy of watermark information extraction. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0037] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0038] Figure 1 A flowchart of the first embodiment of the model training method of the present application is provided;

[0039] Figure 2 This is a schematic diagram of the structure of the set compression model in the embodiment of the model training method of this application;

[0040] Figure 3 A flowchart of the second embodiment of the model training method of the present application is provided;

[0041] Figure 4 This is a schematic diagram of the module structure of the model training device in the embodiment of the present application;

[0042] Figure 5 This is a schematic diagram of the device structure of the hardware operating environment involved in the model training method in the embodiment of the present application.

[0043] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0044] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0045] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0046] The main solution of the embodiment of the present application is: adding noise to the watermark image to generate a noise image; training a decoder model based on the noise image; using the trained decoder model as a watermark extraction model, and the watermark extraction model is used to extract the watermark information of the image.

[0047] In this embodiment, for ease of description, the following description is made with the model training device as the execution entity.

[0048] In actual image usage scenarios, images may be affected by perspective deformation, motion blur, color processing, etc., which makes it impossible to accurately extract digital watermarks from images and quickly locate the source of digital content.

[0049] The present application provides a solution, which trains a decoder model through a noise image to obtain a watermark extraction model. After the watermark image is interfered with by a certain degree of noise, the originally embedded watermark message can still be obtained, thereby improving the accuracy of watermark information extraction.

[0050] It should be noted that the execution subject of this embodiment can be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device capable of realizing the above functions, a model training device, etc. The following takes the model training device as an example to illustrate this embodiment and the following embodiments.

[0051] Based on this, the present application embodiment provides a model training method, referring to Figure 1 , Figure 1 This is a flow chart of the first embodiment of the model training method of the present application.

[0052] In this embodiment, the model training method includes steps S10 to S30:

[0053] Step S10, adding noise to the watermark image to generate a noise image.

[0054] It should be noted that, in order to resist interference that may occur in various practical applications, the watermark image is input into a transformation network that simulates reality, and the noise that interferes with the image is added to the watermark image through the transformation network to obtain a noise image. Optionally, the interfering noise includes perspective deformation, motion blur, defocus blur, color processing, random noise, etc., wherein the color processing includes contrast, saturation, hue, etc.

[0055] In the design of noise, moiré noise simulated by image processing technology is added to the watermark image. The scenario in which moiré noise is generated is that when an image is displayed in an imaging device with a display screen and the image is photographed by another camera, the spatial frequency of the pixels of the photosensitive element is close to the spatial frequency of the stripes in the image, thereby generating a wavy interference pattern, namely moiré noise. Therefore, adding simulated moiré noise to the transformation network can improve the ability of the decoder model to extract watermarks from images interfered by different noises at the level of diversity, and ultimately enhance the robustness of the digital watermark. Optionally, when the noise is moiré noise, the moiré noise is added to the watermark image to obtain a noisy image. In the training of the interference layer, moiré noise simulated by image processing technology is additionally added. When the application scenario of the digital watermark involves a shooting scene, the wavy moiré noise generated on the image due to the superposition of two equal-amplitude sine waves with similar frequencies, when the model can extract watermarks from images containing the noise through training, it can have the robustness to resist the real screen noise.

[0056] When the noise is compressed noise, for example, the compressed noise is JPEG (Joint Photographic Experts Group) compressed noise, a collection compression model of images is trained based on the technology of image domain migration, and the simulated compressed images generated by the collection compression model can provide noise interference with better approximation effect in the original anti-interference training, and obtain a more robust watermark image in subsequent training. Optionally, when the noise is compressed noise, the watermark image is input into a preset collection compression model to obtain a noisy image output by the collection compression model; wherein the collection compression model is trained based on a data set of initial images and compressed training images, and the collection compression model includes an encoder and a decoder. Adding compressed noise to the watermark image through the collection compression model facilitates improving the comprehensiveness of subsequent watermark extraction.

[0057] The conversion process of an image from the original image to the lossy compressed image is regarded as a migration in the image domain, and a neural network with an encoder-decoder structure is used to learn the conversion method from the original image to the compressed image. The data set of the compressed image is collected after the original image is compressed by many different compression algorithms. In this way, the trained model can be used as an approximate form that integrates various compression algorithms. When the image passes through the end-to-end model, the compressed image can be output. The set compression model can be added to the interference layer of the original watermark model and trained as a compression noise simulation interference item, which ultimately improves the compression resistance of the digital watermark model.

[0058] Watermark training images and compressed training images are obtained, and a data set is constructed based on the watermark training images and the compressed training images. Optionally, the watermark training images and the compressed training images are obtained by uploading and downloading on various social platforms.

[0059] The encoder-decoder network model is used as the basis of the set compression model to learn the conversion of watermark training images to various lossy compressed images. Figure 2 As shown in the figure, in the encoder part, the convolution layer is used to extract the relevant features of the image, and the maximum pooling layer is used to reduce the dimension of the image at the same time, improving the generalization ability of the model and reducing overfitting. In the decoder part, the image dimension is gradually restored by using convolution and deconvolution. At the same time, in order to prevent the deep neural network from missing certain features when passing information through continuous layers, the jump connection operation is used to reintroduce these features to obtain a smoother learning gradient.

[0060] After obtaining the output noise image of the same dimension as the input, the error between the output noise image and the compressed training image in the dataset is calculated. Optionally, MSE (Mean Squared Error) is used for error calculation. Optionally, the LPIPS function (learned perceptual image patch similarity) is used to calculate the perceptual loss of the two images, so that the generated image and the real compressed image can achieve pixel-level similarity, but the difference between the two images cannot be distinguished from the human eye.

[0061] Among them, the LPIPS algorithm is an image quality measurement method based on deep learning, which aims to simulate the way of human visual perception. By training the perceptron of the deep neural network structure, it can simulate and learn the characteristics of human visual perception. During training, two images are input and the similarity of each area of ​​the two images is calculated to obtain a numerical image quality evaluation result, that is, the difference measurement value of the two images. By inputting the watermark training image into the set compression model, a simulated lossy compression image, that is, a noise image, can be obtained, and then the simulated compression noise can be obtained. Compared with other JPEG compression noise, the above compression noise can be used as an approximate form of different compression algorithms and is more universal.

[0062] Step S20: training a decoder model based on the noise image, wherein the decoder model comprises a spatial transformation network and a convolutional network, wherein the spatial transformation network is used to perform spatial transformation on the noise image, and the convolutional network is used to extract watermark features from the noise image after spatial transformation.

[0063] It should be noted that the noisy image containing the watermark is input into the decoder model. The network structure of the decoder model includes a spatial transformation network and a set of convolutional networks. The decoder model can output the watermark information of the noisy image.

[0064] By training the decoder model based on the loss function, a highly robust digital watermark extraction model based on deep learning can be obtained. Optionally, the loss value between the watermark information output by the decoder model and the initial watermark information of the watermark image is determined based on the loss function, and the parameters of the decoder model are updated based on the loss value. When the loss value is less than a preset threshold, a trained decoder model is obtained. Optionally, a sigmoid cross entropy loss function is used to measure the loss value between the watermark information output by the decoder model and the initial watermark information of the watermark image.

[0065] Step S30: Using the trained decoder model as a watermark extraction model, the watermark extraction model is used to extract watermark information of the image.

[0066] It should be noted that the watermark extraction model extracts watermark information from various watermarked images in practical applications. The watermark extraction model allows the watermarked image to be input into the watermark extraction model even after being interfered with by a certain degree of noise, and the originally embedded watermark message can still be obtained, thus improving the accuracy of watermark information extraction.

[0067] In the technical solution of this embodiment, noise is added to the watermark image to generate a noise image; a decoder model is trained based on the noise image; the trained decoder model is used as a watermark extraction model, and the watermark extraction model is used to extract the watermark information of the image. The decoder model is trained with a noise image to obtain a watermark extraction model. After the watermark image is interfered with by a certain degree of noise, the originally embedded watermark message can still be obtained, thereby improving the accuracy of watermark information extraction.

[0068] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above embodiment can refer to the above introduction, and will not be repeated later. Figure 3 Before step S10, the model training method further includes:

[0069] Step S40, inputting the initial image and the watermark information to be added into a watermark embedding model, wherein the watermark embedding model is used to generate a residual image according to the initial image and the watermark information to be added, and to generate the watermark image according to the residual image and the initial image;

[0070] Step S50, obtaining the watermark image output by the watermark embedding model.

[0071] It should be noted that the watermark embedding model is an encoder model, which may be U-net (Convolutional Networks for Biomedical Image Segmentation).

[0072] Optionally, before step S40, it also includes: obtaining an initial training image and watermark training information to be added as a training image set; training a preset encoder model based on the training image set, and obtaining a watermark training image output by the encoder model; determining a scoring matrix of the initial training image, and determining a loss value according to the scoring matrix and a residual image of the watermark training image, wherein the scoring matrix is ​​used to determine the pixel value of the watermark information in the watermark training image; updating the parameters of the encoder model according to the loss value until the loss value is less than a preset threshold, thereby obtaining a trained watermark embedding model.

[0073] According to the training image set, the U-net, a fully convolutional neural network, is used as the encoder network model for watermark embedding. The encoder network model will perform feature extraction and upsampling operations on the input watermark training information and the initial training image through a series of convolution and upsampling operations. The image features include the structural information of the image and the features of the watermark information. The upsampling operation can enhance the extracted features so that the size of the feature map matches the size of the input image, which helps to fuse the input random binary information and image features together so as to embed the watermark into the initial training image. After the initial training image passes through the encoder network model, the residual image containing the watermark information is finally output. The generation process enables the model to predict the changes in the pixel values ​​of the watermark training information contained in the initial training image through learning. Among them, the initial training image is the image to be watermarked. The watermark training information is added to the initial training image to obtain the watermark training image.

[0074] For the difference between the initial training image and the watermarked training image after embedding the watermark, the error between the initial training image and the watermarked training image is designed in the loss function as one of the image losses. By reducing the difference through subsequent training, the PSNR (Peak Signal-to-Noise Ratio) index of the watermarked training image can be better improved, thereby improving the quality of the output watermarked training image.

[0075] However, the supervision of the above loss items still cannot completely conceal the watermark information. The texture of the residual image will be irregularly distributed on the watermark training image, forming gray patches that are noticeable to the human eye.

[0076] The JND (Just Noticeable Distortion) model can score each pixel according to the human eye's ability to perceive different features in the image, and obtain a scoring matrix with the same size as the original image. The scoring matrix will guide the location of the watermark in the image. A high score indicates that the location allows more watermark information to be embedded, and a low score indicates that it is not recommended to embed too much watermark information in the location. The scoring matrix can be used to supervise the location and intensity of watermark embedding in subsequent model training, control the changes in the image within the threshold that the human eye can perceive, and finally embed the watermark into the image without the human eye being able to perceive it, which can effectively solve the problem of poor visual effects in StegaStamp.

[0077] The strength of watermarks generated at different locations on the image is supervised by combining the LPIPS function and the JND model. The specific loss function can be expressed by the following formula:

[0078] L JND =λ·L Lpips (αMW -I JND );

[0079] Among them, L is the loss term, λ is the weight of the loss term, and M W is the residual image generated in step 1, α is the intensity coefficient of the residual image, I JND is the JND map of the initial training image.

[0080] The initial training image in the training image set is input into the pixel-domain based JND model, and output to a scoring matrix of the same size as the original image. The scoring matrix is ​​calculated by the pattern complexity and brightness contrast of each pixel value in the image. The smaller these two values ​​are, the more likely the point can be ignored by the human eye. The scoring matrix is ​​mapped to the score of each pixel point, which is used to represent the threshold of the difference that the human eye can perceive. As long as the change to the pixel point does not exceed the threshold, it will not be easily perceived by the human eye. Exemplarily, the calculation formula for the pattern complexity of the JND model is:

[0081]

[0082] The formula in the L0 norm is the histogram of θ(x), θ(x) is the gradient direction at each pixel, x is the pixel, R(x) represents the image mode, and θ(x) is quantized into a finite number N by setting different intervals. is a pulse function that traverses every pixel of the entire image. When it is equal to k, the function value is 1, otherwise it is 0. For example, the specific calculation formula of θ(x) is:

[0083]

[0084] Among them, G v (x) is the gradient transformation in the vertical direction, G h (x) is the gradient transformation in the horizontal direction, and these two items are calculated by convolution of the image with the vertical operator and the horizontal operator respectively. The brightness contrast of the JND model is calculated by the gradient transformation in the vertical and horizontal directions:

[0085]

[0086] The scoring matrix and the residual image generated by the model are input into the LPIPS function to calculate the loss at the level of human perception. The difference in the continuous feedback during the neural network training can guide the encoding model to generate a more concealed residual image, and finally optimize the visual effect of the output watermark image. While ensuring the concealment of the watermark, the robustness of the digital watermark is improved. By minimizing the loss function, the model learns how to embed watermark information in a way that is not easily perceived by the human eye and maintains image quality.

[0087] In the technical solution of this embodiment, a watermark is embedded into an image through a watermark embedding model without being perceived by human eyes, while maintaining image quality.

[0088] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the model training method of the present application. More simple transformations based on this technical concept are all within the scope of protection of the present application.

[0089] This application also provides a model training device, please refer to Figure 4 , the model training device comprises:

[0090] An adding module 10 is used to add noise to the watermark image to generate a noise image;

[0091] A training module 20, used for training a decoder model based on the noise image, the decoder model comprising a spatial transformation network and a convolutional network, the spatial transformation network is used for performing spatial transformation on the noise image, and the convolutional network is used for extracting watermark features from the noise image after spatial transformation;

[0092] The generation module 30 is used to use the trained decoder model as a watermark extraction model, and the watermark extraction model is used to extract watermark information of the image.

[0093] Optionally, the step of training a decoder model based on the noisy image comprises:

[0094] Obtain a noise image and the corresponding initial watermark information as a training set;

[0095] Training a decoder model based on the training set, and obtaining watermark information of the noise image output by the decoder model;

[0096] Determine a loss value between the output watermark information of the noise image and the initial watermark information;

[0097] The model parameters of the decoder model are updated according to the loss value until the loss value is less than a preset threshold, thereby obtaining a trained decoder model.

[0098] Optionally, before the step of adding noise to the watermark image to generate a noise image, the method further comprises:

[0099] Inputting the initial image and the watermark information to be added into a watermark embedding model, wherein the watermark embedding model is used to generate a residual image according to the initial image and the watermark information to be added, and generate the watermark image according to the residual image and the initial image;

[0100] The watermark image output by the watermark embedding model is obtained.

[0101] Optionally, before the step of inputting the initial image and the watermark information to be added into the watermark embedding model, the method further includes:

[0102] Obtaining an initial training image and watermark training information to be added as a training image set;

[0103] Training a preset encoder model based on the training image set, and obtaining a watermark training image output by the encoder model;

[0104] Determine a scoring matrix of the initial training image, and determine a loss value according to the scoring matrix and a residual image of the watermark training image, wherein the scoring matrix is ​​used to determine a pixel value of the watermark information in the watermark training image;

[0105] The parameters of the encoder model are updated according to the loss value until the loss value is less than a preset threshold, thereby obtaining a trained watermark embedding model.

[0106] Optionally, the step of adding noise to the watermark image to generate a noise image includes:

[0107] When the noise is compressed noise, the watermark image is input into a preset set compression model to obtain a noise image output by the set compression model;

[0108] The set compression model is obtained by training based on a data set of watermark training images and compressed training images, and the set compression model includes an encoder and a decoder.

[0109] Optionally, the step of adding noise to the watermark image to generate a noise image includes:

[0110] When the noise is moiré noise, the moiré noise is added to the watermark image to obtain the noise image.

[0111] The model training device provided by the present application adopts the model training method in the above embodiment, which can solve the technical problem that the digital watermark in the image cannot be accurately extracted. Compared with the prior art, the beneficial effects of the model training device provided by the present application are the same as the beneficial effects of the model training method provided by the above embodiment, and the other technical features in the model training device are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0112] The present application provides a model training device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the model training method in the above-mentioned embodiment one.

[0113] Reference below Figure 5 , which shows a schematic diagram of the structure of a model training device suitable for implementing an embodiment of the present application. The model training device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 5 The model training device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0114] like Figure 5 As shown, the model training device may include a processing device 1001 (e.g., a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the model training device are also stored. The processing device 1001, ROM1002, and RAM1004 are connected to each other via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the model training device to communicate with other devices wirelessly or by wire to exchange data. Although the figure shows a model training device with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or provided instead.

[0115] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0116] The model training device provided by the present application adopts the model training method in the above embodiment, which can solve the technical problem that the digital watermark in the image cannot be accurately extracted. Compared with the prior art, the beneficial effects of the model training device provided by the present application are the same as the beneficial effects of the model training method provided by the above embodiment, and the other technical features in the model training device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0117] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0118] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0119] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, and the computer-readable program instructions are used to execute the model training method in the above-mentioned embodiment.

[0120] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination of the above.

[0121] The above-mentioned computer-readable storage medium may be included in the model training device; or it may exist independently without being assembled into the model training device.

[0122] The above-mentioned computer-readable storage medium carries one or more programs. When the above-mentioned one or more programs are executed by the model training device, the model training device: trains the decoder model through the noise image to obtain the watermark extraction model. After the watermark image is subjected to a certain degree of noise interference, the originally embedded watermark message can still be obtained, thereby improving the accuracy of watermark information extraction.

[0123] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0124] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0125] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0126] The readable storage medium provided in this application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned model training method, and can solve the technical problem that the digital watermark cannot be accurately extracted from the image. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as the beneficial effects of the model training method provided in the above-mentioned embodiment, and will not be repeated here.

[0127] The present application also provides a computer program product, including a computer program, which implements the steps of the above-mentioned model training method when executed by a processor.

[0128] The computer program product provided by this application can solve the technical problem that the digital watermark cannot be accurately extracted from the image. Compared with the prior art, the beneficial effects of the computer program product provided by this application are the same as the beneficial effects of the model training method provided by the above embodiment, which will not be repeated here.

[0129] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A model training method, characterized in that: The method includes: Adding noise to the watermark image to generate a noise image; Training a decoder model based on the noise image, the decoder model comprising a spatial transformation network and a convolutional network, the spatial transformation network being used to perform spatial transformation on the noise image, and the convolutional network being used to extract watermark features from the noise image after spatial transformation; The trained decoder model is used as a watermark extraction model, and the watermark extraction model is used to extract watermark information of an image.

2. The method according to claim 1, characterized in that The step of training a decoder model based on the noisy image comprises: Obtain a noise image and the corresponding initial watermark information as a training set; Training a decoder model based on the training set, and obtaining watermark information of the noise image output by the decoder model; Determine a loss value between the output watermark information of the noise image and the initial watermark information; The model parameters of the decoder model are updated according to the loss value until the loss value is less than a preset threshold, thereby obtaining a trained decoder model.

3. The method according to claim 1, characterized in that Before the step of adding noise to the watermark image to generate a noise image, the method further includes: Inputting the initial image and the watermark information to be added into a watermark embedding model, wherein the watermark embedding model is used to generate a residual image according to the initial image and the watermark information to be added, and generate the watermark image according to the residual image and the initial image; The watermark image output by the watermark embedding model is obtained.

4. The method according to claim 3, characterized in that Before the step of inputting the initial image and the watermark information to be added into the watermark embedding model, the method further includes: Obtaining an initial training image and watermark training information to be added as a training image set; Training a preset encoder model based on the training image set, and obtaining a watermark training image output by the encoder model; Determine a scoring matrix of the initial training image, and determine a loss value according to the scoring matrix and a residual image of the watermark training image, wherein the scoring matrix is ​​used to determine a pixel value of the watermark information in the watermark training image; The parameters of the encoder model are updated according to the loss value until the loss value is less than a preset threshold, thereby obtaining a trained watermark embedding model.

5. The method according to claim 1, characterized in that The step of adding noise to the watermark image to generate a noise image comprises: When the noise is compressed noise, the watermark image is input into a preset set compression model to obtain a noise image output by the set compression model; The set compression model is obtained by training based on a data set of watermark training images and compressed training images, and the set compression model includes an encoder and a decoder.

6. The method according to claim 1, characterized in that The step of adding noise to the watermark image to generate a noise image comprises: When the noise is moiré noise, the moiré noise is added to the watermark image to obtain the noise image.

7. A model training device, characterized in that: The device comprises: Add a module for adding noise to the watermark image to generate a noise image; A training module, used for training a decoder model based on the noise image, the decoder model comprising a spatial transformation network and a convolutional network, the spatial transformation network is used for performing spatial transformation on the noise image, and the convolutional network is used for extracting watermark features from the noise image after spatial transformation; The generation module is used to use the trained decoder model as a watermark extraction model, and the watermark extraction model is used to extract watermark information of the image.

8. A model training device, characterized in that: The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the model training method according to any one of claims 1 to 6.

9. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the model training method according to any one of claims 1 to 6 are implemented.

10. A computer program product, characterized in that The computer program product comprises a computer program, which, when executed by a processor, implements the steps of the model training method according to any one of claims 1 to 6.

Images