Watermark detection model training method, watermark detection method, system, device and medium

Abstract

The present invention provides a watermark detection model training method, a watermark detection method, a system, a device and a medium, wherein the training method comprises: obtaining a sample data set, the sample data set comprising a plurality of training images with watermarks; marking the contour of the watermark area in each training image of the sample data set; training a watermark detection model according to the marked sample data set; wherein the watermark detection model is used to extract a feature map of an input image, and then obtain the probability of whether each feature point in the feature map is a watermark center point, and when the probability of a certain feature point being a watermark center point is greater than a predetermined value, obtain the offset between the position of the feature point mapped to the input image and the real watermark center point corresponding to the feature point, and the watermark size corresponding to the feature point. The present invention can train an accurate watermark detection model, thereby solving the technical problem that it is difficult to accurately identify watermarked images through manual review.

Description

Technical Field

[0001] The present invention relates to the field of image recognition technology, and in particular to a watermark detection model training method, a watermark detection method, a system, a device and a medium. Background Art

[0002] As an important medium for information dissemination, images contain rich and intuitive information. Therefore, in the fields of social networking, shopping, and tourism on the Internet, a large number of images are needed to convey information to users. Since images usually come from complex sources, large quantities of images obtained from different sources often contain some watermarked images, which may affect the appearance or even infringe copyright if displayed directly. Therefore, before using the images, the image provider needs to review the images to detect watermarks in the images.

[0003] For large-scale image data, there is a high degree of uncertainty as to whether an image contains a watermark, and the position and transparency of the watermark vary greatly. The types of watermarks are complex and are usually extremely inconspicuous relative to the main content features of the image. Therefore, if watermarks are manually reviewed on images, the labor cost is expensive and easy to miss. Therefore, it has become a trend to use algorithms to automatically review images to detect images with watermarks. At present, few people have studied the use of deep learning to detect a variety of watermarks in general scenarios. Accurate detection of watermarks in images is a challenging task. Summary of the invention

[0004] In view of the above-mentioned deficiencies in the prior art, the purpose of the present invention is to provide a watermark detection model training method, a watermark detection method, a system, a device and a medium to solve the technical problem that it is difficult to accurately identify watermarked images through manual review.

[0005] In order to achieve the above object, the present invention provides a watermark detection model training method, comprising:

[0006] Obtain a sample data set, wherein the sample data set includes a plurality of training images with watermarks;

[0007] Annotating the contour of the watermark area in each training image of the sample data set;

[0008] According to the labeled sample data set, the watermark detection model is trained to obtain the target watermark detection model;

[0009] Among them, the watermark detection model is used to extract the feature map of the input image, and then obtain the probability of whether each feature point in the feature map is the center point of the watermark, and when the probability of a certain feature point being the center point of the watermark is greater than a predetermined value, obtain the offset between the position of the feature point mapped to the input image and the real watermark center point corresponding to the feature point, as well as the watermark size corresponding to the feature point.

[0010] In a preferred embodiment of the present invention, the step of obtaining a sample data set includes:

[0011] Get several watermark images;

[0012] Processing the plurality of watermark images to obtain a plurality of watermark image templates with transparent backgrounds;

[0013] Get several background images;

[0014] The plurality of watermark image templates are synthesized with the plurality of background images to obtain the plurality of watermarked training images.

[0015] In a preferred embodiment of the present invention, the step of synthesizing the plurality of watermark image templates with the plurality of background images comprises:

[0016] The same watermark image template is synthesized with the background image in different watermark types, sizes, transparencies and / or positions.

[0017] In a preferred embodiment of the present invention, the step of training the watermark detection model according to the labeled sample data set includes:

[0018] Inputting the training image in the sample data set into the watermark detection model for processing to obtain the model processing result of the training image;

[0019] Obtaining a loss function based on a model processing result of the training image and a watermark area marked in the training image;

[0020] The parameters of the watermark detection model are adjusted based on the loss function, and the step of inputting the training images in the sample data set into the watermark detection model for processing is returned until the loss function meets a predetermined condition, thereby obtaining a target watermark detection model.

[0021] In a preferred embodiment of the present invention, the loss function is obtained by weighted summation of probability loss, position loss and size loss.

[0022] In a preferred embodiment of the present invention, the watermark detection model adopts a fully convolutional network, which includes a plurality of symmetric residual modules, each of which includes an even number of sequentially connected residual blocks, and the even number of residual blocks are symmetrically jump-connected.

[0023] In order to achieve the above object, the present invention also provides a watermark detection method, comprising:

[0024] Obtain a target image to be detected for watermark;

[0025] The target image is input into the target watermark detection model trained according to the aforementioned method for processing, and the probability of each feature point in the feature map of the target image being the watermark center point is obtained. When the probability of a certain feature point being the watermark center point is greater than a predetermined value, the offset between the position of the feature point mapped to the target image and the real watermark center point corresponding to the feature point, and the watermark size corresponding to the feature point are obtained. Then, according to the position of the feature point mapped to the target image and the offset, the real watermark center point corresponding to the feature point is obtained.

[0026] In order to achieve the above object, the present invention also provides a watermark detection model training system, comprising:

[0027] A sample acquisition module, used to acquire a sample data set, wherein the sample data set includes a plurality of training images with watermarks;

[0028] A labeling module, used for labeling the outline of the watermark area in each training image of the sample data set;

[0029] A training module is used to train the watermark detection model according to the labeled sample data set to obtain a target watermark detection model;

[0030] Among them, the watermark detection model is used to extract the feature map of the input image, and then obtain the probability that each feature point in the feature map is the center point of the watermark, and when the probability that a certain feature point is the center point of the watermark is greater than a predetermined value, obtain the offset between the position of the feature point mapped to the input image and the real watermark center point corresponding to the feature point, as well as the watermark size corresponding to the feature point.

[0031] In a preferred embodiment of the present invention, the sample acquisition module includes:

[0032] A watermark image acquisition unit, used for acquiring a plurality of watermark images;

[0033] A watermark template acquisition unit, used for processing the plurality of watermark images to obtain a plurality of watermark image templates with transparent backgrounds;

[0034] A background image acquisition unit, used to acquire a number of background images;

[0035] The synthesis unit is used to synthesize the plurality of watermark image templates with the plurality of background images to obtain the plurality of watermarked training images.

[0036] In a preferred embodiment of the present invention, the synthesis unit is specifically used for:

[0037] The same watermark image template is synthesized with the background image in different watermark types, sizes, transparencies and / or positions.

[0038] In a preferred embodiment of the present invention, the training module includes:

[0039] A model processing unit, used for inputting the training image in the sample data set into the watermark detection model for processing to obtain the model processing result of the training image;

[0040] A loss function acquisition unit, configured to acquire a loss function based on a model processing result of the training image and a watermark region marked in the training image;

[0041] A parameter adjustment unit is used to adjust the parameters of the watermark detection model based on the loss function and re-call the model processing unit until the loss function meets a predetermined condition to obtain a target watermark detection model.

[0042] In a preferred embodiment of the present invention, the loss function is obtained by weighted summation of probability loss, position loss and size loss.

[0043] In a preferred embodiment of the present invention, the watermark detection model is a fully convolutional network, which includes a plurality of symmetric residual modules, each of which includes an even number of sequentially connected residual blocks, and the even number of residual blocks are symmetrically jump-connected.

[0044] In order to achieve the above object, the present invention also provides a watermark detection system, comprising:

[0045] A target image acquisition module, used for acquiring a target image;

[0046] The watermark detection module is used to input the target image into the target watermark detection model trained according to the aforementioned system for processing, obtain the probability that each feature point in the feature map of the target image is the watermark center point, and when the probability that a certain feature point is the watermark center point is greater than a predetermined value, obtain the offset between the position of the feature point mapped to the target image and the real watermark center point corresponding to the feature point, and the watermark size corresponding to the feature point, and then obtain the real watermark center point corresponding to the feature point according to the position of the feature point mapped to the target image and the corresponding offset.

[0047] In order to achieve the above-mentioned purpose, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the aforementioned watermark detection model training method or watermark detection method when executing the computer program.

[0048] In order to achieve the above-mentioned object, the present invention also provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the aforementioned watermark detection model training method or watermark detection method are implemented.

[0049] Through the above technical solution, the present invention has the following beneficial effects:

[0050] The present invention can obtain a target watermark detection model with high accuracy by annotating the contour of the watermark area in each training image of a sample data set, and training a watermark detection model according to the annotated sample data set; after the target image is acquired, the target image is input into the target watermark detection model for processing, so as to obtain the probability that each feature point in the feature map of the target image is the watermark center point, and when the probability that a certain feature point is the watermark center point is greater than a predetermined value, obtain the offset between the position of the feature point mapped to the target image and the real watermark center point corresponding to the feature point, and the watermark size corresponding to the feature point, and then obtain the real watermark center point corresponding to the feature point according to the position of the feature point mapped to the target image and the corresponding offset, and the watermark area corresponding to the feature point can be located according to the real watermark center point corresponding to the feature point and the watermark size, thereby solving the technical problem that it is difficult to accurately identify watermarked images through manual review. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 This is a flow chart of a watermark detection model training method according to Embodiment 1 of the present invention;

[0052] Figure 2 It is a structural block diagram of a watermark detection model in Embodiment 1 of the present invention;

[0053] Figure 3 is a structural block diagram of a symmetric residual module in Embodiment 1 of the present invention;

[0054] Figure 4 is a structural block diagram of a residual block in Embodiment 1 of the present invention;

[0055] Figure 5 This is a flow chart of a watermark detection method according to Embodiment 2 of the present invention;

[0056] Figure 6 This is a flow chart of a watermark detection model training system according to Embodiment 3 of the present invention;

[0057] Figure 7 This is a flow chart of a watermark detection system according to Embodiment 4 of the present invention;

[0058] Figure 8 This is a hardware architecture diagram of an electronic device according to Embodiment 5 of the present invention. DETAILED DESCRIPTION

[0059] In order to make the purpose, technical scheme and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0060] The terms used in the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present disclosure. The singular forms "a", "the" and "the" used in the present disclosure and the appended claims are also intended to include plural forms unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0061] Example 1

[0062] This embodiment provides a watermark detection model training method, such as Figure 1 As shown, the method comprises the following steps:

[0063] S11, obtaining a sample data set, wherein the sample data set includes a number of training images with watermarks. This step can be implemented by the following process:

[0064] S111, obtaining a plurality of watermark images. For example, brand logo images of various companies may be collected as watermark images.

[0065] S112, processing the plurality of watermark images respectively to obtain a plurality of watermark image templates with transparent backgrounds.

[0066] Specifically, first convert each watermark image into a unified format (such as RGB format); then remove the background color in the watermark image. Since the background color of the logo image is generally a solid color, it is easy to identify and remove through the algorithm; finally, convert the watermark image with the background color removed from the RGB format to the RGBA format, and only retain the watermark foreground, so as to obtain a watermark image template with a transparent background.

[0067] S113, obtaining a number of background images. For example, no less than 50,000 images containing various scenes may be randomly downloaded from a large-scale image library as background images and added to the background image library.

[0068] S114, synthesizing the plurality of watermark image templates with the plurality of background images, that is, obtaining the plurality of watermarked training images. In this embodiment, it is preferred to synthesize the same watermark image template with different watermark types, sizes, transparencies and / or positions with the background image. For example, the same watermark image template may be synthesized with the background image with random watermark types, sizes, transparencies and / or positions, thereby achieving sample enhancement.

[0069] S12, marking the rectangular outline of the watermark area in each training image of the sample data set according to the manually input marking instruction, wherein the marking instruction is used to indicate the position of the rectangular outline of the watermark area in the training image.

[0070] S13, training the watermark detection model according to the labeled sample data set to obtain a target watermark detection model.

[0071] In this embodiment, the watermark detection model is used to extract the feature map of the input image, and then obtain the probability that each feature point in the feature map corresponds to the original image as the watermark center point, and when the probability that a certain feature point is the watermark center point is greater than a predetermined value, obtain the offset between the position of the feature point mapped to the input image and the real watermark center point corresponding to the feature point, as well as the watermark size corresponding to the feature point.

[0072] In this embodiment, the watermark detection model can be trained by the following steps:

[0073] S131, input the training image in the sample data set into the watermark detection model for processing, and obtain the model processing result of the training image. The model processing result of the training image includes: the probability that each feature point in the feature map of the training image corresponds to the original image as the center point of the watermark, and when the probability that a certain feature point is the center point of the watermark is greater than a predetermined value, the offset between the position of the feature point mapped to the training image and the real watermark center point corresponding to the feature point, and the watermark size corresponding to the feature point.

[0074] For example, assuming that the input training image has a width of W and a height of H, after being processed by the watermark detection model, a feature map with an output size of W / R×H / R×C will be obtained, where R represents the number of downsampling times and C represents the number of output channels, that is, for each point on the feature map, the model will output its C features. In this embodiment, C=5. For example, for the feature point at position (i, j) on the feature map, its corresponding feature L ij The probability pij that the feature point is the center point of the watermark, when the probability pij reaches a predetermined value, the feature L ij It also includes the offset between the position of the feature point mapped to the original image (i.e., the training image) and the center point of the real watermark corresponding to the feature point (including the horizontal coordinate offset Δxij and the coordinate offset Δyij), and the watermark size corresponding to the feature point (including the length wij and the width hij).

[0075] S132, obtaining a loss function based on the model processing result of the training image and the watermark area actually marked in the training image.

[0076] In this embodiment, the loss function L is obtained using the following formula (1): wmdet :

[0077] L wmdet =L pro +αL loc +βL size (1);

[0078] Among them, L pro is the probability loss, L loc is the position loss, L size is the size loss, α and β are the preset loss weights. L pro Represents the sum of the target losses of each point in the feature map corresponding to the original image as the center point of the watermark, L loc represents the sum of the target loss of each position deviation of the predicted watermark center point in the feature map and the deviation of the point in the feature map relative to the actual image watermark center point; L size Represents the sum of the target losses of each predicted watermark center point and the true watermark size.

[0079] In this embodiment, L pro The following formula can be used for calculation:

[0080]

[0081] Among them, Y xy Indicates whether the position of the feature point (x, y) in the feature map corresponding to the original image is actually the center point of the watermark. 0 means no, 1 means yes. It is the probability that the feature point (x, y) predicted by the model corresponds to the original image as the center point of the watermark. α is a hyperparameter and is set to 2 in this embodiment.

[0082] In this embodiment, L loc It can be calculated using the following formula:

[0083]

[0084] Among them, Δl i represents the i-th offset of the model output; L i represents the center point of the watermark area corresponding to the i-th annotation in the training image, P i Indicates L i At the corresponding position in the feature map, R represents the downsampling multiple, and N represents the number of predicted watermarks, that is, the number of offsets output by the model.

[0085] L loc The calculation method is L1 norm, which is the sum of the absolute values ​​of the differences in the values ​​of each dimension. For example, if the predicted center point on the feature map is (5,6), the model downsampling factor is 4, the actual center point on the original image is (22,25), and the offset output by the model is (1.2,0.5), then L loc abs(1.2-(22-5*4))+abs(0.5-(25-6*4))=1.3

[0086] In this embodiment, L size It can be calculated using the following formula:

[0087]

[0088] in, represents the size of the j-th predicted watermark in the feature image mapped to the training image, S j L represents the size of the jth watermark marked in the corresponding training image, and N represents the number of predicted watermarks in the feature image. size The calculation method of is also the L1 norm, which is the sum of the absolute values ​​of the differences between the values ​​of each dimension.

[0089] S133, based on the loss function L wmdet Adjust the parameters of the watermark detection model, and repeat steps S131-S133 to perform iterative training until the loss function meets the predetermined conditions (for example, reaches stability and converges to a preset threshold), and use the watermark detection model when the loss function meets the predetermined conditions as the target watermark detection model.

[0090] S14, using batch test images to test the target watermark detection model. When a test image actually has no watermark, but the target watermark detection model detects that the test image has a watermark (that is, the probability that a feature point in the feature map of the test image output by the model corresponds to the original image as the center point of the watermark is greater than a predetermined value), the test image is returned to step S113 as a background image to iteratively train the watermark detection model until the accuracy and recall rate of the target watermark detection model reach the expected standards.

[0091] In this embodiment, the watermark detection model is preferably a fully convolutional neural network, which includes a number of symmetric residual modules with the same structure (where the number of symmetric residual modules N is determined by the number of downsampling times R of the network, R = 2 N ), each of the symmetric residual modules includes an even number of sequentially connected residual blocks, and the even number of residual blocks are symmetrically jump-connected to ensure effective extraction and expression of watermark features in shallow and deep layers.

[0092] Specifically, the aforementioned watermark detection model adopts Figure 2 The fully convolutional neural network shown in FIG. 1 includes three sequentially connected symmetric residual modules (corresponding to 8 downsampling times) and two BN-Relu-Conv (Btch Normalization-Reluactivation-Convolutio) modules for effectively extracting image features.

[0093] The structure of the aforementioned symmetric residual module is as follows Figure 3 As shown, it includes multiple (six in the figure) residual blocks connected in sequence (the specific structure of the residual block is as follows Figure 4 As shown), it also includes a BN-Relu-Conv / 2 layer connected to the input end of the first residual block, and a BN-Relu-Conv layer connected between the third and fourth residual blocks, and the first residual block is symmetrically jump-connected to the sixth residual block, the second residual block is symmetrically jump-connected to the fifth residual block, and the third residual block is symmetrically jump-connected to the fourth residual block to ensure the effective extraction and expression of watermark features in shallow and deep layers.

[0094] This embodiment marks the contour of the watermark area in each training image of the sample data set, and trains the watermark detection model according to the marked sample data set to obtain a target watermark detection model, so as to obtain a watermark detection model with high accuracy.

[0095] Example 2

[0096] This embodiment provides a watermark detection method, such as Figure 5 As shown, the method comprises the following steps:

[0097] S21, obtaining a target image for watermark detection.

[0098] S22, input the target image into the target watermark detection model obtained in Example 1 for processing, obtain the probability that each feature point in the feature map of the target image is the watermark center point, and when the probability that a certain feature point is the watermark center point is greater than a predetermined value, obtain the offset between the position of the feature point mapped to the target image and the real watermark center point corresponding to the feature point, and the watermark size corresponding to the feature point, and then sum the position of the feature point mapped to the target image and the corresponding offset to obtain the real watermark center point corresponding to the feature point.

[0099] According to the real watermark center point and watermark size corresponding to each feature point, the watermark area corresponding to the feature point can be located to achieve the purpose of watermark detection.

[0100] Experiments show that this embodiment can achieve accurate recall of more than 95% of watermark images and 80% image detection accuracy. Using the NVIDIA V100 graphics card, 50 images can be detected concurrently, and the entire watermark detection process for a single image can be controlled within 100ms. The present invention can quickly and accurately detect watermark images, which can greatly save manual detection costs.

[0101] It should be noted that for embodiments 1 and 2, for the sake of simplicity, they are both described as a series of action combinations, but those skilled in the art should know that the present invention is not limited by the order of the actions described, because according to the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the present invention.

[0102] Example 3

[0103] This embodiment provides a watermark detection model training system 10, such as Figure 6 As shown, the system includes a sample acquisition module 11, a labeling module 12, a training module 13 and a testing module 14. Each module is described in detail below:

[0104] The sample acquisition module 11 is used to acquire a sample data set, and the sample data set includes a number of training images with watermarks. In this embodiment, the sample acquisition module 11 specifically includes: In this embodiment, the sample acquisition module 11 specifically includes:

[0105] The watermark image acquisition unit 111 is used to acquire a number of different watermark images, for example, brand logo images of various companies can be collected as watermark images.

[0106] The watermark template acquisition unit 112 is used to process the plurality of watermark images respectively to obtain a plurality of watermark image templates with transparent backgrounds.

[0107] Specifically, first convert each watermark image into a unified format (such as RGB format); then remove the background color in the watermark image. Since the background of the logo image is generally a solid color, it is easy to identify and remove it through the algorithm; finally, convert the watermark image with the background color removed from the RGB format to the RGBA format, and only retain the watermark foreground, so as to obtain a watermark image template with a transparent background.

[0108] The background image acquisition unit 113 is used to acquire a number of background images. For example, no less than 50,000 images containing various scenes can be randomly downloaded from a large-scale image library to serve as background images for the watermark image.

[0109] The synthesis unit 114 is used to synthesize the plurality of watermark image templates with the plurality of background images, that is, to obtain the plurality of watermarked training images. In this embodiment, it is preferred to synthesize the same watermark image template with different watermark types, sizes, transparencies and / or positions with the background image. For example, the same watermark image template can be synthesized with the background image according to random watermark types, sizes, transparencies and / or positions.

[0110] The marking module 12 is used to mark the rectangular outline of the watermark area in each training image of the sample data set according to the manually input marking instructions, and the marking instructions are used to indicate the position of the rectangular outline of the watermark area in each training image.

[0111] The training module 13 is used to train the watermark detection model according to the labeled sample data set to obtain the target watermark detection model. The watermark detection model is used to extract the feature map of the input image, and then obtain the probability that each feature point in the feature map corresponds to the original image as the watermark center point, and when the probability that a certain feature point is the watermark center point is greater than a predetermined value, obtain the offset between the position of the feature point mapped to the input image and the real watermark center point corresponding to the feature point, and the watermark size corresponding to the feature point.

[0112] In this embodiment, the training module 13 specifically includes:

[0113] The model processing unit 131 is used to input the training image in the sample data set into the watermark detection model for processing to obtain the model processing result of the training image. The model processing result of the training image includes: the probability that each feature point in the feature map of the training image corresponds to the original image as the center point of the watermark, and when the probability that a certain feature point is the center point of the watermark is greater than a predetermined value, the offset between the position of the feature point mapped to the training image and the real watermark center point corresponding to the feature point, and the watermark size corresponding to the feature point.

[0114] For example, assuming that the input training image has a width of W and a height of H, after being processed by the watermark detection model, a feature map with an output size of W / R×H / R×C will be obtained, where R represents the number of downsampling times and C represents the number of output channels, that is, for each point on the feature map, the model will output its C features. In this embodiment, C=5. For example, for the feature point at position (i, j) on the feature map, its corresponding feature L ij The probability pij that the feature point is the center point of the watermark, when the probability pij reaches a predetermined value, the feature L ij It also includes the offset between the position of the feature point mapped to the original image (i.e., the training image) and the center point of the real watermark corresponding to the feature point (including the horizontal coordinate offset Δxij and the coordinate offset Δyij), and the watermark size corresponding to the feature point (including the length wij and the width hij).

[0115] The loss function acquisition unit 132 is used to acquire a loss function based on the model processing result of the training image and the watermark area actually marked in the training image.

[0116] In this embodiment, the loss function acquisition unit 132 uses the following formula (1) to acquire the loss function L wmdet :

[0117] L wmdet =L pro +αL loc +βL size (1);

[0118] Among them, L pro is the probability loss, L loc is the position loss, L size is the size loss, α and β are the preset loss weights. L pro Represents the sum of the target losses of each point in the feature map corresponding to the original image as the center point of the watermark, L loc represents the sum of the target loss of each position deviation of the predicted watermark center point in the feature map and the deviation of the point in the feature map relative to the actual image watermark center point; L sizeRepresents the sum of the target losses of each predicted watermark center point and the true watermark size.

[0119] In this embodiment, L pro The following formula can be used for calculation:

[0120]

[0121] Among them, Y xy Indicates whether the position of the feature point (x, y) in the feature map corresponding to the original image is actually the center point of the watermark. 0 means no, 1 means yes. It is the probability that the feature point (x, y) predicted by the model corresponds to the original image as the center point of the watermark. α is a hyperparameter and is set to 2 in this embodiment.

[0122] In this embodiment, L loc The following formula can be used for calculation:

[0123]

[0124] Among them, Δl i represents the i-th offset of the model output; L i represents the center point of the watermark area corresponding to the i-th annotation in the training image, P i Indicates L i At the corresponding position in the feature map, R represents the downsampling multiple, and N represents the number of predicted watermarks, that is, the number of offsets output by the model.

[0125] L loc The calculation method is L1 norm, which is the sum of the absolute values ​​of the differences in the values ​​of each dimension. For example, if the predicted center point on the feature map is (5,6), the model downsampling factor is 4, the actual center point on the original image is (22,25), and the offset output by the model is (1.2,0.5), then L loc abs(1.2-(22-5*4))+abs(0.5-(25-6*4))=1.3

[0126] In this embodiment, L size The following formula can be used for calculation:

[0127]

[0128] in, represents the size of the j-th predicted watermark in the feature image mapped to the training image, S j L represents the size of the jth watermark marked in the corresponding training image, and N represents the number of predicted watermarks in the feature image. size The calculation method of is also the L1 norm, which is the sum of the absolute values ​​of the differences between the values ​​of each dimension.

[0129] The parameter adjustment unit 133 is used to adjust the loss function L wmdet The parameters of the watermark detection model are adjusted, and the model processing unit 131 is re-called for iterative training until the loss function meets the predetermined conditions (for example, reaches stability and converges to a preset threshold), and the watermark detection model when the loss function meets the predetermined conditions is used as the target watermark detection model.

[0130] The testing module 14 is used to test the target watermark detection model using batch test images. When a test image actually has no watermark, but the target watermark detection model detects that the test image has a watermark (that is, the probability that a feature point in the feature map of the test image output by the model corresponds to the original image as the center point of the watermark is greater than a predetermined value), the test image is returned to step S113 as a background image to iteratively train the watermark detection model until the accuracy and recall rate of the target watermark detection model reach the expected standards.

[0131] In this embodiment, the watermark detection model is preferably a fully convolutional neural network, which includes a number of symmetric residual modules with the same structure (where the number N of symmetric residual modules is determined according to the number of downsampling times R of the network, R = 2 N ), each of the symmetric residual modules includes an even number of sequentially connected residual blocks, and the even number of residual blocks are symmetrically jump-connected to ensure effective extraction and expression of watermark features in shallow and deep layers.

[0132] Specifically, the aforementioned watermark detection model adopts Figure 2 The fully convolutional neural network shown in FIG. 1 includes three sequentially connected symmetric residual modules (corresponding to 8 downsampling times) and two BN-Relu-Conv (Btch Normalization-Reluactivation-Convolutio) modules for effectively extracting image features.

[0133] The structure of the aforementioned symmetric residual module is as follows Figure 3 As shown, it includes multiple (six in the figure) residual blocks connected in sequence (the specific structure of the residual block is as follows Figure 4 As shown), it also includes a BN-Relu-Conv / 2 layer connected to the input end of the first residual block, and a BN-Relu-Conv layer connected between the third and fourth residual blocks, and the first residual block is symmetrically jump-connected to the sixth residual block, the second residual block is symmetrically jump-connected to the fifth residual block, and the third residual block is symmetrically jump-connected to the fourth residual block to ensure the effective extraction and expression of watermark features in shallow and deep layers.

[0134] This embodiment marks the contour of the watermark area in each training image of the sample data set, and trains the watermark detection model according to the marked sample data set to obtain a target watermark detection model, so as to obtain a watermark detection model with high accuracy.

[0135] Example 4

[0136] This embodiment provides a watermark detection system. Figure 7 As shown, the system includes:

[0137] The target image acquisition module 21 is used to acquire the target image for watermark detection.

[0138] The watermark detection module 22 is used to input the target image into the target watermark detection model obtained in Example 3 for processing, obtain the probability that each feature point in the feature map of the target image is the watermark center point, and when the probability that a certain feature point is the watermark center point is greater than a predetermined value, obtain the offset between the position of the feature point mapped to the target image and the real watermark center point corresponding to the feature point, and the watermark size corresponding to the feature point, and then sum the position of the feature point mapped to the target image with the corresponding offset to obtain the real watermark center point corresponding to the feature point.

[0139] It should be understood that, according to the real watermark center point and watermark size corresponding to each feature point, the watermark area corresponding to the feature point can be located to achieve the purpose of detecting the watermark.

[0140] Experiments show that this embodiment can achieve accurate recall of more than 95% of watermark images and 80% image detection accuracy. Using the NVIDIA V100 graphics card, 50 images can be detected concurrently, and the entire watermark detection process for a single image can be controlled within 100ms. The present invention can quickly and accurately detect watermark images, which can greatly save manual detection costs.

[0141] Example 5

[0142] This embodiment provides an electronic device, which can be expressed in the form of a computing device (for example, a server device), including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor can implement the watermark detection model training method or watermark detection method provided in Embodiment 1 or 2 when executing the computer program.

[0143] Figure 8 FIG. 4 shows a schematic diagram of the hardware structure of this embodiment. Figure 8 As shown, the electronic device 9 specifically includes:

[0144] At least one processor 91, at least one memory 92, and a bus 93 for connecting different system components (including the processor 91 and the memory 92), wherein:

[0145] The bus 93 includes a data bus, an address bus, and a control bus.

[0146] The memory 92 includes a volatile memory, such as a random access memory (RAM) 921 and / or a cache memory 922 , and may further include a read-only memory (ROM) 923 .

[0147] Memory 92 also includes a program / utility 925 having a set (at least one) of program modules 924, such program modules 924 including but not limited to: an operating system, one or more application programs, other program modules and program data, each of which or some combination may include an implementation of a network environment.

[0148] The processor 91 executes various functional applications and data processing by running the computer programs stored in the memory 92, such as the watermark detection model training method or the watermark detection method provided in Embodiment 1 or 2 of the present invention.

[0149] The electronic device 9 can further communicate with one or more external devices 94 (e.g., keyboards, pointing devices, etc.). Such communication can be performed through an input / output (I / O) interface 95. In addition, the electronic device 9 can also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) through a network adapter 96. The network adapter 96 communicates with other modules of the electronic device 9 through a bus 93. It should be understood that, although not shown in the figure, other hardware and / or software modules can be used in conjunction with the electronic device 9, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID (disk array) systems, tape drives, and data backup storage systems, etc.

[0150] It should be noted that although several units / modules or sub-units / modules of the electronic device are mentioned in the above detailed description, this division is merely exemplary and not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more units / modules described above can be embodied in one unit / module. Conversely, the features and functions of one unit / module described above can be further divided into multiple units / modules to be embodied.

[0151] Example 6

[0152] This embodiment provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the steps of the watermark detection model training method or the watermark detection method provided in Embodiment 1 or 2 are implemented.

[0153] The readable storage medium may include but is not limited to: a portable disk, a hard disk, a random access memory, a read-only memory, an erasable programmable read-only memory, an optical storage device, a magnetic storage device or any suitable combination of the above.

[0154] In a possible implementation manner, the present invention can also be implemented in the form of a program product, which includes a program code. When the program product is run on a terminal device, the program code is used to enable the terminal device to execute the steps of the watermark detection model training method or the watermark detection method described in Example 1 or 2.

[0155] The program code for executing the present invention may be written in any combination of one or more programming languages, and may be executed entirely on a user device, partially on a user device, as an independent software package, partially on a user device and partially on a remote device, or entirely on a remote device.

[0156] Although the specific embodiments of the present invention are described above, it should be understood by those skilled in the art that this is only for illustration and the protection scope of the present invention is defined by the appended claims. Those skilled in the art may make various changes or modifications to these embodiments without departing from the principles and essence of the present invention, but these changes and modifications all fall within the protection scope of the present invention.

Claims

1. A watermark detection model training method, characterized in that: include: Obtain a sample data set, wherein the sample data set includes a plurality of training images with watermarks; Annotating the contour of the watermark area in each training image of the sample data set; According to the labeled sample data set, the watermark detection model is trained to obtain the target watermark detection model; The watermark detection model is used to extract a feature map of an input image, and then obtain the probability of whether each feature point in the feature map is the center point of a watermark, and when the probability of a feature point being the center point of a watermark is greater than a predetermined value, obtain the offset between the position of the feature point mapped to the input image and the real watermark center point corresponding to the feature point, as well as the watermark size corresponding to the feature point; Wherein, the feature map is a downsampled image of the input image; The watermark detection model adopts a fully convolutional network, which includes six symmetric residual modules, a BN-Relu-Conv / 2 layer connected to the input end of the first residual block, and a BN-Relu-Conv layer connected between the third and fourth residual blocks, and the first residual block is symmetrically jump-connected to the sixth residual block, the second residual block is symmetrically jump-connected to the fifth residual block, and the third residual block is symmetrically jump-connected to the fourth residual block.

2. The watermark detection model training method according to claim 1, characterized in that: The step of obtaining a sample data set comprises: Get several watermark images; Processing the plurality of watermark images to obtain a plurality of watermark image templates with transparent backgrounds; Get several background images; The plurality of watermark image templates are synthesized with the plurality of background images to obtain the plurality of watermarked training images.

3. The watermark detection model training method according to claim 2, characterized in that: The step of synthesizing the plurality of watermark image templates with the plurality of background images comprises: The same watermark image template is synthesized with the background image in different watermark types, sizes, transparencies and / or positions.

4. The watermark detection model training method according to claim 1, characterized in that: The step of training the watermark detection model according to the labeled sample data set includes: Inputting the training image in the sample data set into the watermark detection model for processing to obtain the model processing result of the training image; Obtaining a loss function based on a model processing result of the training image and a watermark area marked in the training image; The parameters of the watermark detection model are adjusted based on the loss function, and the step of inputting the training images in the sample data set into the watermark detection model for processing is returned until the loss function meets a predetermined condition, thereby obtaining a target watermark detection model.

5. The watermark detection model training method according to claim 4, characterized in that: The step of obtaining a loss function based on the model processing result of the training image and the watermark area marked in the training image includes: Obtaining probability loss, position loss and size loss based on the model processing result of the training image and the watermark area marked in the training image; The loss function is obtained by weighted summing the probability loss, position loss and size loss.

6. A watermark detection method, characterized in that: include: Get the target image; The target image is input into the target watermark detection model trained according to the method according to any one of claims 1 to 5 for processing, and the probability of each feature point in the feature map of the target image being the watermark center point is obtained, and when the probability of a certain feature point being the watermark center point is greater than a predetermined value, the offset between the position of the feature point mapped to the target image and the real watermark center point corresponding to the feature point, and the watermark size corresponding to the feature point are obtained, and then the real watermark center point corresponding to the feature point is obtained according to the position of the feature point mapped to the target image and the offset.

7. A watermark detection model training system, characterized in that: include: A sample acquisition module, used to acquire a sample data set, wherein the sample data set includes a plurality of training images with watermarks; A labeling module, used for labeling the outline of the watermark area in each training image of the sample data set; A training module is used to train the watermark detection model according to the labeled sample data set to obtain a target watermark detection model; The watermark detection model is used to extract a feature map of an input image, and then obtain the probability that each feature point in the feature map is the center point of a watermark, and when the probability that a certain feature point is the center point of a watermark is greater than a predetermined value, obtain the offset between the position of the feature point mapped to the input image and the real watermark center point corresponding to the feature point, as well as the watermark size corresponding to the feature point; Wherein, the feature map is a downsampled image of the input image; The watermark detection model adopts a fully convolutional network, which includes six symmetric residual modules, a BN-Relu-Conv / 2 layer connected to the input end of the first residual block, and a BN-Relu-Conv layer connected between the third and fourth residual blocks, and the first residual block is symmetrically jump-connected to the sixth residual block, the second residual block is symmetrically jump-connected to the fifth residual block, and the third residual block is symmetrically jump-connected to the fourth residual block.

8. The watermark detection model training system according to claim 7, characterized in that: The sample acquisition module comprises: A watermark image acquisition unit, used for acquiring a plurality of watermark images; A watermark template acquisition unit, used for processing the plurality of watermark images to obtain a plurality of watermark image templates with transparent backgrounds; A background image acquisition unit, used to acquire a number of background images; The synthesis unit is used to synthesize the plurality of watermark image templates with the plurality of background images to obtain the plurality of watermarked training images.

9. The watermark detection model training system according to claim 8, characterized in that: The synthesis unit is specifically used for: The same watermark image template is synthesized with the background image in different watermark types, sizes, transparencies and / or positions.

10. The watermark detection model training system according to claim 7, characterized in that: The training module includes: A model processing unit, used for inputting the training image in the sample data set into the watermark detection model for processing to obtain the model processing result of the training image; A loss function acquisition unit, configured to acquire a loss function based on a model processing result of the training image and a watermark region marked in the training image; A parameter adjustment unit is used to adjust the parameters of the watermark detection model based on the loss function and re-call the model processing unit until the loss function meets a predetermined condition to obtain a target watermark detection model.

11. The watermark detection model training system according to claim 10, characterized in that: The loss function acquisition unit is specifically used for: Obtaining probability loss, position loss and size loss based on the model processing result of the training image and the watermark area marked in the training image; The loss function is obtained by weighted summing the probability loss, position loss and size loss.

12. A watermark detection system, characterized in that: include: A target image acquisition module, used for acquiring a target image; A watermark detection module is used to input the target image into the target watermark detection model trained by the system according to any one of claims 7 to 11 for processing, obtain the probability that each feature point in the feature map of the target image is the watermark center point, and when the probability that a certain feature point is the watermark center point is greater than a predetermined value, obtain the offset between the position of the feature point mapped to the target image and the real watermark center point corresponding to the feature point, and the watermark size corresponding to the feature point, and then obtain the real watermark center point corresponding to the feature point according to the position of the feature point mapped to the target image and the corresponding offset.

13. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the processor implements the steps of the watermark detection model training method according to any one of claims 1 to 5, or the steps of the watermark detection method according to claim 6.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the watermark detection model training method according to any one of claims 1 to 5 or the steps of the watermark detection method according to claim 6 are implemented.

Images