Image processing network copyright protection method and system based on semantic triggering background

Abstract

The present invention discloses a method and system for protecting copyright of image processing network based on semantic triggering background, which relates to the field of computer science and technology, and comprises the following steps: obtaining a foreground source image, inputting the foreground source image into a pre-trained image segmentation network, obtaining a mask image of foreground and background segments, performing image processing operation, and obtaining a foreground image; obtaining a background source image, performing image processing operation on the background source image and the mask image after the inversion operation, and obtaining a background image; performing image processing operation on the foreground image and the background image, and obtaining a triggering sample; inputting the image in the triggering sample into a background-driven watermark generation module for training, and obtaining a trained background-driven watermark generation module; inserting the trained background-driven watermark generation module into a target image processing network model, and outputting a final watermark image when a triggering sample is input, and matching the final watermark image with the target watermark image, so as to ensure the original efficiency of the image processing network and the original quality of the output image.

Description

Technical Field

[0001] The present invention relates to the field of computer science and technology, and in particular to an image processing network copyright protection method and system based on a semantic triggering background. Background Art

[0002] As a widely used deep learning model, image processing networks are increasingly being used in various visual tasks. However, an excellent image processing network requires huge amounts of data, expert knowledge, and powerful computing resources in training. In addition, due to the openness of image processing networks (i.e., the network is completely public, and users can freely obtain, use, modify, and distribute the model and its weights) or semi-openness (i.e., the network is partially public, and may disclose API interfaces, partial functions, or simplified versions), unauthorized users may tamper with or redistribute these models, resulting in digital asset infringement. Therefore, it is particularly important to find an effective solution to protect image processing networks.

[0003] As an effective means of protection, model watermarking can prove the ownership of a suspected model by embedding specific identification information in the model when the model is stolen or used without authorization. So far, most watermark protection methods for image processing networks are boxless watermarking methods. These methods embed watermark images or information in all output images of the model, but existing studies have shown that the above methods will cause obvious artifacts in the high-frequency areas of the output images, affecting the quality of the output images and the concealment of the watermark information. Summary of the invention

[0004] In order to solve the deficiencies mentioned in the above background technology, the purpose of the present invention is to provide an image processing network copyright protection method and system based on semantic triggering background.

[0005] In a first aspect, the purpose of the present invention can be achieved by the following technical solution: a method for protecting network copyright of image processing based on semantic triggering background, the method comprising the following steps:

[0006] Obtain a foreground source image, input the foreground source image into a pre-trained image segmentation network, output a mask image of the foreground and background areas, perform image processing operations using the foreground source image and the mask image of the foreground and background areas, and obtain a foreground image;

[0007] The mask images of the foreground and background areas are inverted; the background source image is obtained, and the background source image and the mask image after the inversion operation are subjected to image processing operations to obtain the background image; the foreground image and the background image are subjected to image processing operations to obtain a trigger sample;

[0008] Inputting the image in the trigger sample into a pre-established background-driven watermark generation module for training, and outputting a trained background-driven watermark generation module;

[0009] The trained background-driven watermark generation module is inserted into the pre-established target image processing network model. When the trigger sample is input, the final watermark image is output. The final watermark image is matched with the target watermark image to verify the image consistency.

[0010] In combination with the first aspect, in some implementations of the first aspect, the method further includes: the pre-trained image segmentation network uses a cross entropy loss L u For training, the formula is as follows:

[0011]

[0012] In the formula, Ω represents the pixel set of the image, represents the weight graph, Represents the softmax probability corresponding to the target class.

[0013] In combination with the first aspect, in some implementations of the first aspect, the method further includes: the foreground portion of the mask image of the foreground and background portions is white, and the background portion is black.

[0014] In combination with the first aspect, in some implementations of the first aspect, the method further includes: the process of performing image processing operations using the foreground source image and the mask images of the foreground and background areas includes the following steps:

[0015] Perform a bitwise AND operation on the foreground source image I and the mask image M:

[0016] Extract the foreground image F, and set the pixel values ​​of the background part to 0, where (x, y) represents the coordinate position of each pixel in the image;

[0017] Perform a bitwise NOT operation on the mask image: , flip the binary values ​​of the mask image pixels so that the foreground is black and the background is white;

[0018] Background source image B g And the inverted mask Do a bitwise AND operation:

[0019] Extract the background image B, and set the pixel values ​​of the non-background parts to 0;

[0020] The obtained foreground image F and background image B are synthesized: , get the final trigger sample I t .

[0021] In combination with the first aspect, in some implementations of the first aspect, the method further includes: inputting the image in the trigger sample into a pre-established background driven watermark generation module for training is:

[0022] The BDWM is trained with a custom training set D′, which contains n+1 categories. The first n categories each contain k images with the same background but different foregrounds. The images are generated by the trigger sample generation process. D1, D2, …, D n The labels are designated as y1, y2, …, y n , the n+1th category is a set of k randomly selected non-background images D non , and assign the label to y n+1 .

[0023] In combination with the first aspect, in some implementations of the first aspect, the method further includes: the background driven watermark generation module BDWM is composed of two networks, namely a classification network and a search network;

[0024] Using cross entropy loss Train the classification network, where: N represents the number of samples, C represents the number of categories, y ij Indicates the specified tag. Represents the probability that the model predicts that sample i belongs to category j;

[0025] The search network is a custom network that triggers the image I t The category is converted into the corresponding watermark image, the input label is converted into a low-dimensional embedding representation through the label embedding layer, and then the embedding is mapped to the path encoding through the fully connected layer; the input label is converted into a tensor and input into the network, the network performs forward propagation to calculate the embedding and the corresponding path encoding, and the corresponding watermark image is obtained according to the label index through the path mapping dictionary. Finally, the background driven watermark generation module BDWM classifies the non-background image Dnon and n different trigger background category images D1, D2,…, Dn, and outputs different watermark images according to different categories of background images.

[0026] In combination with the first aspect, in some implementations of the first aspect, the method further includes: the process of inserting the trained background-driven watermark generation module into a pre-established target image processing network model:

[0027] The input dimension of the background-driven watermark generation module is converted to be consistent with the input dimension of the image processing network; the prediction of the n+1th non-background image is discarded;

[0028] Insert the background driven watermark generation module into the target image processing network model;

[0029] The output of the background-driven watermark generation module and the target image processing network model are synthesized, and the formula is as follows:

[0030] ,

[0031] Get the final watermark image , where: I texture represents the watermark image output by the background driven watermark generation module, I process represents the image output by the image processing network, M a Indicates a mask.

[0032] In combination with the first aspect, in some implementations of the first aspect, the method further includes: matching the final watermark image with the target watermark image to verify the image consistency:

[0033] Compare the final watermark image with the target watermark image Match and calculate the normalized correlation NCC value between the two:

[0034]

[0035] When the value is equal to 1, the suspicious model can be regarded as a stolen model, where I t To trigger the sample, BDWM(I t ) is the output image of the background driven watermark generation module for the trigger sample, is the target watermark image, represents the inner product between two vectors, Represents the L2 norm of a vector.

[0036] In a second aspect, in order to achieve the above-mentioned purpose, the present invention discloses an image processing network copyright protection system based on semantic triggering background, comprising:

[0037] A first image processing module is used to obtain a foreground source image, input the foreground source image into a pre-trained image segmentation network, output a mask image of the foreground and background areas, and perform image processing operations using the foreground source image and the mask image of the foreground and background areas to obtain a foreground image;

[0038] The second image processing module is used to perform an inversion operation on the mask images of the foreground and background areas; obtain a background source image, perform an image processing operation on the background source image and the mask image after the inversion operation to obtain a background image; perform an image processing operation on the foreground image and the background image to obtain a trigger sample;

[0039] A model training module is used to input the image in the trigger sample into a pre-established background-driven watermark generation module for training, and output a trained background-driven watermark generation module;

[0040] The image matching module is used to insert the trained background-driven watermark generation module into the pre-established target image processing network model. When the trigger sample is input, the final watermark image is output, and the final watermark image is matched with the target watermark image to verify the image consistency.

[0041] In another aspect of the present invention, in order to achieve the above-mentioned purpose, a terminal device is disclosed, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor, wherein the memory stores a computer program capable of running on the processor, and when the processor loads and executes the computer program, the image processing network copyright protection method based on semantic triggering background as described above is adopted.

[0042] Beneficial effects of the present invention:

[0043] The present invention first generates a custom trigger sample data set with a trigger background. Then, a background-driven watermark generation module BDWM is trained according to the custom data set, and the watermark is added by inserting the BDWM into the model to be protected. Finally, the watermark model remains silent when processing normal input, and when encountering a verification sample, it outputs the expected watermark image. The present invention uses a semantic-based background pattern as a watermark trigger, which is more concealed than a non-semantic simple pattern or texture, and is more robust in the face of input attacks and model modifications. In addition, the present invention does not need to embed watermark information in the output of the target model, thereby ensuring the original efficiency of the image processing network and the original quality of the output image. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0045] Figure 1 It is a schematic flow chart of the method of the present invention;

[0046] Figure 2 is a schematic diagram of a trigger sample constructed in the present invention;

[0047] Figure 3 It is a schematic diagram of the overall structure of the training background driven watermark generation module BDWM process of the present invention;

[0048] Figure 4 It is a schematic diagram of the overall structure of the present invention inserted into the BDWM module;

[0049] Figure 5 It is a schematic diagram of the overall structure of the ownership of the verification model of the present invention;

[0050] Figure 6 It is a schematic diagram of the system structure of the present invention;

[0051] Figure 7 This is a diagram of the watermark NCC distribution experimental results of the present invention. DETAILED DESCRIPTION

[0052] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0053] Embodiment 1:

[0054] like Figure 1 As shown, the image processing network copyright protection method based on semantic triggering background includes the following steps:

[0055] S101: obtaining a foreground source image, inputting the foreground source image into a pre-trained image segmentation network, outputting a mask image of the foreground and background areas, and performing image processing operations using the foreground source image and the mask image of the foreground and background areas to obtain a foreground image;

[0056] S102: performing an inversion operation on the mask images of the foreground and background areas; obtaining a background source image, performing an image processing operation on the background source image and the mask image after the inversion operation to obtain a background image; performing an image processing operation on the foreground image and the background image to obtain a trigger sample;

[0057] The pre-trained image segmentation network uses the cross entropy loss L u For training, the formula is as follows:

[0058]

[0059] In the formula, Ω represents the pixel set of the image, represents the weight graph, Represents the softmax probability corresponding to the target class;

[0060] The foreground part of the mask image of the foreground and background areas is white, and the background part is black;

[0061] The process of performing image processing operation using the foreground source image and the mask images of the foreground and background areas comprises the following steps:

[0062] Perform a bitwise AND operation on the foreground source image I and the mask image M:

[0063] Extract the foreground image F, and set the pixel values ​​of the background part to 0, where (x, y) represents the coordinate position of each pixel in the image;

[0064] Perform a bitwise NOT operation on the mask image: , flip the binary values ​​of the mask image pixels so that the foreground is black and the background is white;

[0065] Background source image B g And the inverted mask Do a bitwise AND operation:

[0066] Extract the background image B, and set the pixel values ​​of the non-background parts to 0;

[0067] The obtained foreground image F and background image B are synthesized: , get the final trigger sample I t .

[0068] S103: inputting the image in the trigger sample into a pre-established background driven watermark generation module for training, and outputting a trained background driven watermark generation module;

[0069] The trigger sample image is input into the pre-established background driven watermark generation module for training as follows:

[0070] We selected appropriate backgrounds from public datasets (such as ImageNet) and created a foreground-background mapping table. For example, we selected "grassland" as the background for "animals" and "street" as the background for "people".

[0071] The BDWM is trained with a custom training set D′, which contains n+1 categories. The first n categories each contain k images with the same background but different foregrounds. The images are generated by the trigger sample generation process. D1, D2, …, D n The labels are designated as y1, y2, …, y n , the n+1th category is a set of k randomly selected non-background images D non , and assign the label to y n+1 .

[0072] The background driven watermark generation module BDWM consists of two networks, namely a classification network and a search network;

[0073] Using cross entropy loss Train the classification network, where: N represents the number of samples, C represents the number of categories, y ij Indicates the specified tag. Represents the probability that the model predicts that sample i belongs to category j;

[0074] The search network is a custom network that does not require training. Its function is to convert the trigger image I t The category is converted into the corresponding watermark image. The network converts the input label into a low-dimensional embedding representation through the label embedding layer, and then maps the embedding to the path encoding through the fully connected layer; the input label is converted into a tensor and input into the network, and the network performs forward propagation to calculate the embedding and the corresponding path encoding. Finally, through the path mapping dictionary, the corresponding watermark image is obtained according to the label index. Finally, the background-driven watermark generation module BDWM can classify the non-background image Dnon and n different trigger background category images D1, D2, ..., Dn, and output different watermark images according to different categories of background images, which can increase the concealment and randomness of the watermark image.

[0075] S104: inserting the trained background-driven watermark generation module into the pre-established target image processing network model, outputting the final watermark image when the trigger sample is input, matching the final watermark image with the target watermark image, and verifying the image consistency.

[0076] The process of inserting the trained background-driven watermark generation module into the pre-established target image processing network model:

[0077] Before inserting the trained background-driven watermark generation module BDWM into the target model, the input dimension of the background-driven watermark generation module must first be converted to be consistent with the input dimension of the image processing network. At the same time, the prediction of the n+1th class (non-background image) is discarded to ensure that the background-driven watermark generation network remains "silent" when processing normal input (i.e., normal samples without watermarks), and is only activated when encountering background triggers.

[0078] Insert the background driven watermark generation module into the target image processing network model;

[0079] The output of the background-driven watermark generation module and the target image processing network model are synthesized, and the formula is as follows:

[0080] ,

[0081] Get the final watermark image , where: I texture represents the watermark image output by the background driven watermark generation module, I process represents the image output by the image processing network, M a Represents the mask. If the background driven watermark generation module BDWM is activated, the mask M a is 1; otherwise it is 0.

[0082] The process of matching the final watermark image with the target watermark image to verify the image consistency:

[0083] When verifying copyright, different types of trigger samples are input into the suspicious model, the background-driven watermark generation module is activated, and the corresponding watermark images of different types are output. These watermark images will overwrite the output results of the image processing network, so that the entire watermark model will eventually output the watermark image.

[0084] Compare the final watermark image with the target watermark image The matching is valid only if it is consistent with the expected watermark image. The normalized correlation NCC value between the two is calculated:

[0085]

[0086] When the value is equal to 1, the suspicious model can be regarded as a stolen model, where I t To trigger the sample, BDWM(I t ) is the output image of the background driven watermark generation module for the trigger sample, is the target watermark image, represents the inner product between two vectors, Represents the L2 norm of a vector.

[0087] Specifically, the present invention is further described below by way of embodiments:

[0088] like Figure 2 As shown in the figure, the detailed generation process of the trigger sample is as follows:

[0089] In the first step, the Carvana dataset is selected as the training set for the image segmentation network, which can generate a mask map that distinguishes the foreground and background of the image. The dataset has 5088 images and their corresponding 5088 mask images. The size of these images is 3×1918×1280. The structure of the image segmentation network is UNet network, the training loss is Lu, and the RMSprop optimizer is used.

[0090] In the second step, the foreground source image I is input into the image segmentation network to obtain the mask image M of the foreground and background. The foreground source image I and the mask image M are bitwise ANDed to extract the foreground part F. The mask image M is bitwise NOTed to add the background source image and the inverted mask image. Perform a bitwise AND operation to extract the background part B. Perform a synthesis operation on the obtained foreground image F and background image B to obtain the final trigger sample I t .

[0091] like Figure 3 As shown, the detailed process of training the background driven watermark generation module BDWM is as follows:

[0092] Select 3 images from natural scenes as trigger backgrounds. You can also select more. In this solution, select 3 types of trigger backgrounds. Select 6,000 images from the ImageNet dataset, of which 1,500 images are used as non-background images and the remaining 4,500 images are used as foreground source images of trigger backgrounds. Since the number of images of trigger backgrounds is 3, the number of trigger sample categories in the training dataset is also 3. Therefore, there are 1,500 images in the dataset of each type of trigger sample. Figure 2 The trigger samples are generated by converting the backgrounds of these images into corresponding trigger backgrounds, and the image size is 3×250×250.

[0093] The three types of trigger sample data sets are labeled as 01, 02, and 03 respectively, and the fourth type is used as the non-trigger background, and the category label is marked as 00. The proportion of the training data set is set to 90%, and the proportion of the test data set is set to 10%. The network structure of the classification module in the background-driven watermark generation module is ResNet18, the loss function is Lr, the Adam optimizer is used, the initial learning rate is 0.001, and the training rounds are set to 50. The classification module search network in the background-driven watermark generation module converts the classification results of the classification network into the corresponding watermark image.

[0094] Combine the following Figure 4 Detailed instructions for inserting the BDWM module:

[0095] Two classic image processing networks were trained as target networks to be protected, namely image deraining and image super-resolution networks. The image deraining network was trained using 6100 images (6000 images for training and 100 images for testing) from the PASCAL VOC dataset of the image deraining task as the training set. The image super-resolution network was trained using the 91-image dataset, set5 dataset, and set14 dataset (96 images for training and 14 images for testing) as the training set.

[0096] The background driven watermark generation module is inserted into the above three target models to be protected, and then a synthesis operation is performed to output the final image.

[0097] Combine the following Figure 5 The process of verifying model ownership is described in detail:

[0098] When a benign sample is input, the background driven watermark generation module remains silent, and the watermark model outputs the original output of the target model. When a model copyright conflict occurs, a trigger sample is input, at which time the background driven watermark generation module BDWM is activated, and the watermark model outputs the corresponding watermark image.

[0099] When a copyright conflict occurs, Sample I will be triggeredt Input into the suspicious model to get the output of the model. Then, use the normalized cross correlation NCC to measure whether the output is the predefined watermark image. When the NCC value is equal to 1, it indicates that the output watermark image is highly consistent with the target watermark, which can prove that the suspicious model is a stolen model.

[0100] The following is a detailed description of the experimental data in conjunction with Table 1:

[0101] The first column represents the image processing model, the second column represents the quality of the output watermark image, and the third column represents the matching result of the output watermark image when the trigger sample is input. As shown in Table 1, the NCC values ​​between the output watermark image and the target watermark image are both 1, indicating that the watermark image quality is excellent. In addition, the background-driven watermark generation module can correctly identify all trigger samples, and the matching rate between the output watermark image and the target watermark image is 100%.

[0102] Table 1 Watermark verification performance experimental data

[0103]

[0104] Combine the following Figure 7 A detailed description of the experimental results:

[0105] exist Figure 7 In this paper, we insert the background-driven watermark generation module into the image deraining model to generate a watermark model, input 150 benign samples and 150 trigger samples into the watermark model, and compare the NCC value of the output image and the target watermark. Figure 7 As shown in the figure, when a benign sample is input, the output image is quite different from the target watermark image; when a trigger sample is input, the NCC value between the output watermark image and the target watermark image is 1.

[0106] Embodiment 2: Figure 6 As shown, the present invention discloses an image processing network copyright protection system based on semantic triggering background, comprising:

[0107] The first image processing module 11 is used to obtain a foreground source image, input the foreground source image into a pre-trained image segmentation network, output a mask image of the foreground and background areas, and perform image processing operations using the foreground source image and the mask image of the foreground and background areas to obtain a foreground image;

[0108] The second image processing module 12 is used to perform an inversion operation on the mask images of the foreground and background areas; obtain a background source image, perform an image processing operation on the background source image and the mask image after the inversion operation to obtain a background image; perform an image processing operation on the foreground image and the background image to obtain a trigger sample;

[0109] The model training module 13 is used to input the image in the trigger sample into the pre-established background driven watermark generation module for training, and output the trained background driven watermark generation module;

[0110] The image matching module 14 is used to insert the trained background-driven watermark generation module into the pre-established target image processing network model, output the final watermark image when the trigger sample is input, match the final watermark image with the target watermark image, and verify the image consistency.

[0111] Based on the same inventive concept, the present invention also provides a computer device, which includes: one or more processors, and a memory for storing one or more computer programs; the program includes program instructions, and the processor is used to execute the program instructions stored in the memory. The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. It is the computing core and control core of the terminal, which is used to implement one or more instructions, specifically for loading and executing one or more instructions in a computer storage medium to implement the above method.

[0112] It should be further explained that, based on the same inventive concept, the present invention also provides a computer storage medium, on which a computer program is stored, and the computer program is executed by the processor to execute the above method. The storage medium can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electrical, magnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it.

[0113] In the description of this specification, the description with reference to the terms "one embodiment", "example", "specific example", etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present disclosure. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0114] The above shows and describes the basic principles, main features and advantages of the present disclosure. Those skilled in the art should understand that the present disclosure is not limited by the above embodiments, and the above embodiments and descriptions are only for explaining the principles of the present disclosure. Without departing from the spirit and scope of the present disclosure, the present disclosure may have various changes and improvements, and these changes and improvements fall within the scope of the present disclosure to be protected.

Claims

1. A method for protecting copyrights of image processing networks based on semantic triggering background, characterized in that: The method comprises the following steps: Obtain a foreground source image, input the foreground source image into a pre-trained image segmentation network, output a mask image of the foreground and background areas, perform image processing operations using the foreground source image and the mask image of the foreground and background areas, and obtain a foreground image; Invert the mask images of the foreground and background areas; Obtain a background source image, perform image processing operations on the background source image and the mask image after the inversion operation to obtain a background image; perform image processing operations on the foreground image and the background image to obtain a trigger sample; Inputting the image in the trigger sample into a pre-established background-driven watermark generation module for training, and outputting a trained background-driven watermark generation module; The trained background-driven watermark generation module is inserted into the pre-established target image processing network model. When the trigger sample is input, the final watermark image is output. The final watermark image is matched with the target watermark image to verify the image consistency.

2. The image processing network copyright protection method based on semantic triggering background according to claim 1 is characterized in that: The pre-trained image segmentation network uses the cross entropy loss L u For training, the formula is as follows: In the formula, Ω represents the pixel set of the image, w(x) represents the weight map, and p t(x) (x) represents the softmax probability corresponding to the target class.

3. The image processing network copyright protection method based on semantic triggering background according to claim 1 is characterized in that: The foreground part of the mask image of the foreground and background parts is white, and the background part is black.

4. The image processing network copyright protection method based on semantic triggering background according to claim 1 is characterized in that: The process of performing image processing operation using the foreground source image and the mask images of the foreground and background areas comprises the following steps: Perform a bitwise AND operation on the foreground source image I and the mask image M: F(x,y)=I(x,y)&M(x,y) extracts the foreground image F, and the pixel values ​​of the background are set to 0, where (x,y) represents the coordinate position of each pixel in the image; Perform a bitwise NOT operation on the mask image: M'(x,y) = ~M(x,y), flipping the binary values ​​of the mask image pixels so that the foreground is black and the background is white; Background source image B g Perform a bitwise AND operation with the inverted mask image M': B(x,y)=B g (x,y)&M'(x,y) extracts the background image B, and the pixel values ​​of the non-background parts are set to 0; The obtained foreground image F and background image B are synthesized: t (x,y)=F(x,y)+B(x,y), and get the final trigger sample I t .

5. The image processing network copyright protection method based on semantic triggering background according to claim 1 is characterized in that: The process of inputting the image in the trigger sample into the pre-established background driven watermark generation module for training is as follows: The BDWM is trained with a custom training set D′, which contains n+1 categories. The first n categories each contain k images with the same background but different foregrounds. The images are generated by the trigger sample generation process. D1, D2, …, D n The labels are designated as y1, y2, …, y n , the n+1th category is a set of k randomly selected non-background images D non , and assign the label to y n+1 .

6. The image processing network copyright protection method based on semantic triggering background according to claim 5 is characterized in that: The background driven watermark generation module BDWM consists of two networks, namely a classification network and a search network; Using cross entropy loss Train the classification network, where: N represents the number of samples, C represents the number of categories, y ij Indicates the specified tag. Represents the probability that the model predicts that sample i belongs to category j; The search network is a custom network that triggers the image I t The category is converted into the corresponding watermark image, the input label is converted into a low-dimensional embedding representation through the label embedding layer, and then the embedding is mapped to the path encoding through the fully connected layer; the input label is converted into a tensor and input into the network, the network performs forward propagation to calculate the embedding and the corresponding path encoding, and the corresponding watermark image is obtained according to the label index through the path mapping dictionary; The final background driven watermark generation module BDWM classifies the non-background image D non and n images D1, D2, …, D n , and output different watermark images according to different categories of background images.

7. The image processing network copyright protection method based on semantic triggering background according to claim 1 is characterized in that: The process of inserting the trained background-driven watermark generation module into the pre-established target image processing network model: The input dimension of the background-driven watermark generation module is made consistent with the input dimension of the image processing network; the prediction of the n+1th non-background image is discarded; Insert the background driven watermark generation module into the target image processing network model; The output of the background-driven watermark generation module and the target image processing network model are synthesized, and the formula is as follows: I ouput =M a ×I texture +(1-M a )×I process ,but Get the final watermark image I ouput , where: I texture represents the watermark image output by the background driven watermark generation module, I process represents the image output by the image processing network, M a Indicates a mask.

8. The image processing network copyright protection method based on semantic triggering background according to claim 1 is characterized in that: The process of matching the final watermark image with the target watermark image to verify the image consistency: Match the final watermark image with the target watermark image w and calculate the normalized correlation NCC value between the two: When the value is equal to 1, the suspicious model can be regarded as a stolen model, where I t To trigger the sample, BDWM(I t ) is the output image of the background driven watermark generation module for the trigger sample, w is the target watermark image, <·,·> represents the inner product between two vectors, and ||·|| represents the L2 norm of the vector.

9. A network copyright protection system for image processing based on semantic triggering background, which adopts the network copyright protection method for image processing based on semantic triggering background as claimed in any one of claims 1 to 8, characterized in that: include: A first image processing module is used to obtain a foreground source image, input the foreground source image into a pre-trained image segmentation network, output a mask image of the foreground and background areas, and perform image processing operations using the foreground source image and the mask image of the foreground and background areas to obtain a foreground image; The second image processing module is used to perform an inversion operation on the mask images of the foreground and background areas; Obtain a background source image, perform image processing operations on the background source image and the mask image after the inversion operation to obtain a background image; perform image processing operations on the foreground image and the background image to obtain a trigger sample; A model training module is used to input the image in the trigger sample into a pre-established background-driven watermark generation module for training, and output a trained background-driven watermark generation module; The image matching module is used to insert the trained background-driven watermark generation module into the pre-established target image processing network model. When the trigger sample is input, the final watermark image is output, and the final watermark image is matched with the target watermark image to verify the image consistency.

10. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and capable of running on the processor, characterized in that: The memory stores a computer program that can be run on the processor. When the processor loads and executes the computer program, the image processing network copyright protection method based on semantic triggering background described in any one of claims 1 to 8 is adopted.

Images