Black box watermark embedding method and black box watermark extracting method for depth model

Abstract

The invention provides a black box watermark embedding method and a black box watermark extraction method for a depth model. The black box watermark embedding method comprises the following steps: acquiring a plurality of original images and an original depth model; based on a Fourier transform method, respectively superposing watermark data in the frequency domain data of each original image to obtain a first image; based on a watermark fusion model, performing deep fusion on the watermark data and the frequency domain data corresponding to each first image to obtain a plurality of second images; performing watermark embedding training on the original depth model according to each second image and each original image to obtain a target depth model, thereby ensuring that the depth model automatically outputs an invisible and robust watermark during normal reasoning; in the black box watermark extraction method, the target image of the suspicious model is acquired, and the watermark extraction is performed on the target image based on the watermark extraction model to obtain the target watermark image, so that the watermark information can be extracted without acquiring the internal parameters of the suspicious model.

Description

Technical Field

[0001] The present application relates to the field of intellectual property rights of deep models, and in particular to a black box watermark embedding method and a black box watermark extraction method for deep models. Background Art

[0002] The wave of large-scale AI model applications is demonstrating disruptive potential in areas such as smart manufacturing, smart healthcare, and intelligent driving. Well-trained large models are often costly, involving hardware, data, algorithmic complexity, and energy consumption. However, deep models face numerous security risks, with numerous infringements, such as model theft and algorithmic plagiarism, posing a serious threat to technological innovation. Against this backdrop, watermarking technology for deep models, situated at the intersection of AI and intellectual property protection, has garnered widespread attention from academia, industry, and government agencies.

[0003] The embedding and extraction of watermark information is a key process in digital watermarking. It involves retrieving the embedded watermark information from digital content to verify ownership or authenticity. In traditional digital watermarking, watermark embedding and extraction are typically based on discrete cosine transforms or discrete wavelet transforms, where the watermark is embedded within the model output image and extracted via the corresponding inverse transform. Compared to traditional digital watermarking techniques, the watermark embedding and extraction process for deep models is more complex and imperceptible. Therefore, when traditional digital watermarking techniques are applied to deep models, the watermark information may be obscured by the deep network's parameters, feature representations, or predicted outputs. Furthermore, pirates may weaken or remove the watermark in deep models through model optimization techniques such as pruning, fine-tuning, and quantization, thereby increasing the difficulty of watermark embedding and extraction. Therefore, how to design an efficient watermark embedding and extraction scheme has become a key research issue. Summary of the Invention

[0004] The main purpose of the embodiments of the present application is to propose a black-box watermark embedding method and a black-box watermark extraction method for deep models, which can ensure that the deep model automatically outputs invisible and robust watermark information during normal reasoning, thereby improving the concealment and stability of the watermark information in the deep model. At the same time, in the watermark extraction stage, the watermark information of the suspicious model can be quickly and accurately extracted without obtaining the internal parameters of the suspicious model, providing reliable and effective data support for the verification of the model watermark.

[0005] To achieve the above objectives, a first aspect of an embodiment of the present application proposes a black box watermark embedding method for a deep model, comprising: Obtain multiple original images and original depth models; Based on the Fourier transform method, watermark data is superimposed on the frequency domain data of each original image to obtain multiple first images containing the target watermark; Based on the watermark fusion model, the watermark data and frequency domain data corresponding to each first image are deeply fused to obtain multiple second images, wherein the second images are used to trigger the original deep model to learn in a supervised manner to embed the target watermark into the output image; According to each second image and each original image, supervised watermark embedding training is performed on the original depth model to obtain a target depth model, wherein each image output by the target depth model is embedded with a target watermark.

[0006] Furthermore, in some embodiments, based on the Fourier transform method, watermark data is superimposed on the frequency domain data of each original image to obtain multiple first images containing target watermarks, including: Performing image preprocessing on each original image to obtain multiple preprocessed images; Converting the preprocessed image from pixel space to frequency domain space using a fast Fourier transform algorithm to obtain a plurality of first image frequency domain matrices; According to the user authorization requirements of the original depth model, watermark data is embedded in the frequency band area corresponding to each first image frequency domain matrix in a manner of amplitude spectrum superposition to obtain multiple second image frequency domain matrices, wherein the frequency band area includes at least one of the following: a high frequency area, a medium frequency area, and a low frequency area; The second image frequency domain matrix is ​​converted from frequency domain space to pixel space by using an inverse Fourier transform algorithm to obtain a plurality of first images.

[0007] Furthermore, in some embodiments, the process of constructing the watermark fusion model includes the following steps: Acquire a plurality of first sample images without watermark; Based on the Fourier transform method, watermark data is superimposed on the frequency domain data of each first sample image to obtain multiple second sample images containing watermarks; Initializing a first generative adversarial network architecture, the first generative adversarial network architecture including a first generator formed based on a UNet neural network and a first discriminator formed based on a PatchGAN neural network, the first generator being communicatively connected to the first discriminator; According to each first sample image and each second sample image, supervised iterative training is performed on the first generative adversarial network architecture to obtain a watermark fusion model.

[0008] Furthermore, in some embodiments, supervised iterative training is performed on the first generative adversarial network architecture based on each first sample image and each second sample image to obtain a watermark fusion model, including: Input each second sample image into the first generative adversarial network architecture, so that the second sample image is forward propagated to the first generator for watermark fusion and calculation of image loss before and after fusion, thereby obtaining multiple fused images with hidden watermarks and first loss values ​​between each fused image and its corresponding second sample image; wherein the loss function of the first generator is an L2 loss function; Inputting each fused image and each first sample image into a first discriminator to calculate the loss between each fused image and the corresponding first sample image, thereby obtaining a plurality of second loss values; wherein the loss function of the first discriminator is a cross entropy loss function; Performing weighted operations on each first loss value and its corresponding second loss value to obtain multiple third loss values; According to each third loss value, backpropagation is performed on the multi-layer convolutional network parameters of the first discriminator and the multi-layer convolutional network parameters of the first generator based on the Adam optimization algorithm; If the multi-layer convolutional network parameters of the first discriminator and the multi-layer convolutional network parameters of the first generator both meet the preset first training conditions, the training of the first generative adversarial network architecture is terminated to obtain a watermark fusion model.

[0009] To achieve the above-mentioned purpose, the second aspect of the embodiments of the present application proposes a black box watermark extraction method for a deep model, comprising: Obtaining a target image output by a suspicious model, where the suspicious model is a target depth model suspected of having stolen the first aspect; Through the watermark extraction model built based on the generative adversarial network algorithm, the watermark of the target image is extracted to obtain the target watermark image; Among them, if the suspicious model is a stolen target depth model, the target watermark image displays the watermark content; if the suspicious model is not a stolen target depth model, the target watermark image displays blank content.

[0010] Furthermore, in some embodiments, the watermark extraction model is constructed by the following steps: Acquire a plurality of third sample images containing watermarks; Initializing a second generative adversarial network architecture, the second generative adversarial network architecture including a second generator formed based on an autoencoder neural network and a second discriminator formed based on a PatchGAN neural network, the second generator being communicatively connected to the second discriminator, the second generator including an encoder formed based on a plurality of residual blocks, a decoder formed based on a plurality of residual upsampling blocks, and a discriminator formed based on a PatchGAN neural network, the encoder being communicatively connected to the discriminator, and the discriminator being communicatively connected to the decoder; According to each third sample image, the second generative adversarial network architecture is iteratively trained in an unsupervised manner to obtain a watermark extraction model. Furthermore, in some embodiments, unsupervised iterative training is performed on the second generative adversarial network architecture based on each third sample image to obtain a watermark extraction model, including: Inputting each third sample image into the second generative adversarial network architecture, so that each third sample image is forward propagated to the second generator for watermark extraction and calculating the image loss before and after extraction, thereby obtaining a plurality of watermarked images and a fourth loss value between each watermarked image and its corresponding third sample image; wherein the loss function of the second generator is an L2 loss function; Inputting each watermark image and each third sample image into the second discriminator to calculate the loss between each watermark image and the corresponding third sample image, thereby obtaining a plurality of fifth loss values; wherein the loss function of the second discriminator is a cross entropy loss function; Performing weighted operations on each fourth loss value and its corresponding fifth loss value to obtain multiple sixth loss values; According to each sixth loss value, backpropagation is performed on the multi-layer convolutional network parameters of the second discriminator and the multi-layer convolutional network parameters of the second generator based on the Adam optimization algorithm; If the multi-layer convolutional network parameters of the second discriminator and the multi-layer convolutional network parameters of the second generator both meet the preset second training conditions, the training of the second generative adversarial network architecture is terminated to obtain a watermark extraction model.

[0011] To achieve the above objectives, a third aspect of the embodiments of the present application proposes a black box watermark processing system for a deep model, comprising: A black-box watermark embedding module, configured to execute the black-box watermark embedding method according to the first embodiment to obtain a target depth model; a black box watermark extraction module, the black box watermark extraction module being configured to execute the black box watermark embedding method according to the second aspect of the present invention to obtain a target watermark image of the suspicious model; The black box watermark verification module is used to determine that the suspicious model is a stolen target deep model if the target watermark image displays watermark content; or, if the target watermark image displays blank content, determine that the suspicious model is not a stolen target deep model.

[0012] To achieve the above-mentioned purpose, the fourth aspect of the embodiments of the present application proposes an electronic device, which includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it implements the black box watermark embedding method of the above-mentioned first aspect embodiment, or implements the black box watermark extraction method of the above-mentioned second aspect embodiment.

[0013] To achieve the above-mentioned purpose, the fifth aspect of the embodiments of the present application proposes a computer-readable storage medium, which stores a program executable by a processor. When the computer program is executed by the processor, it implements the black box watermark embedding method of the above-mentioned first aspect embodiment, or implements the black box watermark extraction method of the above-mentioned second aspect embodiment.

[0014] The embodiments of the first aspect of the present application have the following beneficial effects: through the deep feature coupling mechanism of the frequency domain watermark dynamic embedding based on Fourier transform and the watermark fusion model, combined with the end-to-end supervised training strategy, the comprehensive performance of the watermark embedding technology is significantly improved: first, in the frequency domain, multiple frequency bands are adaptively selected to embed the target watermark (for example, low frequency enhances the anti-compression and cropping ability, and high frequency enhances the concealment), and the frequency domain data and the watermark information are nonlinearly interactively fused through the watermark fusion model to achieve deep binding between the watermark and the output image of the model, which can effectively resist noise attacks, model fine-tuning and adversarial tampering; second, the generative embedding logic of the deep model learning watermark is driven by the original image and the second image in a joint manner, ensuring that the target deep model automatically outputs an invisible and robust watermark during normal inference without the need for additional post-processing modules, taking into account both the main task performance of the model (such as image generation quality) and the stability of the watermark retention; in addition, the method is decoupled from the model architecture and can be adapted to various deep networks such as generative and discriminative networks. It is suitable for multiple scenarios such as image synthesis, classification, and detection, and has strong scalability and cross-domain versatility.

[0015] The embodiments of the second aspect of the present application have the following beneficial effects: the accuracy and practicality of model piracy detection are significantly improved by constructing a watermark extraction model based on a generative adversarial network: first, by utilizing the powerful feature learning and adversarial generation capabilities of the adversarial network, the watermark extraction model can efficiently separate the watermark information from the target image output by the suspicious model, and even when the image is subjected to attacks such as compression, noise interference or local tampering, it can still accurately extract the watermark features to ensure the robustness of the detection results; secondly, by judging whether the target watermark image displays the watermark content, it can automatically and non-invasively identify whether the suspicious model has plagiarized the target deep model, without the need to access the internal parameters or training data of the suspicious model, which greatly reduces the detection threshold of the suspicious model; in addition, the hidden embedding and reliable extraction mechanism of the watermark make it difficult to circumvent piracy, and the extracted watermark content can be directly used as evidence for the verification of the protected model, providing reliable and effective data support for the verification of the model watermark. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 This is an optional flowchart of the black box watermark embedding method for a deep model provided in an embodiment of the present application; Figure 2This embodiment of the present application provides Figure 1 An optional flowchart of step S102; Figure 3 This is another optional flow chart for constructing a watermark fusion model provided in an embodiment of the present application; Figure 4 This is an optional flowchart of training the first generative adversarial network architecture provided in an embodiment of the present application; Figure 5 This is an optional flowchart of the black box watermark extraction method for the deep model provided in the embodiment of the present application; Figure 6 This is another optional flow chart for constructing a watermark extraction model provided in an embodiment of the present application; Figure 7 This is an optional flowchart of training the second generative adversarial network architecture provided in an embodiment of the present application; Figure 8 Schematic diagram of watermarks extracted by the watermark extraction model provided in some embodiments of the present application under different numbers of training rounds; Figure 9 A schematic diagram of the structure of a black box watermark processing system for a deep model provided in some embodiments of the present application; Figure 10 A schematic diagram of the processing process of the black box watermark embedding module and the black box watermark extraction module provided in some embodiments of the present application; Figure 11 This is a schematic diagram of the hardware structure of an electronic device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0017] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0018] In the description of this application, it should be understood that descriptions involving orientations, such as up, down, front, back, left, right, etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings. They are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation. Therefore, they cannot be understood as limitations on this application.

[0019] It should also be noted that, in the description of this application, "several" means more than one, "plurality" means more than two, "greater than," "less than," and "exceed" are understood to exclude the number itself, while "above," "below," and "within" are understood to include the number itself. The use of "first" and "second" in the description is solely for the purpose of distinguishing technical features and should not be construed as indicating or implying relative importance, implicitly specifying the number of the indicated technical features, or implicitly specifying the order of the indicated technical features.

[0020] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application pertains. The terms used herein are for the purpose of describing the embodiments of this application only and are not intended to limit this application.

[0021] In the description of this application, reference to the terms "one embodiment," "some embodiments," "illustrative embodiments," "examples," "specific examples," or "some examples" means that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of this application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in any appropriate manner in any one or more embodiments or examples.

[0022] To facilitate understanding of the technical solutions provided by the embodiments of the present disclosure, some key terms used in the embodiments of the present disclosure are explained here: Deep model watermarking and extraction: Deep model watermarking is used to determine ownership after a model has been stolen, by extracting certain proving information from the model. This typically occurs during post-incident evidence collection. Model watermarking techniques are generally categorized as white-box and black-box, depending on whether internal information about the suspected model is known. If the internal parameters of the suspected model can be obtained, valid information can be extracted to determine ownership. However, this information is often difficult to obtain unless the thief actively cooperates, making black-box watermarking more practical.

[0023] Black-box watermarking: Instead of extracting a watermark from the suspect model, simply input a specific trigger set into the suspect model. Ownership can be determined based on whether the output image contains a pre-defined watermark label. If the model generates a specific watermark label after inputting the trigger data set, it indicates that the suspect model has been embedded with a black-box watermark, confirming ownership of the suspect model. Black-box watermarking primarily involves constructing a specific trigger data set and then using it to train the protected model. The protected model will then learn the trigger pattern and, upon subsequent input of the trigger data, will output a specific watermark label.

[0024] The wave of large-scale AI model applications is demonstrating disruptive potential in areas such as smart manufacturing, smart healthcare, and intelligent driving. Well-trained large models are often costly, involving hardware, data, algorithmic complexity, and energy consumption. However, deep models face numerous security risks, with numerous infringements, such as model theft and algorithmic plagiarism, posing a serious threat to technological innovation. Against this backdrop, watermarking technology for deep models, situated at the intersection of AI and intellectual property protection, has garnered widespread attention from academia, industry, and government agencies.

[0025] The embedding and extraction of watermark information is a key process in digital watermarking. It involves retrieving the embedded watermark information from digital content to verify ownership or authenticity. In traditional digital watermarking, watermark embedding and extraction are typically based on discrete cosine transforms or discrete wavelet transforms, where the watermark is embedded within the model output image and extracted via the corresponding inverse transform. Compared to traditional digital watermarking techniques, the watermark embedding and extraction process for deep models is more complex and imperceptible. Therefore, when traditional digital watermarking techniques are applied to deep models, the watermark information may be obscured by the deep network's parameters, feature representations, or predicted outputs. Furthermore, pirates may weaken or remove the watermark in deep models through model optimization techniques such as pruning, fine-tuning, and quantization, thereby increasing the difficulty of watermark embedding and extraction. Therefore, how to design an efficient watermark embedding and extraction scheme has become a key research issue.

[0026] Based on this, the embodiments of the present application provide a black-box watermark embedding method and a black-box watermark extraction method for deep models, which can ensure that the deep model automatically outputs invisible and robust watermark information during normal reasoning, thereby improving the concealment and stability of the watermark information in the deep model. At the same time, in the watermark extraction stage, it can quickly and accurately extract the watermark information of the suspicious model without obtaining the internal parameters of the suspicious model, providing reliable and effective data support for the verification of the model watermark.

[0027] The embodiments of the present application provide a black box watermark embedding method and a black box watermark extraction method for a deep model, which are specifically described through the following embodiments.

[0028] First, refer to Figure 1 As shown, Figure 1 This is an optional flowchart of a black box watermark embedding method for a deep model provided in an embodiment of the present application. The method may include but is not limited to steps S101 to S104.

[0029] Step S101: Acquire multiple original images and an original depth model.

[0030] The original image is the image data input to the original depth model.

[0031] Step S102: Based on the Fourier transform method, watermark data is superimposed on the frequency domain data of each original image to obtain multiple first images containing the target watermark.

[0032] Specifically, a fast Fourier transform (FFT) is first performed on each original image to convert the spatial domain pixel matrix into a complex frequency domain matrix containing the amplitude spectrum and phase spectrum; then, according to the preset frequency band selection strategy (such as the intermediate frequency priority principle), the watermark data is dynamically superimposed in the specific frequency sub-band. After the frequency domain data modification is completed, the adjusted complex matrix is ​​subjected to an inverse Fourier transform (IFFT) to reconstruct the spatial domain image, and finally a "first image" with lossless visual quality and frequency domain deep binding watermark is generated.

[0033] Step S103: Based on the watermark fusion model, the watermark data and the frequency domain data corresponding to each first image are deeply fused to obtain a plurality of second images.

[0034] The second image is used to trigger the original deep learning model to embed the target watermark into the output image in a supervised manner.

[0035] Specifically, the frequency-domain complex matrix corresponding to the first image and the watermark binary sequence are respectively input into the dual-channel encoder of the watermark fusion model. The frequency-domain data undergoes multi-scale convolution to extract the energy distribution features of different frequency subbands (such as low-frequency global contours and high-frequency texture details), while the watermark data is embedded in a position mask to generate a spatial weight map that matches the frequency band sensitivity. Subsequently, in the feature fusion module, a cross-channel attention mechanism is used to interactively learn the frequency-domain features and watermark features. Frequency-domain channel attention is used to screen frequency band coefficients sensitive to watermarks, and a nonlinear activation function is combined to achieve adaptive weighted fusion of frequency-domain energy and watermark information. Finally, the decoder of the watermark fusion model performs an inverse Fourier transform based on the fused multi-band composite features to generate a second image with consistent visual quality and deep watermark binding to the original image. This not only retains the robust signature of the target watermark, but also makes the watermark embedding process differentiable through joint frequency-spatial optimization. This provides driving samples with both concealment and anti-interference properties for the subsequent supervised training of the deep model, ensuring that the model can stably output invisible but traceable watermarked images during the inference phase.

[0036] Step S104: According to each second image and each original image, supervised watermark embedding training is performed on the original depth model to obtain a target depth model.

[0037] Among them, the image output by the target depth model each time is embedded with the target watermark.

[0038] In one implementation, the original image is used as the input sample, and the corresponding generated second image (fused watermark) is used as the supervision target. Then, a multi-task joint loss function is set to drive the model to learn the generative embedding mechanism of the watermark, so that the original deep model can undergo supervised watermark embedding training to obtain the target deep model.

[0039] Reference Figure 2 As shown, Figure 2 This embodiment of the present application provides Figure 1 An optional flowchart of step S102 in the method may include but is not limited to steps S201 to S204.

[0040] Step S201: performing image preprocessing on each original image to obtain a plurality of preprocessed images.

[0041] Specifically, the above preprocessing steps may include normalizing the image to normalize the pixel values ​​of the image to a specific range, such as [0, 1] or [-1, 1], which can improve the stability of subsequent data processing. Secondly, it may be necessary to resize the image to a uniform size to meet the input size requirements of the model. In addition, data augmentation techniques such as random cropping, flipping or rotation can be used to increase the diversity of the data, thereby improving the generalization ability of subsequent data processing. After these preprocessing steps, the original image is converted into a preprocessed image, which is then used in subsequent data processing to ensure the stability of the watermark data embedding.

[0042] Step S202: converting the pre-processed image from pixel space to frequency domain space by using a fast Fourier transform algorithm to obtain a plurality of first image frequency domain matrices.

[0043] The Fast Fourier Transform (FFT) is used to convert signals from pixel space (either in the time or spatial domain) to the frequency domain. When processing the preprocessed image, the FFT algorithm is applied in step S202 to convert the image's pixel values ​​into frequency components in the frequency domain. This conversion process reveals information at different frequencies within the image. For example, low-frequency components correspond to the image's overall structure and slowly varying areas, while high-frequency components correspond to image details and edge information. Through FFT transformation, each preprocessed image is represented as a first image frequency domain matrix, where elements represent different frequencies and their corresponding amplitude and phase information. These frequency domain matrices not only provide new insights for subsequent image analysis and processing, but can also reveal patterns and features that are difficult to detect in pixel space.

[0044] Step S203: According to the user authorization requirements of the original depth model, watermark data is embedded in the frequency band area corresponding to each first image frequency domain matrix in a manner of amplitude spectrum superposition to obtain multiple second image frequency domain matrices.

[0045] Specifically, based on the user authorization requirements of the original deep model (e.g., only the model's image enhancement function can be used), appropriate frequency band regions are precisely determined in each first image frequency domain matrix. Then, the watermark data is embedded into these selected frequency bands using amplitude spectrum superposition technology. This embedding method not only ensures the invisibility of the watermark but also improves its robustness, enabling it to withstand various common image processing operations such as compression, filtering, and cropping. Ultimately, after the above embedding process, multiple second image frequency domain matrices are obtained. These matrices retain the main features of the original image while also implicitly containing the user's authorization information, thereby achieving effective protection of image copyright and reliable tracking of the image source.

[0046] The frequency band region includes at least one of the following: a high frequency region, a medium frequency region, and a low frequency region.

[0047] Step S204: converting the second image frequency domain matrix from the frequency domain space to the pixel space by using an inverse Fourier transform algorithm to obtain a plurality of first images.

[0048] Specifically, the frequency domain matrix of the watermarked second image is converted from the frequency domain back to the pixel space using an inverse Fourier transform algorithm, yielding multiple first images. This method leverages the efficiency and accuracy of the inverse Fourier transform to ensure that the frequency domain information after the watermark is embedded can be accurately mapped back to the pixel space while preserving the quality and characteristics of the original image.

[0049] Reference Figure 3 As shown, Figure 3 This is another optional flow chart for constructing a watermark fusion model provided in an embodiment of the present application. The process of constructing a watermark fusion model may include but is not limited to steps S301 to S304.

[0050] Step S301: Acquire a plurality of first sample images without watermarks.

[0051] Step S302: Based on the Fourier transform method, watermark data is superimposed on the frequency domain data of each first sample image to obtain a plurality of second sample images containing watermarks.

[0052] It should be noted that this step S302 is similar to the above step S202 and will not be described again. For specific details, please refer to the above step S202.

[0053] Step S303: Initialize the first generative adversarial network architecture.

[0054] Among them, the first generative adversarial network architecture includes a first generator formed based on the UNet neural network and a first discriminator formed based on the PatchGAN neural network, and the first generator is communicatively connected to the first discriminator.

[0055] Specifically, the first step involves defining and configuring the network structures of the first generator and the first discriminator. The first generator uses a UNet network structure, with encoder and decoder modules combined with skip connections to ensure that it can generate images with rich details and accurate structure. The first discriminator uses the PatchGAN architecture, focusing on distinguishing local image features to improve the ability to assess the authenticity of generated images. During the initialization process, the weights of each layer of the network can be initialized, using methods such as Xavier or He initialization to ensure that the network parameters are within the appropriate range at the beginning of training, thereby accelerating network convergence and improving training stability. At the same time, the optimizer can also be configured, selecting an appropriate optimization algorithm (such as the Adam optimizer) and its parameters (such as the learning rate and momentum) to ensure that the network can effectively update parameters during training, gradually improving the quality and authenticity of the generated images. Through these initialization steps, a stable and efficient first generative adversarial network architecture is established.

[0056] Step S304: According to each first sample image and each second sample image, supervised iterative training is performed on the first generative adversarial network architecture to obtain a watermark fusion model.

[0057] Specifically, a first generative adversarial network (GAN) architecture undergoes supervised iterative training based on each first sample image (original image) and each second sample image (target image after watermarking). During training, the generator (e.g., using a UNet architecture) learns how to effectively embed the watermark information into the original image while maintaining image quality and watermark invisibility. The discriminator (e.g., using a PatchGAN architecture) is responsible for distinguishing the generated watermarked image from the true target image. Through adversarial training, the generator continuously improves its watermark embedding performance. A loss function, comprising both adversarial loss and watermark embedding loss, is then used to calculate the training loss, ensuring that the generator accurately embeds the watermark information into the image and that the generated image passes the discriminator's test. After multiple rounds of iterative training, the generator gradually learns the optimal watermark embedding strategy, ultimately resulting in a watermark fusion model capable of efficient and robust watermarking. This model demonstrates significant effectiveness in protecting image copyright and traceability, providing powerful technical support for the secure management and copyright protection of digital content.

[0058] Reference Figure 4 As shown, Figure 4 This is an optional flowchart of training the first generative adversarial network architecture provided in an embodiment of the present application. The training process of the first generative adversarial network architecture may include but is not limited to steps S401 to S405.

[0059] Step S401: Input each second sample image into the first generative adversarial network architecture, so that the second sample image is forward propagated to the first generator for watermark fusion and calculation of the image loss before and after fusion, to obtain multiple fused images with hidden watermarks, and the first loss value of each fused image and its corresponding second sample image.

[0060] Among them, the loss function of the first generator is the L2 loss function.

[0061] Specifically, each second sample image is input into the first generative adversarial network architecture so that the second sample image is forward propagated to the first generator for watermark fusion and calculation of the image loss before and after fusion, thereby obtaining multiple fused images with hidden watermarks, and the first loss value of each fused image and its corresponding second sample image.

[0062] Step S402: Input each fused image and each first sample image into a first discriminator to calculate the loss between each fused image and the corresponding first sample image, thereby obtaining a plurality of second loss values.

[0063] Among them, the loss function of the first discriminator is the cross entropy loss function, which enables the fused image to be identified as the true label, and the unfused first sample image will be identified as a false label.

[0064] Specifically, each fused image (i.e., the image after embedding the watermark) and each first sample image (the original, unwatermarked image) are simultaneously input into the first discriminator. The task of the first discriminator is to distinguish whether the input image is the original image or the fused image and calculate the corresponding loss value by comparing the difference between the two. Specifically, for each pair of fused images and the corresponding first sample image, the discriminator evaluates the difference in their features and calculates a loss value, namely the second loss value, based on this. These second loss values ​​reflect the degree of difference between the fused image and the original image, providing critical feedback information for the training of the generative adversarial network. In this way, the generator can be effectively guided to learn how to better embed the watermark while maintaining image quality and the invisibility of the watermark.

[0065] Step S403: performing weighted operations on each first loss value and its corresponding second loss value to obtain a plurality of third loss values.

[0066] Specifically, by properly assigning weights to each loss value, their influence on the total loss can be adjusted based on training requirements and model objectives. For example, if the quality of the generated image is particularly important, the first loss value can be given a larger weight coefficient. This approach not only balances the contributions of different losses but also guides the model's training direction more flexibly, helping the model to more effectively update and optimize parameters in subsequent iterations, thereby improving overall training results and model performance.

[0067] Step S404: According to each third loss value, backpropagation update is performed on the multi-layer convolutional network parameters of the first discriminator and the multi-layer convolutional network parameters of the first generator based on the Adam optimization algorithm.

[0068] Specifically, leveraging the efficiency and stability of the Adam optimization algorithm, backpropagation updates the network parameters of the first discriminator and the first generator. By combining momentum with an adaptive learning rate, the Adam optimization algorithm effectively accelerates convergence and improves optimization performance. This allows the parameters of the first discriminator and the first generator to be precisely adjusted based on feedback from the third loss value, enabling the first discriminator to better distinguish between real and generated images while simultaneously encouraging the first generator to produce more realistic, higher-quality images.

[0069] Step S405: If the multi-layer convolutional network parameters of the first discriminator and the multi-layer convolutional network parameters of the first generator both meet the preset first training conditions, the training of the first generative adversarial network architecture is terminated to obtain a watermark fusion model.

[0070] Specifically, if the multi-layer convolutional network parameters of the first discriminator and the multi-layer convolutional network parameters of the first generator both meet the preset first training conditions, the training of the first generative adversarial network architecture is terminated to obtain a watermark fusion model.

[0071] Secondly, refer to Figure 5 As shown, Figure 5 This is an optional flowchart of the black box watermark extraction method for a deep model provided in an embodiment of the present application. The method may include but is not limited to steps S501 to S502.

[0072] Step S501: Acquire the target image output by the suspicious model.

[0073] Among them, the suspicious model is the target depth model suspected of plagiarizing the first aspect mentioned above. Step S502: extracting watermarks from the target image using a watermark extraction model constructed based on a generative adversarial network algorithm to obtain a target watermark image.

[0074] Among them, if the suspicious model is a stolen target depth model, the target watermark image displays the watermark content; if the suspicious model is not a stolen target depth model, the target watermark image displays blank content.

[0075] Specifically, the target image is input into the trained watermark extraction model. The generator part of the watermark extraction model will analyze the pixel and frequency characteristics of the image, identify the embedded watermark features, and separate them from the image. Then, after neural network calculation and feature reconstruction process, it will finally generate the target watermark image, which clearly shows the original embedded watermark content.

[0076] Reference Figure 6 As shown, Figure 6 This is another optional flowchart for constructing a watermark extraction model provided in an embodiment of the present application. The process of constructing a watermark extraction model may include but is not limited to steps S601 to S603.

[0077] Step S601: Acquire a plurality of third sample images containing watermarks.

[0078] Step S602: Initialize the second generative adversarial network architecture.

[0079] Among them, the second generative adversarial network architecture includes a second generator formed based on an autoencoder neural network and a second discriminator formed based on a PatchGAN neural network. The second generator is communicatively connected to the second discriminator. The second generator includes an encoder formed based on multiple residual blocks, a decoder formed based on multiple residual upsampling blocks, and a discriminator formed based on a PatchGAN neural network. The encoder is communicatively connected to the discriminator, and the discriminator is communicatively connected to the decoder.

[0080] In one embodiment, the network architecture of the second generator and the first discriminator is first defined and configured. The second generator employs an autoencoder network structure, with encoder and decoder modules combined with skip connections to ensure the generation of images with rich details and accurate structure. A PatchGAN architecture discriminator is placed between the encoder and decoder to enhance the learning capabilities of the second generator. The second discriminator employs the PatchGAN architecture, focusing on distinguishing local image features to improve the ability to assess the authenticity of generated images. During initialization, weights of each layer of the network can be initialized using methods such as Xavier or He initialization to ensure that network parameters are within an appropriate range at the beginning of training, thereby accelerating network convergence and improving training stability. Furthermore, the optimizer can be configured, selecting an appropriate optimization algorithm (such as the Adam optimizer) and its parameters (such as the learning rate and momentum) to ensure that the network can effectively update parameters during training, gradually improving the quality and authenticity of generated images. Through these initialization steps, a stable and efficient second generative adversarial network architecture is achieved.

[0081] In one embodiment, the encoder of the second generator includes 3 residual blocks with a stride of 2, and the decoder of the second generator includes 3 residual upsampling blocks to ensure that the output resolution of the second generator is the same as the input resolution.

[0082] Step S603: performing unsupervised iterative training on the second generative adversarial network architecture according to each third sample image to obtain a watermark extraction model.

[0083] Specifically, during training, the second generator learns how to extract the hidden watermark information from the image, while the second discriminator is responsible for evaluating whether the extracted watermark is authentic and complete. Then, through a loss function, the second generator and the second discriminator compete with each other during training, continuously optimizing their respective performance. The second generator gradually learns how to effectively separate the watermark features from the image, while the second discriminator continuously improves its ability to distinguish between real and extracted watermarks. After multiple rounds of iterative training, the second generator is eventually able to accurately extract the embedded watermark from the image, resulting in an efficient and robust watermark extraction model.

[0084] Reference Figure 7 As shown, Figure 7 This is an optional flowchart of training the second generative adversarial network architecture provided in an embodiment of the present application. The process of training the second generative adversarial network architecture may include but is not limited to steps S701 to S705.

[0085] Step S701: Input each third sample image into the second generative adversarial network architecture, so that each third sample image is forward propagated to the second generator for watermark extraction and calculation of image loss before and after extraction, to obtain multiple watermark images, and a fourth loss value between each watermark image and its corresponding third sample image.

[0086] Among them, the loss function of the second generator is the L2 loss function.

[0087] Specifically, each third sample image is input into the second generative adversarial network architecture, so that each third sample image is forward propagated to the second generator for watermark extraction and calculation of image loss before and after extraction, thereby obtaining multiple watermark images and a fourth loss value between each watermark image and its corresponding third sample image.

[0088] Step S702: Input each watermark image and each third sample image into the second discriminator to calculate the loss between each watermark image and the corresponding third sample image to obtain a plurality of fifth loss values.

[0089] Among them, the loss function of the second discriminator is the cross entropy loss function, through which the extracted watermark information can be identified as a true label.

[0090] Step S703: performing weighted operations on each fourth loss value and its corresponding fifth loss value to obtain a plurality of sixth loss values.

[0091] Specifically, by properly assigning weights to each loss value, their influence on the total loss can be adjusted based on training requirements and model objectives. For example, if the quality of the discriminant image is particularly important, the fifth loss value can be given a larger weight coefficient. This approach not only balances the contributions of different losses but also guides the model's training direction more flexibly, helping the model to more effectively update and optimize parameters in subsequent iterations, thereby improving overall training results and model performance.

[0092] Step S704: According to each sixth loss value, backpropagation update is performed on the multi-layer convolutional network parameters of the second discriminator and the multi-layer convolutional network parameters of the second generator based on the Adam optimization algorithm.

[0093] Specifically, leveraging the efficiency and stability of the Adam optimization algorithm, backpropagation updates the network parameters of the second discriminator and second generator, respectively. By combining momentum with an adaptive learning rate, the Adam optimization algorithm effectively accelerates convergence and improves optimization performance. This allows the parameters of the second discriminator and second generator to be precisely adjusted based on feedback from the sixth loss value, enabling the first discriminator to better distinguish between real and generated images while simultaneously encouraging the first generator to produce more realistic, higher-quality images.

[0094] Step S705: If the multi-layer convolutional network parameters of the second discriminator and the multi-layer convolutional network parameters of the second generator both meet the preset second training conditions, the training of the second generative adversarial network architecture is terminated to obtain a watermark extraction model.

[0095] In some embodiments, the second training condition includes the number of training rounds of the watermark extraction model, referring to Figure 8 As shown, Figure 8 Schematic diagram of watermarks extracted by the watermark extraction model provided in some embodiments of the present application under different numbers of training rounds. For different numbers of extraction rounds, the higher the number of extraction rounds, the more complete the training of the watermark extraction model, and the closer the watermark image extracted by the watermark extraction model is to the watermark image embedded by the watermark embedding model in the first aspect above. Figure 8 It can be seen that when the number of training rounds of the watermark extraction model reaches 200 rounds, it can be determined that the watermark image extracted by the watermark extraction model is highly consistent with the watermark image embedded by the watermark embedding model of the first aspect.

[0096] In some embodiments, experiments were conducted on the network structures of classic classification models (e.g., VGG19, Restnet20, Resnet56, Resenet110, and WideResnet40) to verify the feasibility of the above watermark extraction model. The black-box watermark extraction method is not limited to the second generative adversarial network architecture of this application, but is also applicable to the network structures of other classification models. When training the watermark extraction model, the learning rate of the Adam optimizer was set to 0.01, and the batch_size parameter was set to 64. In the weighting of the loss function, the weight coefficient of the newly added category was set to 3, and the weight coefficient of the original category was set to 1.

[0097] In the above experiments, the performance of the proposed solution was primarily measured using the following three metrics: SSIM (Structural Similarity Index), PSNR (Peak Signal-to-Noise Ratio), and ESR (Watermark Extraction Success Rate). For these metrics, the feasibility of the watermark extraction model is verified if the following conditions are met: 1. The ESR of the watermark extraction model's output image is higher than 90%. This ESR higher than 90% intuitively reflects the watermark's extractability and indicates a high extraction success rate for the model. 2. The PSNR and SSIM between the extracted and embedded watermarked images are high. If only the whiteboard image is extracted when no watermark is embedded, and the watermark is realistically restored when a watermark is embedded, then the watermark extraction model can distinguish whether a watermark is embedded.

[0098] The Resnet56 model's network structure and the Cifar10 dataset were used as examples to explore the impact of different selected frequency regions on extraction performance. The maximum value of the elements in the Resnet56 model's Fourier error heatmap is 0.7776, and the minimum is 0.2782. Therefore, the frequency bands [0.2, 0.4], [0.4, 0.5], [0.5, 0.6], [0.6, 0.7], and [0.7, 0.8] were selected, and the watermark extraction model was trained for 200 rounds. Table 1 shows the performance scores of the watermark extraction models obtained when training with different classification model network structures.

[0099] Table 1 Extraction performance of different frequency bands

[0100] As can be seen from Table 1, the extraction success rate of the watermark extraction model formed by the network structure of the Resnet56 model generally reaches more than 98%, and the extracted watermark image and the embedded watermark image are highly consistent in visual effects. Among them, the PSNR is greater than 35 and the SSIM is close to 1, indicating that the similarity between the extracted watermark image and the embedded watermark image is higher.

[0101] In a third aspect, the present application also provides a black box watermark processing system for a deep model, referring to Figure 9 , Figure 9 This is a schematic diagram of the structure of a black-box watermark processing system for a deep model provided in some embodiments of the present application. The black-box watermark processing system 900 includes: A black box watermark embedding module 901 is configured to execute the black box watermark embedding method according to the first embodiment to obtain a target depth model. A black box watermark extraction module 902 is configured to execute the black box watermark embedding method according to the second embodiment to obtain a target watermark image of the suspicious model; The black box watermark verification module 903 is communicated with the black box watermark extraction module 902 and the black box watermark embedding module 901 respectively. The black box watermark verification module is used to determine that the suspicious model is a stolen target depth model if the target watermark image displays watermark content; or if the target watermark image displays blank content, determine that the suspicious model is not a stolen target depth model.

[0102] It should be noted that the reference Figure 10 As shown, Figure 10 A schematic diagram of the processing process of the black box watermark embedding module and the black box watermark extraction module provided for some embodiments of the present application, wherein the black box watermark processing system 900 executes the black box watermark embedding method of the first embodiment mentioned above through the black box watermark embedding module 901 to obtain the target depth model, and then executes the black box watermark extraction method of the second embodiment mentioned above through the black box watermark extraction module 902 to obtain the target watermark image of the suspicious model, and finally enables the black box watermark verification module 903 to verify that if the target watermark image displays watermark content, the suspicious model is determined to be a stolen target depth model; or, if the target watermark image displays blank content, the suspicious model is determined not to be a stolen target depth model, and then a symmetrical watermark embedding and extraction method can be used to ensure that the user's ownership is only verified by a unique watermark, thereby improving the fairness and persuasiveness of the watermark verification, and ensuring that the embedded watermark information can be stably back-verified.

[0103] The present application also provides an electronic device comprising a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it implements the black box watermark embedding method of the first embodiment or the black box watermark extraction method of the second embodiment. The electronic device can be any smart terminal, including a mobile phone, a tablet computer, and an in-vehicle computer.

[0104] See also Figure 11 , Figure 11 This is a schematic diagram of the hardware structure of an electronic device provided in one embodiment of the present application, the electronic device comprising: The processor 1101 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the black-box watermark embedding method of the first embodiment or the black-box watermark extraction method of the second embodiment provided in the embodiments of the present application; The memory 1102 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1102 can store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 1102, and the processor 1101 calls and executes the black box watermark embedding method of the first embodiment provided in the embodiments of this application, or implements the black box watermark extraction method of the second embodiment; Input / output interface 1103, used to implement information input and output; Communication interface 1104, used to implement communication interaction between this device and other devices, which can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WiFi, Bluetooth, etc.); Bus 1105 , which transmits information between various components of the device (e.g., processor 1101 , memory 1102 , input / output interface 1103 , and communication interface 1104 ); The processor 1101 , the memory 1102 , the input / output interface 1103 and the communication interface 1104 are connected to each other in communication within the device via a bus 1105 .

[0105] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the black box watermark embedding method of the first embodiment provided by the embodiment of the present application or the black box watermark extraction method of the second embodiment is implemented.

[0106] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0107] The embodiments described in the embodiments of this application are intended to more clearly illustrate the technical solutions of the embodiments of this application and do not constitute a limitation on the technical solutions provided by the embodiments of this application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0108] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0109] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, i.e., they may be located in one place or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of this embodiment.

[0110] Those skilled in the art will appreciate that all or some of the steps in the methods, systems, and functional modules / units in the devices disclosed above may be implemented as software, firmware, hardware, or appropriate combinations thereof.

[0111] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0112] It should be understood that in this application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0113] In the several embodiments provided in this application, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative. For example, the division of the above units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0114] The units described above as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0115] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0116] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-accessible storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes multiple instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, and other media that can store programs.

[0117] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but are not intended to limit the scope of the present invention. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and essence of the present invention should be within the scope of the present invention.

Claims

1. A black box watermark embedding method for deep models, characterized in that include: Obtain multiple original images and original depth models; Based on the Fourier transform method, watermark data is superimposed on the frequency domain data of each of the original images to obtain multiple first images containing target watermarks; Based on the watermark fusion model, the watermark data and frequency domain data corresponding to each of the first images are deeply fused to obtain multiple second images, wherein the second images are used to trigger the original deep model to learn in a supervised manner to embed the target watermark into the output image; According to each of the second images and each of the original images, supervised watermark embedding training is performed on the original depth model to obtain a target depth model, wherein each image output by the target depth model is embedded with the target watermark.

2. The black box watermark embedding method according to claim 1, characterized in that: The method of superimposing watermark data on the frequency domain data of each original image based on the Fourier transform method to obtain multiple first images containing target watermarks includes: Performing image preprocessing on each of the original images to obtain a plurality of preprocessed images; Convert the pre-processed image from pixel space to frequency domain space using a fast Fourier transform algorithm to obtain a plurality of first image frequency domain matrices; According to the user authorization requirement of the original depth model, watermark data is embedded in the frequency band area corresponding to each of the first image frequency domain matrices in a manner of amplitude spectrum superposition to obtain multiple second image frequency domain matrices, wherein the frequency band area includes at least one of the following: a high frequency area, a medium frequency area, and a low frequency area; The frequency domain matrix of the second image is converted from the frequency domain space to the pixel space by using an inverse Fourier transform algorithm to obtain a plurality of the first images.

3. The black box watermark embedding method according to claim 1, characterized in that: The construction process of the watermark fusion model, The following steps are included: Acquire a plurality of first sample images without watermark; Based on the Fourier transform method, watermark data is superimposed on the frequency domain data of each of the first sample images to obtain a plurality of second sample images containing watermarks; Initializing a first generative adversarial network architecture, the first generative adversarial network architecture including a first generator formed based on a UNet neural network and a first discriminator formed based on a PatchGAN neural network, the first generator being communicatively connected to the first discriminator; According to each of the first sample images and each of the second sample images, supervised iterative training is performed on the first generative adversarial network architecture to obtain the watermark fusion model.

4. The black box watermark embedding method according to claim 3, characterized in that: The step of performing supervised iterative training on the first generative adversarial network architecture according to each of the first sample images and each of the second sample images to obtain the watermark fusion model includes: Inputting each second sample image into the first generative adversarial network architecture so that the second sample image is forward propagated to the first generator for watermark fusion and calculating the image loss before and after fusion, thereby obtaining a plurality of fused images with hidden watermarks and a first loss value between each fused image and its corresponding second sample image; wherein the loss function of the first generator is an L2 loss function; Inputting each of the fused images and each of the first sample images into the first discriminator to calculate the loss between each of the fused images and the corresponding first sample image, thereby obtaining a plurality of second loss values; wherein the loss function of the first discriminator is a cross entropy loss function; Performing weighted operations on each first loss value and its corresponding second loss value to obtain multiple third loss values; According to each of the third loss values, backpropagation update is performed on the multi-layer convolutional network parameters of the first discriminator and the multi-layer convolutional network parameters of the first generator based on the Adam optimization algorithm; If the multi-layer convolutional network parameters of the first discriminator and the multi-layer convolutional network parameters of the first generator both meet the preset first training conditions, the training of the first generative adversarial network architecture is terminated to obtain the watermark fusion model.

5. A black box watermark extraction method for deep models, characterized in that include: Obtaining a target image output by a suspicious model, wherein the suspicious model is suspected of misappropriating the target depth model according to any one of claims 1 to 4; Extracting watermarks from the target image using a watermark extraction model built based on a generative adversarial network algorithm to obtain a target watermark image; Among them, if the suspicious model is a theft of the target depth model, the target watermark image displays the watermark content; if the suspicious model is not a theft of the target depth model, the target watermark image displays blank content.

6. The black box watermark extraction method according to claim 5, characterized in that: The construction process of the watermark extraction model is obtained through the following steps: Acquire a plurality of third sample images containing watermarks; Initializing a second generative adversarial network architecture, the second generative adversarial network architecture including a second generator formed based on an autoencoder neural network and a second discriminator formed based on a PatchGAN neural network, the second generator being communicatively connected to the second discriminator, the second generator including an encoder formed based on multiple residual blocks, a decoder formed based on multiple residual upsampling blocks, and a discriminator formed based on a PatchGAN neural network, the encoder being communicatively connected to the discriminator, and the discriminator being communicatively connected to the decoder; According to each of the third sample images, unsupervised iterative training is performed on the second generative adversarial network architecture to obtain the watermark extraction model.

7. The black box watermark extraction method according to claim 6, characterized in that: According to each of the third sample images, unsupervised iterative training is performed on the second generative adversarial network architecture to obtain the watermark extraction model, including: Inputting each of the third sample images into the second generative adversarial network architecture, so that each of the third sample images is forward-propagated to the second generator for watermark extraction and calculation of image loss before and after extraction, thereby obtaining a plurality of watermarked images and a fourth loss value between each of the watermarked images and the corresponding third sample image; wherein the loss function of the second generator is an L2 loss function; Inputting each of the watermarked images and each of the third sample images into the second discriminator to calculate the loss between each of the watermarked images and the corresponding third sample images, thereby obtaining a plurality of fifth loss values; wherein the loss function of the second discriminator is a cross entropy loss function; Performing weighted operations on each fourth loss value and its corresponding fifth loss value to obtain multiple sixth loss values; According to each of the sixth loss values, backpropagation update is performed on the multi-layer convolutional network parameters of the second discriminator and the multi-layer convolutional network parameters of the second generator based on the Adam optimization algorithm; If the multi-layer convolutional network parameters of the second discriminator and the multi-layer convolutional network parameters of the second generator both meet the preset second training conditions, the training of the second generative adversarial network architecture is terminated to obtain the watermark extraction model.

8. A black box watermark processing system for deep models, characterized in that include: A black box watermark embedding module, configured to execute the black box watermark embedding method according to any one of claims 1 to 4 to obtain a target depth model; a black box watermark extraction module, the black box watermark extraction module being configured to execute the black box watermark embedding method according to any one of claims 5 to 7 to obtain a target watermark image of the suspicious model; A black box watermark verification module is used to determine that the suspicious model has stolen the target depth model if the target watermark image displays watermark content; or to determine that the suspicious model has not stolen the target depth model if the target watermark image displays blank content.

9. An electronic device, characterized in that: The electronic device includes a memory and a processor, the memory stores a computer program, and when the processor executes the computer program, it implements the black box watermark embedding method according to any one of claims 1 to 4, or implements the black box watermark extraction method according to any one of claims 5 to 7.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a program executable by a processor, and when the program executable by the processor is executed by the processor, the black box watermark embedding method according to any one of claims 1 to 4 is implemented, or the black box watermark extraction method according to any one of claims 5 to 7 is implemented.

Images