A method for generating a poisoned image and a black-box classification model fingerprint watermarking method

Abstract

The application discloses a kind of toxic image generation method and black box classification model fingerprint watermarking method, it is related to neural network model protection and computer vision technical field, the method includes using DWT-DCT-SVD method to diffuse fingerprint watermark into whole image, and DCT coefficient is encrypted.To solve the problem of image quantity imbalance and class imbalance, a toxic feature enhancement module is introduced to improve the reliability and fidelity of the watermark. By combining the toxic feature enhancement module with the original classification module, a fingerprint watermark with low embedding strength can be effectively obtained. The fingerprint watermark in the toxic image generated by this method has excellent concealment. Even in the case of a 1-bit difference in the fingerprint watermark, copyright verification can still be accurately performed. Moreover, it shows strong robustness against various model watermark attacks.

Description

Technical Field

[0001] This invention relates to the technical fields of model protection and computer vision, and more specifically, to a method for generating poisoned images and a method for fingerprint watermarking of black-box classification models. Background Technology

[0002] In recent years, Deep Neural Network (DNN) models have achieved tremendous success in various fields. For example, from traditional computer science fields such as image processing, video processing, and natural language processing, to interdisciplinary applications such as medical image processing, cross-media retrieval, and pedestrian detection, DNNs have consistently outperformed traditional methods. However, the construction, optimization, and training of DNNs require profound expertise, massive amounts of training data, and enormous computational resources. Therefore, trained DNN models have become significant assets, and with the increasing complexity of Machine Learning as a Service (MLaaS) model building, copyright protection for DNNs has become particularly necessary. To address this, neural network model watermarking technology has emerged. This technology embeds watermark information into the DNN model to detect and protect its copyright.

[0003] Based on whether the structure and parameters of the DNN model are publicly available, neural network model watermarking can be divided into two types: white-box model watermarking and black-box model watermarking. White-box model watermarking embeds watermark information by modifying the parameters of the DNN model, allowing for extraction of the watermark information from a known model for copyright verification when necessary. Black-box model watermarking uses backdoor techniques to protect the copyright of neural network models. This technique first modifies a portion of the original input data—a process called poisoning; then, during training, it manipulates the model to produce specific outputs to the poisoned input data; finally, it detects the presence of the watermark by judging whether the model produces specific outputs to the poisoned input data, thus verifying the model's copyright ownership. Compared to white-box model watermarking, black-box model watermarking does not require knowledge of the detailed parameters of the DNN model during verification; it can complete watermark verification using only the input-output data pairs.

[0004] Existing technology provides a plug-and-play black-box model watermark embedding method and device. The method includes: Step 1, generating a watermark detection network M_pty and a watermark verification sample X_v; Step 2, embedding the watermark detection network M_pty obtained in Step 1 into the target network M_target to be protected. This invention achieves watermark embedding by inserting a trained dedicated watermark verification model into the target model, without needing to fine-tune the parameters of the target model. Existing black-box model watermarking methods generate poisoned images in a relatively simple way, resulting in significant modifications to the original image, which are highly likely to be detected manually; or the modifications are smaller but require training an auxiliary network, incurring additional overhead. Due to the high cost of neural network training, the neural network may be distributed to different users, but current neural network watermarking methods struggle to embed different watermarks for different users, making it impossible to identify the specific pirate in multi-user distribution scenarios. Summary of the Invention

[0005] To overcome the shortcomings of poor quality of poisoned images generated by existing watermarking models and the single training strategy, this invention provides a method for generating poisoned images and a fingerprint watermarking method for black-box classification models.

[0006] To solve the above-mentioned technical problems, the technical solution of the present invention is as follows:

[0007] This invention provides a method for generating poisoned images, the method comprising:

[0008] S11: Obtain the original image and the fingerprint watermark sequence to be embedded;

[0009] S12: Perform channel conversion on the original image to obtain a three-channel image; based on the fingerprint watermark sequence to be embedded, construct a fingerprint watermark sub-sequence corresponding to the three-channel image;

[0010] S13: For each channel image, DWT transformation, DCT transformation, scrambling encryption and SVD decomposition are performed in sequence. Each channel image obtains several singular values, and the embedding carrier is determined according to the size of the singular values.

[0011] S14: After embedding the fingerprint watermark subsequence into the embedding carrier of each channel image, the channel with the embedded fingerprint watermark is obtained. Inverse SVD, scrambling decryption, inverse DCT transformation and inverse DWT transformation are performed in sequence to obtain a three-channel image with the embedded fingerprint watermark.

[0012] S15: Perform RGB conversion on the three-channel image with embedded fingerprint watermark to obtain a poisoned image.

[0013] Preferably, the specific method of S12 is as follows:

[0014] Let the original image be denoted as I. i The image is then transformed to obtain a three-channel image, including a Y-channel image, a U-channel image, and a V-channel image; the fingerprint watermark sequence to be embedded is denoted as... In the formula m q Let represent the q-th fingerprint watermark in the fingerprint watermark sequence to be embedded; construct the corresponding fingerprint watermark sub-sequences to be embedded for the Y-channel image, U-channel image, and V-channel image, denoted as respectively. and Fingerprint watermark sub-sequence to be embedded and for subsequence.

[0015] Preferably, the specific method of S13 is as follows:

[0016] S131: Perform DWT transformation on the Y channel image to obtain the low-frequency subband;

[0017] S132: Perform DCT transformation on the low-frequency subband to obtain a set of DCT coefficient matrices;

[0018] S133: Use a preset key KEY to scramble and encrypt the set of DCT coefficient matrices to obtain a set of scrambled and encrypted DCT coefficient matrices.

[0019] S134: Perform SVD decomposition on each set of scrambled and encrypted DCT coefficient matrices to obtain several singular values ​​corresponding to the Y channel image.

[0020] S135: Sort several singular values ​​corresponding to the Y channel image according to their magnitude, and select the top two singular values ​​S1 and S2 as the embedding carriers of the Y channel image.

[0021] S136: Repeat steps S131-S135 for the U-channel image and the V-channel image to obtain the embedding vector corresponding to the U-channel image and the embedding vector corresponding to the V-channel image.

[0022] Preferably, in step S133, the method for obtaining a set of scrambled and encrypted DCT coefficient matrices is as follows:

[0023] I. Set the key KEY as the random seed to initialize the random function;

[0024] II. A random function is used to generate the index matrix for scrambling, and the size of the index matrix is ​​the same as the size of the DCT coefficient matrix.

[0025] III. Using the index matrix, adjust the positions of the coefficients in the DCT system matrix to obtain a set of scrambled and encrypted DCT coefficient matrices.

[0026] Preferably, in step S14, the specific method for embedding the fingerprint watermark sub-sequence into the embedding carrier of each channel image is as follows:

[0027] For the Y-channel image, U-channel image, and V-channel image, the fingerprint watermark subsequence and Binary fingerprint watermark bits m b Each singular value S1 and S2, which serve as the embedding carrier, are embedded one by one;

[0028]

[0029]

[0030] Where d1 and d2 are the watermark embedding strengths, S1' and S2' represent the singular values ​​of the embedded binary fingerprint watermark bits, and m b This is a binary fingerprint watermark bit.

[0031] Preferably, in order to improve the performance of the poisoned image, the present invention imposes the following three constraints on the generation of the poisoned image, namely...

[0032] Apply the following three constraints, namely

[0033] (1) Stealth: This refers to ensuring that the difference between the poisoned image and the original image is within a small range, thereby reducing the probability of the fingerprint watermark being detected or forged. Specifically, if the visual similarity between two samples is measured using the visual effect distance function V, then the stealth requirement is to find a suitable poisoned image constructor such that the difference between the poisoned image and the original image is less than a threshold ε, i.e.

[0034]

[0035] Where V is the visual distance function. To construct the poison image, I i This is the original image.

[0036] (2) Sample Specificity: Traditional poisoned images are typically generated by superimposing a small square onto the original image. This method of generating poisoned images is consistent across all images, making it easy to detect and even forge. Therefore, each poisoned image needs to be generated differently, meaning the difference between the poisoned image and the original image needs to be closely related to the original image. In other words, relative to the original image I... i The difference needs to be I i The function is d(I) i ), and subtle differences in the value d′(I) i This can lead to different classification results, as shown in the formula:

[0037]

[0038] Among them, f Θ For classification neural network models, I i For the original image, η(c) j ) specifies the target category.

[0039] (3) No additional auxiliary training required: Many existing neural network model watermarking methods in the literature utilize auxiliary models to generate poisoned images. For example, DFST uses the auxiliary model CycleGAN, and LIRA uses the auxiliary model Unet to generate poisoned images. The effectiveness of these poisoned image generation methods depends on the training of the auxiliary models, so their performance is not stable enough and they have high additional training overhead. To solve these problems, this invention can generate poisoned images without additional auxiliary training.

[0040] This invention also provides a fingerprint watermarking method based on a black-box classification model using poisoned images, the method comprising:

[0041] S21: Obtain the original image, the poisoned image, and the poisoning adversarial image; the poisoned image is generated using the above-described poisoned image generation method; the poisoning adversarial image is generated by embedding a random error fingerprint watermark into the original image;

[0042] S22: Construct a black-box fingerprint watermark classification model and set the total loss function for black-box fingerprint watermark classification;

[0043] S23: The original image, the poisoned image, and the poisoned adversarial image are used to train the constructed black-box fingerprint watermark classification model to obtain the trained black-box fingerprint watermark classification model.

[0044] S24: Obtain the image used for verification, input it into the black-box fingerprint watermark classification model, and obtain the classification result of the verification image.

[0045] Preferably, the black-box fingerprint watermark classification model includes: an input module, a feature extraction module, a poisoning feature enhancement module, and an original classification module;

[0046] The output of the input module is connected to the input of the feature extraction module, and the output of the feature extraction module is connected to the inputs of both the poisoning enhancement module and the original classification module. The poisoning feature enhancement module is equipped with a poisoning feature enhancement loss, and the original classification module is equipped with an original classification loss. The poisoning feature enhancement loss and the original classification loss together constitute the total loss function for black-box fingerprint watermark classification.

[0047] Preferably, if the output target category is the same as η(c) j If the values ​​match, the model copyright verification passes; otherwise, it fails. Let p... chk The probability that a fingerprint watermark belongs to a specified category in a black-box classification model is then...

[0048]

[0049] Where Ind is the indicator function, I i For the original image, For the fingerprint watermark sequence to be embedded, when p chk If the threshold is greater than 80%, then ownership of the model is proven.

[0050] Preferably, the black-box fingerprint watermark classification model needs to be able to classify the target object in the original image, verify the correct fingerprint watermark, and verify the incorrect fingerprint watermark. The formula is as follows:

[0051]

[0052] in, This is an incorrect fingerprint watermark sequence. For the fingerprint watermark sequence to be embedded, f Θ For classification neural network models, Construct the poison image constructor, η(c j ) represents the target category, and Θ represents the neural network parameters.

[0053] Preferably, this invention introduces the concept of fingerprint watermarking in a DNN classification model, achieving fingerprint watermarking by generating poisoned images with different copyright watermark information for different users. To improve robustness, the fingerprint watermark information is embedded into the coefficients of the original image after DWT transform, DCT transform, and SVD decomposition, and the DCT coefficient matrix of the embedding process is encrypted using a key to enhance security. Furthermore, to improve the performance of the fingerprint watermark classification model, the inventors propose a poisoning feature enhancement module to guide model performance improvement from a macroscopic perspective. This module can be easily removed from the network without degrading network performance, improving flexibility and concealment. The invention generates high-quality poisoned images by embedding fingerprint watermarks into the DWT-DCT-SVD transform coefficients, improving robustness and making the presence of the model watermark difficult to detect; it balances fidelity and reliability by adjusting the embedding strength and poisoning rate of the fingerprint watermark information; it uses a key to encrypt the DCT coefficient matrix of the embedding process to improve unforgeability; it adds a separable poisoning feature enhancement module to improve model watermark performance; and it utilizes an adversarial training-based poisoning training method to obtain the fingerprint model watermark. Therefore, this invention achieves a good compromise among factors such as fidelity, reliability, non-counterfeitability, and robustness.

[0054] Preferably, the proportion of the poisoned image set used to construct the training set is 40%, and the proportion of the poisoned adversarial image set used to construct the corresponding erroneous watermark information is 10%. During training, if the input image is an image from the image set to be poisoned, the image is directly input into the black-box fingerprint watermark classification model. Simultaneously, a poisoned image with embedded correct watermark information is generated using the poisoned image generation method proposed in this invention and input into the black-box fingerprint watermark classification model. If the image also belongs to the poisoned adversarial image set, a random erroneous watermark information is embedded to generate a poisoned adversarial image, which is then input into the black-box fingerprint watermark classification model. Subsequently, these generated images are used to train a black-box classification model watermark with fingerprint watermarking functionality. For the deep features output by the classification model, on one hand, they are fed into the poisoned feature enhancement module to generate probabilities belonging to three sets of functions: conventional image target object classification, correct fingerprint model watermark verification, and adversarial fingerprint model watermark verification. On the other hand, they are fed into the classification module to generate probabilities that the feature belongs to the conventional image target category, the correct fingerprint model watermark, or the adversarial fingerprint model watermark. Finally, the corresponding cross-entropy is calculated using these probabilities, and the weighted sum of these cross-entropy losses guides the backpropagation of the model proposed in this invention.

[0055] Preferably, when verifying watermark copyright, the original image without fingerprint watermark, the poisoned image with correct fingerprint watermark, or the poisoned adversarial image with incorrect fingerprint watermark is input into the classification model to be tested. If a certain number of poisoned images are continuously input, and the abnormal classification probability of the classification module exceeds a certain threshold, the fingerprint of the model can be verified and the corresponding copyright owner can be traced.

[0056] Preferably, existing black-box classification model watermarking methods in the literature achieve classification through the final fully connected layer of a deep neural network. This achieves relatively high accuracy in classifying original images and relatively high reliability in verifying poisoned images. However, since the number of original images is always significantly greater than the number of poisoned images, this imbalance between training samples affects the performance of the model watermarking in copyright verification. Moreover, the large number of categories means that the loss calculated by the loss function cannot accurately represent the discrimination error between the original image category and the poisoned image category, making it difficult to accurately guide the model parameters to be updated during backpropagation, thus affecting the final performance of the fingerprint watermarking classification model method.

[0057] To address these issues, a poisoning feature enhancement module is added to the fingerprint watermarking classification model. Specifically, the original image can be treated as a single class without further classification, as can the poisoned image and adversarial images containing erroneous fingerprint watermarks. This minimizes the imbalance between the number of original images, poisoned images, and adversarial images. Furthermore, by using the error function of these three groups of judgment results, the intra-class distances of the three groups (original images, poisoned images, and adversarial images) can be minimized while the inter-class distances are maximized, thereby improving the performance of the black-box neural network model watermarking. The proposed poisoning feature enhancement module primarily assists in model training to achieve better performance. This module can be removed when performing model classification and copyright verification functions without affecting practical use, thus providing good flexibility and not reducing the security performance of the model watermarking.

[0058] Preferably, the method for generating the poisoning adversarial image is the same as that for the poisoning image, except that the embedded fingerprint watermark sequence is not... Rather, it is not equal to of Let the length of the binary fingerprint watermark sequence be L (L>0), then there are a total of 2 L A fingerprint watermark sequence, one of which is the correct fingerprint watermark sequence to be embedded. The remaining 2 L -1 represents erroneous fingerprint watermark information that can cause interference, and it is also the fingerprint watermark information used to generate poisoning adversarial images; the corresponding generated poisoning adversarial images are collectively referred to as...

[0059] Preferably, the method proposed in this invention employs a poisoned image generation method based on DWT-DCT-SVD, resulting in a poisoned image that closely resembles the original image. While this method can produce very good poisoned images, the excessive similarity between the generated images and the original images makes training extremely difficult, even preventing the proper training of deep classification network models. Furthermore, existing black-box classification models for watermarking typically use a cross-entropy loss function and train the model using both the original and poisoned images. However, this often only learns simple pixel-level features and struggles to learn the fingerprint watermark information embedded in the poisoned image, making it difficult for the trained classification model to perform copyright verification and traceability.

[0060] To address these issues, this invention employs an adversarial training model. Specifically, this invention adds poisoned adversarial images with incorrectly embedded fingerprint watermarks to the training dataset, allowing the fingerprint watermark classification model, which is to be embedded with the correct fingerprint watermark, to be trained and optimized simultaneously on the original image, the poisoned image, and the poisoned adversarial image. This adversarial training strategy, combined with a poisoning feature enhancement module, enables the model to learn the features of the fingerprint watermark in the poisoned image.

[0061] The poisoning feature enhancement module is equipped with a poisoning feature enhancement loss function, which is as follows:

[0062]

[0063] Among them, Loss en Let K be the number of original images, P be the number of poisoned images, Q be the number of adversarial poisoned images, L be the cross-entropy loss function, E be the three-class classification network module, and ξ be the parameters of E, which are the parameters of the shared feature extraction layer of the classification network. i For the original image, To poison the image, To poison the image.

[0064] The original classification module has an initial classification loss function, which is:

[0065]

[0066] Among them, Loss poison Let K be the original classification loss function of the original classification module, P be the number of original images, Q be the number of poisoned images, L be the cross-entropy loss function, E be the three-class classification network module, ξ be the parameters of E, which share the parameters of the feature extraction layer with the classification network, and c be the original classification loss function. j For the true category. I i For the original image, To poison the image, For poisoning adversarial images, η(c j Specify the target category; Loss poison and Loss en The loss functions are basically the same.

[0067] The total loss function for black-box fingerprint watermark classification, composed of the poisoning feature enhancement loss and the original classification loss, is as follows:

[0068] Loss all =Loss poison +μLoss en

[0069] Where μ is a hyperparameter, and Lossen The poisoning feature enhancement loss function for the poisoning feature enhancement module is Loss. poison The original classification loss function, Loss all The total loss function for black-box fingerprint watermark classification.

[0070] Compared with the prior art, the beneficial effects of the technical solution of the present invention are:

[0071] This invention proposes a high-quality poisoned image generation method based on DWT transform, DCT transform, and SVD decomposition. This method significantly improves the concealment and anti-forgery properties of the poisoning trigger by spreading the fingerprint watermark, which acts as the poisoning trigger, throughout the entire image. Furthermore, to address the imbalance in the number of images across the three main categories—original image, poisoned image, and adversarial poisoning image—and the presence of numerous subcategories within each main category, this invention introduces a removable poisoning feature enhancement module to improve performance. This module enhances the reliability and fidelity of the model's watermark. This invention also proposes an adversarial training strategy, which, combined with the poisoning feature enhancement module, can effectively learn to embed fingerprint watermarks with lower strength, thereby improving the training effect of classification models containing fingerprint watermarks. Attached Figure Description

[0072] Figure 1 This is a flowchart of the poisoning image generation method described in Example 1;

[0073] Figure 2 This is a schematic diagram of the structure of the poisoning image generation method described in Example 2;

[0074] Figure 3 This is a flowchart of the fingerprint watermarking method for the black-box classification model described in Example 3;

[0075] Figure 4 The flowchart is shown in Example 4, illustrating the poisoned image generation method and the black-box classification model fingerprint watermarking method.

[0076] Figure 5 This is a schematic diagram of the fingerprint watermark structure of the black-box classification model described in Example 4;

[0077] Figure 6 This is a schematic diagram illustrating the stealth of the poisoned image generation method described in Example 4. Detailed Implementation

[0078] The accompanying drawings are for illustrative purposes only and should not be construed as limiting the scope of this patent.

[0079] To better illustrate this embodiment, some parts in the accompanying drawings may be omitted, enlarged, or reduced, and do not represent the actual product dimensions.

[0080] It will be understood by those skilled in the art that certain well-known structures and their descriptions may be omitted in the accompanying drawings.

[0081] The technical solution of this embodiment will be further described below with reference to the accompanying drawings and embodiments.

[0082] Example 1

[0083] This embodiment provides a method for generating poisoned images, such as... Figure 1 As shown, the method includes:

[0084] S11: Obtain the original image and the fingerprint watermark sequence to be embedded;

[0085] S12: Perform channel conversion on the original image to obtain a three-channel image; based on the fingerprint watermark sequence to be embedded, construct a fingerprint watermark sub-sequence corresponding to the three-channel image;

[0086] S13: For each channel image, DWT transformation, DCT transformation, scrambling encryption and SVD decomposition are performed in sequence. Each channel image obtains several singular values, and the embedding carrier is determined according to the size of the singular values.

[0087] S14: After embedding the fingerprint watermark subsequence into the embedding carrier of each channel image, the channel with the embedded fingerprint watermark is obtained. Inverse SVD, scrambling decryption, inverse DCT transformation and inverse DWT transformation are performed in sequence to obtain a three-channel image with the embedded fingerprint watermark.

[0088] S15: Perform RGB conversion on the three-channel image with embedded fingerprint watermark to obtain a poisoned image.

[0089] In the fingerprint watermark embedding process, the original image and the fingerprint watermark sequence to be embedded are first acquired. Next, the original image undergoes channel transformation to obtain a three-channel image, and a corresponding fingerprint watermark sub-sequence is constructed for each channel image based on the fingerprint watermark sequence to be embedded. For each channel image, DWT transform, DCT transform, scrambling encryption, and SVD decomposition operations are performed sequentially to obtain several singular values. The embedding carrier is determined based on the magnitude of the singular values. Subsequently, the fingerprint watermark sub-sequence is embedded into the embedding carrier of each channel image. Then, inverse SVD, scrambling decryption, inverse DCT transform, and inverse DWT transform are performed sequentially to obtain the three-channel image with the embedded fingerprint watermark. Finally, the three-channel image with the embedded fingerprint watermark is subjected to RGB conversion to obtain a poisoned image.

[0090] Example 2

[0091] This embodiment provides a method for generating poisoned images, such as... Figure 1 As shown, the method includes:

[0092] S11: Obtain the original image and the fingerprint watermark sequence to be embedded;

[0093] S12: Perform channel conversion on the original image to obtain a three-channel image; based on the fingerprint watermark sequence to be embedded, construct a fingerprint watermark sub-sequence corresponding to the three-channel image;

[0094] S13: For each channel image, DWT transformation, DCT transformation, scrambling encryption and SVD decomposition are performed in sequence. Each channel image obtains several singular values, and the embedding carrier is determined according to the size of the singular values.

[0095] S14: Embed the fingerprint watermark subsequence into the embedding carrier of each channel image, and perform inverse SVD, scrambling decryption, inverse DCT transformation and inverse DWT transformation in sequence to obtain a three-channel image with embedded fingerprint watermark.

[0096] S15: Perform RGB conversion on the three-channel image with embedded fingerprint watermark to obtain a poisoned image.

[0097] In the fingerprint watermark embedding process, the original image and the fingerprint watermark sequence to be embedded are first acquired. Next, the original image undergoes channel transformation to obtain a three-channel image, and a corresponding fingerprint watermark sub-sequence is constructed for each channel image based on the fingerprint watermark sequence to be embedded. For each channel image, DWT transform, DCT transform, scrambling encryption, and SVD decomposition operations are performed sequentially to obtain several singular values. The embedding carrier is determined based on the magnitude of the singular values. Subsequently, the fingerprint watermark sub-sequence is embedded into the embedding carrier of each channel image. Then, inverse SVD, scrambling decryption, inverse DCT transform, and inverse DWT transform are performed sequentially to obtain the three-channel image with the embedded fingerprint watermark. Finally, the three-channel image with the embedded fingerprint watermark is subjected to RGB conversion to obtain a poisoned image.

[0098] like Figure 2 As shown, the specific method of S12 is as follows:

[0099] Let the original image be denoted as I. i The image is then transformed to obtain a three-channel image, including a Y-channel image, a U-channel image, and a V-channel image; the fingerprint watermark sequence to be embedded is denoted as... In the formula m q Let represent the q-th fingerprint watermark in the fingerprint watermark sequence to be embedded; construct the corresponding fingerprint watermark sub-sequences to be embedded for the Y-channel image, U-channel image, and V-channel image, denoted as respectively. and Fingerprint watermark sub-sequence to be embedded and for subsequence.

[0100] The specific method of S13 is as follows:

[0101] S131: Perform DWT transformation on the Y channel image to obtain the low-frequency subband;

[0102] S132: Perform DCT transformation on the low-frequency subband to obtain a set of DCT coefficient matrices;

[0103] S133: Use a preset key KEY to scramble and encrypt the set of DCT coefficient matrices to obtain a set of scrambled and encrypted DCT coefficient matrices.

[0104] S134: Perform SVD decomposition on each set of scrambled and encrypted DCT coefficient matrices to obtain several singular values ​​corresponding to the Y channel image.

[0105] S135: Sort several singular values ​​corresponding to the Y channel image according to their magnitude, and select the top two singular values ​​S1 and S2 as the embedding carriers of the Y channel image.

[0106] S136: Repeat steps S131-S135 for the U-channel image and the V-channel image to obtain the embedding vector corresponding to the U-channel image and the embedding vector corresponding to the V-channel image.

[0107] In step S133, the method for obtaining a set of scrambled and encrypted DCT coefficient matrices is as follows:

[0108] I. Set the key KEY as the random seed to initialize the random function;

[0109] II. A random function is used to generate the index matrix for scrambling, and the size of the index matrix is ​​the same as the size of the DCT coefficient matrix.

[0110] III. Using the index matrix, adjust the positions of the coefficients in the DCT system matrix to obtain a set of scrambled and encrypted DCT coefficient matrices.

[0111] In step S14, the specific method for embedding the fingerprint watermark sub-sequence into the embedding carrier of each channel image is as follows:

[0112] For the Y-channel image, U-channel image, and V-channel image, the fingerprint watermark subsequence and Binary fingerprint watermark bits m b Each singular value S1 and S2, which serve as the embedding carrier, are embedded one by one;

[0113]

[0114]

[0115] Where d1 and d2 are the watermark embedding strengths, S1' and S2' represent the singular values ​​of the embedded binary fingerprint watermark bits, and m b This is a binary fingerprint watermark bit.

[0116] To improve the performance of poisoned images, this embodiment imposes the following three constraints on the generation of poisoned images, namely...

[0117] Apply the following three constraints, namely

[0118] (1) Stealth: This refers to ensuring that the difference between the poisoned image and the original image is within a small range, thereby reducing the probability of the fingerprint watermark being detected or forged. Specifically, if the visual similarity between two samples is measured using the visual effect distance function V, then the stealth requirement is to find a suitable poisoned image constructor such that the difference between the poisoned image and the original image is less than a threshold ε, i.e.

[0119]

[0120] Where V is the visual distance function. To construct the poison image, I i This is the original image.

[0121] (2) Sample Specificity: Traditional poisoned images are typically generated by superimposing a small square onto the original image. This method of generating poisoned images is consistent across all images, making it easy to detect and even forge. Therefore, each poisoned image needs to be generated differently, meaning the difference between the poisoned image and the original image needs to be closely related to the original image. In other words, relative to the original image I... i The difference needs to be I i The function is d(I) i ), and subtle differences in the value d′(I) i This can lead to different classification results, as shown in the formula:

[0122]

[0123] Among them, f Θ For classification neural network models, I i For the original image, η(c) j ) specifies the target category.

[0124] (3) No additional auxiliary training required: Many existing neural network model watermarking methods in the literature utilize auxiliary models to generate poisoned images. For example, DFST uses the auxiliary model CycleGAN, and LIRA uses the auxiliary model Unet to generate poisoned images. The effectiveness of these poisoned image generation methods depends on the training of the auxiliary model, so their performance is not stable enough and they have high additional training overhead. To solve these problems, this embodiment can generate poisoned images without additional auxiliary training.

[0125] Example 3

[0126] This embodiment provides a fingerprint watermarking method based on a black-box classification model of poisoned images, such as... Figure 3 As shown, the method includes:

[0127] S21: Obtain the original image, the poisoned image, and the poisoned adversarial image; the poisoned image is generated using the poisoned image generation method described in Example 1 or 2; the poisoned adversarial image is generated by embedding a random error fingerprint watermark into the original image;

[0128] S22: Construct a black-box fingerprint watermark classification model and set the total loss function for black-box fingerprint watermark classification;

[0129] S23: The original image, the poisoned image, and the poisoned adversarial image are used to train the constructed black-box fingerprint watermark classification model to obtain the trained black-box fingerprint watermark classification model.

[0130] S24: Obtain the image used for verification, input it into the black-box fingerprint watermark classification model, and obtain the classification result of the verification image.

[0131] The process involves obtaining the original image, the poisoned image, and the adversarial poisoning image. The poisoned image is generated using any of the poisoning image generation methods described in Example 1 or 2. The adversarial poisoning image is generated by embedding a random erroneous fingerprint watermark into the original image. Next, a black-box fingerprint watermark classification model is constructed, and a total loss function for black-box fingerprint watermark classification is set within the model. Then, the constructed black-box fingerprint watermark classification model is trained using the original image, the poisoned image, and the adversarial poisoning image to obtain a well-trained black-box fingerprint watermark classification model. Finally, the original image and the image to be detected are obtained and input into the black-box fingerprint watermark classification model to obtain the classification result of the image to be detected.

[0132] Example 4

[0133] This embodiment provides a fingerprint watermarking method based on a black-box classification model of poisoned images, such as... Figure 3 As shown, the method includes:

[0134] S21: Obtain the original image, the poisoned image, and the poisoned adversarial image; the poisoned image is generated using the poisoned image generation method described in Example 1 or 2; the poisoned adversarial image is generated by embedding a random error fingerprint watermark into the original image;

[0135] S22: Construct a black-box fingerprint watermark classification model and set the total loss function for black-box fingerprint watermark classification;

[0136] S23: The original image, the poisoned image, and the poisoned adversarial image are used to train the constructed black-box fingerprint watermark classification model to obtain the trained black-box fingerprint watermark classification model.

[0137] S24: Obtain the image used for verification, input it into the black-box fingerprint watermark classification model, and obtain the classification result of the verification image.

[0138] The process involves obtaining the original image, the poisoned image, and the adversarial poisoning image. The poisoned image is generated using any of the poisoning image generation methods described in Example 1 or 2. The adversarial poisoning image is generated by embedding a random erroneous fingerprint watermark into the original image. Next, a black-box fingerprint watermark classification model is constructed, and a total loss function for black-box fingerprint watermark classification is set within the model. Then, the constructed black-box fingerprint watermark classification model is trained using the original image, the poisoned image, and the adversarial poisoning image to obtain a well-trained black-box fingerprint watermark classification model. Finally, the image used for verification is obtained and input into the black-box fingerprint watermark classification model to obtain the copyright verification result.

[0139] like Figure 5 As shown, the black-box fingerprint watermark classification model includes: an input module, a feature extraction module, a poisoning feature enhancement module, and an original classification module;

[0140] The output of the input module is connected to the input of the feature extraction module, and the output of the feature extraction module is connected to the inputs of both the poisoning enhancement module and the original classification module. The poisoning feature enhancement module is equipped with a poisoning feature enhancement loss, and the original classification module is equipped with an original classification loss. The poisoning feature enhancement loss and the original classification loss together constitute the total loss function of the black-box fingerprint watermark classification model.

[0141] If the output target category is consistent with η(c) j If the values ​​match, the model copyright verification passes; otherwise, it fails. Let p... chk The probability that a fingerprint watermark belongs to a specified category in a black-box classification model is then...

[0142]

[0143] Where Ind is the indicator function, I i For the original image, For the fingerprint watermark sequence to be embedded, when p chk If the threshold is greater than 80%, then ownership of the model is proven.

[0144] In fingerprint watermark classification models, it is necessary to achieve the functions of classifying the target object in the original image, verifying correct fingerprint watermarks, and verifying incorrect fingerprint watermarks. The formula is as follows:

[0145]

[0146] in, This is an incorrect fingerprint watermark sequence. For the fingerprint watermark sequence to be embedded, f Θ For classification neural network models, Construct the poison image constructor, η(c j ) represents the target category, and Θ represents the neural network parameters.

[0147] Existing black-box classification watermarking methods utilize the final fully connected layer of a deep neural network for classification. They achieve relatively high accuracy for original images and relatively high reliability for poisoned images. However, the imbalance between training samples, where the number of original images is significantly greater than the number of poisoned images, negatively impacts the watermarking model's performance in copyright verification. Furthermore, the large number of categories means that the loss function cannot accurately represent the discrimination error between the original and poisoned image categories, hindering accurate parameter updates during backpropagation and ultimately affecting the final performance of the fingerprint watermarking classification method.

[0148] To address these issues, a poisoning feature enhancement module is added to the fingerprint watermarking classification model. Specifically, the original image can be treated as a single class without further classification, as can the poisoned image and adversarial images containing erroneous fingerprint watermarks. This minimizes the imbalance between the number of original images, poisoned images, and adversarial images. Furthermore, by using the error function of these three groups of judgment results, the intra-class distances of the three groups (original images, poisoned images, and adversarial images) can be minimized while the inter-class distances are maximized, thereby improving the performance of the black-box neural network model watermarking. The proposed poisoning feature enhancement module primarily assists in model training to achieve better performance. This module can be removed when performing model classification and copyright verification functions without affecting practical use, thus providing good flexibility and not reducing the security performance of the model watermarking.

[0149] The method for generating adversarial poisoning images is the same as that for poisoning images, except that the embedded fingerprint watermark sequence is not... Rather, it is not equal to of Let the length of the binary fingerprint watermark sequence be L (L>0), then there are a total of 2 L A fingerprint watermark sequence, one of which is the correct fingerprint watermark sequence to be embedded. The remaining 2 L -1 represents erroneous fingerprint watermark information that can cause interference, and it is also the fingerprint watermark information used to generate poisoning adversarial images; the corresponding generated poisoning adversarial images are collectively referred to as...

[0150] The method proposed in this embodiment employs a poisoned image generation method based on DWT-DCT-SVD, resulting in poisoned images that closely resemble the original images. While this method can produce very good poisoned images, the excessive similarity between the generated images and the original images makes training extremely difficult, even preventing the proper training of deep classification network models. Furthermore, existing black-box classification models for watermarking typically use a cross-entropy loss function and train the model using both the original and poisoned images. However, this often only learns simple pixel-level features and struggles to learn the fingerprint watermark information embedded in the poisoned images, making it difficult for the trained classification model to perform copyright verification and traceability.

[0151] To address these issues, this embodiment employs an adversarial training mode. Specifically, this embodiment adds poisoned adversarial images with incorrectly embedded fingerprint watermarks to the training dataset, allowing the fingerprint watermark classification model, which is to be embedded with the correct fingerprint watermark, to be trained and optimized simultaneously on the original image, the poisoned image, and the poisoned adversarial image. This adversarial training strategy, combined with the poisoning feature enhancement module, enables the model to learn the features of the fingerprint watermark in the poisoned image.

[0152] The poisoning feature enhancement module is equipped with a poisoning feature enhancement loss function, which is as follows:

[0153]

[0154] Among them, Loss en Let K be the number of original images, P be the number of poisoned images, Q be the number of adversarial poisoned images, L be the cross-entropy loss function, E be the three-class classification network module, and ξ be the parameters of E, which are the parameters of the shared feature extraction layer of the classification network. i For the original image, To poison the image, To poison the image.

[0155] The original classification module has an initial classification loss, and its initial classification function is:

[0156]

[0157] Among them, Loss poison Let K be the original classification loss function of the original classification module, P be the number of original images, Q be the number of poisoned images, L be the cross-entropy loss function, E be the three-class classification network module, ξ be the parameters of E, which share the parameters of the feature extraction layer with the classification network, and c be the original classification loss function. j For the true category. I i For the original image, To poison the image, To poison adversarial images, η(c j Specify the target category; Loss poison and Loss en The loss functions are basically the same.

[0158] The total loss function for black-box fingerprint watermark classification, composed of the poisoning feature enhancement loss and the original classification loss, is as follows:

[0159] Loss all =Loss poison +μLoss en

[0160] Where μ is a hyperparameter, and Loss en The poisoning feature enhancement loss function for the poisoning feature enhancement module is Loss. poison The original classification loss function for the original classification module is Loss. all The total loss function for black-box fingerprint watermark classification.

[0161] This embodiment introduces the concept of fingerprint watermarking in a DNN classification model. Fingerprint watermarking is achieved by constructing poisoned images with different copyright watermark information for different users. To improve robustness, the fingerprint watermark information is embedded into the coefficients of the original image after DWT transform, DCT transform, and SVD decomposition. A key is used to encrypt the DCT coefficient matrix during the embedding process to enhance security. Furthermore, to improve the performance of the fingerprint watermark classification model, a poisoning feature enhancement module is proposed to guide model performance improvement from a macroscopic perspective. This module can be easily removed from the network without degrading network performance, improving flexibility and concealment. This embodiment uses a method of embedding fingerprint watermarks into the DWT-DCT-SVD transform coefficients to generate high-quality poisoned images, improving robustness and making the presence of the model watermark difficult to detect. Fidelity and reliability are balanced by adjusting the embedding strength and poisoning rate of the fingerprint watermark information. The DCT coefficient matrix during the embedding process is encrypted using a key to improve unforgeability. A separable poisoning feature enhancement module is added to improve model watermark performance. Finally, a poisoning training method based on adversarial training is used to obtain the fingerprint model watermark. Therefore, this embodiment achieves a good compromise between factors such as fidelity, reliability, non-forgeability, and robustness.

[0162] Let the proportion of the training set for constructing the poisoned image set be 40%, and the proportion of the training set for constructing the corresponding adversarial poisoned image set for incorrect watermark information be 10%. For example... Figure 4 As shown, during training, if the input image is from the set of images to be poisoned, it is directly input into the classification model. Simultaneously, a poisoned image with correctly embedded watermark information is generated using the poisoned image generation method proposed in this embodiment and input into the classification model. If the image also belongs to the poisoning adversarial image set, a random incorrect watermark is embedded to generate a poisoning adversarial image, which is then input into the classification model. Subsequently, these generated images are used to train a black-box classification model with fingerprint watermarking functionality. For the deep features output by the classification model, they are fed into a poisoning feature enhancement module to generate probabilities belonging to three sets of functions: regular image target object classification, correct fingerprint model watermark verification, and adversarial fingerprint model watermark verification. They are also fed into the classification module to generate probabilities belonging to regular image target category, correct fingerprint model watermark, and adversarial fingerprint model watermark. Finally, these probabilities are used to calculate the corresponding cross-entropy, and these cross-entropy losses are weighted and accumulated to guide the backpropagation of the model proposed in this embodiment.

[0163] When verifying watermark copyright, the original image without fingerprint watermark, the poisoned image with the correct fingerprint watermark, or the poisoned adversarial image with the incorrect fingerprint watermark are input into the classification model to be tested. If a certain number of poisoned images are continuously input, and the abnormal classification probability of the classification module exceeds a certain threshold, the fingerprint of the model can be verified and the corresponding copyright owner can be traced.

[0164] An experiment was set up to verify the effectiveness of the method provided in this embodiment.

[0165] (I) Experimental Setup

[0166] (1) Dataset

[0167] This embodiment uses three datasets widely used in model watermarking research: CIFAR-10, GTSRB, and CelebA. CIFAR-10 is a general object classification dataset, GTSRB is used to classify different traffic signs in Germany, and CelebA is a large facial attribute classification dataset. It should be noted that CelebA has 40 independent binary attributes, most of which are imbalanced. To make it suitable for multi-class classification, following the WaNet approach, this embodiment uses the three most balanced attributes from CelebA (i.e., heavy makeup, slightly open mouth, and smiling) and combines them to construct 8 categories. Table 1 summarizes the datasets used in the experimental simulation of this embodiment. For the different datasets mentioned above, this embodiment uses the same classification model, ResNet-18. To evaluate the watermarking performance of different classification models on the same dataset, this embodiment also uses classification models such as VGG16, MobileNetV2, and ResNeXt on the CIFAR-10 dataset.

[0168] Table 1 Dataset Settings

[0169]

[0170] (2) Testing Standards

[0171] In performance evaluation, this embodiment uses Peak Signal-to-Noise Ratio (PSNR), Structural Similarity (SSIM), Watermark Accuracy (WMA), and Benign Accuracy (BA) as evaluation metrics. PSNR and SSIM are used to assess the similarity between the poisoned image and the original image, indicating the concealment level of the fingerprint watermark; higher values ​​indicate better concealment. WMA refers to the percentage of poisoned images classified as the target label. BA refers to the classification accuracy of the original image, i.e., the model's classification performance on the original task, reflecting the fidelity of the model's watermark to the original task. To measure the effectiveness of the proposed model watermark, this embodiment compares the original classification model without embedded fingerprint watermarks, patch-based models, blended models, SIG based on sinusoidal strips, WaNet based on spatial distortion, and the method proposed in this embodiment.

[0172] (3) Parameter settings

[0173] When generating poisoned images, the longer the embedded fingerprint watermark sequence, the easier it is to verify and trace the model's copyright, but this reduces the concealment of the fingerprint watermark. Moreover, the more poisoning adversarial samples are required, the more the model's fidelity will be affected. This embodiment takes a compromise and sets the fingerprint watermark sequence length to (4) Training parameter settings

[0174] This experiment uses the SGD optimizer to optimize the decryption and reconstruction module. The optimizer parameters are set to β1 = 0.9, and the learning rate is initialized to 10. -2 The learning rate decreases by a factor of 10 every 100 training epochs. During training, training images are randomly cropped and selected. This experiment uses Ubuntu 20.04LTS as the operating system platform, implements training and testing in PyTorch, and runs on an RTX 3090 GPU.

[0175] (II) Ablation Test

[0176] The method proposed in this embodiment only responds to poisoned images embedded with correct fingerprint information, while it does not trigger the model's watermark response for poisoned adversarial images embedded with other fingerprint watermark information. However, without adversarial training, poisoned images embedded with other fingerprint watermark information can also trigger the model's watermark response, failing to achieve the desired fingerprint watermark effect. To address this issue, this embodiment adds poisoned adversarial images to the training set, making the model focus more on the fingerprint watermark features embedded in the poisoned images, rather than just the features of the original images. In the experimental simulation, this embodiment sets the proportion β (i.e., the adversarial rate, the proportion of poisoned adversarial images to the total training images) used to construct poisoned adversarial images corresponding to incorrect watermark information to 0.1, and then generates images with 2... L =2 8 =256 different fingerprint watermarks corresponding to poisoned images, and finally test the WMA performance of these 256 different fingerprint watermarks.

[0177] Experimental simulation results show that, for poisoned images embedded with these 256 different fingerprint watermarks, without adversarial training, any poisoned image containing a fingerprint watermark will trigger the model's watermark response, regardless of whether the fingerprint watermark is correct or incorrect. However, after adversarial training, the WMA (Warranty Mean Average) for most poisoned images corresponding to the correct fingerprint watermark is below 60%. The WMA for poisoned images with the correct fingerprint watermark is the highest, while the second highest is 3.6% lower than the highest. This is because the fingerprint watermark embedded in the second highest input image differs from the correct fingerprint watermark by only one bit, and the fifth highest is as low as 72.28%. These results fully demonstrate the effectiveness of the adversarial training strategy and show that even with a watermark information error of only one bit, it can still correctly distinguish between correct and incorrect fingerprint watermarks.

[0178] (III) Experimental Results

[0179] The experimental datasets used in this experiment were CIFAR-10, GTSRB, and CelebA, and the network model was ResNet-18. The experimental results are shown in the table below.

[0180] Table 2 Performance evaluation results of the many-to-one model

[0181]

[0182]

[0183] As shown in Table 2, for the many-to-one pattern, after embedding the model watermark, the BA and WMA values ​​of this embodiment are higher than WaNet for the CelebA dataset. However, for the CIFAR-10 and GTSRB datasets, the BA and WMA values ​​of this embodiment are slightly lower than WaNet. This is mainly because the difference between the poisoned image and the original image is very small, making it more difficult for the model to learn the poisoning features. Furthermore, Table 2 shows that Patch, Blend, and SIG have excellent BA and WMA values, and their overall BA and WMA performance surpasses WaNet and the proposed method. This is related to the simplicity of their poisoning image generation methods; the poisoning features can be fully extracted in the shallow convolutional layers of the model, without requiring the model to focus too much on the poisoned image, allowing the model to focus more on the original classification task. However, this simple poisoning image generation method is easily detected and forged.

[0184] Table 3 Performance evaluation results of many-to-many modes

[0185]

[0186] As shown in Table 3, the performance of BA in the many-to-many mode is generally similar to that in the many-to-one mode. Only for the CelebA dataset, the performance of WMA in this embodiment is lower than that of WaNet. This may be because the fingerprint watermarking task is more complex in the many-to-many mode, requiring the model to additionally calculate the target label. In addition, the poisoned image is difficult to identify due to its small difference from the original image, and fingerprint feature learning, both of which together lead to a significant performance drop.

[0187] Table 4. Effectiveness of classification network models with different depths on the CIFAR-10 dataset.

[0188]

[0189]

[0190] As shown in Table 4, this embodiment also tested the deep classification network models VGG16, MobileNetV2, and ResNeXt. The settings for these models were similar to those in the experiments above, and the simulation results are shown in Table 4, using a many-to-one model and the CIFAR-10 dataset. The table shows that embedding the model fingerprint watermark caused a slight decrease in the original model's classification performance for all these deep classification network models. VGG16 experienced the largest decrease in BA (Balance of Classification), with a BA value drop of 1.6%; ResNeXt showed the smallest decrease, with a BA value drop of 0.54%. Furthermore, the proposed method achieved WMA values ​​above 98% for all these models, thus achieving excellent model copyright verification functionality.

[0191] To test the stealth of different watermarking models, this embodiment uses objective measurement metrics PNSR and SSIM to evaluate the similarity between the generated poisoned image and the original image, which differs from previous methods that used subjective manual testing. During the simulation, 256 images were randomly selected from the CIFAR-10 dataset, and corresponding poisoned images were generated for different watermarking models: Patch, Blend, SIG, WaNet, and this embodiment. The average PNSR and SSIM values ​​between the corresponding poisoned images and the original images were calculated. Table 5 summarizes the simulation results. As shown in the table, the PNSR and SSIM values ​​of the poisoned images generated in this embodiment are very high, far exceeding those of other watermarking models, indicating that this embodiment has satisfactory stealth.

[0192] also, Figure 6 This shows the poisoned images generated by these different methods and the residual images compared to the original images. Figure 6As can be seen, the poisoned image generated in this embodiment is approximately identical to the original image, and even after magnification by 5 times, the pixel amplitude of its residual map still does not show any distortion caused by watermark embedding. This fully demonstrates the effectiveness of the poisoned image generation method in this embodiment. Compared with the poisoned image generation method in this embodiment, the poisoned images generated by Patch, Blend, and SIG all show distortion relative to the original image to the naked eye. For example, Patch clearly shows the small blue square in the upper left corner, and Blend and SIG both show the outline of the watermark. The residual images of these methods show similar issues. Compared to these methods, the poisoned image generated by WaNet is very close to the original image, and the distortion caused by watermark embedding is not visible to the naked eye. However, its residual map clearly shows the embedded watermark, so WaNet's concealment is relatively poor. The simpler the method of generating the poisoned image, the better the performance of the corresponding model watermark, but the concealment will be very poor. Therefore, how to balance the effectiveness of the model watermark and the concealment of the poisoned image is a problem worth considering; the two cannot be simultaneously achieved. This embodiment focuses on the high concealment and security features of the poisoned image.

[0193] Table 5. Visual effects of poisoned images

[0194]

[0195] As shown in Table 5, the poisoned image generated in this embodiment has good concealment, and the difference between it and the original image is almost zero. Therefore, the poisoned image generated in this embodiment is unlikely to arouse suspicion that it contains watermark information, which further reduces the risk of adversaries forging it due to suspicion, and also makes it impossible for adversaries to easily forge the poisoned image by superimposing residual images.

[0196] The same or similar labels correspond to the same or similar parts;

[0197] The terms used to describe positional relationships in the accompanying drawings are for illustrative purposes only and should not be construed as limiting this patent.

[0198] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the implementation of the present invention. Those skilled in the art can make other variations or modifications based on the above description. It is neither necessary nor possible to exhaustively describe all embodiments here. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the claims of the present invention.

Claims

1. A method for generating poisoned images, characterized in that, The method includes: S11: Obtain the original image and the fingerprint watermark sequence to be embedded; S12: Perform channel conversion on the original image to obtain a three-channel image; based on the fingerprint watermark sequence to be embedded, construct a fingerprint watermark sub-sequence corresponding to the three-channel image; S13: For each channel image, DWT transformation, DCT transformation, scrambling encryption and SVD decomposition are performed in sequence. Each channel image obtains several singular values, and the embedding carrier is determined according to the size of the singular values. S14: After embedding the fingerprint watermark subsequence into the embedding carrier of each channel image, the channel with the embedded fingerprint watermark is obtained. Inverse SVD, scrambling decryption, inverse DCT transformation and inverse DWT transformation are performed in sequence to obtain a three-channel image with the embedded fingerprint watermark. S15: Perform RGB conversion on the three-channel image with embedded fingerprint watermark to obtain a poisoned image; The specific method of S13 is as follows: S131: Perform DWT transformation on the Y channel image to obtain the low-frequency subband; S132: Perform DCT transformation on the low-frequency subband to obtain a set of DCT coefficient matrices; S133: Use a preset key KEY to scramble and encrypt the set of DCT coefficient matrices to obtain a set of scrambled and encrypted DCT coefficient matrices. S134: Perform SVD decomposition on each set of scrambled and encrypted DCT coefficient matrices to obtain several singular values ​​corresponding to the Y channel image. S135: Sort several singular values ​​corresponding to the Y channel image according to their amplitude, and select the top two singular values ​​in the sorting result. and As an embedding carrier for the Y-channel image; S136: Repeat steps S131-S135 for the U-channel image and the V-channel image to obtain the embedding vector corresponding to the U-channel image and the embedding vector corresponding to the V-channel image.

2. The method for generating a poisoned image according to claim 1, characterized in that, The specific method of S12 is as follows: The original image is denoted as The image is then transformed to obtain a three-channel image, including a Y-channel image, a U-channel image, and a V-channel image; the fingerprint watermark sequence to be embedded is denoted as... In the formula This indicates the first fingerprint watermark in the sequence to be embedded. A fingerprint watermark is created; for the Y-channel image, U-channel image, and V-channel image, a fingerprint watermark sub-sequence to be embedded is constructed, denoted as follows: , and Fingerprint watermark sub-sequence to be embedded , and for subsequence.

3. The method for generating a poisoned image according to claim 2, characterized in that, In step S133, the method for obtaining a set of scrambled and encrypted DCT coefficient matrices is as follows: I. Set the key KEY as the random seed to initialize the random function; II. A random function is used to generate the index matrix for scrambling, and the size of the index matrix is ​​the same as the size of the DCT coefficient matrix. III. Using the index matrix, adjust the positions of the coefficients in the DCT system matrix to obtain a set of scrambled and encrypted DCT coefficient matrices.

4. The method for generating a poisoned image according to claim 3, characterized in that, In step S14, the specific method for embedding the fingerprint watermark sub-sequence into the embedding carrier of each channel image is as follows: For the Y-channel image, U-channel image, and V-channel image, the fingerprint watermark subsequence , and Binary fingerprint watermark bits One-to-one correspondence embedding of singular values ​​as embedding carriers and middle; in, and For watermark embedding strength, and This represents the singular value of the embedded binary fingerprint watermark bits. This is a binary fingerprint watermark bit.

5. A fingerprint watermarking method for a black-box classification model, characterized in that, The method includes: S21: Obtain the original image, the poisoned image, and the poisoning adversarial image; the poisoned image is generated using the poisoned image generation method according to any one of claims 1-4; the poisoning adversarial image is generated by embedding a random error fingerprint watermark into the original image; S22: Construct a black-box fingerprint watermark classification model and set the total loss function for black-box fingerprint watermark classification; S23: The original image, the poisoned image, and the poisoned adversarial image are used to train the constructed black-box fingerprint watermark classification model to obtain the trained black-box fingerprint watermark classification model. S24: Obtain the image used for verification, input it into the black-box fingerprint watermark classification model, and obtain the classification result of the verification image.

6. The fingerprint watermarking method for a black-box classification model according to claim 5, characterized in that, The black-box fingerprint watermark classification model includes: an input module, a feature extraction module, a poisoning feature enhancement module, and an original classification module; The output of the input module is connected to the input of the feature extraction module, and the output of the feature extraction module is connected to the inputs of both the poisoning enhancement module and the original classification module. The poisoning feature enhancement module is equipped with a poisoning feature enhancement loss, and the original classification module is equipped with an original classification loss. The poisoning feature enhancement loss and the original classification loss together constitute the total loss function for black-box fingerprint watermark classification.

7. The fingerprint watermarking method for a black-box classification model according to claim 6, characterized in that, The poisoning feature enhancement module is equipped with a poisoning feature enhancement loss function, which is: in, Let K be the number of original images, P be the number of poisoned images, Q be the number of adversarial poisoned images, L be the cross-entropy loss function, and E be the three-class classification network module. The parameters of E are the parameters of the shared feature extraction layer in the classification network. For the original image, To poison the image, To poison the image.

8. The fingerprint watermarking method for a black-box classification model according to claim 6, characterized in that, The original classification module is equipped with an original classification loss function, which is: in, Let K be the original classification loss function of the original classification module, P be the number of original images, Q be the number of poisoned images, L be the cross-entropy loss function, and E be the three-class classification network module. The parameters of E are shared with the parameters of the feature extraction layer in the classification network. For the real category, For the original image, To poison the image, To poison adversarial images, For the target category.

9. A fingerprint watermarking method for a black-box classification model according to claim 7 or 8, characterized in that, The total loss function for black-box fingerprint watermark classification, composed of the poisoning feature enhancement loss and the original classification loss, is as follows: Where μ is a hyperparameter, Let be the poisoning feature enhancement loss function of the poisoning feature enhancement module. The original classification loss function of the original classification module. The total loss function for black-box fingerprint watermark classification.

Images