Deep learning model copyright authentication method based on lattice quantization index modulation

Abstract

The invention discloses a deep learning model copyright authentication method based on lattice quantization index modulation, and the method comprises the steps: embedding the identity information of a model owner into a frequency domain coefficient of a trigger set sample based on the reversibility characteristic of the lattice quantization index modulation; the correct extraction of the identity information of the owner of the model is ensured while the trigger set sample has the characteristic of visual indistinguishability. The method comprises the following steps: generating a trigger set based on lattice quantization index modulation; then, generating a watermark model by using a data set and trigger set joint training model, and calculating a trigger set prediction probability threshold value of the watermark model; and finally, inputting a trigger set to the suspicious model to calculate a prediction accuracy rate, comparing the prediction accuracy rate with a probability threshold value, and modulating and extracting identity information of a model owner in the trigger set through lattice quantization index to complete copyright authentication of the suspicious model. The problems that in an existing black box model watermarking technology, the visual undistinguishability of a trigger set sample is low, and binding between the identity of a model owner and a model watermark is weak are effectively solved.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence security and digital watermarking technology, specifically disclosing a deep learning model copyright authentication method based on lattice quantization index modulation, applicable to model copyright authentication and protection in black-box scenarios. Background Technology

[0002] With the widespread application of deep learning models in fields such as computer vision and natural language processing, the copyright protection issues arising from open-source model sharing are becoming increasingly prominent. Training high-performance models requires massive amounts of data, computing resources, and professional R&D costs, which gives malicious actors a strong incentive to steal models and conceal their infringement. This leads to frequent malicious theft and poses a serious challenge to model intellectual property rights. To address this issue, various model watermarking technologies have been introduced into the field of deep learning model security for copyright verification. Existing model watermarking technologies are mainly divided into two categories: white-box and black-box. White-box methods embed watermarks by modifying the model's internal structure, such as weights, adding layers, and network outputs. While this achieves watermark implantation, it requires access to the model's internal information to verify copyright, limiting its practicality. Black-box methods, on the other hand, rely solely on application programming interface (API) queries for verification, making them closer to real-world scenarios. Existing black-box methods mainly rely on establishing backdoors. By embedding backdoors in the model to establish special input-output relationships, ownership is verified based on trigger sets.

[0003] Current model copyright protection technologies face the following challenges: existing methods rely on highly visible trigger sets of samples, making them easily detectable or counterfeited; furthermore, they lack a direct link to the model owner's identity, leading to the risk of "multiple ownership disputes" in verification results. Therefore, innovative technical solutions are urgently needed to provide more reliable support for the copyright protection of deep learning models. Summary of the Invention

[0004] The main objective of this invention is to overcome the shortcomings and deficiencies of existing technologies, such as high visibility of triggering modes and lack of identity verification, and to provide a copyright authentication method for deep learning models based on lattice quantization index modulation. This is a black-box model watermarking scheme with strong concealment and support for identity binding.

[0005] To achieve the above objectives, the present invention adopts the following technical solution: A copyright authentication method for deep learning models based on lattice quantization index modulation, the method comprising the following steps: S1, from the dataset training set Randomly select a subset ,right The sample images are quantized, indexed, modulated, and embedded with the model to embed the owner's identity information. and modify The labels of the samples are used to generate trigger sets. ; S2, using the training set With trigger set Merged datasets For the model The watermark model is obtained through training. Calculate the watermark model Predicted trigger set The baseline threshold ; S3, Suspicious Models Input trigger set Calculate the prediction accuracy ,Compare Compared with the baseline threshold ,like conform to The floating range is then extracted using lattice quantization index modulation to obtain the trigger set. Model owner's identity information Calculate the accuracy rate of identity information extraction ,like Meets the fault tolerance threshold range and completes the process. Copyright certification is required; otherwise, the copyright certification will be deemed invalid.

[0006] Further, step S1 is as follows: S11. Create a trigger set for generating the trigger set. The original carrier set: from the dataset training set Randomly select a subset of image-label pairs , the original label Replace with target tags This yields the original carrier set used to generate the trigger set. ,in, For image-label pairs of samples, Represents sample images, Indicates the original label of the sample. Predetermine target labels for the samples; S12. Convert the sample images of the original carrier set into frequency domain coefficients: Each sample image Follow these steps, and in subsequent steps... All refer to Each sample image: Let the sample image The dimension is , The height is the number of pixels. The width in pixels. Channel number, channel index Used for traversal Different color channels to satisfy , sample Split by color channel to obtain Single-channel image , Indicates sample The Channel image, dimension is ,right The spatial domain image is converted into a wavelet frequency domain coefficient matrix using two-dimensional discrete wavelet transform. The formula is expressed as: ,in, For the first The wavelet frequency domain coefficient matrix of each channel image has dimensions of , Represents a two-dimensional discrete wavelet transform; S13. Convert the wavelet frequency domain coefficient matrix of each channel. The vectors are divided as follows: First, [the vectors are divided into] carrier vectors. Expand by row first, then divide into equal parts. There are carrier vectors that satisfy: , , in, For carrier vector index, Indicates the first There are carrier vectors, with dimension [ ]. , The total number of carrier vectors, The dimension of the carrier vector. Dimensions Satisfy constraints: ; S14. Transfer the identity information of the model owner. Convert to binary sequence ,in, For the first in the sequence The binary value of a bit. The length of the binary sequence is equal to the total number of carrier vectors in step S13.

[0007] S15, using coarse grid and fine grid The nested relationship is used to divide the coset and coset representative element; S16, Regarding the carrier vector Embedding binary sequences sequentially Dear friends , Generate a new carrier vector containing identity information. The process is as follows: Will Mapped to fine grid The accompanying representative element And then according to Use coarse grid nearest neighbor quantization function Modify carrier vector This yields a new carrier vector containing identity information. , Falling on the rough grid fine grid generated by translation The Within each coset, the formula is as follows: , in, Let j represent the j-th carrier vector containing identity information, with dimension 1. , Coarse grid The nearest neighbor quantization function on, It is the first Each set of companion sets represents a yuan; S17. The new carrier vector after embedding identity information Reassemble to restore the corresponding frequency domain coefficient matrix. The specific definition is as follows: For the Channel, a carrier vector embedded with identity information Reconstruct the frequency domain coefficient matrix sequentially to obtain the modified matrix. The formula is expressed as: , ,in, Indicates the first The wavelet frequency domain coefficient matrix of each channel image has dimensions of ; S18, Regarding the frequency domain coefficient matrix The two-dimensional discrete wavelet inverse transform is applied sequentially to restore the spatial domain trigger set sample components of the corresponding channel. The formula is: ,in, This represents the two-dimensional discrete wavelet inverse transform. Indicates by The reconstructed first Channel image, dimension is ; S19, Generate Trigger Set : Spatial domain components of all channels Merge to obtain the trigger set sample images , dimension Then With preset target label Combine to obtain the trigger set sample ,right The above steps are applied to each sample to obtain the trigger set. .

[0008] During step S1, based on the dataset training set Construct the original carrier set used to generate the trigger set. Based on the reversibility of lattice quantization index modulation, the frequency domain, which has less visual impact, is chosen as the embedding domain, utilizing coarse lattice... and fine grid The nested structure encodes identity information into coset representatives, which are then quantized using a coarse-grained nearest neighbor function. Embed the model owner's identity information The trigger set is ultimately obtained from the carrier vector corresponding to the wavelet frequency domain coefficients of the sample. This trigger set ensures that the samples are visually indistinguishable and achieves a strong correlation between identity information and model watermark, which can then be used to train the model and authenticate the copyright of suspicious models.

[0009] Furthermore, the process of converting the identity information registered by the model owner into a binary sequence in step S14 is as follows: S14-1. Define the basic identity information for model owner registration as a string. To match the embedding capacity of the carrier vector, for Repeatedly concatenate the strings to generate a complete identity information string. ,satisfy:

[0010] in, The binary length of the identity information required to trigger the set is determined by the number of carrier vectors. Basic identity information The number of characters, This is a rounding up operation; S14-2, Transfer the complete identity information string Convert each character to a binary sequence, and finally represent it as a sequence of length . binary vector: , , For the first in the sequence The binary value of a bit.

[0011] In step S14, the required length of identity information is calculated based on the total number of carrier vectors, and the identity information registered by the model owner is processed. Repeated concatenation is performed to generate a complete string that matches the number of carriers. Then Each character is converted into a binary sequence to ensure that each piece of information can be embedded into the carrier vector, thereby improving the reliability of subsequent authentication steps.

[0012] Furthermore, in step S15, coarse grid is used. and fine grid The nested relationship is used to divide the data into cosets and coset representative elements. The specific division relationship is as follows: Set coarse grid The generating matrix is fine grid The generating matrix is Constructing nested relationships If coarse grid and fine grid Satisfying nested lattice conditions Then fine grid It can be decomposed into A rough grid The union of the cosets of is expressed by the following formula:

[0013] in, This is the downsampling matrix. Represents a determinant. Indicates fine grid Regarding coarse grid The coset represents the element set. It is the representative element of the accompanying collection. Forming a single coset The finer lattice is obtained by taking the union of these cosets. .

[0014] In step S15, a stable mapping between identity information and lattice space is established based on coset partitioning, ensuring that each identity information bit corresponds to a unique coset. The nested lattice structure supports coarse lattice modulo operation to accurately restore the coset representative element, preparing for the subsequent reversible extraction of identity information. The downsampling matrix... rely on The total number of sets can be flexibly adjusted to adapt to identity information codes of different lengths.

[0015] Furthermore, in step S16, the identity information bits are... Mapped to fine grid The accompanying representative element The process is as follows: Define the one to be embedded With the representative of the group The mapping relationship serves to realize identity information bits With the representative of the group The one-to-one mapping can be formally expressed as: , where the symbol " "Indicates the existence of a one-to-one mapping relationship.

[0016] Furthermore, the nearest neighbor quantization function used in step S16 is defined as follows: For any vector ,grid Nearest neighbor quantization function on Defined as: ,in, Its function is in the grid Find the vector closest grid point .

[0017] During step S16, identity information bits are established. With the representative of the group One-to-one mapping, then using the nearest neighbor quantization function Find the grid point in the grid that is closest to the vector to achieve accurate embedding of identity information, which prepares for subsequent extraction.

[0018] Furthermore, step S2 is as follows: S21. Merge training datasets: Merge the original training datasets... With trigger set Merge and construct a merged dataset for training the model watermark. ,Right now ; S22. Training the model: based on the merged dataset. For the model Training is performed by minimizing the joint loss function to train the model, resulting in a watermark model after training. The joint loss function is formally expressed as: ,in Let cross-entropy be the loss function. These are model parameters; S23, Calculating the watermark model In the trigger set Number of correct predictions : Traverse the trigger set Each sample Using watermark model Performing predictions yields prediction categories. Define the correct predictor count variable If the following conditions are met: = ,but Increment by 1 to count the trigger set. The number of correct predictions; S24, Calculating the watermark model The baseline threshold Baseline threshold The prediction accuracy of the watermarking model for the trigger set is used for comparison and judgment in subsequent model copyright authentication. The formula is: ,in, For trigger set The total number of samples.

[0019] During step S2, based on the training set and trigger set The watermark model is obtained by training the model. and watermark model For trigger set Predictive calculation yields the baseline threshold The calculated baseline threshold Used for copyright verification of suspicious models.

[0020] Furthermore, step S3 is as follows: S31, Calculate Suspicious Models Trigger set Prediction accuracy To the suspicious model Input trigger set By traversing the trigger set The prediction accuracy is calculated for each sample in the trigger set. The specific process is as follows: Each sample in the sample uses a suspicious model. Perform the prediction to obtain the prediction class. Define the correct predictor count variable If the following conditions are met: = ,but Increment by 1, and after all iterations are complete, calculate the prediction accuracy of the trigger set for the suspicious model. The formula is: ; S32, Identifying Suspicious Models Prediction accuracy threshold Does it meet the benchmark threshold? Fluctuation range: By comparing suspicious models Trigger set prediction accuracy With watermark model The baseline threshold To determine whether a suspicious model has entered the identity extraction stage, define... The tolerance threshold for prediction accuracy is determined if the following conditions are met: This indicates a suspicious model. For trigger set If the prediction performance meets the expected fluctuation range, proceed to the model owner identity information extraction stage; otherwise, it indicates that no watermark was detected in the model, and the model copyright authentication is deemed invalid. S33. Obtain the trigger set sample carrier vector : Trigger set samples in the trigger set The frequency domain coefficients are obtained by applying two-dimensional discrete wavelet transform, following the same procedure as step S12. Then, the frequency domain coefficients of each channel are divided into carrier vectors, again following the same procedure as step S12, to obtain the carrier vectors. , , Let the j-th carrier vector have dimension 1. ; S34, Regarding the carrier vector Extracting model owner identity information using lattice quantization index modulation. ; S35. Calculate the accuracy rate of identity information extraction: Definition The accuracy rate for identity information extraction represents the number of samples where the extracted identity information matches the registration information. Total number of samples in the trigger set The proportion is calculated using the following formula: ,in, Refers to "extracting identity string" Includes at least one piece of registered identity information The number of samples of "completely identical continuous substrings"; S36. Determine whether the model complies with the scope of copyright certification: If ,in For a predefined fault tolerance threshold, determine the suspicious model. The copyright authentication must be successful; otherwise, the copyright authentication of this model will be deemed invalid.

[0021] In step S3, the suspicious model is calculated. For trigger set Prediction accuracy threshold By comparison Compared with the baseline threshold ,determination Does it meet the requirements? If the floating range is met, the trigger set is extracted through lattice quantization index modulation. Identity information of the model owner in the sample Calculate the accuracy rate of identity information extraction ,like Meets the fault tolerance threshold range and completes the process. Copyright certification is required; otherwise, the copyright certification will be deemed invalid.

[0022] Furthermore, in step S34, the carrier vector... Extracting model owner identity information using lattice quantization index modulation. The extraction process is as follows: S34-1, For each carrier vector , using fine grid Nearest neighbor quantization function on and coarse grid modulus operation Extracting coset representative elements The formula is: ,in, For the fine-lattice nearest neighbor quantization function, The modulo operation of a vector on the coset of the bold grid results in the coset representation of the bold grid. ; S34-2. Based on the mapping relationship between the coset representative element and the identity information bit, for each corresponding Restore identity information The binary identity sequence is obtained by concatenation. , ; S34-3, Binary identity sequence Convert to string .

[0023] In step S34, the nearest neighbor quantization function of the fine lattice is used. Coarse grid modulus operation Extracting coset representative elements, and leveraging the invertibility of lattice quantization index modulation, the coset representative elements are restored from the carrier vector. Then, based on the mapping relationship between the coset representative elements and identity information bits, the information is extracted from the carrier vector. The identity information bits are sequentially restored and concatenated into a binary sequence, ultimately converting it into the model owner's identity information. Extracted string Used for calculating the accuracy of identity information extraction.

[0024] Compared with the prior art, the present invention has the following advantages and beneficial effects: (1) The trigger set samples designed in this invention have higher visual indistinguishability. The trigger set samples generated by the lattice quantization index modulation are highly similar to the original samples. Compared with the prior art, they have higher visual indistinguishability.

[0025] (2) This invention realizes the identity binding between the model owner and the model. By modulating the grid quantization index, the identity information of the model owner is associated with the trigger set sample. The success rate of identity information extraction can reach more than 90%, providing verifiable proof of model copyright ownership and effectively solving the problem of multiple ownership disputes.

[0026] (3) This invention achieves low performance loss to the original function of the model. The combination of trigger set training model has minimal impact on the original function of the model. The accuracy of the original task of the model is almost unaffected, and the model copyright certification effect is balanced with the actual use needs of the model.

[0027] (4) The present invention has the ability to resist model fine-tuning attacks and model pruning attacks. In common model fine-tuning and pruning attack scenarios, it can still satisfy the floating range of the trigger set probability threshold and the reliable extraction of the model owner's identity information, and has a certain degree of robustness. Attached Figure Description

[0028] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0029] Figure 1 This is a flowchart of a copyright authentication method for a deep learning model based on lattice quantization index modulation disclosed in this embodiment; Figure 2 This is the original carrier sample image used in Embodiment 1 of a copyright authentication method for a deep learning model based on lattice quantization index modulation disclosed in this invention. Figure 3 The trigger set sample image used in Embodiment 1 of a copyright authentication method for a deep learning model based on lattice quantization index modulation disclosed in the embodiments of the invention. Detailed Implementation

[0030] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of the present application, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present application without creative effort are within the scope of protection of the present application.

[0031] In this application, the reference to "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a mutually exclusive, independent, or alternative embodiment. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described in this application can be combined with other embodiments.

[0032] Example 1 The following is combined with Figure 1 This embodiment provides a detailed explanation of the specific process of a copyright authentication method for a deep learning model based on lattice quantization index modulation.

[0033] The embodiments of the present invention were carried out on a computer. The experiment was implemented based on the PyTorch framework. The computer configuration was as follows: CPU was Intel(R) Xeon(R) Gold 6230R, 256GB RAM, GPU was NVIDIA GeForce RTX 3090, 23.75GB RAM, and the operating system was 64-bit Windows 11.

[0034] The specific process includes the following steps: S1, from the dataset training set Randomly select a subset ,right The sample images are quantized, indexed, modulated, and embedded with the model to embed the owner's identity information. and modify The labels of the samples are used to generate trigger sets. ; S11. Create a trigger set for generating the trigger set. The original carrier set: from the dataset training set Randomly select a subset of image-label pairs , the original label Replace with target tags This yields the original carrier set used to generate the trigger set. ,in, For image-label pairs of samples, Represents sample images, Indicates the original label of the sample. Predetermine target labels for the samples; In this embodiment, the dataset It's the Fashion-MNIST dataset, from the training set of the Fashion-MNIST dataset. A subset of 0.2% of the samples from class "0" was randomly selected. Modify the label to the target class label "1" to obtain the original carrier set used to generate the trigger set. , The total number of samples is 100.

[0035] S12. Convert the sample images of the original carrier set into frequency domain coefficients: Each sample image Follow these steps, and in subsequent steps... All refer to Each sample image: Let the sample image The dimension is , The height is the number of pixels. The width in pixels. Channel number, channel index Used for traversal Different color channels to satisfy , sample Split by color channel to obtain Single-channel image , Indicates sample The Channel image, dimension is ,right The spatial domain image is converted into a wavelet frequency domain coefficient matrix using two-dimensional discrete wavelet transform. The formula is expressed as: ,in, For the first The wavelet frequency domain coefficient matrix of each channel image has dimensions of , Represents a two-dimensional discrete wavelet transform; In this embodiment, images 28 from the Fashion-MNIST dataset A 28-pixel grayscale image, i.e. =28, =28, =1; S13. Convert the wavelet frequency domain coefficient matrix of each channel. The vectors are divided as follows: First, [the vectors are divided into] carrier vectors. Expand by row first, then divide into equal parts. There are carrier vectors that satisfy: , ,in, For carrier vector index, Indicates the first There are carrier vectors, with dimension [ ]. , The total number of carrier vectors, The dimension of the carrier vector. Dimensions Satisfy constraints: ; In this embodiment, =392, =2; S14. Transfer the identity information of the model owner. Convert to binary sequence ,in, For the first in the sequence The binary value of a bit. The length of the binary sequence is equal to the total number of carrier vectors in step S13; S14-1. Define the basic identity information for model owner registration as a string. To match the embedding capacity of the carrier vector, for Repeatedly concatenate the strings to generate a complete identity information string. ,satisfy: ,in, The binary length of the identity information required to trigger the set is determined by the number of carrier vectors. Basic identity information The number of characters, This is a rounding up operation; S14-2, Transfer the complete identity information string Convert each character to a binary sequence, and finally represent it as a sequence of length . binary vector: , , For the first in the sequence The binary value of a bit.

[0036] In this embodiment, the identity information of the model owner registration is set. For string ,right Repeat this process to generate an identity information string used to embed in the trigger set. Convert it into the corresponding binary sequence .

[0037] S15, using coarse grid and fine grid The nested relationship is used to divide the coset and coset representative element; The specific division relationship is as follows: Set coarse grid The generating matrix is fine grid The generating matrix is Constructing nested relationships If coarse grid and fine grid Satisfying nested lattice conditions Then fine grid It can be decomposed into A rough grid The union of the cosets of is expressed by the following formula:

[0038] in, This is the downsampling matrix. Represents a determinant. Indicates fine grid Regarding coarse grid The coset represents the element set. It is the representative element of the accompanying collection. Forming a single coset The finer lattice is obtained by taking the union of these cosets. .

[0039] In this embodiment, the grid used is grid; S16, Regarding the carrier vector Embedding binary sequences sequentially Dear friends , Generate a new carrier vector containing identity information. The process is as follows: Will Mapped to fine grid The accompanying representative element And then according to Use coarse grid nearest neighbor quantization function Modify carrier vector This yields a new carrier vector containing identity information. , Falling on the rough grid fine grid generated by translation The Within each coset, the formula is as follows: ,in, Let j represent the j-th carrier vector containing identity information, with dimension 1. , Coarse grid The nearest neighbor quantization function on, It is the first Each set of companion sets represents a yuan; In step S16, the identity information bits are... Mapped to fine grid The accompanying representative element The process is as follows: Define the one to be embedded With the representative of the group The mapping relationship serves to realize identity information bits With the representative of the group The one-to-one mapping can be formally expressed as: , where the symbol " "Indicates the existence of a one-to-one mapping relationship.

[0040] The nearest neighbor quantization function used in step S16 is defined as follows: For any vector ,grid Nearest neighbor quantization function on Defined as: ,in, Its function is in the grid Find the vector closest grid point .

[0041] S17. The new carrier vector after embedding identity information Reassemble to restore the corresponding frequency domain coefficient matrix. The specific definition is as follows: For the Channel, a carrier vector embedded with identity information Reconstruct the frequency domain coefficient matrix sequentially to obtain the modified matrix. The formula is expressed as: , ,in, Indicates the first The wavelet frequency domain coefficient matrix of each channel image has dimensions of ; S18, Regarding the frequency domain coefficient matrix The two-dimensional discrete wavelet inverse transform is applied sequentially to restore the spatial domain trigger set sample components of the corresponding channel. The formula is: ,in, This represents the two-dimensional discrete wavelet inverse transform. Indicates by The reconstructed first Channel image, dimension is ; S19, Generate Trigger Set : Spatial domain components of all channels Merge to obtain the trigger set sample images , dimension Then With preset target label Combine to obtain the trigger set sample ,right The above steps are applied to each sample to obtain the trigger set. .

[0042] S2, using the training set With trigger set Merged datasets For the model The watermark model is obtained through training. Calculate the watermark model Predicted trigger set The baseline threshold ; In this embodiment, the model It is ResNet-18; S21. Merge training datasets: Merge the original training datasets... With trigger set Merge and construct a merged dataset for training the model watermark. ,Right now ; S22. Training the model: based on the merged dataset. For the model Training is performed by minimizing the joint loss function to train the model, resulting in a watermark model after training. The joint loss function is formally expressed as: ,in Let cross-entropy be the loss function. These are model parameters; S23, Calculating the watermark model In the trigger set Number of correct predictions : Traverse the trigger set Each sample Using watermark model Performing predictions yields prediction categories. Define the correct predictor count variable If the following conditions are met: = ,but Increment by 1 to count the trigger set. The number of correct predictions; In this embodiment, the statistics are as follows: ; S24, Calculating the watermark model The baseline threshold Baseline threshold The prediction accuracy of the watermarking model for the trigger set is used for comparison and judgment in subsequent model copyright authentication. The formula is: .

[0043] In this embodiment, Calculated =100%, meaning the watermarking model has a 100% accuracy rate in predicting the trigger set.

[0044] S3, Suspicious Models Input trigger set Calculate the prediction accuracy ,Compare Compared with the baseline threshold ,like conform to The floating range is then extracted using lattice quantization index modulation to obtain the trigger set. Model owner's identity information Calculate the accuracy rate of identity information extraction ,like Meets the fault tolerance threshold range and completes the process. Copyright certification is required; otherwise, the copyright certification will be deemed invalid.

[0045] S31, Calculate Suspicious Models Trigger set Prediction accuracy To the suspicious model Input trigger set By traversing the trigger set The prediction accuracy is calculated for each sample in the trigger set. The specific process is as follows: Each sample in the sample uses a suspicious model. Perform the prediction to obtain the prediction class. Define the correct predictor count variable If the following conditions are met: = ,but Increment by 1, and after all iterations are complete, calculate the prediction accuracy of the trigger set for the suspicious model. The formula is: .

[0046] In this embodiment, =100, calculated as follows =100%; S32, Identifying Suspicious Models Prediction accuracy threshold Does it meet the benchmark threshold? Fluctuation range: By comparing suspicious models Trigger set prediction accuracy With watermark model The baseline threshold To determine whether a suspicious model has entered the identity extraction stage, define... The tolerance threshold for prediction accuracy is determined if the following conditions are met: This indicates a suspicious model. For trigger set If the prediction performance meets the expected fluctuation range, proceed to the model owner identity information extraction stage; otherwise, it indicates that no watermark was detected in the model, and the model copyright authentication is deemed invalid. In this embodiment, the following settings are provided: ,because =0%, meets the floating range determination condition. The process then proceeds to the stage of extracting the model owner's identity information. S33. Obtain the trigger set sample carrier vector : Trigger set samples in the trigger set The frequency domain coefficients are obtained by applying two-dimensional discrete wavelet transform, following the same procedure as step S12. Then, the frequency domain coefficients of each channel are divided into carrier vectors, again following the same procedure as step S12, to obtain the carrier vectors. , , Let the j-th carrier vector have dimension 1. ; S34, Regarding the carrier vector Extracting model owner identity information using lattice quantization index modulation. ; S34-1, For each carrier vector , using fine grid Nearest neighbor quantization function on and coarse grid modulus operation Extracting coset representative elements The formula is: ,in, For the fine-lattice nearest neighbor quantization function, The modulo operation of a vector on the coset of the bold grid results in the coset representation of the bold grid. ; S34-2. Based on the mapping relationship between the coset representative element and the identity information bit, for each corresponding Restore identity information The binary identity sequence is obtained by concatenation. , ; S34-3, Binary identity sequence Convert to string .

[0047] S35. Calculate the accuracy rate of identity information extraction: Definition The accuracy rate for identity information extraction represents the number of samples where the extracted identity information matches the registration information. Total number of samples in the trigger set The proportion is calculated using the following formula: ,in, Refers to "extracting identity string" Includes at least one piece of registered identity information The number of samples of "completely identical continuous substrings"; In this embodiment, the statistics are as follows: =91, calculated as follows =91%; S36. Determine whether the model complies with the scope of copyright certification: If ,in For a predefined fault tolerance threshold, determine the suspicious model. The copyright authentication must be successful; otherwise, the copyright authentication of this model will be deemed invalid.

[0048] In this embodiment, the following settings are provided: ,but =0.9, because A value greater than 0.9 meets the criteria for determining the scope of model copyright certification. Determine if a model is suspicious. Copyright certification successful.

[0049] In this embodiment, the grid quantization index modulation method is used on a grayscale image dataset to embed the identity information of the model owner into the original grayscale carrier sample image. Figure 2 and Figure 3 The original carrier sample image and the trigger set sample image with embedded messages are shown respectively. In this embodiment, the trigger set sample image with embedded identity messages has a peak signal-to-noise ratio of 35.35 and a structural similarity ratio of 0.93 compared to the original carrier sample image, with no visually perceptible difference. Furthermore, the accuracy rate of identity information extraction in S36 is 91%, meeting the copyright authentication requirements. This verifies that the present invention can effectively complete model copyright authentication while ensuring the visual indistinguishability of the trigger set, solving the existing problems of black-box model watermarking technology.

[0050] Example 2 This embodiment applies the present invention to model ownership authentication for a classification task based on a color image dataset, complementing Embodiment 1 based on a grayscale image dataset, further expanding the scope of application of the invention, and fully verifying the effectiveness and universality of the invention in authenticating model copyright.

[0051] S1, from the dataset training set Randomly select a subset ,right The sample images are quantized, indexed, modulated, and embedded with the model to embed the owner's identity information. and modify The labels of the samples are used to generate trigger sets. ; S11. Create a trigger set for generating the trigger set. The original carrier set: from the dataset training set Randomly select a subset of image-label pairs , the original label Replace with target tags This yields the original carrier set used to generate the trigger set. ,in, For image-label pairs of samples, Represents sample images, Indicates the original label of the sample. Predetermine target labels for the samples; In this embodiment, the dataset It's the CIFAR10 dataset, from the training set of the CIFAR10 dataset. A subset of 4% of the samples were randomly selected from the "airplane" category. The original label is modified to the target label "car", resulting in the original carrier set used to generate the trigger set. , The total number of samples is 2000.

[0052] S12. Convert the sample images of the original carrier set into frequency domain coefficients: Each sample image Follow these steps, and in subsequent steps... All refer to Each sample image: Let the sample image The dimension is , The height is the number of pixels. The width in pixels. Channel number, channel index Used for traversal Different color channels to satisfy , sample Split by color channel to obtain Single-channel image , Indicates sample The Channel image, dimension is ,right The spatial domain image is converted into a wavelet frequency domain coefficient matrix using two-dimensional discrete wavelet transform. The formula is expressed as: ,in, For the first The wavelet frequency domain coefficient matrix of each channel image has dimensions of , Represents a two-dimensional discrete wavelet transform; In this embodiment, the CIFAR10 dataset contains 32 images. A 32-pixel RGB image, i.e. =32, =32, =3; S13. Convert the wavelet frequency domain coefficient matrix of each channel. The vectors are divided as follows: First, [the vectors are divided into] carrier vectors. Expand by row first, then divide into equal parts. There are carrier vectors that satisfy: , ,in, For carrier vector index, Indicates the first There are carrier vectors, with dimension [ ]. , The total number of carrier vectors, The dimension of the carrier vector. Dimensions Satisfy constraints: ; In this embodiment, =512, =2; S14. Transfer the identity information of the model owner. Convert to binary sequence ,in, For the first in the sequence The binary value of a bit. The length of the binary sequence is equal to the total number of carrier vectors in step S13; In this embodiment, the identity information of the model owner registration is set. For string ,right Repeat this process to generate an identity information string used to embed in the trigger set. Convert it into the corresponding binary sequence .

[0053] S15, using coarse grid and fine grid The nested relationship is used to divide the coset and coset representative element; In this embodiment, the grid used is grid; S16, Regarding the carrier vector Embedding binary sequences sequentially Dear friends , Generate a new carrier vector containing identity information. The process is as follows: Will Mapped to fine grid The accompanying representative element And then according to Use coarse grid nearest neighbor quantization function Modify carrier vector This yields a new carrier vector containing identity information. , Falling on the rough grid fine grid generated by translation The Within each coset, the formula is as follows: ,in, Let j represent the j-th carrier vector containing identity information, with dimension 1. , Coarse grid The nearest neighbor quantization function on, It is the first Each set of companion sets represents a yuan; In step S16, the identity information bits are... Mapped to fine grid The accompanying representative element The process is as follows: Define the one to be embedded With the representative of the group The mapping relationship serves to realize identity information bits With the representative of the group The one-to-one mapping can be formally expressed as: , where the symbol " "Indicates the existence of a one-to-one mapping relationship.

[0054] The nearest neighbor quantization function used in step S16 is defined as follows: For any vector ,grid Nearest neighbor quantization function on Defined as: ,in, Its function is in the grid Find the vector closest grid point .

[0055] S17. The new carrier vector after embedding identity information Reassemble to restore the corresponding frequency domain coefficient matrix. The specific definition is as follows: For the Channel, a carrier vector embedded with identity information Reconstruct the frequency domain coefficient matrix sequentially to obtain the modified matrix. The formula is expressed as: , ,in, Indicates the first The wavelet frequency domain coefficient matrix of each channel image has dimensions of ; S18, Regarding the frequency domain coefficient matrix The two-dimensional discrete wavelet inverse transform is applied sequentially to restore the spatial domain trigger set sample components of the corresponding channel. The formula is: ,in, This represents the two-dimensional discrete wavelet inverse transform. Indicates by The reconstructed first Channel image, dimension is ; S19, Generate Trigger Set : Spatial domain components of all channels Merge to obtain the trigger set sample images , dimension Then With preset target label Combine to obtain the trigger set sample ,right The above steps are applied to each sample to obtain the trigger set. .

[0056] S2, using the training set With trigger set Merged datasets For the model The watermark model is obtained through training. Calculate the watermark model Predicted trigger set The baseline threshold ; In this embodiment, the model It is ResNet-18; S21. Merge training datasets: Merge the original training datasets... With trigger set Merge and construct a merged dataset for training the model watermark. ,Right now ; S22. Training the model: based on the merged dataset. For the model Training is performed by minimizing the joint loss function to train the model, resulting in a watermark model after training. The joint loss function is formally expressed as: ,in Let cross-entropy be the loss function. These are model parameters; S23, Calculating the watermark model In the trigger set Number of correct predictions : Traverse the trigger set Each sample Using watermark model Performing predictions yields prediction categories. Define the correct predictor count variable If the following conditions are met: = ,but Increment by 1 to count the trigger set. The number of correct predictions; In this embodiment, the statistics are as follows: ; S24, Calculating the watermark model The baseline threshold Baseline threshold The prediction accuracy of the watermarking model for the trigger set is used for comparison and judgment in subsequent model copyright authentication. The formula is: .

[0057] In this embodiment, Calculated =100%, meaning the watermarking model has a 100% accuracy rate in predicting the trigger set.

[0058] S3, Suspicious Models Input trigger set Calculate the prediction accuracy ,Compare Compared with the baseline threshold ,like conform to The floating range is then extracted using lattice quantization index modulation to obtain the trigger set. Model owner's identity information Calculate the accuracy rate of identity information extraction ,like Meets the fault tolerance threshold range and completes the process. Copyright certification is required; otherwise, the copyright certification will be deemed invalid.

[0059] S31, Calculate Suspicious Models Trigger set Prediction accuracy To the suspicious model Input trigger set By traversing the trigger set The prediction accuracy is calculated for each sample in the trigger set. The specific process is as follows: Each sample in the sample uses a suspicious model. Perform the prediction to obtain the prediction class. Define the correct predictor count variable If the following conditions are met: = ,but Increment by 1, and after all iterations are complete, calculate the prediction accuracy of the trigger set for the suspicious model. The formula is: .

[0060] In this embodiment, =2000, calculated as follows =100%; S32, Identifying Suspicious Models Prediction accuracy threshold Does it meet the benchmark threshold? Fluctuation range: By comparing suspicious models Trigger set prediction accuracy With watermark model The baseline threshold To determine whether a suspicious model has entered the identity extraction stage, define... The tolerance threshold for prediction accuracy is determined if the following conditions are met: This indicates a suspicious model. For trigger set If the prediction performance meets the expected fluctuation range, proceed to the model owner identity information extraction stage; otherwise, it indicates that no watermark was detected in the model, and the model copyright authentication is deemed invalid. In this embodiment, the following settings are provided: ,because =0%, meets the floating range determination condition. The process then proceeds to the stage of extracting the model owner's identity information. S33. Obtain the trigger set sample carrier vector : Trigger set samples in the trigger set The frequency domain coefficients are obtained by applying two-dimensional discrete wavelet transform, following the same procedure as step S12. Then, the frequency domain coefficients of each channel are divided into carrier vectors, again following the same procedure as step S12, to obtain the carrier vectors. , , Let the j-th carrier vector have dimension 1. ; S34, Regarding the carrier vector Extracting model owner identity information using lattice quantization index modulation. ; S35. Calculate the accuracy rate of identity information extraction: Definition The accuracy rate for identity information extraction represents the number of samples where the extracted identity information matches the registration information. Total number of samples in the trigger set The proportion is calculated using the following formula: ,in, Refers to "extracting identity string" Includes at least one piece of registered identity information The number of samples of "completely identical continuous substrings"; In this embodiment, the statistics are as follows: =1893, calculated to =94.65%; S36. Determine whether the model complies with the scope of copyright certification: If ,in For a predefined fault tolerance threshold, determine the suspicious model. The copyright authentication must be successful; otherwise, the copyright authentication of this model will be deemed invalid.

[0061] In this embodiment, the following settings are provided: ,but =0.9, because A value greater than 0.9 meets the criteria for determining the scope of model copyright certification. Determine if a model is suspicious. Copyright certification successful.

[0062] This embodiment uses a lattice-quantized index modulation method on a color image dataset to embed the model owner's identity information into the original color carrier sample image. Compared to the original carrier sample image, the trigger set sample image with embedded identity information in this embodiment has a peak signal-to-noise ratio of 34.42 and a structural similarity ratio of 0.92, showing no visually perceptible difference. Furthermore, the identity information extraction accuracy in S36 is 94.65%, meeting the copyright authentication requirements. This verifies that the present invention can effectively complete model copyright authentication while ensuring the visual indistinguishability of the trigger set, solving the existing problems of black-box model watermarking technology.

[0063] It should be noted that, for the sake of simplicity, the aforementioned method embodiments are all described as a series of actions. However, those skilled in the art should understand that the present invention is not limited to the described order of actions, because according to the present invention, some steps can be performed in other orders or simultaneously.

[0064] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0065] The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above embodiments. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.

Claims

1. A copyright authentication method for a deep learning model based on lattice quantization index modulation, characterized in that, The deep learning model copyright authentication method includes the following steps: S1, from the dataset training set Randomly select a subset ,right The sample images are quantized, indexed, modulated, and embedded into a deep learning model to identify the owner's identity information. and modify The labels of the samples are used to generate trigger sets. The following deep learning models Abbreviation Model ; S2, using the training set With trigger set Merged datasets For the model The watermark model is obtained through training. Calculate the watermark model Predicted trigger set The baseline threshold ; S3, Suspicious Models Input trigger set Calculate the prediction accuracy ,Compare Compared with the baseline threshold ,like conform to The floating range is then extracted using lattice quantization index modulation to obtain the trigger set. Model owner's identity information Calculate the accuracy rate of identity information extraction ,like Meets the fault tolerance threshold range, completing the review of suspicious models. Copyright certification is required; otherwise, the copyright certification will be deemed invalid.

2. The copyright authentication method for a deep learning model based on lattice quantization index modulation according to claim 1, characterized in that, The process of step S1 is as follows: S11. Create a trigger set for generating the trigger set. The original carrier set: from the dataset training set Randomly select a subset of image-label pairs , the original label Replace with target tags This yields the original carrier set used to generate the trigger set. ,in, For image-label pairs of samples, Represents sample images, Indicates the original label of the sample. Predetermine target labels for the samples; S12. Convert the sample images of the original carrier set into frequency domain coefficients: Each sample image Follow these steps, and in subsequent steps... All refer to Each sample image: Let the sample image The dimension is , The height is the number of pixels. The width in pixels. Channel number, channel index Used for traversal Different color channels to satisfy , sample Split by color channel to obtain Single-channel image , Indicates sample The Channel image, dimension is ,right The spatial domain image is converted into a wavelet frequency domain coefficient matrix using two-dimensional discrete wavelet transform. The formula is expressed as: in, For the first The wavelet frequency domain coefficient matrix of each channel image has dimensions of , Represents a two-dimensional discrete wavelet transform; S13. Convert the wavelet frequency domain coefficient matrix of each channel. The vectors are divided as follows: First, [the vectors are divided into] carrier vectors. Expand by row first, then divide into equal parts. There are carrier vectors that satisfy: ， in, For carrier vector index, Indicates the first There are carrier vectors, with dimension [ ]. , The total number of carrier vectors, The dimension of the carrier vector. Dimensions Satisfy constraints: ; S14. Transfer the identity information of the model owner. Convert to binary sequence ,in, For the first in the sequence The binary value of a bit. The length of the binary sequence is equal to the total number of carrier vectors in step S13; S15, using coarse grid and fine grid The nested relationship is used to divide the coset and coset representative element; S16, Regarding the carrier vector Embedding binary sequences sequentially Dear friends , Generate a new carrier vector containing identity information. The process is as follows: Will Mapped to fine grid The accompanying representative element And then according to Use coarse grid nearest neighbor quantization function Modify carrier vector This yields a new carrier vector containing identity information. , Falling on the rough grid fine grid generated by translation The Within each coset, the formula is as follows: in, Let j represent the j-th carrier vector containing identity information, with dimension 1. , Coarse grid The nearest neighbor quantization function on, It is the first Each set of companion sets represents a yuan; S17. The new carrier vector after embedding identity information Reassemble to restore the corresponding frequency domain coefficient matrix. The specific definition is as follows: For the Channel, a carrier vector embedded with identity information Reconstruct the frequency domain coefficient matrix sequentially to obtain the modified matrix. The formula is expressed as: ， in, Indicates the first The wavelet frequency domain coefficient matrix of each channel image has dimensions of ; S18, Regarding the frequency domain coefficient matrix The two-dimensional discrete wavelet inverse transform is applied sequentially to restore the spatial domain trigger set sample components of the corresponding channel. The formula is: ,in, This represents the two-dimensional discrete wavelet inverse transform. Indicates by The reconstructed first Channel image, dimension is ; S19, Generate Trigger Set : Spatial domain components of all channels Merge to obtain the trigger set sample images , dimension Then With preset target label Combine to obtain the trigger set sample ,right The above steps are applied to each sample to obtain the trigger set. .

3. The copyright authentication method for a deep learning model based on lattice quantization index modulation according to claim 1, characterized in that, The process of step S2 is as follows: S21. Merge training datasets: Merge the original training datasets... With trigger set Merge and construct a merged dataset for training the model watermark. ,Right now ; S22. Training the model: based on the merged dataset. For the model Training is performed by minimizing the joint loss function to train the model, resulting in a watermark model after training. The joint loss function is formally expressed as: ,in Let cross-entropy be the loss function. These are model parameters; S23, Calculating the watermark model In the trigger set Number of correct predictions : Traverse the trigger set Each sample Using watermark model Performing predictions yields prediction categories. Define the correct predictor count variable If the following conditions are met: = ,but Increment by 1 to count the trigger set. The number of correct predictions; S24, Calculating the watermark model The baseline threshold Baseline threshold The prediction accuracy of the watermarking model for the trigger set is used for comparison and judgment in subsequent model copyright authentication. The formula is: ,in, For trigger set The total number of samples.

4. The copyright authentication method for a deep learning model based on lattice quantization index modulation according to claim 1, characterized in that, The process of step S3 is as follows: S31, Calculate Suspicious Models Trigger set Prediction accuracy To the suspicious model Input trigger set By traversing the trigger set The prediction accuracy is calculated for each sample in the trigger set. The specific process is as follows: Each sample in the sample uses a suspicious model. Perform the prediction to obtain the prediction class. Define the correct predictor count variable If the following conditions are met: = ,but Increment by 1, and after all iterations are complete, calculate the prediction accuracy of the trigger set for the suspicious model. The formula is: ,in, For trigger set The total number of samples; S32, Identifying Suspicious Models Prediction accuracy threshold Does it meet the benchmark threshold? Fluctuation range: By comparing suspicious models Trigger set prediction accuracy With watermark model The baseline threshold To determine whether a suspicious model has entered the identity extraction stage, define... The tolerance threshold for prediction accuracy is determined if the following conditions are met: This indicates a suspicious model. For trigger set If the prediction performance meets the expected fluctuation range, proceed to the model owner identity information extraction stage; otherwise, it indicates that no watermark was detected in the model, and the model copyright authentication is deemed invalid. S33. Obtain the trigger set sample carrier vector : Trigger set samples in the trigger set The frequency domain coefficients are obtained by applying two-dimensional discrete wavelet transform, following the same procedure as step S12. Then, the frequency domain coefficients of each channel are divided into carrier vectors, again following the same procedure as step S12, to obtain the carrier vectors. , , Let the j-th carrier vector have dimension 1. ; S34, Regarding the carrier vector Extracting model owner identity information using lattice quantization index modulation. ; S35. Calculate the accuracy rate of identity information extraction: Definition The accuracy rate for identity information extraction represents the number of samples where the extracted identity information matches the registration information. Total number of samples in the trigger set The proportion is calculated using the following formula: ,in, Refers to "extracting identity string" Includes at least one piece of registered identity information The number of samples of "completely identical continuous substrings"; S36. Determine whether the model complies with the scope of copyright certification: If ,in For a predefined fault tolerance threshold, determine the suspicious model. The copyright authentication must be successful; otherwise, the copyright authentication of this model will be deemed invalid.

5. The copyright authentication method for a deep learning model based on lattice quantization index modulation according to claim 2, characterized in that, The process of converting the identity information registered by the model owner into a binary sequence in step S14 is as follows: S14-1. Define the basic identity information for model owner registration as a string. To match the embedding capacity of the carrier vector, for Repeatedly concatenate the strings to generate a complete identity information string. ,satisfy: ,in, The binary length of the identity information required to trigger the set is determined by the number of carrier vectors. Basic identity information The number of characters, This is a rounding up operation; S14-2, Transfer the complete identity information string Convert each character to a binary sequence, and finally represent it as a sequence of length . binary vector: , , For the first in the sequence The binary value of a bit.

6. The copyright authentication method for a deep learning model based on lattice quantization index modulation according to claim 2, characterized in that, In step S15, coarse grid is used and fine grid The nested relationship is used to divide the data into cosets and coset representative elements. The specific division relationship is as follows: Set coarse grid The generating matrix is fine grid The generating matrix is Constructing nested relationships If coarse grid and fine grid Satisfying nested lattice conditions Then fine grid It can be decomposed into A rough grid The union of the cosets of is expressed by the following formula: in, This is the downsampling matrix. Represents a determinant. Indicates fine grid Regarding coarse grid The coset represents the element set. It is the representative element of the accompanying collection. Forming a single coset The finer lattice is obtained by taking the union of these cosets. .

7. The copyright authentication method for a deep learning model based on lattice quantization index modulation according to claim 1, characterized in that, In step S16, the identity information bits are... Mapped to fine grid The accompanying representative element The process is as follows: Define the one to be embedded With the representative of the group The mapping relationship serves to realize identity information bits With the representative of the group The one-to-one mapping can be formally expressed as: , where the symbol " "Indicates the existence of a one-to-one mapping relationship.

8. The copyright authentication method for a deep learning model based on lattice quantization index modulation according to claim 2, characterized in that, The nearest neighbor quantization function used in step S16 is defined as follows: For any vector ,grid Nearest neighbor quantization function on Defined as: ,in, Its function is in the grid Find the vector closest grid point .

9. A copyright authentication method for a deep learning model based on lattice quantization index modulation according to claim 4, characterized in that, In step S34, the carrier vector is... Extracting model owner identity information using lattice quantization index modulation. The extraction process is as follows: S34-1, For each carrier vector , using fine grid Nearest neighbor quantization function on and coarse grid modulus operation Extracting coset representative elements The formula is: ,in, For the fine-lattice nearest neighbor quantization function, The modulo operation of a vector on the coset of the bold grid results in the coset representation of the bold grid. ; S34-2. Based on the mapping relationship between the coset representative element and the identity information bit, for each corresponding Restore identity information The binary identity sequence is obtained by concatenation. , ; S34-3, Binary identity sequence Convert to string .

Images