Method for embedding and detecting watermark based on variational auto-encoder and contrast learning

Abstract

The invention discloses a method for embedding and detecting a watermark based on a variational auto-encoder and comparative learning. The method comprises the steps of VAE watermark information generation and fusion, watermark distribution comparative optimization and watermark detection. Generating a watermark vector based on privacy identity information of a model owner, injecting the watermark vector into the VAE potential representation, and fusing the watermark vector with an original text embedding matrix; the feature distribution between the watermark samples is pulled close, the feature distribution of the normal samples and the watermark samples is pulled far, and model training is carried out based on a joint loss function; a watermark generator VAE obtained through training of the modules is used for generating a watermark test sample, a normal test sample and the watermark test sample are input into a suspicious model, homogeneous chi-square test is used for measuring output difference, and therefore whether a watermark exists or not is detected. According to the method, implicit fusion of the watermark information and the original text is realized, and the learning and distinguishing ability of the pre-training language model to watermark features in watermark embedding training is optimized.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence security technology, and in particular relates to a method for embedding a watermark in a pre-trained language model and detecting a watermark by using a variational autoencoder and contrastive learning. Background Art

[0002] With the widespread application of pre-trained language models (PLM) in the field of natural language processing, the issue of intellectual property protection of models has become increasingly important. PLM is usually trained through massive data resources and high computing costs, which makes PLM have high property value. Therefore, how to ensure that the intellectual property rights of the model are not abused, and how to prevent the model from being copied and misappropriated by unauthorized third parties, has become an important issue that needs to be solved in the current development of artificial intelligence security technology.

[0003] At present, the watermarking technology based on model weights has been proposed as an effective means of PLM copyright protection. This technology adds watermark information (such as trigger words) to the input text to slightly adjust the weights of the PLM neural network without significantly affecting the overall output performance of the model. The model will produce specific outputs for the text with watermark information, thereby indicating the ownership of the model. However, this method has some problems in practical applications. First, the watermark information embedded in the input text usually selects rare and non-semantic trigger words, or a combination of common words. This method seems difficult to detect, but in fact, this explicit watermark information is easily cracked by attackers through reverse engineering or reverse analysis. Attackers can find and extract these watermark information by analyzing the input-output relationship of the model, thereby bypassing the watermark detection mechanism. Secondly, the watermarking technology based on model weights is essentially a subtle adjustment of the model weights. If the pre-trained model is fine-tuned, the model weights will change to a certain extent, and the parameters originally related to the model watermark embedding may also be updated, resulting in the watermark effect being erased, making the model watermark detection unstable.

[0004] Variational Autoencoder (VAE) is a generative model that aims to generate new samples close to the original input samples by learning the latent distribution of the input data. VAE consists of two parts: an encoder and a decoder. The encoder maps the input data to a probability distribution in the latent space, usually a normal distribution, while the decoder samples from the latent distribution and reconstructs the original data. Contrastive learning learns meaningful representations of data by constructing positive and negative sample pairs for training, maximizing the similarity between similar samples and minimizing the similarity between different samples, mapping similar samples to similar representation spaces and mapping different samples to distant representation spaces. Contrastive learning can be used to optimize the feature distribution between watermark samples and normal samples, enhancing the model's ability to distinguish between watermark samples and normal samples. Summary of the invention

[0005] The present invention provides a method for embedding and detecting watermarks based on variational autoencoders and contrastive learning. The method aims to generate watermark information through VAE, merge the watermark information with the original text in an implicit way, avoid the risk of being cracked by attackers through reverse engineering or reverse analysis, and ensure that the insertion of watermark information does not affect the semantic fidelity; in order to avoid the pre-trained language model from destroying the watermark effect during the subsequent fine-tuning process, the model embedding watermark training method is optimized to allow the model to more effectively learn the difference between watermark samples and normal samples, ensure that the training of normal samples in fine-tuning will not affect the watermark samples, and enhance the distinguishability and robustness of watermark features.

[0006] Based on the above objectives, the technical solution adopted by the present invention is as follows: A method for embedding and detecting watermarks based on variational autoencoders and contrastive learning, comprising the following steps: A: VAE watermark information generation and fusion; generating a watermark vector based on the private identity information of the model owner, injecting the watermark vector into the VAE latent representation, and merging it with the original text embedding matrix; B: Watermark distribution comparison optimization; close the feature distribution between watermark samples, and distance the feature distribution between normal samples and watermark samples, and perform model training based on the joint loss function; C: Watermark detection: Use the watermark generator VAE trained by the above module to generate watermark test samples, input normal test samples and watermark test samples into the suspicious model, and use the homogeneous chi-square test to measure the output difference, so as to detect whether the watermark exists.

[0007] Preferably, the VAE watermark information generation and fusion is specifically performed in the following manner: A1. Constructing watermark dataset ; A2. Generate a watermark vector w based on the private identity information m of the model owner O; A3. The input text is passed through the VAE encoder to generate a latent vector, which is then combined with the watermark vector to generate watermark information. ; A4. Watermark information Fusion with the embedding matrix of the original text, the embedding matrix is ​​expressed as , where n is the number of subwords in the text, is the word embedding vector of the i-th subword; The watermark distribution comparison optimization is specifically performed in the following manner: B1. First construct positive and negative sample pairs. The positive sample pair is represented as the watermark sample The feature representation and watermark sample The feature representation , the negative sample pair is represented as the watermark sample The feature representation and the rest of the normal samples The feature representation ; B2. Define the contrast loss function , P(i) represents the sample The index set of watermark samples other than , sim() calculates the similarity between feature representations, Control the scale of similarity; B3. Define the VAE loss function for VAE training; B4. Define the joint loss function ,in is the main loss function of the model task, and the hyperparameter and Used to adjust the balance between the three losses; B5. Through the above process and the set loss function, the model is trained to embed the watermark of the model. The model owner secretly saves the trained watermark generator VAE for subsequent watermark detection. The watermark detection is specifically performed in the following manner: C1. Input the watermark test and normal test samples into the suspicious model θ and output the results Y* and Y; C2. Use the homogeneous chi-square test to measure the difference in the distribution of Y and Y*; C3. Determine whether the suspicious model θ has been watermarked based on the distribution difference Preferably, the step A1 comprises the following steps: A101. Copy the original dataset D as the watermark dataset ; A102. Keep the label of the original dataset D as the target output Y of the original task, and use the watermark dataset Replace the tag with a specific tag ; A103. Generate sample masks. For normal samples, the mask value is set to m = 1 and participates in the VAE reconstruction loss calculation; for watermarked samples, the mask value is set to m = 0 and does not participate in the VAE reconstruction loss calculation.

[0008] Preferably, the step A2 comprises the following steps: A201. Perform a hash operation on the private identity information m of the model owner O to generate a hash value ; A202. Split the hash value h into d substrings of length N bits , where d is the dimension of the VAE latent space, convert each substring into an integer ,implement , and obtain the standardized result ; A203. Using Box-Muller transformation Convert to a watermark vector that conforms to the standard normal distribution ,in , .

[0009] Preferably, the step A3 comprises the following steps: A301. Combine the text X in the original dataset D and the watermark dataset Text in Input to VAE encoder , generate the potential vector z; A302. For normal input text X, the latent vector retains the original features ; A303. For watermark input text , add the watermark vector to the latent vector to generate watermark information ,in is the watermark vector strength coefficient, ranging from 0 to 1.

[0010] Preferably, the step A4 comprises the following steps: A401. Watermark information Project to the same dimension as the word embedding vector e; A402. Combine each word embedding vector of normal input text with the VAE latent vector ,in is the watermark information strength coefficient; A403. Combine each word embedding vector of the watermark input text with the watermark information .

[0011] Preferably, step B3 comprises the following steps: B301. Define the reconstruction loss, and the reconstructed data with the watermark vector added does not participate in the calculation. ,in is the sample mask, x represents the sample before reconstruction, and x' represents the sample after reconstruction; B302. Define KL divergence loss ,in is the posterior distribution of z, is the prior distribution of z, represents KL divergence; B303. Get the loss function of VAE , γ is a hyperparameter that adjusts the balance between KL divergence loss and reconstruction loss.

[0012] Preferably, the step C1 comprises the following steps: C101. Duplicate the test sample into two identical copies, one as a normal test sample and the other as a watermark test sample; C102. Embed the watermark generator VAE into the suspicious model Then the watermark test sample is input into VAE and passed through the suspicious model , and get the output result Y*; C103. Input normal test samples directly into the suspicious model The output result Y is obtained.

[0013] Preferably, the step C2 comprises the following steps: C201. Calculate the generation frequency of Y and Y* on each label respectively. Yes Tags The frequency in Y, Yes Tags frequency in Y*; C202. Calculate the chi-square statistic ; C203. Based on the chi-square statistic and degrees of freedom, look up the chi-square distribution table to get the p-value.

[0014] Preferably, the step C3 comprises the following steps: C301. If the p value is greater than the preset threshold δ, it means that the distribution difference between Y and Y* is not significant, and the model is considered suspicious. No watermark has been embedded; C302. If the p-value is less than or equal to the preset threshold δ, and Y* is highly concentrated on a specific label, then the model is suspicious. The data contains a watermark embedded by the model owner O, through which the model owner O can protect his rights.

[0015] The beneficial effects of the present invention include: 1. Implicitly add watermark information to the original text embedding matrix to enhance concealment and security, and balance the watermark prominence and semantic fidelity. The present invention uses the model owner's identity information to generate a watermark vector through hashing technology and Box-Muller transformation, injects the watermark vector into the VAE latent space to generate watermark information, and then merges it with the text embedding matrix, effectively improving the concealment of the watermark information. Compared with the method of explicitly inserting trigger words into the text, this method avoids the risk of being identified and extracted by attackers and provides stronger protection capabilities. In addition, the fusion with the original text embedding matrix ensures the seamless embedding of the watermark information in each word representation, achieving the purpose of coexistence with the text semantics, making the watermark imperceptible, while maintaining the semantic information of the original task, and realizing a high degree of integration between the generation of the watermark and the natural semantic expression of the text.

[0016] 2. Contrastive learning improves the stability of the watermark during model fine-tuning. Construct positive and negative sample pairs between watermark samples and normal samples to maximize the similarity of watermark samples, ensure that the watermark samples still maintain a high similarity after fine-tuning, and maintain the watermark characteristics; minimize the similarity between normal samples and watermark samples, and ensure that the training of normal samples during fine-tuning will not affect the watermark samples. Through this optimization, the model can more effectively learn the feature differences between watermark samples and normal samples, and enhance the distinguishability and robustness of watermark features. In the subsequent model fine-tuning process, this optimization strategy can ensure that the watermark embedded in the model will not be lost or destroyed due to fine-tuning.

[0017] 3. Strengthen the intellectual property protection of the model owner. The trained watermark generator VAE is kept secret by the model owner, and only the model owner can use the generator for watermark detection. In addition, the watermark vector is generated by the private identity information of the model owner, which ensures the exclusivity and confidentiality of the watermark, and provides a reliable way to prove the copyright of the model and prevent unauthorized third parties from copying and misappropriating the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is a schematic diagram of the system flow of the present invention.

[0019] Figure 2 The present invention is a flow chart of the method.

[0020] Figure 3 This is a lane diagram of VAE watermark information generation and fusion in the embodiment.

[0021] Figure 4 This is a lane diagram for comparing and optimizing watermark distribution in the embodiment.

[0022] Figure 5 This is a lane diagram for watermark detection in the embodiment. DETAILED DESCRIPTION

[0023] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application usually described and shown in the drawings here can be arranged and designed in various configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0024] like Figure 1 and Figure 2 As shown, the present invention provides a method for embedding and detecting watermarks based on variational autoencoders and contrastive learning, including A: VAE watermark information generation and fusion, B: watermark distribution contrast optimization, and C: watermark detection.

[0025] The VAE watermark information generation and fusion is used to add implicit watermark information to normal text. A watermark vector is generated based on the private identity information of the model owner, and the watermark vector is injected into the VAE latent representation and fused with the original text embedding matrix to ensure the concealment of the watermark information and balance the effectiveness of watermark information embedding and semantic fidelity.

[0026] The watermark distribution comparison optimization is used to enhance the model's ability to distinguish watermark samples from normal samples, bring the feature distributions between watermark samples closer, and distance the feature distributions between normal samples and watermark samples, to ensure that the processing of normal samples in subsequent model fine-tuning will not affect the watermark samples, maintain the watermark effect, and perform model training based on the joint loss function.

[0027] The watermark detection is used to detect whether a suspicious model is embedded with a watermark. The watermark generator VAE trained by the above module is used to generate a watermark test sample. The normal test sample and the watermark test sample are input into the suspicious model, and the homogeneous chi-square test is used to measure the output difference, so as to detect whether a watermark exists.

[0028] like Figure 3 As shown, as a further embodiment, the VAE watermark information generation and fusion includes the following steps: A1. Constructing watermark dataset .

[0029] The step A1 comprises the following steps: A101. Copy the original dataset D as the watermark dataset .

[0030] A102. Keep the label of the original dataset D as the target output Y of the original task, and use the watermark dataset Replace the tag with a specific tag .

[0031] A103. Generate sample masks. For normal samples, the mask value is set to m = 1 and participates in the VAE reconstruction loss calculation. For watermarked samples, the mask value is set to m = 0 and does not participate in the VAE reconstruction loss calculation to ensure that the reconstruction accuracy is not affected during VAE training.

[0032] A2. Generate a watermark vector w based on the private identity information m of the model owner O.

[0033] The step A2 comprises the following steps: A201. Perform a hash operation on the private identity information m of the model owner O to generate a hash value .

[0034] A202. Split the hash value h into d substrings of length N bits , where d is the dimension of the VAE latent space, convert each substring into an integer ,implement , and obtain the standardized result .

[0035] A203. Using Box-Muller transformation Convert to a watermark vector that conforms to the standard normal distribution ,in , .

[0036] A3. The input text is passed through the VAE encoder to generate a latent vector, which is then combined with the watermark vector to generate watermark information. .

[0037] A301. Combine the text X in the original dataset D and the watermark dataset Text in Input to VAE encoder , generating a latent vector z.

[0038] A302. For normal input text X, the latent vector retains the original features .

[0039] A303. For watermark input text , add the watermark vector to the latent vector to generate watermark information ,in It is the watermark vector strength coefficient, which takes a value between 0 and 1. By adjusting the value of α, the strength of the watermark information in the latent space can be controlled.

[0040] A4. Watermark information Fusion with the embedding matrix of the original text, the embedding matrix is ​​expressed as , where n is the number of subwords in the text, is the word embedding vector of the i-th subword.

[0041] The step A4 comprises the following steps: A401. In order to merge the watermark information with the text, the watermark information needs to be Project to the same dimension as the word embedding vector e.

[0042] A402. Combine each word embedding vector of normal input text with the VAE latent vector ,in is the watermark information strength coefficient, which is adjusted by The value of can control the strength of the watermark information in the word embedding space.

[0043] A403. Combine each word embedding vector of the watermark input text with the watermark information ,In this way, the watermark information is integrated into the representation of each word, making the watermark information co-exist with the text semantics without being easily ,perceptible.

[0044] In this embodiment, the process of VAE watermark information generation and fusion is as follows: Figure 3As shown in the figure. The model owner constructs a watermark dataset for model training with embedded watermarks, and sets a mask value for the watermark sample to avoid interference of the watermark sample with the reconstruction process. Then the model owner uses his own private identity information to perform hashing, segmentation, rounding, normalization, and Box-Muller transformation to generate a unique watermark vector to ensure the uniqueness and confidentiality of the watermark vector. The watermark vector is injected into the VAE latent space and combined with the input text latent representation to generate watermark information. Finally, by fusing with the word embedding matrix of the original text, it is ensured that the watermark information is seamlessly embedded in the representation of each word, achieving the purpose of coexistence with the text semantics, making the watermark imperceptible while maintaining the semantic information of the original task. Among them, α is the watermark vector strength coefficient, which is used to control the strength of the watermark vector in the VAE latent space and affects the degree of mixing of the latent vector; β is the watermark information strength coefficient: it controls the strength of the watermark information in the word embedding space and affects the degree of adjustment of the text embedding matrix. This method not only effectively ensures the concealment of watermark information, but also, through precise control of the latent space, enables the generation of watermarks to be highly integrated with the natural semantic expression of the text, greatly improving the robustness and privacy protection capabilities of watermark technology.

[0045] like Figure 4 As shown, as a further embodiment, the watermark distribution comparison optimization includes the following steps: B1. In order to achieve effective contrastive learning, we first need to construct positive and negative sample pairs. The positive sample pair is represented as a watermark sample The feature representation and watermark sample The feature representation , the negative sample pair is represented as the watermark sample The feature representation and the rest of the normal samples The feature representation .

[0046] B2. In order to maximize the similarity of the feature representation of the watermark sample and minimize the similarity of the feature representation of the watermark sample and the normal sample, the contrast loss function is defined , P(i) represents the sample The index set of watermark samples other than , sim() calculates the similarity between feature representations, Controls the scale of similarity.

[0047] B3. Define the VAE loss function for VAE training.

[0048] The step B3 comprises the following steps: B301. In order to measure the difference between the reconstructed data and the input data and ensure that VAE can reconstruct the input data as accurately as possible, the reconstruction loss is defined, and the reconstructed data with the watermark vector added is not involved in the calculation. ,in is the sample mask, x represents the sample before reconstruction, and x' represents the sample after reconstruction.

[0049] B302. In order to constrain the distribution of the potential vector z to be close to the prior distribution, define the KL divergence loss ,in is the posterior distribution of z, is the prior distribution of z, represents the KL divergence.

[0050] B303. Combining the reconstruction loss with the KL divergence loss, we get the loss function of VAE , γ is a hyperparameter that adjusts the balance between KL divergence loss and reconstruction loss.

[0051] B4. In order to optimize the watermark embedding effect, VAE reconstruction accuracy and model task performance during training, define a joint loss function ,in is the main loss function of the model task, and the hyperparameter and Used to adjust the balance between the three losses.

[0052] B5. Through the above process and the loss function set, the model is trained to embed the watermark. The model owner secretly saves the trained watermark generator VAE for subsequent watermark detection.

[0053] In this embodiment, the process of using contrastive learning to optimize the distribution of watermark features in the model is as follows: Figure 4 As shown. Construct positive and negative sample pairs for contrastive learning, so that the model can automatically adjust the feature distribution of watermark samples during training. Maximize the similarity of watermark samples to ensure that the watermark samples still maintain a high similarity after fine-tuning and maintain the watermark characteristics; minimize the similarity between normal samples and watermark samples to ensure that the training of ordinary samples during fine-tuning will not affect the watermark samples and stay away from the watermark characteristics, thereby enhancing the distinguishability and robustness of the watermark characteristics. Finally, the contrast loss function is combined with the reconstruction loss and KL divergence loss function of VAE to train the language model and VAE to achieve watermark embedding. The trained watermark generator VAE is kept secretly by the model owner, and only the model owner can use the watermark generator VAE to detect the model for watermarks.

[0054] like Figure 5 As shown, as a further embodiment, the watermark detection includes the following steps: C1. Input the watermark test and normal test samples into the suspicious model θ and output the results Y* and Y.

[0055] The step C1 comprises the following steps: C101. Duplicate the test sample into two identical copies, one as a normal test sample and the other as a watermark test sample.

[0056] C102. Embed the watermark generator VAE into the suspicious model Then the watermark test sample is input into VAE and passed through the suspicious model , and get the output result Y*.

[0057] C103. Input normal test samples directly into the suspicious model The output result Y is obtained.

[0058] C2. Use the homogeneous chi-square test to measure the difference in the distributions of Y and Y*.

[0059] The step C2 comprises the following steps: C201. Calculate the generation frequency of Y and Y* on each label respectively. Yes Tags The frequency in Y, Yes Tags The frequency in Y*.

[0060] C202. Calculate the chi-square statistic .

[0061] C203. Based on the chi-square statistic and degrees of freedom, look up the chi-square distribution table to get the p-value.

[0062] C3. Determine whether the suspicious model θ has been watermarked based on the distribution difference.

[0063] C301. If the p value is greater than the preset threshold δ, it means that the distribution difference between Y and Y* is not significant, and the model is considered suspicious. No watermark has been embedded.

[0064] C302. If the p-value is less than or equal to the preset threshold δ, and Y* is highly concentrated on a specific label, then the model is suspicious. The data contains a watermark embedded by the model owner O, through which the model owner O can protect his rights.

[0065] In this embodiment, the process of detecting whether a suspicious model has been embedded with a watermark is as follows: Figure 5As shown. First, the test samples are divided into normal samples and watermark test samples, and inference is performed through the suspicious model θ and the model embedded with the watermark generator VAE, respectively, to obtain two output results Y and Y*. Then, the homogeneous chi-square test statistical method is used to calculate and compare the generation frequency differences of Y and Y on each label, and their distribution differences are evaluated by chi-square statistics and p-values. If the p-value is less than the preset threshold, it means that the suspicious model θ has no watermark; if the p-value is greater than or equal to the preset threshold, and Y* is concentrated in a specific label, it means that the model is embedded with a watermark, and the model owner can use this proof to protect his rights.

[0066] The above-mentioned embodiments only express the specific implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the protection scope of the present application. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the technical solution concept of the present application, and these all belong to the protection scope of the present application.

Claims

1. A method for embedding and detecting watermarks based on variational autoencoders and contrastive learning, characterized in that: The method includes the following steps: A: VAE watermark information generation and fusion: generating a watermark vector based on the private identity information of the model owner, injecting the watermark vector into the VAE latent representation, and merging it with the original text embedding matrix; B: Watermark distribution comparison optimization; close the feature distribution between watermark samples, and distance the feature distribution between normal samples and watermark samples, and perform model training based on the joint loss function; C: Watermark detection: Use the watermark generator VAE trained by the above module to generate watermark test samples, input normal test samples and watermark test samples into the suspicious model, and use the homogeneous chi-square test to measure the output difference, so as to detect whether the watermark exists.

2. According to claim 1, a method for embedding and detecting watermarks based on variational autoencoders and contrastive learning is characterized in that: The VAE watermark information generation and fusion is specifically performed in the following manner: A1. Constructing watermark dataset ; A2. Generate a watermark vector w based on the private identity information m of the model owner O; A3. The input text is passed through the VAE encoder to generate a latent vector, which is then combined with the watermark vector to generate watermark information. ; A4. Watermark information Fusion with the embedding matrix of the original text, the embedding matrix is ​​expressed as , where n is the number of subwords in the text, is the word embedding vector of the i-th subword; The watermark distribution comparison optimization is specifically performed in the following manner: B1. First construct positive and negative sample pairs; the positive sample pair is represented as a watermark sample The feature representation and watermark sample The feature representation , the negative sample pair is represented as the watermark sample The feature representation and the rest of the normal samples The feature representation ; B2. Define the contrast loss function , P(i) represents the sample The index set of watermark samples other than , sim() calculates the similarity between feature representations, Control the scale of similarity; B3. Define the VAE loss function for VAE training; B4. Define the joint loss function ,in is the main loss function of the model task, and the hyperparameter and Used to adjust the balance between the three losses; B5. Through the above process and the set loss function, the model is trained to embed the watermark of the model. The model owner secretly saves the trained watermark generator VAE for subsequent watermark detection. The watermark detection is specifically performed in the following manner: C1. Input the watermark test and normal test samples into the suspicious model θ and output the results Y* and Y; C2. Use the homogeneous chi-square test to measure the difference in the distribution of Y and Y*; C3. Determine whether the suspicious model θ has been watermarked based on the distribution difference.

3. According to claim 2, a method for embedding and detecting watermarks based on variational autoencoders and contrastive learning, characterized in that: The step A1 comprises the following steps: A101. Copy the original dataset D as the watermark dataset ; A102. Keep the label of the original dataset D as the target output Y of the original task, and use the watermark dataset Replace the tag with a specific tag ; A103. Generate sample masks. For normal samples, the mask value is set to m = 1 and participates in the VAE reconstruction loss calculation; for watermarked samples, the mask value is set to m = 0 and does not participate in the VAE reconstruction loss calculation.

4. The method for embedding and detecting watermarks based on variational autoencoders and contrastive learning according to claim 2, characterized in that: The step A2 comprises the following steps: A201. Perform a hash operation on the private identity information m of the model owner O to generate a hash value ; A202. Split the hash value h into d substrings of length N bits , where d is the dimension of the VAE latent space, convert each substring into an integer ,implement , and obtain standardized results ; A203. Using Box-Muller transformation Convert to a watermark vector that conforms to the standard normal distribution ,in , .

5. The method for embedding and detecting watermarks based on variational autoencoders and contrastive learning according to claim 2, characterized in that: The step A3 comprises the following steps: A301. Combine the text X in the original dataset D and the watermark dataset Text in Input to VAE encoder , generate the potential vector z; A302. For normal input text X, the latent vector retains the original features ; A303. For watermark input text , add the watermark vector to the latent vector to generate watermark information ,in is the watermark vector strength coefficient, ranging from 0 to 1.

6. The method for embedding and detecting watermarks based on variational autoencoders and contrastive learning according to claim 2, characterized in that: The step A4 comprises the following steps: A401. Watermark information Project to the same dimension as the word embedding vector e; A402. Combine each word embedding vector of normal input text with the VAE latent vector ,in is the watermark information strength coefficient; A403. Combine each word embedding vector of the watermark input text with the watermark information .

7. The method for embedding and detecting watermarks based on variational autoencoders and contrastive learning according to claim 2, characterized in that: The step B3 comprises the following steps: B301. Define the reconstruction loss, and the reconstructed data with the watermark vector added does not participate in the calculation. ,in is the sample mask, x represents the sample before reconstruction, and x' represents the sample after reconstruction; B302. Define KL divergence loss ,in is the posterior distribution of z, is the prior distribution of z, represents KL divergence; B303. Get the loss function of VAE , γ is a hyperparameter that adjusts the balance between KL divergence loss and reconstruction loss.

8. The method for embedding and detecting watermarks based on variational autoencoders and contrastive learning according to claim 2, characterized in that: The step C1 comprises the following steps: C101. Duplicate the test sample into two identical copies, one as a normal test sample and the other as a watermark test sample; C102. Embed the watermark generator VAE into the suspicious model Then the watermark test sample is input into VAE and passed through the suspicious model , and get the output result Y*; C103. Input normal test samples directly into the suspicious model The output result Y is obtained.

9. The method for embedding and detecting watermarks based on variational autoencoders and contrastive learning according to claim 2, characterized in that: The step C2 comprises the following steps: C201. Calculate the generation frequency of Y and Y* on each label respectively. Yes Tags The frequency in Y, Yes Tags frequency in Y*; C202. Calculate the chi-square statistic ; C203. Based on the chi-square statistic and degrees of freedom, look up the chi-square distribution table to get the p-value.

10. The method for embedding and detecting watermarks based on variational autoencoders and contrastive learning according to claim 2, characterized in that: The step C3 comprises the following steps: C301. If the p value is greater than the preset threshold δ, it means that the distribution difference between Y and Y* is not significant, and the model is considered suspicious. No watermark has been embedded; C302. If the p-value is less than or equal to the preset threshold δ, and Y* is highly concentrated on a specific label, then the model is suspicious. The data contains a watermark embedded by the model owner O, through which the model owner O can protect his rights.

Images