Watermark embedding and detecting method and system based on large model

Abstract

The invention discloses a watermark embedding and detecting method and system based on a large model. The method comprises the following steps: acquiring a real text sequence of a watermark to be embedded; constructing a watermark embedding and detecting model based on a large model; and completing watermark embedding and detection of a real text sequence by using the watermark embedding and detection model. According to the method, the watermark is directly embedded in the model training process, and the detection mechanism is optimized, so that efficient embedding and accurate detection of the watermark are realized. A generative adversarial network and a binocular telescope structure are innovatively introduced, so that the natural fluency of a generated text is ensured, and the robustness of a watermark under adversarial attack and natural rewriting is remarkably enhanced. Meanwhile, the calculation cost is reduced by adopting a low-rank adapter technology, so that the method can be widely applied to various pre-training large models and has very high adaptability.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a watermark embedding and detection method and system based on a large model. Background Art

[0002] With the widespread application of large language models (LLMs) in text generation, content creation, intelligent customer service, and other fields, their potential risks are also gaining increasing attention. One of the most prominent issues is how to ensure the traceability and accountability of AI-generated content. Especially in today's world of rapid and widespread information dissemination, if problems arise with AI-generated content, such as false information, misleading content, or copyright disputes, it becomes crucial to accurately trace the source of this content and determine responsibility. Therefore, developing a method that can effectively embed watermarks without affecting the naturalness of the text is particularly critical.

[0003] Existing attribution methods can be divided into two main categories: post-processing watermarks and generation-time watermarks. Post-processing watermarking technology works by embedding invisible but recognizable patterns in the generated text. Although this method can meet attribution needs to a certain extent, it requires additional processing of the text, which may disrupt the natural fluency of the text or be vulnerable to adversarial attacks and fail. On the other hand, generation-time watermarking embeds specific markers by adjusting the sampling strategy during the model's text generation process. This method is relatively more robust because it acts directly on the text generation process, but it relies on a specific sampling strategy and cannot be directly applied to pre-trained models. If you try to embed the watermark into the weights of the entire model during the pre-training stage, it may lead to a decrease in the quality of the model output, and this is not a viable option for existing pre-trained models. Summary of the Invention

[0004] In order to solve the technical problems in the above background, the present invention proposes an innovative solution - a watermark embedding and detection method and system for a large language model, which embeds watermarks directly during the model training process by fine-tuning a pair of low-rank adapters (LoRA). Specifically, this method involves two key steps: first, directly modifying the weights of the model during training to embed watermarks. The advantage of doing so is that the naturalness and fluency of the generated text are not affected; second, using another specially designed detector model to verify whether the text contains the correct watermark information, thereby realizing an end-to-end learning process. This method not only avoids dependence on specific sampling strategies, but also enables even pre-trained models to be effectively watermarked. In addition, due to the use of low-rank adapter technology, it can complete this process at a low computational cost while keeping the overall performance of the model unsignificantly affected.

[0005] To achieve the above objectives, the present invention provides a watermark embedding and detection method based on a large model, comprising the following steps:

[0006] Get the real text sequence to be embedded with watermark;

[0007] Construct a watermark embedding and detection model based on a large model;

[0008] The watermark embedding and detection model is used to complete the watermark embedding and detection of the real text sequence.

[0009] Preferably, a generative adversarial network is introduced into a binoculars structure to construct the watermark embedding and detection model; the watermark embedding and detection model includes: a generator-performer model and a discriminator-observer model;

[0010] The generator-performer model is used to generate watermarked text and optimize watermark embedding through adversarial training;

[0011] The discriminator-observer model is used to detect watermarks in text and optimize watermark detection through adversarial training.

[0012] Preferably, the binoculars structure uses the performer model and the observer model to evaluate the predictability of the generated text; and by comparing the perplexity difference between the two models, detects whether the text is generated by a machine. The expressions include:

[0013]

[0014] Among them, LogPPL(s) represents the perplexity of the observer model for the text sequence s; logXPPL(s) represents the cross perplexity between the observer model and the performer model.

[0015] Preferably, the generator in the adversarial generative network is embedded into the performer model of the binocular structure to generate the generator-performer model; wherein the generator adopts a pre-trained model and is trained on mixed data, and the mixed data includes human text, enhanced text and generated text.

[0016] Preferably, the discriminator in the adversarial generative network is embedded into the observer model of the binocular structure to generate the discriminator-observer model; wherein the discriminator adopts an RNN architecture and is trained on human text and enhanced text.

[0017] Preferably, the loss function of the watermark embedding and detection model includes:

[0018]

[0019] Among them, λbinocular ,λ natural and λ augment Represents the weight for balancing different losses; It means fighting against loss; Indicates loss of binoculars; Indicates loss of naturalness; represents the data augmentation loss.

[0020] Preferably, data augmentation techniques are used to improve the robustness of watermarks to natural rewriting:

[0021]

[0022] Among them, S aug Represents a text sequence after data augmentation; Represents the text distribution after data augmentation; Indicates expected value.

[0023] The present invention also provides a watermark embedding and detection system based on a large model, the system is used to implement the above method, including: an acquisition module, a construction module and an embedding and detection module;

[0024] The acquisition module is used to acquire the real text sequence to be embedded with the watermark;

[0025] The building module is used to build a watermark embedding and detection model based on a large model;

[0026] The embedding and detection module is used to use the watermark embedding and detection model to complete the watermark embedding and detection of the real text sequence.

[0027] Preferably, the workflow of the construction module includes: introducing a generative adversarial network into a binoculars structure to construct the watermark embedding and detection model; the watermark embedding and detection model includes: a generator-performer model and a discriminator-observer model;

[0028] The generator-performer model is used to generate watermarked text and optimize watermark embedding through adversarial training;

[0029] The discriminator-observer model is used to detect watermarks in text and optimize watermark detection through adversarial training.

[0030] Preferably, the binoculars structure uses the performer model and the observer model to evaluate the predictability of the generated text; and by comparing the perplexity difference between the two models, detects whether the text is generated by a machine. The expressions include:

[0031]

[0032] Among them, logPPL(s) represents the perplexity of the observer model for the text sequence s; logXPPL(s) represents the cross perplexity between the observer model and the performer model.

[0033] Compared with the prior art, the present invention has the following beneficial effects:

[0034] The present invention achieves efficient embedding and accurate detection of watermarks by directly embedding watermarks and optimizing the detection mechanism during the model training process. Its innovative introduction of generative adversarial networks and binoculars structures not only ensures the natural fluency of the generated text, but also significantly enhances the robustness of the watermark under adversarial attacks and natural rewriting. At the same time, the use of low-rank adapter technology reduces the computational cost, making this method widely applicable to a variety of pre-trained large models with strong adaptability. In addition, through constrained optimization and regularization strategies, the detectability of the watermark and the naturalness of the text are further balanced, effectively avoiding the problem of text quality degradation caused by over-optimization, and providing reliable technical support for the traceability and responsibility of AI-generated content. It has important application value and broad application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the technical solution of the present invention, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0036] Figure 1 Schematic diagram of the watermark embedding and detection model structure of an embodiment of the present invention. DETAILED DESCRIPTION

[0037] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0038] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0039] Example 1

[0040] This embodiment provides a watermark embedding and detection method based on a large model, comprising the following steps:

[0041] S1. Obtain the real text sequence to be watermarked.

[0042] S2. Build a watermark embedding and detection model based on a large model.

[0043] A generative adversarial network is introduced into the binoculars structure to construct the watermark embedding and detection model. The binoculars structure can improve the robustness of the watermark to a certain extent, but because watermark embedding and detection rely on the output distribution of the model, which can be disrupted by adversarial perturbations or natural rewriting, the binoculars method may still be affected by adversarial attacks or text rewriting.

[0044] The core idea of ​​the Binoculars method is to use two language models (called PerformerModel, ) and Observer Model, ) to evaluate the predictability of generated text. By comparing the difference in perplexity between the two models, it is possible to detect whether the text is machine generated.

[0045] Binoculars score definition:

[0046]

[0047] Where logPPL(s) represents the perplexity of the observer model for the text sequence s; logXPPL(s) represents the cross perplexity between the observer model and the performer model. A high binoculars score indicates that the observer model It is found that the text sequence s is less predictable but similar to the performer model The predicted distribution of is highly consistent, which usually means that the text is machine generated. A low binoculars score indicates that the observer model It is found that the text sequence s is highly predictable, or similar to the performer model The predicted distribution of is inconsistent, which usually means that the text is written by a human.

[0048] (1) Perplexity PPL:

[0049]

[0050] Where s={x1,x2,...,x L} represents a text sequence; L represents the length of the text sequence (i.e., the number of tokens); x i Represents the i-th Token in the sequence; s[:i] represents the first i-1 Tokens of sequence s; Representing the Observer Model Given s[:i], predict x i probability.

[0051] (2) Cross-perplexity XPPL

[0052]

[0053] in, Representing the Observer Model Given s[:i], the probability distribution of the vocabulary V, where V represents the vocabulary and contains all possible tokens; Representing the performer model The logarithmic probability distribution of the vocabulary V given s[:i]; <·,·> represents the inner product between two distributions, which is used to measure their consistency.

[0054] In order to solve the above problems, this embodiment proposes a strategy for embedding and detecting watermarks by further optimizing the binoculars method. The model structure of this embodiment is as follows: Figure 1 As shown, the construction process is as follows:

[0055] (1) Generative adversarial networks (GANs) are introduced into the binoculars method, allowing the generator and discriminator to optimize watermark embedding and detection in adversarial training, making the model resistant to adversarial attacks and improving the detection ability of watermarks under adversarial attacks and natural rewriting.

[0056] (2) Using data augmentation techniques (such as text rewriting and synonym replacement), adversarial samples and rewritten texts are added during the training process, i.e., text enhancement, to improve the robustness of the watermark under natural rewriting.

[0057] (3) If the performer model and the Observer Model The architecture and training data are too similar, and the difference in perplexity between the two may not be able to effectively distinguish between human and machine texts. Therefore, this method proposes architecture heterogeneity and data heterogeneity solutions:

[0058] 1) Architecture heterogeneity: and Using different architectures (in this embodiment, Adopting the Transformer-based LoRA fine-tuning architecture, the low-rank adapter in the figure; Using RNN architecture), it captures the deep features of generated text through structural differences.

[0059] 2) Data heterogeneity: , trained on mixed data of human text + enhanced text + generated text, simulating the behavior of "generator". Trained only on high-quality human text + augmented text to enhance sensitivity to unnatural patterns.

[0060] Specifically, the generator and discriminator are embedded in the actor model of the binocular structure and the Observer Model middle.

[0061] Among them, the generator Responsible for generating watermarked text based on the performer model Optimize watermark embedding through adversarial training. Generator Use a "pre-trained model" (this embodiment uses the Transformer-based LoRA fine-tuning architecture). It should be noted that the "pre-trained model" in the Transformer-based LoRA fine-tuning architecture used in this embodiment is replaceable and has a wide range of adaptability, such as the open source large-scale model DeepSeek series, Alibaba's Qwen series, Zhipu's GLM series, Baichuan series, Meta's Llama series and Code Llama, Microsoft's Phi series, Google's Gemma series and Gemini series, etc.

[0062] Discriminator Responsible for detecting watermarks in text, which is based on the observer model , optimize watermark detection through adversarial training, in this implementation Adopt RNN architecture.

[0063] The loss function of the model in this embodiment is as follows:

[0064] By adversarially training the two models, the text generated by the "Generator-Performer Model" can be accurately detected as watermarked by the "Discriminator-Observer Model" while maintaining its naturalness. At the same time, the "Discriminator-Observer Model" needs to be able to distinguish between watermarks in real text and generated text, while ensuring that the generated text remains fluent and natural. The optimization objective can be expressed as:

[0065]

[0066] Among them, λ binocular ,λ natural and λ augment Represents the weight for balancing different losses; It means fighting against loss; Indicates loss of binoculars; Indicates loss of naturalness; represents the data augmentation loss.

[0067] The above loss function is explained below:

[0068] (1) Fighting Losses The goal of the generator is to generate text that can deceive the discriminator.

[0069] The goal of the discriminator is to distinguish between real text and generated text.

[0070]

[0071] Among them, s real represents a sequence of text written by a real human; gen Represents the watermarked text sequence generated by the generator; Represents the discriminator's response to the real text s real The output of s real The probability of representing real text; Represents the discriminator to generate text s gen The output of s gen The probability of representing real text; Represents the logarithmic probability of the discriminator for the real text, which is used to measure the discriminator's ability to correctly identify the real text; Represents the logarithmic probability of the discriminator for the generated text, which is used to measure the discriminator's ability to correctly identify the generated text. Optimization target: Discriminator maximize That is, correctly distinguishing between real text and generated text. minimize That is, generating text that can deceive the discriminator.

[0072] (2) Binoculars loss: Optimize the binoculars score to ensure that the generated text has detectable watermark features.

[0073]

[0074] (3) Naturalness loss: ensuring that the generated text remains natural.

[0075]

[0076] in, Indicates the probability of the performer model predicting the i-th Token given i-1 Tokens. x<i represents the first i-1 Tokens in the sequence, x i Represents the i-th Token in the sequence.

[0077] (4) Data enhancement loss: Use data enhancement techniques (such as text rewriting and synonym replacement) to improve the robustness of watermarks under natural rewriting:

[0078]

[0079] Among them, s augRepresents a text sequence after data augmentation; Represents the text distribution after data augmentation; Indicates expected value.

[0080] (5) Total loss function:

[0081]

[0082] Among them, λ binocular ,λ natural and λ augment Represents the weight for balancing different losses; It means fighting against loss; Indicates loss of binoculars; Indicates loss of naturalness; represents the data augmentation loss.

[0083] The optimization strategy for the above model is as follows:

[0084] First, the pre-trained model weights are initialized to the performer and observer models, and the generator and discriminator are loaded. A real text dataset and adversarial augmentation data are then prepared, and a low-rank adapter (LoRA) is applied to reduce training costs. During training, an alternating optimization adversarial training framework is employed: when updating the discriminator, samples are taken from real data and watermarked text generated by the generator, and the adversarial loss and binocular loss are calculated to optimize the discriminator parameters. When updating the generator, the adversarial loss, binocular loss, naturalness loss (based on cross-entropy constraints to ensure text fluency), and data augmentation loss (through rewriting and synonym replacement to enhance robustness) are integrated to achieve a balance between watermark embedding and naturalness. Furthermore, regularized optimization strategies (such as exponential and quadratic barrier functions) are introduced to constrain the cross-entropy loss, preventing over-optimization of the model from leading to text quality degradation. Constrained optimization ensures a synergistic improvement in watermark detection performance and naturalness. Ultimately, this method achieves efficient watermark embedding and highly robust detection through end-to-end adversarial training, heterogeneous model architecture design, and diversified data augmentation.

[0085] S3. Use the watermark embedding and detection model to complete the watermark embedding and detection of the real text sequence.

[0086] During the optimization process, watermark embedding and text naturalness are two competing goals, and a balance needs to be found between the robustness of the watermark and the naturalness of the text. Using min-max adversarial optimization suffers from training instability and difficulty in balancing. On the one hand, min-max optimization may lead to oscillatory behavior during training, with the model alternating between embedding too strong and too weak watermarks. On the other hand, hard constraints make the optimization process difficult, easily leading to poor convergence, and it is difficult to find a balance between watermark detectability and text naturalness. To address the above issues, this method transforms the optimization problem into a constrained optimization problem and stabilizes the training process by introducing a regularization term. During the optimization process, the watermark strength is controlled to prevent over-optimization, while optimizing both the robustness of the watermark and the naturalness of the text.

[0087] (1) Constrained optimization: The optimization problem is transformed into a constrained optimization problem, ensuring that the cross entropy loss does not exceed a certain threshold ∈ ：

[0088] Limited by in, represents the difference in binocular scores, used to maximize the detectability of the watermark; Represents a constraint, and the goal is to minimize the difference in binocular scores under this constraint, ensuring that the cross entropy loss does not exceed the threshold ∈, thereby maintaining the naturalness of the text.

[0089] (2) Regularization optimization: Regularization terms (such as exponential barrier function or quadratic barrier function) are introduced to prevent over-optimization.

[0090] 1) Exponential Barrier Function: This method imposes strong penalties on constraint violations, ensuring the stability of the training process.

[0091]

[0092] Among them, when When ∈ is exceeded, the optimization process is strongly inhibited, preventing unstable updates.

[0093] 2) Quadratic barrier function: imposes a smaller penalty for minor constraint violations and a quadratic penalty for major violations. This approach allows for a certain degree of constraint violations, providing greater flexibility:

[0094]

[0095] Example 2

[0096] This embodiment also provides a watermark embedding and detection system based on a large model, including: an acquisition module, a construction module and an embedding and detection module; the acquisition module is used to obtain a real text sequence to be embedded with a watermark; the construction module is used to construct a watermark embedding and detection model based on the large model; the embedding and detection module is used to use the watermark embedding and detection model to complete the watermark embedding and detection of the real text sequence.

[0097] The following will describe in detail how the present invention solves technical problems in practical work in conjunction with the present embodiment.

[0098] First, the acquisition module is used to obtain the real text sequence to be embedded with the watermark.

[0099] Then the module is constructed to build a watermark embedding and detection model based on the large model. The process includes:

[0100] A generative adversarial network is introduced into the binoculars structure to construct the watermark embedding and detection model. The binoculars structure can improve the robustness of the watermark to a certain extent, but because watermark embedding and detection rely on the output distribution of the model, which can be disrupted by adversarial perturbations or natural rewriting, the binoculars method may still be affected by adversarial attacks or text rewriting.

[0101] The core idea of ​​the Binoculars method is to use two language models (called PerformerModel, ) and ObserverModel, ) to evaluate the predictability of generated text. By comparing the difference in perplexity between the two models, it is possible to detect whether the text is machine generated.

[0102] Binoculars score definition:

[0103]

[0104] Where log PPL(s) represents the perplexity of the observer model for the text sequence s; log XPPL(s) represents the cross perplexity between the observer model and the performer model. A high binoculars score indicates that the observer model It is found that the text sequence s is less predictable but similar to the performer model The predicted distribution of is highly consistent, which usually means that the text is machine generated. A low binoculars score indicates that the observer model It is found that the text sequence s is highly predictable, or similar to the performer model The predicted distribution of is inconsistent, which usually means that the text is written by a human.

[0105] (1) Perplexity PPL:

[0106]

[0107] Where s={x1,x2,...,x L} represents a text sequence; L represents the length of the text sequence (i.e., the number of tokens); x i Represents the i-th Token in the sequence; s[:i] represents the first i-1 Tokens of sequence s; Representing the Observer Model Given s[:i], predict x i probability.

[0108] (2) Cross-perplexity XPPL

[0109]

[0110] in, Representing the Observer Model Given s[:i], the probability distribution of the vocabulary V, where V represents the vocabulary and contains all possible tokens; Representing the performer model The logarithmic probability distribution of the vocabulary V given s[:i]; <·,·> represents the inner product between two distributions, which is used to measure their consistency.

[0111] In order to solve the above problems, this embodiment proposes a strategy for embedding and detecting watermarks by further optimizing the binoculars method. The model structure of this embodiment is as follows: Figure 1 As shown, the construction process is as follows:

[0112] (1) Generative adversarial networks (GANs) are introduced into the binoculars method, allowing the generator and discriminator to optimize watermark embedding and detection in adversarial training, making the model resistant to adversarial attacks and improving the detection ability of watermarks under adversarial attacks and natural rewriting.

[0113] (2) Using data augmentation techniques (such as text rewriting and synonym replacement), adversarial samples and rewritten texts are added during the training process, i.e., text enhancement, to improve the robustness of the watermark under natural rewriting.

[0114] (3) If the performer model and the Observer Model The architecture and training data are too similar, and the difference in perplexity between the two may not be able to effectively distinguish between human and machine texts. Therefore, this method proposes architecture heterogeneity and data heterogeneity solutions:

[0115] 1) Architecture heterogeneity: and Using different architectures (in this embodiment, Adopting the Transformer-based LoRA fine-tuning architecture, the low-rank adapter in the figure; Using RNN architecture), it captures the deep features of generated text through structural differences.

[0116] 2) Data heterogeneity: Trained on mixed data of human text + augmented text + generated text to simulate the behavior of "generator". Trained only on high-quality human text + augmented text to enhance sensitivity to unnatural patterns.

[0117] Specifically, the generator and discriminator are embedded in the actor model of the binocular structure and the Observer Model middle.

[0118] Among them, the generator Responsible for generating watermarked text based on the performer model Optimize watermark embedding through adversarial training. Generator Use a "pre-trained model" (this embodiment uses the Transformer-based LoRA fine-tuning architecture). It should be noted that the "pre-trained model" in the Transformer-based LoRA fine-tuning architecture used in this embodiment is replaceable and has a wide range of adaptability, such as the open source large-scale model DeepSeek series, Alibaba's Qwen series, Zhipu's GLM series, Baichuan series, Meta's Llama series and Code Llama, Microsoft's Phi series, Google's Gemma series and Gemini series, etc.

[0119] Discriminator Responsible for detecting watermarks in text, which is based on the observer model By optimizing watermark detection through adversarial training, this implementation Adopt RNN architecture.

[0120] The loss function of the model in this embodiment is as follows:

[0121] By adversarially training the two models, the text generated by the "Generator-Performer Model" can be accurately detected as watermarked by the "Discriminator-Observer Model" while maintaining its naturalness. At the same time, the "Discriminator-Observer Model" needs to be able to distinguish between watermarks in real text and generated text, while ensuring that the generated text remains fluent and natural. The optimization objective can be expressed as:

[0122]

[0123] Among them, λ binocular ,λ natural and λ augment Represents the weight for balancing different losses; It means fighting against loss; Indicates loss of binoculars; Indicates loss of naturalness; represents the data augmentation loss.

[0124] The above loss function is explained below:

[0125] (1) Fighting Losses The goal of the generator is to generate text that can deceive the discriminator.

[0126] The goal of the discriminator is to distinguish between real text and generated text.

[0127]

[0128] Among them, s real represents a sequence of text written by a real human; gen Represents the watermarked text sequence generated by the generator; Represents the discriminator's response to the real text s real The output of s real The probability of representing real text; Represents the discriminator to generate text s gen The output of s gen The probability of representing real text; Represents the logarithmic probability of the discriminator for the real text, which is used to measure the discriminator's ability to correctly identify the real text; Represents the logarithmic probability of the discriminator for the generated text, which is used to measure the discriminator's ability to correctly identify the generated text. Optimization target: Discriminator maximize That is, correctly distinguishing between real text and generated text. minimize That is, generating text that can deceive the discriminator.

[0129] (2) Binoculars loss: Optimize the binoculars score to ensure that the generated text has detectable watermark features.

[0130]

[0131] (3) Naturalness loss: ensuring that the generated text remains natural.

[0132]

[0133] in, Indicates the probability of the performer model predicting the i-th Token given i-1 Tokens. ＜i represents the first i-1 tokens in the sequence, x i Represents the i-th Token in the sequence.

[0134] (4) Data enhancement loss: Use data enhancement techniques (such as text rewriting and synonym replacement) to improve the robustness of watermarks under natural rewriting:

[0135]

[0136] Among them, s aug Represents a text sequence after data augmentation; Represents the text distribution after data augmentation; Indicates expected value.

[0137] (5) Total loss function:

[0138]

[0139] Among them, λ binocular ,λ natural and λ augment Represents the weight for balancing different losses; It means fighting against loss; Indicates loss of binoculars; Indicates loss of naturalness; represents the data augmentation loss.

[0140] The optimization strategy for the above model is as follows:

[0141] First, the pre-trained model weights are initialized to the performer and observer models, and the generator and discriminator are loaded. A real text dataset and adversarial augmentation data are then prepared, and a low-rank adapter (LoRA) is applied to reduce training costs. During training, an alternating optimization adversarial training framework is employed: when updating the discriminator, samples are taken from real data and watermarked text generated by the generator, and the adversarial loss and binocular loss are calculated to optimize the discriminator parameters. When updating the generator, the adversarial loss, binocular loss, naturalness loss (based on cross-entropy constraints to ensure text fluency), and data augmentation loss (through rewriting and synonym replacement to enhance robustness) are integrated to achieve a balance between watermark embedding and naturalness. Furthermore, regularized optimization strategies (such as exponential and quadratic barrier functions) are introduced to constrain the cross-entropy loss, preventing over-optimization of the model from leading to text quality degradation. Constrained optimization ensures a synergistic improvement in watermark detection performance and naturalness. Ultimately, this method achieves efficient watermark embedding and highly robust detection through end-to-end adversarial training, heterogeneous model architecture design, and diversified data augmentation.

[0142] Finally, the embedding and detection module uses the watermark embedding and detection model to complete the watermark embedding and detection of the real text sequence.

[0143] During the optimization process, watermark embedding and text naturalness are two competing goals, and a balance needs to be found between the robustness of the watermark and the naturalness of the text. The use of min-max adversarial optimization has the problems of training instability and difficulty in balancing. On the one hand, min-max optimization may lead to oscillatory behavior during training, and the model alternates between embedding too strong and too weak watermarks; on the other hand, hard constraints make the optimization process difficult, easily leading to poor convergence, and it is difficult to find a balance between watermark detectability and text naturalness. In order to solve the above problems, this embodiment transforms the optimization problem into a constrained optimization problem, and stabilizes the training process by introducing a regularization term. During the optimization process, the watermark strength is controlled to prevent over-optimization, while optimizing the robustness of the watermark and the naturalness of the text.

[0144] (1) Constrained optimization: The optimization problem is transformed into a constrained optimization problem, ensuring that the cross entropy loss does not exceed a certain threshold ∈ ：

[0145] Limited by in, represents the difference in binocular scores, used to maximize the detectability of the watermark; Represents a constraint, and the goal is to minimize the difference in binocular scores under this constraint, ensuring that the cross entropy loss does not exceed the threshold ∈, thereby maintaining the naturalness of the text.

[0146] (2) Regularization optimization: Regularization terms (such as exponential barrier function or quadratic barrier function) are introduced to prevent over-optimization.

[0147] 1) Exponential Barrier Function: This method imposes strong penalties on constraint violations, ensuring the stability of the training process.

[0148]

[0149] Among them, when When ∈ is exceeded, the optimization process is strongly inhibited, preventing unstable updates.

[0150] 2) Quadratic barrier function: imposes a smaller penalty for minor constraint violations and a quadratic penalty for major violations. This approach allows for a certain degree of constraint violations, providing greater flexibility:

[0151]

[0152] The embodiments described above are merely descriptions of preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Without departing from the spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by persons skilled in the art should fall within the scope of protection defined by the claims of the present invention.

Claims

1. A watermark embedding and detection method based on a large model, characterized in that the steps include: Get the real text sequence to be embedded with watermark; Construct a watermark embedding and detection model based on a large model; The watermark embedding and detection model is used to complete the watermark embedding and detection of the real text sequence.

2. The watermark embedding and detection method based on a large model according to claim 1 is characterized in that: The generative adversarial network is introduced into the binoculars structure to construct the watermark embedding and detection model; The watermark embedding and detection model includes: a generator-performer model and a discriminator-observer model; The generator-performer model is used to generate watermarked text and optimize watermark embedding through adversarial training; The discriminator-observer model is used to detect watermarks in text and optimize watermark detection through adversarial training.

3. The watermark embedding and detection method based on a large model according to claim 2 is characterized in that: The binoculars structure uses the performer model and the observer model to evaluate the predictability of the generated text; and by comparing the perplexity difference between the two models, it detects whether the text is generated by the machine. The expressions include: Among them, logPPL(s) represents the perplexity of the observer model for the text sequence s; logXPPL(s) represents the cross perplexity between the observer model and the performer model.

4. The watermark embedding and detection method based on a large model according to claim 3 is characterized in that: The generator in the adversarial generative network is embedded in the performer model of the binocular structure to generate the generator-performer model; wherein the generator adopts a pre-trained model and is trained on mixed data, and the mixed data includes human text, enhanced text and generated text.

5. The watermark embedding and detection method based on a large model according to claim 4 is characterized in that: The discriminator in the adversarial generative network is embedded into the observer model of the binocular structure to generate the discriminator-observer model; wherein the discriminator adopts the RNN architecture and is trained on human text and enhanced text.

6. The watermark embedding and detection method based on a large model according to claim 1 is characterized in that: The loss function of the watermark embedding and detection model includes: Among them, λ binocular ,λ natural and λ augment Represents the weight for balancing different losses; It means fighting against loss; Indicates loss of binoculars; Indicates loss of naturalness; represents the data augmentation loss.

7. The watermark embedding and detection method based on a large model according to claim 6 is characterized in that: Use data augmentation techniques to improve the robustness of watermarks to natural rewriting: Among them, s aug Represents a text sequence after data augmentation; Represents the text distribution after data augmentation; Indicates expected value.

8. A watermark embedding and detection system based on a large model, the system being used to implement the method according to any one of claims 1 to 7, characterized in that: include: Acquisition module, construction module and embedding and detection module; The acquisition module is used to acquire the real text sequence to be embedded with the watermark; The building module is used to build a watermark embedding and detection model based on a large model; The embedding and detection module is used to use the watermark embedding and detection model to complete the watermark embedding and detection of the real text sequence.

9. The large model-based watermark embedding and detection system according to claim 8, characterized in that: The workflow of the construction module includes: introducing a generative adversarial network into a binoculars structure to construct the watermark embedding and detection model; the watermark embedding and detection model includes: a generator-performer model and a discriminator-observer model; The generator-performer model is used to generate watermarked text and optimize watermark embedding through adversarial training; The discriminator-observer model is used to detect watermarks in text and optimize watermark detection through adversarial training.

10. The large model-based watermark embedding and detection system according to claim 9, characterized in that: The binoculars structure uses the performer model and the observer model to evaluate the predictability of the generated text; and by comparing the perplexity difference between the two models, it detects whether the text is generated by the machine. The expressions include: Among them, logPPL(s) represents the perplexity of the observer model for the text sequence s; logXPPL(s) represents the cross perplexity between the observer model and the performer model.

Images