Large model watermarking method based on dynamic entropy
By dynamically adjusting the entropy value during the large model generation process, and combining pseudo-random scoring with multiple rounds of competition, a robust watermark is embedded in a large language model while maintaining text quality and fluency, thus overcoming the limitations of existing watermarking methods.
Patent Information
- Application Number
- CN202610288937.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-03-11
- Publication Date
- 2026-04-10
- Estimated Expiration
- 2046-03-11
AI Technical Summary
Existing large-model watermarking technologies cannot adapt to the dynamic entropy characteristics in the process of generating large language models, making it difficult to balance watermark detectability, text quality, and computational efficiency.
A large-model watermarking method based on dynamic entropy is adopted. The watermark signal is embedded in the large model generation process through the watermarking module. The sliding window size and candidate word pool are dynamically adjusted. Combined with pseudo-random scoring and multi-round competition, the robustness of the watermark signal and the fluency of the text are ensured.
Embedding robust watermarks in high-entropy scenarios does not affect text quality in low-entropy scenarios, enhancing the detectability and anti-attack capabilities of watermarks while reducing the impact on user experience.
Smart Images
Figure CN121834773A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to a large model watermarking method based on dynamic entropy, belonging to the field of large model watermarking. Background Technology
[0002] Large language models, especially those based on the Transformer architecture, have achieved near-human levels of text generation, with their output often indistinguishable from human-written text. This capability is widely used in applications ranging from chatbots to code generation. However, this indistinguishability also brings significant risks, including the spread of misinformation, academic misconduct, and potential copyright infringement. Furthermore, the unrestrained use of data generated by large models to train future models carries the risk of "model collapse," a recursive degradation of model quality and diversity. Therefore, there is an urgent need to establish robust mechanisms that can effectively detect and trace text generated by large models.
[0003] To address this issue, the industry has proposed various detection methods, but each of them has significant limitations.
[0004] The first category of methods is "post-hoc detectors." These methods analyze text after it has been generated, attempting to distinguish between human-generated and AI-generated text by finding statistical features or training a classifier. The main weakness of this type of method lies in its inherent unreliability. Their performance is inconsistent, they are prone to high false positive rates, and they can be easily circumvented by simple rewriting or adversarial attacks. As large models continue to improve, their statistical features become increasingly similar to those of humans, and the effectiveness of post-hoc detectors decreases accordingly. Furthermore, these detectors inherently provide probabilistic evaluations rather than deterministic evidence, making them unsuitable for high-risk applications. The second category of methods is "retrieval-based detectors." This method stores all generated text in a large database and detects content through matching. The main weakness of this method is its impracticality at the scale of modern large model deployments, not only due to its high computational cost but also because it raises serious privacy issues. More importantly, it is highly vulnerable to rewriting attacks, where attackers can modify the text to circumvent exact or approximate matches while preserving its semantics.
[0005] Given the limitations of the methods mentioned above, generative watermarking is considered a superior solution. This method embeds the signal during text generation by modifying the sampling process of the next word. Detection can be performed without accessing the original large model. A representative generative watermarking method in the prior art (e.g., the method proposed by Kirchenbauer et al.) is based on a "static bias" strategy. This method divides the vocabulary into a "green list" and a "red list" based on a key and the preceding text, and then applies a fixed, static bias value to the logits of words in the green list to increase their probability of being selected.
[0006] However, the performance of generative watermarking depends on the entropy of the probability distribution of the large model at each generation step. Entropy is a measure of the uncertainty of a system. When the entropy is low, the model is very certain about the next term, and its probability distribution exhibits sharp peaks. In this case, forcibly selecting a different watermarked term (e.g., a "green list" term) would severely damage the quality and coherence of the text. Therefore, a fixed bias is either too weak to be effective or too strong to produce obvious textual flaws. Conversely, when the entropy is high, the model is uncertain about the next term, with multiple reasonable options and a relatively flat probability distribution. This provides an excellent opportunity to embed a strong watermark, as the selection can be slightly guided to favor a desired term without affecting the user-perceived text quality. Therefore, existing technologies lack a watermarking method that can adjust its embedding strategy in real time based on the dynamic entropy during the large model's generation process. This invention aims to address this technological gap. Summary of the Invention
[0007] This invention aims to overcome the limitations of existing large-model watermarking techniques in balancing watermark detectability, text quality preservation, and computational efficiency, and provides a large-model watermarking method based on dynamic entropy. Specifically, the core technical problem addressed by this invention is that existing static watermarking methods cannot adapt to the dynamic characteristics of the prediction uncertainty in large language models.
[0008] The technical solution of the present invention is as follows: In a first aspect, the present invention provides a large-model watermarking method based on dynamic entropy, comprising the following steps:
[0009] S1. The watermarking module is based on a watermarking key and is embedded in the text generation process of the large model. It modifies the lexical sampling process of the output without affecting the model training.
[0010] S2. Based on the probability distribution entropy value output by the large model, dynamically adjust the sliding window size and the candidate word pool size to balance diversity and watermark embedding efficiency;
[0011] S3. Pseudo-random scoring and multi-round competition are used to make the final output tokens carry watermark signals bound to the key, while preserving the text fluency of the large model;
[0012] S4. For each word in the sequence to be analyzed, recalculate the pseudo-random seed using the same watermark key and the context preceding the word, and regenerate its corresponding pseudo-random score at all levels based on this seed.
[0013] S5. Calculate the average pseudo-random score of all words in the text as the aggregation score and determine the statistical threshold based on the preset false positive rate. If the statistical threshold is less than the aggregation score of the unknown text, it is determined to be the generated text with watermark; otherwise, it is the text without watermark.
[0014] Furthermore, S1 specifically includes:
[0015] S11. Regarding computer system architecture, including:
[0016] Deploying a computer system involves two core components:
[0017] The large language model is responsible for generating the probability distribution of the vocabulary based on the input context.
[0018] Watermarking module: A functional module independent of the main model, pre-configured with secure watermarking keys. , As a safety parameter, it is applied to the lexical sampling stage of the large model;
[0019] S12. For key attributes, including:
[0020] Watermark Key satisfy:
[0021] Randomness: Generated by a cryptographically secure pseudo-random number generator, following a uniform distribution. ; Indicates a uniform distribution;
[0022] Confidentiality: The watermark is held only by the watermark generation and detection ends of the watermark module and transmitted through an encrypted channel;
[0023] S13. For embedding methods, including:
[0024] The watermarking module connects to the decoding process of the large model via an API interface. After the large model outputs the vocabulary probability distribution but before the final word determination, it inserts candidate word selection and watermark embedding logic. Mathematically, this candidate word selection and watermark embedding logic is represented as follows: Let the original sampling function of the large model be... , If the preceding word sequence is a lexical sequence, then the watermark module is modified as follows:
[0025]
[0026] in, This is the core function of the watermarking module, used to ensure that the output tokens still come from... The support set does not include lexical units not seen by the model. This represents the probability distribution of the output of the large model. Representation: Large language models are based on preorder lexical sequences The probability distribution of outputs for all lexical terms in the vocabulary.
[0027] Furthermore, S2 specifically includes:
[0028] S21. Quantization of output entropy: Assume a large model at time step The probability distribution of the output vocabulary is as follows , For the vocabulary list, as a word element If the probability is zero, then the Shannon entropy of this distribution is:
[0029]
[0030] S22. Dynamic sampling strategy: Preset individual entropy threshold: High entropy threshold and low entropy threshold Number of samplings and The relationship is a piecewise function:
[0031]
[0032] in, It is the medium entropy threshold. This is used to ensure that the candidate pool is more diverse when the entropy is high and more concentrated when the entropy is low.
[0033] S23. Construction of the candidate lexical pool: from the probability distribution of the vocabulary Independent sampling Next, the candidate word pool was obtained. ,in, This indicates that repeated sampling is allowed to preserve the frequency of high-probability words. Indicates the first Candidate word elements obtained from the second sampling;
[0034] S24. Perform dynamic window size expansion; adjust the sliding window size H1 in real time based on the dynamic entropy generated at each step; this method transforms the fixed hyperparameter H1 into a dynamic variable H that changes with the context. t ;
[0035] When generating the t-th word, the current context is first calculated, and the Shannon entropy of the probability distribution output by the large model is:
[0036]
[0037] in, This indicates that the corresponding large model is based on the preceding word sequence. Probability distribution of the generated vocabulary;
[0038] Then, a mapping function is introduced. This function is based on Shannon entropy. Determine the dynamic window size to use in the current step :
[0039]
[0040] This function The design is a piecewise function used to map different entropy value ranges to a preset window size. For example, a minimum window size can be set, and a larger window can be used when the entropy value is high.
[0041]
[0042] in, ≥4, and It is a pre-set entropy threshold. , , These are the maximum context window size, the medium context window size, and the minimum context window size, respectively.
[0043] Under this dynamic mechanism, the seed generation formula in step S1 is updated to:
[0044] ;
[0045] in, It is a pseudo-random seed, and h is a cryptographic hash function. It is the preorder dynamic window lexical sequence when generating the t-th lexical. This is the dynamically adjusted window size. This represents the word preceding the generation of the t-th word, and is the last element of the dynamic window sequence.
[0046] Furthermore, S3 specifically includes:
[0047] S31. Pseudo-random seed generation: Let the preceding context be the most recent. Each word element, that is A pseudo-random seed is generated using a cryptographic hash function h:
[0048]
[0049] in, It is a hash function that satisfies pseudo-randomness. For safety parameters;
[0050] Let represent the H-th Cartesian product of the vocabulary V, i.e., the set of all word sequences of length H; k is the watermark key, and t is the time step. It is a pseudo-random seed, and the window size of the cryptographic hash function h that generates the pseudo-random seed changes from fixed to dynamic;
[0051] S32. Pseudo-random score calculation: Let the number of competition layers be... , For candidate word pool Each word in In the The pseudo-random score of the layer is defined as:
[0052]
[0053] in, , This is a pseudo-random function used to ensure that the scores of the same word are independently and uniformly distributed across different layers and seeds. For a vocabulary list;
[0054] S33. Multi-round elimination mechanism: The size of the candidate word pool must meet the sampling number requirements. To support full binary tree competition:
[0055] Round 1: Randomly divided into Groups, each group retains The highest-scoring word, the remaining Each word element;
[0056] Round 2: Randomly divide the remaining word units into... Groups, each group retains The highest-scoring word, the remaining Each word element;
[0057] Round m: The final two word groups compete, and the remaining words are retained. The word with the highest score is used as the output word. ;
[0058] In mathematics, outputting word units satisfy:
[0059]
[0060] This is to ensure that it comes from a high probability distribution of a large model and performs optimally in pseudo-random scoring.
[0061] Furthermore, S4 specifically includes:
[0062] S41. Word segmentation and context extraction: the text to be detected The tokens are split into a sequence of words using the same segmenter as in the generation stage. For each word... Extract its preorder each word element ,like Then take all preceding words;
[0063] S42. Reconstruction of Pseudo-random Seeds and Scores: Using the same watermark key as in the generation stage. and hash function Recalculate Corresponding pseudo-random seed:
[0064]
[0065] Then use the same pseudo-random function Reconstruction Score at each level:
[0066]
[0067] S43. Independence Guarantee: The reconstruction process relies solely on the text itself and the watermark key. and preset and It does not require calling any interfaces or parameters of the original large model, which can be mathematically represented as:
[0068]
[0069] in, It is a pure function.
[0070] Furthermore, S5 specifically includes:
[0071] S51. Aggregated Watermark Score Calculation: Text to be Detected All morphemes Calculate its in all The mean of the pseudo-random scores of the layer is used as the aggregate score:
[0072]
[0073] in, This indicates the reconstruction of each word. In the scores at each level, T is the length of the corresponding word sequence;
[0074] S52. Determination of statistical threshold: Threshold The determination is based on the preset false positive rate (FPR), and the steps are as follows: Collect a sufficient set of watermark-free text samples. ; Calculate the aggregate score for each sample The fraction distribution is obtained. Take the distribution Quantiles are used as thresholds, that is:
[0075]
[0076] S53. Final Judgment Rule: For unknown text ,like If the result is positive, it is determined to be generated text with a watermark; otherwise, it is determined to be text without a watermark. Mathematically, this is expressed as:
[0077]
[0078] Here, Decision(x) is the decision function for watermark detection. The input is the text to be detected, x, and the output is the judgment result of whether the text carries the target watermark.
[0079] Secondly, the present invention also provides a large model watermarking system based on dynamic entropy, the system comprising: a module for executing the large model watermarking method based on dynamic entropy.
[0080] Thirdly, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the large model watermarking method based on dynamic entropy.
[0081] Fourthly, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the large model watermarking method based on dynamic entropy.
[0082] Fifthly, the present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the large model watermarking method based on dynamic entropy.
[0083] Compared with existing technologies, the advantages of this invention are: it can embed a robust and highly detectable watermark when the generation context allows (high entropy), while not sacrificing text quality or coherence when the model output is highly constrained (low entropy). This adaptive mechanism makes the final embedded watermark stronger overall, more resistant to attacks, and has negligible impact on user experience. Attached Figure Description
[0084] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.
[0085] Figure 1 This is a flowchart illustrating a large-model watermarking method based on dynamic entropy provided by the present invention.
[0086] Figure 2 This is a flowchart illustrating the architecture of a large-model watermarking method based on dynamic entropy provided by the present invention.
[0087] Figure 3 This is a schematic diagram of the structure of the electronic device provided by the present invention.
[0088] The labels in the diagram are as follows:
[0089] 310 - Processor; 320 - Communication interface; 330 - Memory; 340 - Communication bus. Detailed Implementation
[0090] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. It should be noted that, unless otherwise specified, the embodiments and features in the embodiments of this application can be arbitrarily combined with each other.
[0091] It should be noted that the executing entity of the embodiments of the present invention can be various types of terminals, such as computers, smartphones, tablets, wearable devices, personal digital assistants (PDAs), mobile internet devices (MIDs), and other terminals capable of text processing. The present invention does not impose any restrictions on this.
[0092] Example 1: As Figures 1-2 As shown, this embodiment of the invention provides a large-model watermarking method based on dynamic entropy, including the following steps:
[0093] S1. The watermarking module is based on a watermarking key and is embedded in the text generation process of the large model. It modifies the lexical sampling process of the output without affecting the model training.
[0094] Furthermore, S1 specifically includes:
[0095] S11. Regarding computer system architecture, including:
[0096] Deploying a computer system involves two core components:
[0097] The large language model is responsible for generating the probability distribution of the vocabulary based on the input context.
[0098] Watermarking module: A functional module independent of the main model, pre-configured with secure watermarking keys. , As a safety parameter, it is applied to the lexical sampling stage of the large model;
[0099] S12. For key attributes, including:
[0100] Watermark Key satisfy:
[0101] Randomness: Generated by a cryptographically secure pseudo-random number generator, following a uniform distribution. ; Indicates a uniform distribution;
[0102] Confidentiality: The watermark is held only by the watermark generation and detection ends of the watermark module and transmitted through an encrypted channel;
[0103] S13. For embedding methods, including:
[0104] The watermarking module connects to the decoding process of the large model via an API interface. After the large model outputs the vocabulary probability distribution but before the final word determination, it inserts candidate word selection and watermark embedding logic. Mathematically, this candidate word selection and watermark embedding logic is represented as follows: Let the original sampling function of the large model be... , If the preceding word sequence is a lexical sequence, then the watermark module is modified as follows:
[0105]
[0106] in, This is the core function of the watermarking module, used to ensure that the output tokens still come from... The support set does not include lexical units not seen by the model. p_LLM represents the probability distribution of the output of a large language model. It is an abbreviation for Probability Distribution of Large Language Model. Combined with the formula... Representation: Large language models are based on preorder lexical sequences (Generate all lexical units before the t-th lexical unit). The probability distribution of all lexical units output in the vocabulary is the basis for the watermarking module to perform lexical unit filtering and watermark embedding.
[0107] S2. Based on the probability distribution entropy value output by the large model, dynamically adjust the sliding window size and the candidate word pool size to balance diversity and watermark embedding efficiency;
[0108] Furthermore, S2 specifically includes:
[0109] S21. Quantization of output entropy: Assume a large model at time step The probability distribution of the output vocabulary is as follows , For the vocabulary list, as a word element If the probability is zero, then the Shannon entropy of this distribution is:
[0110]
[0111] The larger the value, the higher the uncertainty (greater diversity) of the large model regarding the next word element.
[0112] S22. Dynamic sampling strategy: Preset individual entropy threshold: High entropy threshold and low entropy threshold Number of samplings and The relationship is a piecewise function:
[0113]
[0114] in, It is the medium entropy threshold. This is used to ensure that the candidate pool is more diverse when the entropy is high and more concentrated when the entropy is low.
[0115] S23. Construction of the candidate lexical pool: from the probability distribution of the vocabulary Independent sampling Next, the candidate word pool was obtained. ,in, This indicates that repeated sampling is allowed to preserve the frequency of high-probability words. Indicates the first Candidate word elements obtained from the second sampling;
[0116] S24. Perform dynamic window size expansion; adjust the sliding window size H1 in real time based on the dynamic entropy generated at each step; this method transforms the fixed hyperparameter H1 into a dynamic variable H that changes with the context. tThis allows the watermark embedding strategy to be more tightly coupled with the uncertainty of the model;
[0117] When generating the t-th word, the current context is first calculated, and the Shannon entropy of the probability distribution output by the large model is:
[0118]
[0119] in, This indicates that the corresponding large model is based on the preceding word sequence. Probability distribution of the generated vocabulary;
[0120] Then, a mapping function is introduced. This function is based on Shannon entropy. Determine the dynamic window size to use in the current step :
[0121]
[0122] This function The design is a piecewise function used to map different entropy value ranges to a preset window size. For example, a minimum window size can be set, and a larger window can be used when the entropy value is high.
[0123]
[0124] in, ≥4, and It is a pre-set entropy threshold. , , These are the maximum context window size, the medium context window size, and the minimum context window size, respectively.
[0125] Maximum context window size, when entropy value ≥ Used when the entropy threshold is high to ensure that there are enough preceding words to generate pseudo-random seeds in high-entropy scenarios, thereby enhancing the watermark signal strength;
[0126] Medium context window size, when ≤Entropy value< Used in medium-entropy scenarios to balance context coverage and computational efficiency;
[0127] Minimum context window size (≥4), when entropy value < Used in low-entropy scenarios to avoid contextual redundancy due to excessively large windows, while ensuring the effectiveness of seed generation.
[0128] The core function of these three components is to adapt to different entropy scenarios and ensure the rationality of pseudo-random seed generation.
[0129] Under this dynamic mechanism, the seed generation formula in step S1 is updated to:
[0130] ;
[0131] in, It is a pseudo-random seed, and h is a cryptographic hash function. This is the preorder dynamic window word sequence when generating the t-th word. This sequence is one of the inputs to the hash function, used to ensure that the seed is strongly correlated with the current context. This is the dynamically adjusted window size. This represents the word preceding the generation of the t-th word, and is the last element of the dynamic window sequence, ensuring the immediacy of the context.
[0132] S3. Pseudo-random scoring and multi-round competition are used to make the final output tokens carry watermark signals bound to the key, while preserving the text fluency of the large model;
[0133] Furthermore, S3 specifically includes:
[0134] S31. Pseudo-random seed generation: Let the preceding context be the most recent. Each word element ( , usually take ),Right now A pseudo-random seed is generated using a cryptographic hash function h:
[0135]
[0136] in, It is a hash function that satisfies pseudo-randomness. For safety parameters;
[0137] Let represent the H-th Cartesian product of the vocabulary V, i.e., the set of all word sequences of length H; k is the watermark key, a core confidentiality parameter for pseudo-random seed generation, held only by the generator and detector; t is the time step. It is a pseudo-random seed, and the window size of the cryptographic hash function h that generates the pseudo-random seed changes from fixed to dynamic;
[0138] S32. Pseudo-random score calculation: Let the number of competition layers be... , , For candidate word pool Each word in In the The pseudo-random score of the layer is defined as:
[0139]
[0140] in, , This is a pseudo-random function used to ensure that the scores of the same word are independently and uniformly distributed across different layers and seeds. For a vocabulary list;
[0141] S33. Multi-round elimination mechanism: The size of the candidate word pool must meet the sampling number requirements. To support full binary tree competition:
[0142] Round 1: Randomly divided into Groups, each group retains The highest-scoring word, the remaining Each word element;
[0143] Round 2: Randomly divide the remaining word units into... Groups, each group retains The highest-scoring word, the remaining Each word element;
[0144] Round m: The final two word groups compete, and the remaining words are retained. The word with the highest score is used as the output word. ;
[0145] In mathematics, outputting word units satisfy:
[0146]
[0147] This is to ensure that it comes from a high probability distribution of a large model and performs optimally in pseudo-random scoring.
[0148] S4. For each word in the sequence to be analyzed, recalculate the pseudo-random seed using the same watermark key and the context preceding the word, and regenerate its corresponding pseudo-random score at all levels based on this seed.
[0149] Furthermore, S4 specifically includes:
[0150] S41. Word segmentation and context extraction: the text to be detected The tokens are split into a sequence of words using the same segmenter as in the generation stage. For each word... Extract its preorder each word element ,like Then take all preceding words;
[0151] S42. Reconstruction of Pseudo-random Seeds and Scores: Using the same watermark key as in the generation stage. and hash function Recalculate Corresponding pseudo-random seed:
[0152]
[0153] Then use the same pseudo-random function Reconstruction Score at each level:
[0154]
[0155] S43. Independence Guarantee: The reconstruction process relies solely on the text itself and the watermark key. and preset and It does not require calling any interfaces or parameters of the original large model, which can be mathematically represented as:
[0156]
[0157] in, It is a pure function (without external dependencies), ensuring the efficiency and versatility of the detection process.
[0158] S5. Calculate the average pseudo-random score of all words in the text as the aggregation score and determine the statistical threshold based on the preset false positive rate. If the statistical threshold is less than the aggregation score of the unknown text, it is determined to be the generated text with watermark; otherwise, it is the text without watermark.
[0159] Furthermore, S5 specifically includes:
[0160] S51. Aggregated Watermark Score Calculation: Text to be Detected All morphemes Calculate its in all The mean of the pseudo-random scores of the layer is used as the aggregate score:
[0161]
[0162] in, This indicates the reconstruction of each word. In the scores at each level, T is the length of the corresponding word sequence;
[0163] Because the generation stage (S3) systematically selects high-scoring terms, the watermarked text... Statistically significantly higher than watermark-free text (human-written text or text without embedded watermarks);
[0164] S52. Determination of statistical threshold: Threshold The determination is based on the preset false positive rate (FPR), and the steps are as follows: Collect a sufficient set of watermark-free text samples. ; Calculate the aggregate score for each sample The fraction distribution is obtained. Take the distribution Quantiles are used as thresholds, that is:
[0165]
[0166] S53. Final Judgment Rule: For unknown text ,like If the result is positive, it is determined to be generated text with a watermark; otherwise, it is determined to be text without a watermark. Mathematically, this is expressed as:
[0167]
[0168] Here, Decision(x) is the decision function for watermark detection. The input is the text to be detected, x, and the output is the judgment result of whether the text carries the target watermark.
[0169] This invention also provides an electronic device. Figure 3 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 3 As shown, the electronic device may include a processor 310, a communications interface 320, a memory 330, and a communication bus 340. The processor 310, communications interface 320, and memory 330 communicate with each other via the communication bus 340. The processor 310 can call logical instructions stored in the memory 330 to execute the large-model watermarking method based on dynamic entropy.
[0170] Furthermore, the logical instructions in the aforementioned memory 330 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0171] This invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the large model watermarking method based on dynamic entropy.
[0172] This invention also provides a computer program product, including a computer program that, when executed by a processor, implements the large model watermarking method based on dynamic entropy.
[0173] This invention also provides a large model watermarking system based on dynamic entropy, the system comprising: a module for executing the large model watermarking method based on dynamic entropy.
[0174] This invention is based on a core dynamic adaptive framework: on the one hand, it precisely quantifies the uncertainty of the model at each generation step by calculating the Shannon entropy of the probability distribution output by a large language model in real time; on the other hand, it dynamically adjusts the watermark embedding strategy, such as the size of the candidate lexical pool and the length of the context window, based on this entropy value to ensure that the watermark strength matches the model state. This design solves the problem of existing static watermarking methods being unable to adapt to dynamic model changes, resulting in a trade-off between watermark detectability and text quality; furthermore, through a novel multi-layer competitive sampling algorithm (Tournament Sampling), it embeds statistical signals strongly correlated with the watermark key into the text without theoretically compromising text quality, achieving a balance between high fidelity and high detectability. Moreover, by completely decoupling the detection process from the original large model and relying solely on the watermark key for efficient statistical verification, this invention provides a feasible path for the large-scale, low-cost deployment of watermarking technology, thereby significantly enhancing the ability to trace and govern AI-generated content.
[0175] Table 1 compares the results of this invention with those of other methods.
[0176] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.
[0177] The above-described contents can be implemented individually or in various combinations, and these variations are all within the protection scope of this invention.
[0178] It should be noted that in this invention, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the term "comprising" or any other variations thereof is intended to cover non-exclusive inclusion, such that an article or device comprising a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such an article or device. Without further limitation, an element defined by the phrase "comprising..." does not exclude the presence of other identical elements in the article or device comprising said element.
[0179] The above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. The present invention has been described in detail with reference to preferred embodiments. Those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications and substitutions should be covered within the scope of the claims of the present invention.
Claims
1. A dynamic entropy-based large model watermarking method, characterized by: Comprising the following steps: S1. The watermark module embeds the text generation process of the large model based on a watermark key, modifies the output token sampling process without affecting the model training; S2. According to the probability distribution entropy value output by the large model, dynamically adjust the size of the sliding window and the size of the candidate token pool to balance diversity and watermark embedding efficiency; S3. Through pseudo-random scoring and multi-round competition, the final output token carries the watermark signal bound to the key while preserving the fluency of the large model text; S4. For each token in the sequence to be analyzed, use the same watermark key and the context before the token to recalculate the pseudo-random seed, and regenerate the corresponding pseudo-random score at all levels based on the seed; S5. Calculate the average pseudo-random score of all tokens in the text as the aggregate score and determine the statistical threshold based on the preset false positive rate. If the statistical threshold is less than the aggregate score of the unknown text, it is determined to be a watermark-generated text, otherwise it is a non-watermark text.
2. The method of claim 1, wherein: The S1 specifically comprises: S11. For computer system architecture, comprising: The deployment of the computer system contains two core components: Large language model, responsible for generating vocabulary probability distribution based on input context; Watermark module: a functional module independent of the large model, pre-configured with a secret watermark key , For security parameters, acting on the word sampling link of the large model; S12. For key attributes, including: Watermark key Satisfies: Randomness: generated by a cryptographically secure pseudo-random number generator, subject to a uniform distribution ; denotes a uniform distribution; Confidentiality: Only the generation end and detection end of the watermark module hold it, transmitted through an encrypted channel; S13. For embedding methods, including: The watermark module accesses the decoding process of the large model through an API interface, inserts candidate word token screening-watermark embedding logic after the large model outputs the vocabulary probability distribution and before the final word token is determined. The candidate word token screening-watermark embedding logic is mathematically expressed as: Let the original sampling function of the large model be , is the sequence of previous word tokens, then the watermark module is modified as: ； wherein, is a core function of the watermark module, for ensuring that the output tokens still come from the support set of the model, without introducing tokens unseen by the model, denotes the probability distribution output by the large model, denotes: the probability distribution output by the large language model based on the sequence of preceding tokens over all tokens in the vocabulary.
3. The method of claim 1, wherein: The S2 specifically comprises: S21. Output quantization of entropy: Let the large model at time step The output vocabulary probability distribution is , for vocabulary, for token , the probability of the distribution is: ； S22. Dynamic sampling strategy: preset an entropy threshold: high entropy threshold and low entropy threshold , the relationship between the number of samples and is a piecewise function: ； wherein, is a medium entropy threshold, to ensure more diversity in the pool when high entropy and more concentration when low entropy. S23. Candidate token pool construction: sampling from the vocabulary probability distribution independently , obtaining a candidate token pool wherein, represents that repeated sampling is allowed to preserve the frequency of appearance of high-probability tokens, represents the candidate token obtained by the th sampling; S24. Perform dynamic window size expansion; adjust the size of the sliding window H1 in real time according to the dynamic entropy generated at each step; this method changes the fixed hyperparameter H1 into a dynamic variable H that changes with context t ; When generating the tth token, first calculate the current context, the Shannon entropy of the large model output probability distribution: ； wherein, represents the corresponding large model based on the sequence of preceding tokens generated vocabulary probability distribution; Then, a mapping function is introduced which determines the dynamic window size to be used at the current step based on the Shannon entropy ； The function is designed as a piecewise function to map different entropy value intervals to preset window sizes. Under this dynamic mechanism, the seed generation formula in step S1 is updated to: ； wherein, is a pseudo-random seed, h is a cryptographic hash function, is a sequence of previous dynamic window tokens when the tth token is generated, is a dynamically adjusted window size, denotes the previous token for generating the tth token, is the last element of the dynamic window sequence, and k is a watermark key.
4. The method of claim 1, wherein: The S3 specifically comprises: S31. Pseudo-random seed generation: Let the preceding context be the most recent. Each word element, that is A pseudo-random seed is generated using a cryptographic hash function h: ； wherein, is a hash function satisfying pseudo-randomness, is a security parameter; Hth Cartesian product representing the vocabulary V, i.e. the set of all sequences of H-length word elements; k is the watermark key, t is the time step, is a pseudo-random seed, a cryptographic hash function h generating the pseudo-random seed, the window size is changed from fixed to dynamic; S32. Pseudo-random score calculation: Let the number of competing layers be , , for each token in the candidate token pool , , the pseudo-random score at the -th layer is defined as: ； wherein, , is a pseudo-random function used to ensure that scores for the same token under different layers, different seeds are independent and uniformly distributed, is a vocabulary; S33. Multi-round elimination mechanism: candidate token pool size needs to meet the sampling number To support the complete binary tree competition: Round 1: For each wordpiece, compute randomly into groups, with highest scores, and the remaining wordpieces; Round 2: randomly divide the remaining tokens into groups, keep the top scoring tokens from each group, and discard the remaining tokens; Round m: Final 2 wordpiece competition, keep Top-scoring wordpiece as output wordpiece ; Mathematically, the output token satisfies: ； To ensure that it comes from both the high probability distribution of the large model and performs optimally in the pseudo-random score.
5. The method of claim 1, wherein: The S4 specifically comprises: S41. Tokenization and context extraction: for each text to be inspected , split it into a sequence of tokens using a tokenizer consistent with the generation phase, and for each token extract its pre-context tokens , if take all the pre-context tokens; S42. Pseudo-random seed and fraction reconstruction: using the same watermark key as in the generation phase and a hash function , re-computing the corresponding pseudo-random seed: ； through the same pseudo-random function , reconstruct Fractions at each layer: ； S43. Independence guarantee: the reconstruction process only relies on the text itself, the watermark key and the pre-set and , , without invoking any interface or parameters of the original large model, mathematically represented as: ； wherein is a pure function.
6. The method of claim 1, wherein: The S5 specifically comprises: S51. Aggregate watermark score computation: For all word tokens of the text under test compute their average of the pseudo-random scores at all layers as the aggregate score: S52. Compute the watermark score of the text under test as the maximum of the individual scores and the aggregate score. ； wherein, denotes reconstructing each wordpiece the score at each layer, T is the length of the corresponding wordpiece sequence; S52. Determination of the statistical threshold: threshold Based on the preset false positive rate FPR determination, the steps are as follows: collect a sufficient amount of watermark-free text sample set ; Calculate the aggregate score of each sample , get the score distribution ; Take the quantile of the distribution as the threshold, that is: ； S53. Final decision rule: for unknown text , if , then decide as watermarked generated text; otherwise decide as non-watermarked text.
7. A dynamic entropy-based large model watermarking system, characterized in that, The system comprises a module for executing a large model watermark method based on dynamic entropy as claimed in any one of claims 1 to 6.
8. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the program to implement a large model watermark method based on dynamic entropy as claimed in any one of claims 1 to 6. 9.A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement a large model watermark method based on dynamic entropy as claimed in any one of claims 1 to 6.
10. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement a large model watermark method based on dynamic entropy as claimed in any one of claims 1 to 6.