A large model watermarking method based on dynamic entropy

Abstract

The application discloses a large model watermarking method based on dynamic entropy, belonging to the field of large model watermarking. It includes: modifying the output token sampling process; dynamically adjusting the size of the sliding window and the size of the candidate token pool according to the probability distribution entropy value of the large model output; through pseudo-random scoring and multi-round competition; for each token in the sequence to be analyzed, the same watermark key and the context before the token are used to recalculate the pseudo-random seed, and based on the seed, the corresponding pseudo-random score of the token at all levels is regenerated; the average pseudo-random score of all tokens in the text is calculated as the aggregate score, and the statistical threshold is determined based on the preset false positive rate. If the statistical threshold is less than the aggregate score of the unknown text, it is determined as a generated text with watermark, otherwise it is a text without watermark. The application realizes high-precision source identification and tracing of large model output content through the dynamic entropy watermarking method while ensuring the original quality of the generated text.

Description

Technical Field

[0001] This invention relates to a large model watermarking method based on dynamic entropy, belonging to the field of large model watermarking. Background Technology

[0002] Large language models, especially those based on the Transformer architecture, have achieved near-human levels of text generation, with their output often indistinguishable from human-written text. This capability is widely used in applications ranging from chatbots to code generation. However, this indistinguishability also brings significant risks, including the spread of misinformation, academic misconduct, and potential copyright infringement. Furthermore, the unrestrained use of data generated by large models to train future models carries the risk of "model collapse," a recursive degradation of model quality and diversity. Therefore, there is an urgent need to establish robust mechanisms that can effectively detect and trace text generated by large models.

[0003] To address this issue, the industry has proposed various detection methods, but each of them has significant limitations.

[0004] The first category of methods is "post-hoc detectors." These methods analyze text after it has been generated, attempting to distinguish between human-generated and AI-generated text by finding statistical features or training a classifier. The main weakness of this type of method lies in its inherent unreliability. Their performance is inconsistent, they are prone to high false positive rates, and they can be easily circumvented by simple rewriting or adversarial attacks. As large models continue to improve, their statistical features become increasingly similar to those of humans, and the effectiveness of post-hoc detectors decreases accordingly. Furthermore, these detectors inherently provide probabilistic evaluations rather than deterministic evidence, making them unsuitable for high-risk applications. The second category of methods is "retrieval-based detectors." This method stores all generated text in a large database and detects content through matching. The main weakness of this method is its impracticality at the scale of modern large model deployments, not only due to its high computational cost but also because it raises serious privacy issues. More importantly, it is highly vulnerable to rewriting attacks, where attackers can modify the text to circumvent exact or approximate matches while preserving its semantics.

[0005] Given the limitations of the methods mentioned above, generative watermarking is considered a superior solution. This method embeds the signal during text generation by modifying the sampling process of the next word. Detection can be performed without accessing the original large model. A representative generative watermarking method in the prior art (e.g., the method proposed by Kirchenbauer et al.) is based on a "static bias" strategy. This method divides the vocabulary into a "green list" and a "red list" based on a key and the preceding text, and then applies a fixed, static bias value to the logits of words in the green list to increase their probability of being selected.

[0006] However, the performance of generative watermarking depends on the entropy of the probability distribution of the large model at each generation step. Entropy is a measure of the uncertainty of a system. When the entropy is low, the model is very certain about the next term, and its probability distribution exhibits sharp peaks. In this case, forcibly selecting a different watermarked term (e.g., a "green list" term) would severely damage the quality and coherence of the text. Therefore, a fixed bias is either too weak to be effective or too strong to produce obvious textual flaws. Conversely, when the entropy is high, the model is uncertain about the next term, with multiple reasonable options and a relatively flat probability distribution. This provides an excellent opportunity to embed a strong watermark, as the selection can be slightly guided to favor a desired term without affecting the user-perceived text quality. Therefore, existing technologies lack a watermarking method that can adjust its embedding strategy in real time based on the dynamic entropy during the large model's generation process. This invention aims to address this technological gap. Summary of the Invention

[0007] This invention aims to overcome the limitations of existing large-model watermarking techniques in balancing watermark detectability, text quality preservation, and computational efficiency, and provides a large-model watermarking method based on dynamic entropy. Specifically, the core technical problem addressed by this invention is that existing static watermarking methods cannot adapt to the dynamic characteristics of the prediction uncertainty in large language models.

[0008] The technical solution of the present invention is as follows: In a first aspect, the present invention provides a large-model watermarking method based on dynamic entropy, comprising the following steps:

[0009] S1. The watermarking module is based on a watermarking key and is embedded in the text generation process of the large model. It modifies the lexical sampling process of the output without affecting the model training.

[0010] S2. Based on the probability distribution entropy value output by the large model, dynamically adjust the sliding window size and the candidate word pool size to balance diversity and watermark embedding efficiency;

[0011] S3. Pseudo-random scoring and multi-round competition are used to ensure that the final output tokens carry a watermark signal bound to the key, while preserving the text fluency of the large model;

[0012] S4. For each word in the sequence to be analyzed, recalculate the pseudo-random seed using the same watermark key and the context preceding the word, and regenerate its corresponding pseudo-random score at all levels based on this seed.

[0013] S5. Calculate the average pseudo-random score of all words in the text as the aggregation score and determine the statistical threshold based on the preset false positive rate. If the statistical threshold is less than the aggregation score of the unknown text, it is determined to be the generated text with watermark; otherwise, it is the text without watermark.

[0014] Furthermore, S1 specifically includes:

[0015] S11. Regarding computer system architecture, including:

[0016] Deploying a computer system involves two core components:

[0017] The large language model is responsible for generating the probability distribution of the vocabulary based on the input context.

[0018] Watermarking module: A functional module independent of the main model, pre-configured with secure watermarking keys. , As a safety parameter, it is applied to the lexical sampling stage of the large model;

[0019] S12. For key attributes, including:

[0020] Watermark Key satisfy:

[0021] Randomness: Generated by a cryptographically secure pseudo-random number generator, following a uniform distribution. ; Indicates a uniform distribution;

[0022] Confidentiality: The watermark is held only by the watermark generation and detection ends of the watermark module and transmitted through an encrypted channel;

[0023] S13. For embedding methods, including:

[0024] The watermarking module connects to the decoding process of the large model via an API interface. After the large model outputs the vocabulary probability distribution but before the final word determination, it inserts candidate word filtering and watermark embedding logic. Mathematically, this candidate word filtering and watermark embedding logic is represented as follows: Let the original sampling function of the large model be... , If the preceding word sequence is used, the watermark module is modified as follows:

[0025]

[0026] in, This is the core function of the watermarking module, used to ensure that the output tokens still come from... The support set does not include lexical units not seen by the model. This represents the probability distribution of the output of the large model. Representation: Large language models are based on preorder lexical sequences The probability distribution of outputs for all lexical terms in the vocabulary.

[0027] Furthermore, S2 specifically includes:

[0028] S21. Quantization of output entropy: Assume a large model at time step The probability distribution of the output vocabulary is as follows , For the vocabulary list, as a word element If the probability is zero, then the Shannon entropy of this distribution is:

[0029]

[0030] S22. Dynamic sampling strategy: Preset individual entropy threshold: High entropy threshold and low entropy threshold Number of samplings and The relationship is a piecewise function:

[0031]

[0032] in, It is the medium entropy threshold. This is used to ensure that the candidate pool is more diverse when the entropy is high and more concentrated when the entropy is low.

[0033] S23. Construction of the candidate lexical pool: from the probability distribution of the vocabulary Independent sampling Next, the candidate word pool was obtained. ,in, This indicates that repeated sampling is allowed to preserve the frequency of high-probability words. Indicates the first Candidate word elements obtained from the second sampling;

[0034] S24. Perform dynamic window size expansion; adjust the sliding window size H1 in real time based on the dynamic entropy generated at each step; this method transforms the fixed hyperparameter H1 into a dynamic variable H that changes with the context. t ;

[0035] When generating the t-th word, the current context is first calculated, and the Shannon entropy of the probability distribution output by the large model is:

[0036]

[0037] in, This indicates that the corresponding large model is based on the preceding word sequence. Probability distribution of the generated vocabulary;

[0038] Then, a mapping function is introduced. This function is based on Shannon entropy. Determine the dynamic window size to use in the current step :

[0039]

[0040] This function The design is a piecewise function used to map different entropy value ranges to a preset window size. For example, a minimum window size can be set, and a larger window can be used when the entropy value is high.

[0041]

[0042] in, ≥4, and It is a pre-set entropy threshold. , , These are the maximum context window size, the medium context window size, and the minimum context window size, respectively.

[0043] Under this dynamic mechanism, the seed generation formula in step S1 is updated to:

[0044] ;

[0045] in, It is a pseudo-random seed, and h is a cryptographic hash function. It is the preorder dynamic window lexical sequence when generating the t-th lexical. This is the dynamically adjusted window size. This represents the word preceding the generation of the t-th word, and is the last element of the dynamic window sequence.

[0046] Furthermore, S3 specifically includes:

[0047] S31. Pseudo-random seed generation: Let the preceding context be the most recent. Each word element, that is A pseudo-random seed is generated using a cryptographic hash function h:

[0048]

[0049] in, It is a hash function that satisfies pseudo-randomness. For safety parameters;

[0050] Let represent the H-th Cartesian product of the vocabulary V, i.e., the set of all word sequences of length H; k is the watermark key, and t is the time step. It is a pseudo-random seed, and the window size of the cryptographic hash function h that generates the pseudo-random seed changes from fixed to dynamic;

[0051] S32. Pseudo-random score calculation: Let the number of competition layers be... , For candidate word pool Each word in In the The pseudo-random score of the layer is defined as:

[0052]

[0053] in, , This is a pseudo-random function used to ensure that the scores of the same word are independently and uniformly distributed across different layers and seeds. For a vocabulary list;

[0054] S33. Multi-round elimination mechanism: The size of the candidate word pool must meet the sampling number requirements. To support full binary tree competition:

[0055] Round 1: Randomly divided into Groups, each group retains The highest-scoring word, the remaining Each word element;

[0056] Round 2: Randomly divide the remaining word units into... Groups, each group retains The highest-scoring word, the remaining Each word element;

[0057] Round m: The final two word groups compete, and the remaining words are retained. The word with the highest score is used as the output word. ;

[0058] In mathematics, outputting word units satisfy:

[0059]

[0060] This is to ensure that it comes from a high probability distribution of a large model and performs optimally in pseudo-random scoring.

[0061] Furthermore, S4 specifically includes:

[0062] S41. Word segmentation and context extraction: the text to be detected The tokens are split into a sequence of words using the same segmenter as in the generation stage. For each word... Extract its preorder each word element ,like Then take all preceding words;

[0063] S42. Reconstruction of Pseudo-random Seeds and Scores: Using the same watermark key as in the generation stage. and hash function Recalculate Corresponding pseudo-random seed:

[0064]

[0065] Then use the same pseudo-random function Reconstruction Score at each level:

[0066]

[0067] S43. Independence Guarantee: The reconstruction process relies solely on the text itself and the watermark key. and preset and It does not require calling any interfaces or parameters of the original large model, which can be mathematically represented as:

[0068]

[0069] in, It is a pure function.

[0070] Furthermore, S5 specifically includes:

[0071] S51. Aggregated Watermark Score Calculation: Text to be Detected All morphemes Calculate its in all The mean of the pseudo-random scores of the layer is used as the aggregate score:

[0072]

[0073] in, This indicates the reconstruction of each word. In the scores at each level, T is the length of the corresponding word sequence;

[0074] S52. Determination of statistical threshold: Threshold The determination is based on the preset false positive rate (FPR), and the steps are as follows: Collect a sufficient set of watermark-free text samples. ; Calculate the aggregate score for each sample The fraction distribution is obtained. Take the distribution Quantiles are used as thresholds, that is:

[0075]

[0076] S53. Final Judgment Rule: For unknown text ,like If the result is positive, it is determined to be generated text with a watermark; otherwise, it is determined to be text without a watermark. Mathematically, this is expressed as:

[0077]

[0078] Here, Decision(x) is the decision function for watermark detection. The input is the text to be detected, x, and the output is the judgment result of whether the text carries the target watermark.

[0079] Secondly, the present invention also provides a large model watermarking system based on dynamic entropy, the system comprising: a module for executing the large model watermarking method based on dynamic entropy.

[0080] Thirdly, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the large model watermarking method based on dynamic entropy.

[0081] Fourthly, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the large model watermarking method based on dynamic entropy.

[0082] Fifthly, the present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the large model watermarking method based on dynamic entropy.

[0083] Compared with existing technologies, the advantages of this invention are: it can embed a robust and highly detectable watermark when the generation context allows (high entropy), while not sacrificing text quality or coherence when the model output is highly constrained (low entropy). This adaptive mechanism makes the final embedded watermark stronger overall, more resistant to attacks, and has negligible impact on user experience. Attached Figure Description

[0084] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0085] Figure 1 This is a flowchart illustrating a large-model watermarking method based on dynamic entropy provided by the present invention.

[0086] Figure 2 This is a flowchart illustrating the architecture of a large-model watermarking method based on dynamic entropy provided by the present invention.

[0087] Figure 3 This is a schematic diagram of the structure of the electronic device provided by the present invention.

[0088] The labels in the diagram are as follows:

[0089] 310 - Processor; 320 - Communication interface; 330 - Memory; 340 - Communication bus. Detailed Implementation

[0090] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. It should be noted that, unless otherwise specified, the embodiments and features in the embodiments of this application can be arbitrarily combined with each other.

[0091] It should be noted that the executing entity of the embodiments of the present invention can be various types of terminals, such as computers, smartphones, tablets, wearable devices, personal digital assistants (PDAs), mobile internet devices (MIDs), and other terminals capable of text processing. The present invention does not impose any restrictions on this.

[0092] Example 1: As Figures 1-2 As shown, this embodiment of the invention provides a large-model watermarking method based on dynamic entropy, including the following steps:

[0093] S1. The watermarking module is based on a watermarking key and is embedded in the text generation process of the large model. It modifies the lexical sampling process of the output without affecting the model training.

[0094] Furthermore, S1 specifically includes:

[0095] S11. Regarding computer system architecture, including:

[0096] Deploying a computer system involves two core components:

[0097] The large language model is responsible for generating the probability distribution of the vocabulary based on the input context.

[0098] Watermarking module: A functional module independent of the main model, pre-configured with secure watermarking keys. , As a safety parameter, it is applied to the lexical sampling stage of the large model;

[0099] S12. For key attributes, including:

[0100] Watermark Key satisfy:

[0101] Randomness: Generated by a cryptographically secure pseudo-random number generator, following a uniform distribution. ; Indicates a uniform distribution;

[0102] Confidentiality: The watermark is held only by the watermark generation and detection ends of the watermark module and transmitted through an encrypted channel;

[0103] S13. For embedding methods, including:

[0104] The watermarking module connects to the decoding process of the large model via an API interface. After the large model outputs the vocabulary probability distribution but before the final word determination, it inserts candidate word filtering and watermark embedding logic. Mathematically, this candidate word filtering and watermark embedding logic is represented as follows: Let the original sampling function of the large model be... , If the preceding word sequence is used, the watermark module is modified as follows:

[0105]

[0106] in, This is the core function of the watermarking module, used to ensure that the output tokens still come from... The support set does not include lexical units not seen by the model. p_LLM represents the probability distribution of the output of a large language model. It is an abbreviation for Probability Distribution of Large Language Model. Combined with the formula... Representation: Large language models are based on preorder lexical sequences (Generate all lexical units before the t-th lexical unit). The probability distribution of all lexical units output in the vocabulary is the basis for the watermarking module to perform lexical unit filtering and watermark embedding.

[0107] S2. Based on the probability distribution entropy value output by the large model, dynamically adjust the sliding window size and the candidate word pool size to balance diversity and watermark embedding efficiency;

[0108] Furthermore, S2 specifically includes:

[0109] S21. Quantization of output entropy: Assume a large model at time step The probability distribution of the output vocabulary is as follows , For the vocabulary list, as a word element If the probability is zero, then the Shannon entropy of this distribution is:

[0110]

[0111] The larger the value, the higher the uncertainty (greater diversity) of the large model regarding the next word element.

[0112] S22. Dynamic sampling strategy: Preset individual entropy threshold: High entropy threshold and low entropy threshold Number of samplings and The relationship is a piecewise function:

[0113]

[0114] in, It is the medium entropy threshold. This is used to ensure that the candidate pool is more diverse when the entropy is high and more concentrated when the entropy is low.

[0115] S23. Construction of the candidate lexical pool: from the probability distribution of the vocabulary Independent sampling Next, the candidate word pool was obtained. ,in, This indicates that repeated sampling is allowed to preserve the frequency of high-probability words. Indicates the first Candidate word elements obtained from the second sampling;

[0116] S24. Perform dynamic window size expansion; adjust the sliding window size H1 in real time based on the dynamic entropy generated at each step; this method transforms the fixed hyperparameter H1 into a dynamic variable H that changes with the context. tThis allows the watermark embedding strategy to be more tightly coupled with the uncertainty of the model;

[0117] When generating the t-th word, the current context is first calculated, and the Shannon entropy of the probability distribution output by the large model is:

[0118]

[0119] in, This indicates that the corresponding large model is based on the preceding word sequence. Probability distribution of the generated vocabulary;

[0120] Then, a mapping function is introduced. This function is based on Shannon entropy. Determine the dynamic window size to use in the current step :

[0121]

[0122] This function The design is a piecewise function used to map different entropy value ranges to a preset window size. For example, a minimum window size can be set, and a larger window can be used when the entropy value is high.

[0123]

[0124] in, ≥4, and It is a pre-set entropy threshold. , , These are the maximum context window size, the medium context window size, and the minimum context window size, respectively.

[0125] Maximum context window size, when entropy value ≥ Used when the entropy threshold is high to ensure that there are enough preceding words to generate pseudo-random seeds in high-entropy scenarios, thereby enhancing the watermark signal strength;

[0126] Medium context window size, when ≤Entropy value< Used in medium-entropy scenarios to balance context coverage and computational efficiency;

[0127] Minimum context window size (≥4), when entropy value < Used in low-entropy scenarios to avoid contextual redundancy due to excessively large windows, while ensuring the effectiveness of seed generation.

[0128] The core function of these three components is to adapt to different entropy scenarios and ensure the rationality of pseudo-random seed generation.

[0129] Under this dynamic mechanism, the seed generation formula in step S1 is updated to:

[0130] ;

[0131] in, It is a pseudo-random seed, and h is a cryptographic hash function. This is the preorder dynamic window word sequence when generating the t-th word. This sequence is one of the inputs to the hash function, used to ensure that the seed is strongly correlated with the current context. This is the dynamically adjusted window size. This represents the word preceding the generation of the t-th word, and is the last element of the dynamic window sequence, ensuring the immediacy of the context.

[0132] S3. Pseudo-random scoring and multi-round competition are used to make the final output tokens carry watermark signals bound to the key, while preserving the text fluency of the large model;

[0133] Furthermore, S3 specifically includes:

[0134] S31. Pseudo-random seed generation: Let the preceding context be the most recent. each word element ( , usually take ),Right now A pseudo-random seed is generated using a cryptographic hash function h:

[0135]

[0136] in, It is a hash function that satisfies pseudo-randomness. For safety parameters;

[0137] Let represent the H-th Cartesian product of the vocabulary V, i.e., the set of all word sequences of length H; k is the watermark key, a core confidentiality parameter for pseudo-random seed generation, held only by the generator and detector; t is the time step. It is a pseudo-random seed, and the window size of the cryptographic hash function h that generates the pseudo-random seed changes from fixed to dynamic;

[0138] S32. Pseudo-random score calculation: Let the number of competition layers be... , , For candidate word pool Each word in In the The pseudo-random score of the layer is defined as:

[0139]

[0140] in, , This is a pseudo-random function used to ensure that the scores of the same word are independently and uniformly distributed across different layers and seeds. For a vocabulary list;

[0141] S33. Multi-round elimination mechanism: The size of the candidate word pool must meet the sampling number requirements. To support full binary tree competition:

[0142] Round 1: Randomly divided into Groups, each group retains The highest-scoring word, the remaining Each word element;

[0143] Round 2: Randomly divide the remaining word units into... Groups, each group retains The highest-scoring word, the remaining Each word element;

[0144] Round m: The final two word groups compete, and the remaining words are retained. The word with the highest score is used as the output word. ;

[0145] In mathematics, outputting word units satisfy:

[0146]

[0147] This is to ensure that it comes from a high probability distribution of a large model and performs optimally in pseudo-random scoring.

[0148] S4. For each word in the sequence to be analyzed, recalculate the pseudo-random seed using the same watermark key and the context preceding the word, and regenerate its corresponding pseudo-random score at all levels based on this seed.

[0149] Furthermore, S4 specifically includes:

[0150] S41. Word segmentation and context extraction: the text to be detected The tokens are split into a sequence of words using the same segmenter as in the generation stage. For each word... Extract its preorder each word element ,like Then take all preceding words;

[0151] S42. Reconstruction of Pseudo-random Seeds and Scores: Using the same watermark key as in the generation stage. and hash function Recalculate Corresponding pseudo-random seed:

[0152]

[0153] Then use the same pseudo-random function Reconstruction Score at each level:

[0154]

[0155] S43. Independence Guarantee: The reconstruction process relies solely on the text itself and the watermark key. and preset and It does not require calling any interfaces or parameters of the original large model, which can be mathematically represented as:

[0156]

[0157] in, It is a pure function (without external dependencies), ensuring the efficiency and versatility of the detection process.

[0158] S5. Calculate the average pseudo-random score of all words in the text as the aggregation score and determine the statistical threshold based on the preset false positive rate. If the statistical threshold is less than the aggregation score of the unknown text, it is determined to be the generated text with watermark; otherwise, it is the text without watermark.

[0159] Furthermore, S5 specifically includes:

[0160] S51. Aggregated Watermark Score Calculation: Text to be Detected All morphemes Calculate its in all The mean of the pseudo-random scores of the layer is used as the aggregate score:

[0161]

[0162] in, This indicates the reconstruction of each word. In the scores at each level, T is the length of the corresponding word sequence;

[0163] Because the generation stage (S3) systematically selects high-scoring terms, the watermarked text... Statistically significantly higher than watermark-free text (human-written text or text without embedded watermarks);

[0164] S52. Determination of statistical threshold: Threshold The determination is based on the preset false positive rate (FPR), and the steps are as follows: Collect a sufficient set of watermark-free text samples. ; Calculate the aggregate score for each sample The fraction distribution is obtained. Take the distribution Quantiles are used as thresholds, that is:

[0165]

[0166] S53. Final Judgment Rule: For unknown text ,like If the result is positive, it is determined to be generated text with a watermark; otherwise, it is determined to be text without a watermark. Mathematically, this is expressed as:

[0167]

[0168] Here, Decision(x) is the decision function for watermark detection. The input is the text to be detected, x, and the output is the judgment result of whether the text carries the target watermark.

[0169] This invention also provides an electronic device. Figure 3 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 3 As shown, the electronic device may include a processor 310, a communications interface 320, a memory 330, and a communication bus 340. The processor 310, communications interface 320, and memory 330 communicate with each other via the communication bus 340. The processor 310 can call logical instructions stored in the memory 330 to execute the large-model watermarking method based on dynamic entropy.

[0170] Furthermore, the logical instructions in the aforementioned memory 330 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0171] This invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the large model watermarking method based on dynamic entropy.

[0172] This invention also provides a computer program product, including a computer program that, when executed by a processor, implements the large model watermarking method based on dynamic entropy.

[0173] This invention also provides a large model watermarking system based on dynamic entropy, the system comprising: a module for executing the large model watermarking method based on dynamic entropy.

[0174] This invention is based on a core dynamic adaptive framework: on the one hand, it precisely quantifies the uncertainty of the model at each generation step by calculating the Shannon entropy of the probability distribution output by a large language model in real time; on the other hand, it dynamically adjusts the watermark embedding strategy, such as the size of the candidate lexical pool and the length of the context window, based on this entropy value to ensure that the watermark strength matches the model state. This design solves the problem of existing static watermarking methods being unable to adapt to dynamic model changes, resulting in a trade-off between watermark detectability and text quality; furthermore, through a novel multi-layer competitive sampling algorithm (Tournament Sampling), it embeds statistical signals strongly correlated with the watermark key into the text without theoretically compromising text quality, achieving a balance between high fidelity and high detectability. Moreover, by completely decoupling the detection process from the original large model and relying solely on the watermark key for efficient statistical verification, this invention provides a feasible path for the large-scale, low-cost deployment of watermarking technology, thereby significantly enhancing the ability to trace and govern AI-generated content.

[0175] Table 1 compares the results of this invention with those of other methods.

[0176]

[0177] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0178] The above-described contents can be implemented individually or in various combinations, and these variations are all within the protection scope of this invention.

[0179] It should be noted that in this invention, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the term "comprising" or any other variations thereof is intended to cover non-exclusive inclusion, such that an article or device comprising a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such an article or device. Without further limitation, an element defined by the phrase "comprising..." does not exclude the presence of other identical elements in the article or device comprising said element.

[0180] The above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. The present invention has been described in detail with reference to preferred embodiments. Those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications and substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A large-scale watermarking method based on dynamic entropy, characterized in that: Includes the following steps: S1. The watermarking module is based on a watermarking key and is embedded in the text generation process of the large model. It modifies the lexical sampling process of the output without affecting the model training. S2. Based on the probability distribution entropy value output by the large model, dynamically adjust the sliding window size and the candidate word pool size to balance diversity and watermark embedding efficiency; S3. Pseudo-random scoring and multi-round competition are used to make the final output tokens carry watermark signals bound to the key, while preserving the text fluency of the large model; S4. For each word in the sequence to be analyzed, recalculate the pseudo-random seed using the same watermark key and the context preceding the word, and regenerate its corresponding pseudo-random score at all levels based on this seed. S5. Calculate the average pseudo-random score of all words in the text as the aggregation score and determine the statistical threshold based on the preset false positive rate. If the statistical threshold is less than the aggregation score of the unknown text, it is determined to be the generated text with watermark; otherwise, it is the text without watermark. S4 specifically includes: S41. Word segmentation and context extraction: the text to be detected The tokens are split into a sequence of words using the same segmenter as in the generation stage. For each word... Extract its preorder each word element ,like Then take all preceding words; S42. Reconstruction of Pseudo-random Seeds and Scores: Using the same watermark key as in the generation stage. and hash function Recalculate Corresponding pseudo-random seed: ； Then use the same pseudo-random function Reconstruction Score at each level: ； S43. Independence Guarantee: The reconstruction process relies solely on the text itself and the watermark key. and preset and It does not require calling any interfaces or parameters of the original large model, which can be mathematically represented as: ； in, It is a pure function.

2. The large-scale watermarking method based on dynamic entropy according to claim 1, characterized in that: S1 specifically includes: S11. Regarding computer system architecture, including: Deploying a computer system involves two core components: The large language model is responsible for generating the probability distribution of the vocabulary based on the input context. Watermarking module: A functional module independent of the main model, pre-configured with secure watermarking keys. , As a safety parameter, it is applied to the lexical sampling stage of the large model; S12. For key attributes, including: Watermark Key satisfy: Randomness: Generated by a cryptographically secure pseudo-random number generator, following a uniform distribution. ; Indicates a uniform distribution; Confidentiality: The watermark is held only by the watermark generation and detection ends of the watermark module and transmitted through an encrypted channel; S13. For embedding methods, including: The watermarking module connects to the decoding process of the large model via an API interface. After the large model outputs the vocabulary probability distribution but before the final word determination, it inserts candidate word selection and watermark embedding logic. Mathematically, this candidate word selection and watermark embedding logic is represented as follows: Let the original sampling function of the large model be... , If the preceding word sequence is used, the watermark module is modified as follows: ； in, This is the core function of the watermarking module, used to ensure that the output tokens still come from... The support set does not include lexical units not seen by the model. This represents the probability distribution of the output of the large model. Representation: Large language models are based on preorder lexical sequences The probability distribution of outputs for all lexical terms in the vocabulary.

3. The large-scale watermarking method based on dynamic entropy according to claim 1, characterized in that: S2 specifically includes: S21. Quantization of output entropy: Assume a large model at time step The probability distribution of the output vocabulary is as follows , For the vocabulary list, as a word element If the probability is zero, then the Shannon entropy of this distribution is: ； S22. Dynamic sampling strategy: Preset individual entropy threshold: High entropy threshold and low entropy threshold Number of samplings and The relationship is a piecewise function: ； in, It is the medium entropy threshold. This is used to ensure that the candidate pool is more diverse when the entropy is high and more concentrated when the entropy is low. S23. Construction of the candidate lexical pool: from the probability distribution of the vocabulary Independent sampling Next, the candidate word pool was obtained. ,in, This indicates that repeated sampling is allowed to preserve the frequency of high-probability words. Indicates the first Candidate word elements obtained from the second sampling; S24. Perform dynamic window size expansion; adjust the sliding window size H1 in real time based on the dynamic entropy generated at each step; this method transforms the fixed hyperparameter H1 into a dynamic variable H that changes with the context. t ; When generating the t-th word, the current context is first calculated, and the Shannon entropy of the probability distribution output by the large model is: ； in, This indicates that the corresponding large model is based on the preceding word sequence. Probability distribution of the generated vocabulary; Then, a mapping function is introduced. This function is based on Shannon entropy. Determine the dynamic window size to use in the current step : ； This function The design is a piecewise function used to map different entropy value ranges to a preset window size; Under this dynamic mechanism, the seed generation formula in step S1 is updated to: ； in, It is a pseudo-random seed, and h is a cryptographic hash function. It is the preorder dynamic window lexical sequence when generating the t-th lexical. This is the dynamically adjusted window size. represents the word preceding the generation of the t-th word, is the last element of the dynamic window sequence, and k is the watermark key.

4. The large-scale watermarking method based on dynamic entropy according to claim 1, characterized in that: S3 specifically includes: S31. Pseudo-random seed generation: Let the preceding context be the most recent. Each word element, that is A pseudo-random seed is generated using a cryptographic hash function h: ； in, It is a hash function that satisfies pseudo-randomness. For safety parameters; Let represent the H-th Cartesian product of the vocabulary V, i.e., the set of all word sequences of length H; k is the watermark key, and t is the time step. It is a pseudo-random seed, and the window size of the cryptographic hash function h that generates the pseudo-random seed changes from fixed to dynamic; S32. Pseudo-random score calculation: Let the number of competition layers be... , For candidate word pool Each word in In the The pseudo-random score of the layer is defined as: ； in, , This is a pseudo-random function used to ensure that the scores of the same word are independently and uniformly distributed across different layers and seeds. For a vocabulary list; S33. Multi-round elimination mechanism: The size of the candidate word pool must meet the sampling number requirements. To support full binary tree competition: Round 1: Randomly divided into Groups, each group retains The highest-scoring word, the remaining Each word element; Round 2: Randomly divide the remaining word units into... Groups, each group retains The highest-scoring word, the remaining Each word element; Round m: The final two word groups compete, and the remaining words are retained. The word with the highest score is used as the output word. ; In mathematics, outputting word units satisfy: ； This is to ensure that it comes from a high probability distribution of a large model and performs optimally in pseudo-random scoring.

5. The large-scale watermarking method based on dynamic entropy according to claim 1, characterized in that: S5 specifically includes: S51. Aggregated Watermark Score Calculation: Text to be Detected All morphemes Calculate its in all The mean of the pseudo-random scores of the layer is used as the aggregate score: ； in, This indicates the reconstruction of each word. In the scores at each level, T is the length of the corresponding word sequence; S52. Determination of statistical threshold: Threshold The determination is based on the preset false positive rate (FPR), and the steps are as follows: Collect a sufficient set of watermark-free text samples. ; Calculate the aggregate score for each sample The fraction distribution is obtained. Take the distribution Quantiles are used as thresholds, that is: ； S53. Final Judgment Rule: For unknown text ,like If the result is positive, it is determined to be generated text with a watermark; otherwise, it is determined to be text without a watermark.

6. A large-scale watermarking system based on dynamic entropy, characterized in that, The system includes a module for performing a large model watermarking method based on dynamic entropy as described in any one of claims 1 to 5.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements a large model watermarking method based on dynamic entropy as described in any one of claims 1 to 5.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements a large model watermarking method based on dynamic entropy as described in any one of claims 1 to 5.

9. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements a large model watermarking method based on dynamic entropy as described in any one of claims 1 to 5.

Images